Sign in with
Sign up | Sign in
Your question

High-throughput data acquisition build

Last response: in Systems
Share
September 16, 2010 9:20:40 PM

Hello,

I'm looking to build a data acquisition system that will be writing up to 100MB/s sustained for a few hours every day. The data comes in through 1 or 2 gigabit ethernet ports. In parallel with the data acquisition, some experimental control and monitoring (display) processes will also be running.

What would be some considerations for doing this reliably? Some I thought of:
- No RAID, onboard (Intel ACH10R?), or full hardware RAID? Internal or external (eSATA) HDD cage?
- Chipset platform: is memory bandwidth going to make a difference?
- 2, 4, 6 cores?
- how to gauge the reliability of current mainboards?

I'd be grateful for suggestions.

Thanks!

- Mizumi
September 17, 2010 4:41:49 PM

I was thinking of buying the following (all items and prices from newegg.ca)

GIGABYTE GA-890FXA-UD5 AM3 AMD 890FX SATA 6Gb/s USB 3.0 ATX AMD Motherboard ($190; has dual gigabit ethernet)
AMD Phenom II X4 955 Black Edition Deneb 3.2GHz 4 x 512KB L2 Cache 6MB L3 Cache Socket AM3 125W Quad-Core Processor HDZ955FBGMBOX ($160)
Gskill Ripjaws 2x2GB memory kit ($90)
PNY VCQFX380-PCIE-PB Quadro FX 380 256MB 128-bit GDDR3 PCI Express 2.0 x16 Workstation Video Card ($135; to get dual monitors)
boot hdd: wd green 640GB in mirroring RAID ($120 for the pair, don't need that much space but green series is cheap & reliable?)
data hd: Samsung Spinpoint 3 1tb in striping RAID ($180 for the pair)
CORSAIR HX Series CMPSU-750HX 750W PSU ($160)
COOLER MASTER Centurion 5 CAC-T05-UW case ($60)
areca ARC-1110 PCI-X 64bit/133MHz SATA II RAID Controller Card - OEM ($300)
SNT SNT-SAC3141TL 4x3.5" Hard drive in 3x5.25" Bay SAS/SATA Trayless Backplane Hot Swap RAID Cage ($69)
Windows 7 OEM

Does that look reasonable?


m
0
l
September 17, 2010 6:13:57 PM

Would it be possible for you to describe a little more what the purpose of the computer is? Is this a database-like application (that will both write & query data), or is it just being fed data by some other application and writing it to flat files or something?

You can easily get dual (even triple) monitor support for much less, if I understand the purpose of your build correctly. An ATI 5450 should run about $40-50, and can support up to 3 monitors. The workstation card you have selected is probably overkill, unless your software can utilize CUDA, then you might want to stick with an nVidia card. Even then, you would probably be better off with a GTS 400 or GTS 200 series card, which can be had for around $100 or less.

Depending on whether or not your application is well-threaded, you might want to consider a Phenom II X6 1055T or 1090T. The 1055T should be about the same price as the X4 955, while the 1090T will be close to double the price.

Based on the price of the memory, it looks like you selected either 1066 MHz or 1333 MHz CL9, which will work fine, though if you anticipate long runs of number-crunching of some sort, you might consider 1333/1600 MHz CL7, which should get 1-2% improvement for another $10 or so.

The power supply is overkill for your needs...you could easily go with a 550W PSU. The one you chose is very good, but the major power draw on a PSU is usually the graphics card, then the CPU. Without a gaming graphics card, there's little need for more than 450-550W.

The WD Caviar Green series is generally considered a storage drive, not a boot drive. Consider getting a Samsung Spinpoint F3 500 GB instead. The Caviar Green is a 5400 rpm drive with power-saving spin-down features, not necessarily what you want for your boot drive.

RAID card seems like a good idea.
m
0
l
Related resources
September 17, 2010 9:36:08 PM

Thanks for the reply & helpful suggestions!

coldsleep said:
Would it be possible for you to describe a little more what the purpose of the computer is? Is this a database-like application (that will both write & query data), or is it just being fed data by some other application and writing it to flat files or something?


It is being fed data, through the ethernet port(s) by a purpose-built data acquisition system that records and digitizes wideband signals from large electrode arrays. The data is written to (as far as I know) nearly flat files with relatively little structure, just block headers and summary statistics every once in a while.

In parallel, though, some features about the data are extracted, displayed, and used to trigger some experimental control signals.

coldsleep said:
You can easily get dual (even triple) monitor support for much less, if I understand the purpose of your build correctly. An ATI 5450 should run about $40-50, and can support up to 3 monitors. [...]

Depending on whether or not your application is well-threaded, you might want to consider a Phenom II X6 1055T or 1090T. The 1055T should be about the same price as the X4 955, while the 1090T will be close to double the price.

Based on the price of the memory, it looks like you selected either 1066 MHz or 1333 MHz CL9, which will work fine, though if you anticipate long runs of number-crunching of some sort, you might consider 1333/1600 MHz CL7, which should get 1-2% improvement for another $10 or so.

The power supply is overkill for your needs...you could easily go with a 550W PSU. The one you chose is very good, but the major power draw on a PSU is usually the graphics card, then the CPU. Without a gaming graphics card, there's little need for more than 450-550W.

The WD Caviar Green series is generally considered a storage drive, not a boot drive. Consider getting a Samsung Spinpoint F3 500 GB instead. The Caviar Green is a 5400 rpm drive with power-saving spin-down features, not necessarily what you want for your boot drive.


All good points, I'll come up with a revised build soon. To support 3 monitors, the 5450 would need DisplayPort though, right?
m
0
l
September 17, 2010 10:41:29 PM

mizumi said:
All good points, I'll come up with a revised build soon. To support 3 monitors, the 5450 would need DisplayPort though, right?


Interesting info on the data.

I believe ATI has recently come out with passive DisplayPort to DVI adapters that go for $20 or $30, so should be an option. But yes, in order to use 3 monitors, you have to use the Displayport connector on the card.
m
0
l
September 17, 2010 11:54:27 PM

stupid question: if you need high speed data writing, why not a large-ish SSD? Expensive, but nothing is faster than SSDs in read/write.

Spinpoint F4 2 TB drives might be your best bet for large amounts of data on high speed mechanical drives
m
0
l
!