roles of memory, CPUs, solid state drive and hybrid drives in data processing

duquetter · Dec 31, 2014

I'm doing a custom build on a Dell Precision T7500 to process data efficiently. it will have dual Xeon x5670 6 core CPUs, 96GB of PC3-10600R memory, a 240GB Kingston solid state drive and two Seagate 4TB Hybrid drives. It will be used to process text files in batches of 50-75GB. the data will first be loaded onto the SSD then the processing started. I would think the data would end up being moved into the 96gB of memory for processing with some activity on the SSD, but I would think the hybrid 4TB hard drives would get used much and would not slow the processing down. Does that sound right? any comments or suggestions? Thank you.

Ron

geofelt · Dec 31, 2014

If you will be doing intensive production on a ssd, you might be worried about ssd longevity.
A SSD has a limited number of rewrite cycles.
You can reduce this exposure in two ways.
1. A larger ssd will have more nand chips and will last longer.
2. A better ssd will be more efficient in nand management.
Currently, I think the Samsung 850 PRO is the current best. I would look at a 500gb or possibly 1tb version.

I would not bother with hybrid hard drives. They do not have enough high performance storage to matter much. Hard drive performance is limited by mechanical data transfer rates. Look to larger drives which have denser platters that can transfer more data per revolution.

duquetter · Dec 31, 2014

geofelt :

If you will be doing intensive production on a ssd, you might be worried about ssd longevity.
A SSD has a limited number of rewrite cycles.
You can reduce this exposure in two ways.
1. A larger ssd will have more nand chips and will last longer.
2. A better ssd will be more efficient in nand management.
Currently, I think the Samsung 850 PRO is the current best. I would look at a 500gb or possibly 1tb version.

I would not bother with hybrid hard drives. They do not have enough high performance storage to matter much. Hard drive performance is limited by mechanical data transfer rates. Look to larger drives which have denser platters that can transfer more data per revolution.

duquetter · Dec 31, 2014

thank you. I am aware of the limited writes on an SSD, but didn't factor it in for long term. the unit will have 96GB of PC3-10600R memory and we'll be processing 50-75GB of data at a time. Do you think a lot of the data processing will be done in memory or have an idea how the work load will take place or be shared between the memory, SSD and hard drives? Lets assume I put in two 512GB SSDs. I would think that would be more efficient than the larger SATA/hybrid drives.

Search

roles of memory, CPUs, solid state drive and hybrid drives in data processing

duquetter

Distinguished

geofelt

Titan

duquetter

Distinguished

duquetter

Distinguished

TRENDING THREADS

Latest posts

Moderators online

Share this page