Building a 10000 dollar xeon workstation

wzhouiup · Feb 14, 2016

Hi, I am thinking of building a 10k xeon workstation for a guy who studies computational physics. Does this build look good?

[PCPartPicker part list](http://pcpartpicker.com/p/7xjRZL) / [Price breakdown by merchant](http://pcpartpicker.com/p/7xjRZL/by_merchant/)

Type|Item|Price
:----|:----|:----
**CPU** | [Intel Xeon E5-2697 V3 2.6GHz 14-Core Processor](http://pcpartpicker.com/part/intel-cpu-bx80644e52697v3) | $3024.27 @ Newegg
**CPU** | [Intel Xeon E5-2697 V3 2.6GHz 14-Core Processor](http://pcpartpicker.com/part/intel-cpu-bx80644e52697v3) | $3024.27 @ Newegg
**CPU Cooler** | [Noctua NH-D15 82.5 CFM CPU Cooler](http://pcpartpicker.com/part/noctua-cpu-cooler-nhd15) | $97.82 @ Amazon
**CPU Cooler** | [Noctua NH-D15 82.5 CFM CPU Cooler](http://pcpartpicker.com/part/noctua-cpu-cooler-nhd15) | $97.82 @ Amazon
**Motherboard** | [Asus Z10PE-D16 WS SSI EEB Dual-CPU LGA2011-3 Motherboard](http://pcpartpicker.com/part/asus-motherboard-z10ped16ws) |-
**Memory** | [Kingston 64GB (4 x 16GB) Registered DDR4-2133 Memory](http://pcpartpicker.com/part/kingston-memory-kvr21r15d4k464) | $474.13 @ Amazon
**Memory** | [Kingston 64GB (4 x 16GB) Registered DDR4-2133 Memory](http://pcpartpicker.com/part/kingston-memory-kvr21r15d4k464) | $474.13 @ Amazon
**Memory** | [Kingston 64GB (4 x 16GB) Registered DDR4-2133 Memory](http://pcpartpicker.com/part/kingston-memory-kvr21r15d4k464) | $474.13 @ Amazon
**Memory** | [Kingston 64GB (4 x 16GB) Registered DDR4-2133 Memory](http://pcpartpicker.com/part/kingston-memory-kvr21r15d4k464) | $474.13 @ Amazon
**Storage** | [Samsung 850 EVO-Series 250GB 2.5" Solid State Drive](http://pcpartpicker.com/part/samsung-internal-hard-drive-mz75e250bam) | $95.34 @ Amazon
**Storage** | [Hitachi Deskstar NAS 4TB 3.5" 7200RPM Internal Hard Drive](http://pcpartpicker.com/part/hitachi-internal-hard-drive-0s03664) | $174.38 @ Newegg
**Video Card** | [\*EVGA GeForce GTX 960 2GB Superclocked Video Card](http://pcpartpicker.com/part/evga-video-card-02gp42962kr) | $190.17 @ Newegg
**Case** | [Phanteks Enthoo Pro ATX Full Tower Case](http://pcpartpicker.com/part/phanteks-case-phes614pbk) | $108.98 @ Amazon
**Power Supply** | [EVGA 1000W 80+ Platinum Certified Fully-Modular ATX Power Supply](http://pcpartpicker.com/part/evga-power-supply-220ps1000v1) | $171.27 @ Newegg
**Optical Drive** | [Samsung SH-224DB/BEBE DVD/CD Writer](http://pcpartpicker.com/part/samsung-optical-drive-sh224dbbebe) | $20.69 @ Amazon
**Case Fan** | [Cougar Vortex PWM 70.5 CFM 120mm Fan](http://pcpartpicker.com/part/cougar-case-fan-cfv12hp) | $16.33 @ Amazon
**Case Fan** | [Cougar Vortex PWM 70.5 CFM 120mm Fan](http://pcpartpicker.com/part/cougar-case-fan-cfv12hp) | $16.33 @ Amazon
**Case Fan** | [Cougar Vortex PWM 70.5 CFM 120mm Fan](http://pcpartpicker.com/part/cougar-case-fan-cfv12hp) | $16.33 @ Amazon
**Other**| Windows Server 2012| $699.99
| *Prices include shipping, taxes, rebates, and discounts* |
| Total (before mail-in rebates) | $9680.51
| Mail-in rebates | -$30.00
| **Total** | **$9650.51**
| \*Lowest price parts chosen from parametric criteria |
| Generated by [PCPartPicker](http://pcpartpicker.com) 2016-02-14 12:36 EST-0500 |

cdabc123 · Feb 14, 2016

can his program take advantage of cards like nvidia tesla or intel phi?

BadActor · Feb 14, 2016

Two of those Noctuas would be a lot of weight hanging off the board.

wzhouiup · Feb 14, 2016

cdabc123 :

according to him, his program is 100% cpu based, but he also needs a gpu for some other MIT work

wzhouiup · Feb 14, 2016

BadActor :

Hmmm, i can see that being an issue. What would be a better option?

cdabc123 · Feb 14, 2016

wzhouiup :

then that may not be the best build ill come up with one in a few min. is their a max amount of threads or cores his programs handle?

wzhouiup · Feb 14, 2016

his previous computer had 14 cores (not sure if including logical) and he said he wants more. Note that his computer was from 2009, so this xeon build does not have too many cores. He originally wanted 1tb of ram but that was way too overkill.

joex444 · Feb 14, 2016

As an actual physicist who runs actual computationally intensive things, my guess is he's better off with a small cluster. Mine is a 9 machine setup, each having two quad core Xeons, 24GB RAM, and 4TB in RAID. The entire cluster is 72 cores, 216GB RAM, and 36TB. Granted, that went for about $25k

My very first question is whether the computation can be split into multiple subtasks or whether the entire thing is linear. If it's splittable, then they'll benefit from parallelization. It could be the same task with different parameters that are all independent. In this case, 24 cores at 3.0GHz is better than 12 cores at 4.0GHz. Even more so, 48 cores at 2.0GHz is better than 24 at 3.0GHz. The easiest way to reach such core counts is to go with separate physical PCs. This gets you a couple things: you can get cheaper CPUs that have fewer cores, but you'll spend extra on cases, PSUs. You save by going with lower density RAM modules though, as each machine only needs 16GB rather than 256GB. You save with single CPU motherboards versus dual CPU boards, and you save on single CPU processors rather than SMP enabled Xeons. You also have the ability to pool the storage together, so it's possible to use, eg, two 2TB drives in each machine as RAID1, and with, say, 8 machines you end up with 16TB usable. It would also be possible to give each machine just a SSD, give it some work space and have it store the large data on an NFS machine which has a RAID5 or RAID6 in there. We also have an NFS machine with 16x1TB in a combination RAID10 and RAID6... so that's 10TB on the RAID6 and 2TB on the RAID10.

If instead the tasks are all linear, then having a bunch of cores is pointless. They would get by with a 4790K overclocked. It's extremely likely that if they think their task is all linear then they are not a very good programmer. It should be extremely parallelized, which I think is why your initial build had 28 cores and 56 threads in it.

The next question is whether it's GPU assisted or not. If not, the GPUs are superfluous. In my case, between these machine there are zero GPUs and only one monitor which is physically plugged and removed if, for some reason, physical administration is necessary. Most of the time it can be remotely administered and these run 24/7 completely headless. They're rebooted remotely as well, if needed. This is typically once a year or so.

What I can definitely tell you is that the single 4TB drive in there is absolutely incorrect. A task like this *requires* RAID. My guess is you're dealing with a researcher or graduate student and neither one of them is OK with losing their work due to a hard drive failure. No amount of backups are going to help here, it needs to be an active mirror or RAID5/6 array. Spending $10k and putting $250 into storage is just so incomprehensible to me.

If you do go with SMP, be sure the PSU has two EPS connectors.

cdabc123 · Feb 14, 2016

quad E7-8870 15 core
supermicro X10QBL-4

on this build you can save alot of money if you get a used version of the latest stepping. this mobo has 32 ddr4 slots you get 120 threads from this build. and up to 2 tb of ram

wzhouiup · Feb 14, 2016

From what I know, he says that a cluster will not benefit him at all. This is the reason why he wants one extremely powerful computer. (plus the grant money he is receiving is for one computer only not for two or three) His "problem" is not enough cores and not enough ram.

Another thing to know is that i am not acutally building the computer, my job is to list out the parts based on his wants and needs. (i am a student, not a physicist)

To answer your second question, he has almost no idea about hardware, but he wants a gpu because he runs windows server.

wzhouiup · Feb 14, 2016

joex444 :

As an actual physicist who runs actual computationally intensive things, my guess is he's better off with a small cluster. Mine is a 9 machine setup, each having two quad core Xeons, 24GB RAM, and 4TB in RAID. The entire cluster is 72 cores, 216GB RAM, and 36TB. Granted, that went for about $25k

My very first question is whether the computation can be split into multiple subtasks or whether the entire thing is linear. If it's splittable, then they'll benefit from parallelization. It could be the same task with different parameters that are all independent. In this case, 24 cores at 3.0GHz is better than 12 cores at 4.0GHz. Even more so, 48 cores at 2.0GHz is better than 24 at 3.0GHz. The easiest way to reach such core counts is to go with separate physical PCs. This gets you a couple things: you can get cheaper CPUs that have fewer cores, but you'll spend extra on cases, PSUs. You save by going with lower density RAM modules though, as each machine only needs 16GB rather than 256GB. You save with single CPU motherboards versus dual CPU boards, and you save on single CPU processors rather than SMP enabled Xeons. You also have the ability to pool the storage together, so it's possible to use, eg, two 2TB drives in each machine as RAID1, and with, say, 8 machines you end up with 16TB usable. It would also be possible to give each machine just a SSD, give it some work space and have it store the large data on an NFS machine which has a RAID5 or RAID6 in there. We also have an NFS machine with 16x1TB in a combination RAID10 and RAID6... so that's 10TB on the RAID6 and 2TB on the RAID10.

If instead the tasks are all linear, then having a bunch of cores is pointless. They would get by with a 4790K overclocked. It's extremely likely that if they think their task is all linear then they are not a very good programmer. It should be extremely parallelized, which I think is why your initial build had 28 cores and 56 threads in it.

The next question is whether it's GPU assisted or not. If not, the GPUs are superfluous. In my case, between these machine there are zero GPUs and only one monitor which is physically plugged and removed if, for some reason, physical administration is necessary. Most of the time it can be remotely administered and these run 24/7 completely headless. They're rebooted remotely as well, if needed. This is typically once a year or so.

What I can definitely tell you is that the single 4TB drive in there is absolutely incorrect. A task like this *requires* RAID. My guess is you're dealing with a researcher or graduate student and neither one of them is OK with losing their work due to a hard drive failure. No amount of backups are going to help here, it needs to be an active mirror or RAID5/6 array. Spending $10k and putting $250 into storage is just so incomprehensible to me.

If you do go with SMP, be sure the PSU has two EPS connectors.

Well, i believe the data is going to be in the cloud, so raid is probably out of the question

tom_spinach · Feb 14, 2016

Get a case where the Board is horizontally mounted, so the cooler weight isn't a problem anymore.

Search

Building a 10000 dollar xeon workstation

wzhouiup

Reputable

cdabc123

Honorable

BadActor

Expert

wzhouiup

Reputable

wzhouiup

Reputable

cdabc123

Honorable

wzhouiup

Reputable

joex444

Distinguished

cdabc123

Honorable

wzhouiup

Reputable

wzhouiup

Reputable

tom_spinach

Reputable

TRENDING THREADS

Latest posts

Moderators online

Share this page