Hi Experts,
I am in the process of building a DUAL CPU system to host two Tesla K20 GPUs (http://www.newegg.com/Product/Product.aspx?Item=N82E16814132008)
this is what I decided on so far:
The choice of CPU was basically what I could afford for a dual system given their performance.
Thw choice of motherboard was done having in mind that this system will run for weeks on end without shutting down, so i thought it wiser to prefer a server-grade supermicro board than the Asus workstaion one, Z9PED8 WS (http://www.asus.com/Motherboards/Z9PED8_WS/)
The motherboard has onboard video and lots of PCIe to slot the 2 Teslas in (they require dual width PCIe 16). And I will not need an extra Videocard because I will be using onboard video. However, homebuilts with supermicro are not easy...
I have already avoided one pitfall by getting supermicro CPU cooler (http://www.newegg.com/Product/Product.aspx?Item=N82E16816101683) as opposed to my favourite silent one, as per the comments here: http://www.newegg.com/Product/Product.aspx?Item=N82E16813182348
I am also prepared to drill a few holes to any computer case i may buy or re-cycle
However, I am sure there are things I have not thought about and I am totally inexperienced. So I am asking for any opinions regarding this setup and anyone who had experience with this motherboard or the Asus one to please share.
One actual question I have is about Power supply. I have thought of using a 1000W Platinum powersupply from one of the reputable and silent vendors, e.g. antec. i dont care much about redundancy. BUT in the mother board reviews I have read that it reuqires 1x24pin and 2x8pin connectors. The Tesla cards require 1x8pin and 1x6pin PCI express connector, each. I have no idea whether standard powersupplies support this.
So can someone please suggest appropriate power supply for this specifiic X9DR3 Mobo at 1000W peferably Platinum?
Do you think i will need additional cooling? The system will be running at an ambient temperature of 40 C(104F) for a few months per year and I am not keen on air-conditioning. Some kind of closed-loop hydro cooling perhaps, additional to the 2 coolers on the CPUs I already mentioned? I am open to suggestions. The Teslas have their own cooling fans.
Can I "switch off" the Teslas without removing them from the motherboard? In order to save power when not needed? They have their own PCIe power pins. Maybe I can install a switch to turn them off? Or power consumption will be minimal when idle?
Finally, what is your opinion about recycling computer cases, improvising or even welding one from scrap? Obviously the motherboard must screw on it and not shortcircuit, but empty discarded computer cases is something I see a lot in rubbish bins and it pains me...
So to summarise:
best regards and thanking you i advance for any thoughts you may have on the above,
bliko
p.s. I have already asked a similar question 1.5 years ago but I have not proceeded in building any system at the end, http://www.tomshardware.co.uk/forum/373710-31-need-advice-scientific-computing-workstation-built-2xk20
I was obviously a keen AMD follwer back then as I am still very happy with my athlon, but for this system it will be xeon.
I am in the process of building a DUAL CPU system to host two Tesla K20 GPUs (http://www.newegg.com/Product/Product.aspx?Item=N82E16814132008)
this is what I decided on so far:
2 x E5-2630v2 intel cpu (http://www.google.co.uk/url?sa=t&rct=j&q=e5%20%202630%20v2&source=web&cd=1&ved=0CB0QFjAA&url=http%3A%2F%2Fark.intel.com%2Fproducts%2F75790%2FIntel-Xeon-Processor-E5-2630-v2-15M-Cache-2_60-GHz&ei=UpesU-rNIcmP0AXaj4DYCA&usg=AFQjCNGnBiTgxVFprX1E-z7aHWJzzuv0vw)
Supermicro X9DR3-F motherboard (http://www.supermicro.com/products/motherboard/Xeon/C600/X9DR3-F.cfm)
8x8GB Hynix PC3L-12800 ECC at 1600MHz or Samsung (as prescribed by Supermicro)
A couple of WD Caviar Black / 2TB.
OS: exclusively LINUX.
The choice of CPU was basically what I could afford for a dual system given their performance.
Thw choice of motherboard was done having in mind that this system will run for weeks on end without shutting down, so i thought it wiser to prefer a server-grade supermicro board than the Asus workstaion one, Z9PED8 WS (http://www.asus.com/Motherboards/Z9PED8_WS/)
The motherboard has onboard video and lots of PCIe to slot the 2 Teslas in (they require dual width PCIe 16). And I will not need an extra Videocard because I will be using onboard video. However, homebuilts with supermicro are not easy...
I have already avoided one pitfall by getting supermicro CPU cooler (http://www.newegg.com/Product/Product.aspx?Item=N82E16816101683) as opposed to my favourite silent one, as per the comments here: http://www.newegg.com/Product/Product.aspx?Item=N82E16813182348
I am also prepared to drill a few holes to any computer case i may buy or re-cycle
However, I am sure there are things I have not thought about and I am totally inexperienced. So I am asking for any opinions regarding this setup and anyone who had experience with this motherboard or the Asus one to please share.
One actual question I have is about Power supply. I have thought of using a 1000W Platinum powersupply from one of the reputable and silent vendors, e.g. antec. i dont care much about redundancy. BUT in the mother board reviews I have read that it reuqires 1x24pin and 2x8pin connectors. The Tesla cards require 1x8pin and 1x6pin PCI express connector, each. I have no idea whether standard powersupplies support this.
So can someone please suggest appropriate power supply for this specifiic X9DR3 Mobo at 1000W peferably Platinum?
Do you think i will need additional cooling? The system will be running at an ambient temperature of 40 C(104F) for a few months per year and I am not keen on air-conditioning. Some kind of closed-loop hydro cooling perhaps, additional to the 2 coolers on the CPUs I already mentioned? I am open to suggestions. The Teslas have their own cooling fans.
Can I "switch off" the Teslas without removing them from the motherboard? In order to save power when not needed? They have their own PCIe power pins. Maybe I can install a switch to turn them off? Or power consumption will be minimal when idle?
Finally, what is your opinion about recycling computer cases, improvising or even welding one from scrap? Obviously the motherboard must screw on it and not shortcircuit, but empty discarded computer cases is something I see a lot in rubbish bins and it pains me...
So to summarise:
1) 2 x E5-2630v2 : provided this is all I can afford, any opinions on price/performance within E5 range?
2) Tips for the Supermicro motherboard X9DR3-F, especially about how to power it (approx 1000W, redundancy is not a must) without buying a supermicro case (super expensive). I believe it is better choise than the Asus Z9PED8 WS for a long-running system, Opinions?
3) Tips about cooling the system given that it will be running for the summer months in 40C (104F) ambient temperature without air-conditioning?
4) Can I switch Tesla GPUs off somehow in order to save power when idle, or consumed power would be negligible at no-load, idle state?
5) Can I "weld" my own organic case to host the system from scrap? Any tips, experience on this?
best regards and thanking you i advance for any thoughts you may have on the above,
bliko
p.s. I have already asked a similar question 1.5 years ago but I have not proceeded in building any system at the end, http://www.tomshardware.co.uk/forum/373710-31-need-advice-scientific-computing-workstation-built-2xk20
I was obviously a keen AMD follwer back then as I am still very happy with my athlon, but for this system it will be xeon.
Facebook
Twitter
Instagram