Hussain36 :
Hello,
I'm planning to building a render farm in the coming a few months, somewhere within 2016.
That will be for my 3D studio rendering.
but also I may use it commercially proving render service to other small studio. (That is just better than being there doing nothing.)
As a starting stage, I'll start with hardware budget of $26,000. But will expand it over the time.
Mainly that will be for animation in
Maya 2016/2017 with
mental ray. It will a dedicated standalone render farm.
I don't have that much knowledge on hardware, but it least basic enough to understand.
So, I'm thinking of
Intel Xeon E5-2699 v4.
I wonder is dual CPU gives double the performance (200%)?
Or is it better to have two computers with single CPU than a single computer with two CPUs?
What I'm thinking of now is to have two computers, one with dual CPU, and the other with single CPU but a motherboard for dual CPU for future upgrade.
I need you help and support to choose the other parts of the computer
CPU: Intel Xeon E5-2699 v4
(suggested)
CPU Cooler: ???
(I'd prefer liquid cooling for the luxury of silence) Also, as far as I know, cooler CPU better performance. The cooler, the better.
Motherboard: ???
Memory: ???
(I'm thinking of 64GB for each CPU. Speed: 2400) (If 128GB is worth having, I can go for that) & ECC for sure...
Memory Cooler: ???
(If worth having)
GPU: ???
(Note that mental ray is CPU based renderer with GPU accelerated)
Storage: ???
I'm thinking of PCI-e - NVM
Power Supply: ???
OS: Windows 10 Pro
(I wonder if it is better to have Server version than consumer 10 Pro version )
If there is any other part that I've missed, please mention it.
Any other hints and suggests (for example: UPS, NAS Storage Farm, & etc...), please do.
All suggestions and opinions are welcome.
Thanks in advance...
Hussain36,
In my view, there should be separate systems for 1. Modeling /Animation and 2. Rendering /Processing
SYSTEM 1 > Modeling /Animation > Focuses on single threaded performance for modeling, and fast memory and disk systems, based on the Xeon E5-1680 v3 8-core @ 3.2 / 3.8GHz which has in Passmark baselines, the highest single-threaded performance of Xeon E5 1600 series of 2153. There is not a benefit to providing dual CPU's for this system. The ASUS X99-E WS motherboard, using the X99 chipset, produces very high CPU scores and has an M.2 Drive socket supporting extremely fast NVMe memory drive, and the GPU/ SLI. The GPUs are a pair of Quadro M4000's in SLI that together provide 2X GPU's, 16GB of video memory, and 3,328 CUDA cores. 3D performance would have extraordinary bandwidth, plus very high anti-aliasing (x64 instead of x16 of a GTX, and very high single and double precision 10-bit colour, and OpenGL support.
SYSTEM 2 > Rendering /Processing Focuses on calculation intense processing power. Mental Ray is fully thread-scalar- the more cores the better. This concept uses a pair of E5-2640 v4 (10-core @2.40 / 3.4 GHz with 128GB of DD4 2133 on a Supermicro X10 DAi motherboard that supports three GPU's. The first GPU is a Quadro K5200 (8GB) which has superb bandiwth and 3D performance.
For processor-based rendering, there is a better balance obtained with the use of GPU CUDA coprocessing, that is a Maximus Quadro /Tesla combination. There are CPU with slightly fewer cores but higher turbo speeds that power the coprocessors and have good enough single-threaded performance that they are useful for 3D modeling, animation and computation processing /simulation.
This is arranged in an NVIDIA Maximus configuration with a pair of (used) Tesla K20 coprocessors. The K5200 plus the two K20's provide 2304 + (2X 2496) = 7,296 CUDA cores and 18GB of graphahics memory. The prcomputing power for Mental Ray should be enormous. Tthis system can be built using Maxwell series Quadro M5000 and a pair of Tesla M40. for an approximate additional $9,000, but there are Pascal series Quadros and already a Pascal P100 ($12,400). In my view, the suggested configuration would provide excellent rendering/processing but at a far better cost /performance ratio.
The disk systems of both systems are not at all refined, but it's a start.
BambiBoom PixelCannon Modelanimetricgrapharific iWork TurboBlast ExtremeSignaure SuperModel 9600 ®©$$™®£™©™_ 5.2.16
SYSTEM 1 > Modeling and Animation
1. CPU: Intel Xeon E5-1680 v3 8-core @ 3.2 / 3.8GHz, 20M Cache, 140W > $2057
____
http://ark.intel.com/products/82767 > (Passmark No, 10 CPU average CPU score =17166)
2. CPU Cooler: Cooler Master Hyper 212 EVO CPU Fan > $32.
3. Motherboard: ASUS X99-E WS LGA2011-v3/ Intel X99/ DDR4/ 4-Way CrossFireX & 4-Way SLI/ SATA3&USB3.0/ M.2&SATA Express/ A&2GbE/ CEB Workstation Motherboard > $488
____
http://www.superbiiz.com/detail.php?name=MB-X99E-WS
4. RAM: Samsung DDR4-2133 16GB/2Gx72 ECC CL15 Samsung Chip Server Memory > $880 ($110 each)
____
http://www.superbiiz.com/detail.php?name=D416GE21S
5. GPU 1, 2: PNY NVIDIA Quadro M4000 8GB GDDR5 4DisplayPorts PCI-Express Video Card > $1,574 ( $787 each)
___
http://www.superbiiz.com/detail.php?name=PNY-M4000
6. Drive 1: Samsung SM951 512GB (NVMe) MZVPV512HDGL-00000 MZ-VPV5120 Gen3 M.2 80mm PCIe 3.0 x4 512G SSD with a SSD protective case - OEM> $370
____http://www.newegg.com/Product/Product.aspx?Item=9SIA12K3MH3372&cm_re=samsung_nvme-_-9SIA12K3MH3372-_-Product
7. Drive 2: Intel 750 Series AIC 400GB PCI-Express 3.0 x4 MLC Internal Solid State Drive (SSD) SSDPEDMW400G4X1 > $350 (Active Projects)
____
http://www.newegg.com/Product/Product.aspx?Item=N82E16820167359
8. Drive 3,4: 2X Seagate Constellation ES.3 ST4000NM0033 4TB 7200RPM SATA3/SATA 6.0 GB/s 128MB Enterprise Hard Drive (3.5 inch) > $406 ($203 each) (RAID 1)(Files, Backup, System Image)
____
http://www.superbiiz.com/detail.php?name=HD-ST40NM3
8. PSU: CORSAIR RMx RM1000X 1000W ATX12V / EPS12V 80 PLUS GOLD Certified Full Modular Nvidia Sli ready and crossfire support Power Supply> $160
____
http://www.newegg.com/Product/Product.aspx?Item=N82E16817139140
9. Optical Drive: Pioneer Black 16X BD-R 2X BD-RE 16X DVD+R 12X BD-ROM 4MB Cache SATA Blu-ray Burner BDR-209DBK > $64
10. Case: LIAN LI PC-A75X No Power Supply ATX Full Tower Case (Black) CA-A75 $179.99
11. Operating System: Microsoft Windows 7 Professional 64-bit w/ SP1 (1-Pack, DVD), OEM MSFQC04649 $138.99
________________________________________________________
TOTAL = $6,700
SYSTEM 2 > Rendering
BambiBoom PixelCannon <Rendermodeagraphalicious iWork TurboSignature ExtremeRay 9900 ®©$$™®£™©™ _ 5.1.16
CPU: 2X Intel Xeon Processor E5-2640 v4 (10-core @2.40 / 3.4 GHz, 25M Cache) > $1,840 ($920 each)
($920 each)(Superbiiz)
CPU Coolers: 2X Supermicro SNK-P0048AP4 CPU Heatsink For LGA2011 >$64 ($32 each)
Motherboard: Supermicro X10DAI-O Dual LGA2011/ Intel C612/ DDR4/ SATA3&USB3.0/ A&2GbE/ EATX Server Motherboard > $365
Memory:128GB (8 X 16GB) Crucial DDR4-2133 16GB/2Gx72 ECC/REG CL15 Server Memory > $800 (Superbiiz)
GPU 1: PNY NVIDIA Quadro K5200 8GB GDDR5 2DVI/2DisplayPorts PCI-Express Video Card > $2,000
GPU 2,3: 2X TESLA K20 GPU ACCELERATOR > $1,800 (used about $900 each)
Drive 1: Intel 750 Series AIC 400GB PCI-Express 3.0 x4 MLC PCIe Internal Solid State Drive (SSD) SSDPEDMW400G4X1 > $350
Drive 2, 3: 2X Seagate Constellation ES.3 ST3000NM0023 3TB 7200RPM SAS3/SAS 6.0 GB/s 128MB Enterprise Hard Drive (3.5 inch) $370 > ($185 each)
Power Supply: CORSAIR HXi HX1200i CP-9020070-NA 1200W ATX12V / EPS12V 80 PLUS PLATINUM Certified Full Modular Power Supply Nvidia Sli ready and crossfire support with C-link > $239
Case: LIAN LI PC-P80NB No Power Supply ATX Full Tower (Black) > $360
Operating System: Microsoft Windows 7 Professional SP1 64-bit English (1-Pack), OEM > $139.
____________________________________
TOTAL = about $8,327
Notes:
This would be a highly capable system and for your budget, there would be far more advantage in having two of these systems for $17,000 than a single $26,000 system.In fact, unless there is a gigantic project of astounding time limitations, starting with one dedicated system of this level. In a single-user situation may be already more than is necessary. I make 3180 X 2140 renderings (single source lighting) in 7-8 minutes on the 12-core Dell Precision T5500 and in about the the same time on the HP z420 listed below. The Telsa K20 GPU coprocessors are used but these are extremely durable- a MTBF rating of 72,000 means they can be expected to run continuously for more than 9 years. I was at a research facility yesterday that had 10 dual Xeon computing nodes, each containing 4X Tesla K20 and these had run since 2012 continuously with no failures.
1. The Xeon E5-2640 v4 has not been benchmarked extensively, but performance should be very good. There are two systems using the E5-2640 v4 on Passmark and the CPU score for a single CPU is 15776 and for a dual configuration-
25080. That would place it at No. 8 in the Passmark dual CPU list. With 1 total of 20-cores /40 threads, provide a lot calculation power. The 3.4Ghz turbo speed has a Passmark of single-threaded rating of 1861 which is sufficient capability in combination with the Quadro K4200 for quite demanding visualizations, so besides rendering, animation, Arc/GIS, 3D structural design /analysis, particle and thermal simulation, and visualizations of these should be very good.
2. The Quadro K5200 was chosen as the GPU's in a NVIDIA Maximus configuration (Quadro + Tesla) have to have the same series processor- in this case all have to be Kepler series.
4. Benchmarks of GPU's:
_________NVIDIA Tesla K20[/color][/b]
OS OS X
API OpenCL
OS Windows
API OpenCL
Face Detection
_________________32.104 mPixels/s
TV-L1 Optical Flow
_________________ 11.228 mPixels/s
Ocean Surface Simulation
______________ 1427.912 Frames/s
Particle Simulation - 64k
___________ 377.433 mInteraction/s
T-Rex
__________________4.129 Frames/s
Video Composition
_________________ 64.479 Frames/s
Bitcoin Mining
________________ 179.058 mHash/s
5. The disk system is unrefined and could benefit from a RAID controller and some form of 1+0 RAID.
6. I'm not yet convinced that Windows 10 is ready for professional use.
______________________________________________________________
Cheers,
BambiBoom
Modeling:
1. HP z420 (2015) > Xeon E5-1660 v2 (6-core @ 3.7 / 4.0GHz) > 32GB DDR3 1866 ECC RAM > Quadro K4200 (4GB) > Intel 730 480GB (9SSDSC2BP480G4R5) > Western Digital Black WD1003FZEX 1TB> M-Audio 192 sound card > 600W PSU> > Windows 7 Professional 64-bit > Logitech z2300 speakers > 2X Dell Ultrasharp U2715H (2560 X 1440)>
[ Passmark Rating = 5064 > CPU= 13989 / 2D= 819 / 3D= 4596 / Mem= 2772 / Disk= 4555]
[Passmark V9.0 Beta Rating = 5019.1 > CPU= 14206 / 2D= 779 / 3D= 5032 / Mem= 2707 / Disk= 4760] 3.31.16
[Cinebench R15 > CPU = 1014 OpenGL= 126.59 FPS] 7.8.15
Rendering:
2. Dell Precision T5500 (2011) (Revised) > 2X Xeon X5680 (6-core @ 3.33 / 3.6GHz), 48GB DDR3 1333 ECC Reg. > Quadro K2200 (4GB ) > PERC H310 / Samsung 840 250GB / WD RE4 Enterprise 1TB > M-Audio 192 sound card > Logitech z313 > 875W PSU > Windows 7 Professional 64> HP 2711x (27", 1920 X 1080)
[ Passmark system rating = 3844 / CPU = 15047 / 2D= 662 / 3D= 3550 / Mem= 1785 / Disk= 2649] (12.30.15)