Sign in with
Sign up | Sign in
Your question

Virtualization Host Build

Last response: in Systems
Share
February 3, 2012 5:24:24 PM

Virtualization Host Build

I am a Pre-Sales Systems Engineer, and I sell a fairly complex software product. Our physical Demo/Test facilities are 1000 miles away from me, and our Virtual ones suffer from lack of resources, clogged schedules (my environment's performance suffers when several others are working concurrently), plus we can't show how our product works to manage/support Virtual environments from within a shared Virtual environment.

So, I need my own. I'm building a NAS (I have one already that I bought, but am not overly happy with), the contents of which I'm dealing with on the FreeNAS forums, as that's the Open Source product I'm going to use to run it (I'd prefer OpenFiler, but that project seems to be stalled, and the Dev Team is completely silent to the user community).

The other piece I need is the Virtualization host. The last time I built my own system was around 1997 or 1998 - up to then I had built numerous systems, but things have gotten a little more complicated.

This host will hopefully run VMware VSphere 5 (aka ESX 5) although VMware Hypervisor (ESXi 5) will be sufficient. I can't run Workstation or Server, since I need VCenter for my product. One of my concerns is that ESX won't run, or pieces of it, on the hardware platform I'm considering. There's a site, ESXWhitebox.com, that details various whitebox builds and their ESX compatibility issues. But they don't have any info on ESX 5 . . . I'd also like a dual-socket solution to get more cores running, and that puts me into the "Server Motherboard" realm, which is fine from a cost perspective, but I need something that will fit comfortably on top of a bookcase. A mid-Tower ATX Case is as big as I'm willing to get, and when I look at the pre-Packaged servers (Dell T610) and the like, they are sorely lacking in features (no PCIe 3, USB3, built in SAS support, etc.).

My fallback, should ESX not run on it, will be to run MS Windows Server 2008 R2 EE, and use HyperV. Our product supports both, but most of what we see in the field is VMware ESX.

Relevant Info:

Approximate Purchase Date: Sunday, Feb 5th, 2011
Budget Range: up to $4000
System Usage from Most to Least Important: Virtualization Platform
Parts Not Required: No Keyboard/Mouse/Monitor/Speakers or OS Required
Preferred Website(s) for Parts: Provantage, Newegg, Amazon
Country: ;USA
Parts Preferences: by brand or type: I am biased towards Intel processors, Asus motherboards (from long ago) and Fractal Design cases
Overclocking: Not now, Maybe in the future
SLI or Crossfire: Nope
Monitor Resolution: all my Monitors are 1920x1200, but whatever video is built in is fine
Additional Comments: See above, and VM List at the bottom

So, my general requirements are:

Many, fast Cores (6 at a minimum)
16 GB of RAM Minimum (to start), 32 GB RAM in the near future, possibly needing more than 32 GB down the road, but with only a single socket, 6 core CPU, probably not more than that. And I don't want to have to throw out RAM to go from 16 GB to 32 GB, or 16 GB to 48 GB.
Dual 1 GB Ethernet NICs - they won't be bonded - one will run to the NAS storage, the other will provide "front end" connectivity to the VMs.
6 gb/sec SATA

Nice to Haves:

6 gb/sec SAS (I might toss in a 1 TB 6 gb/sec SAS drive to get started building all the VMs locally until I get the NAS solution finalized)
USB 3
Overclocking - I'm not planning on Overclocking this system, but I'd like to be situated to where I can if I decide I want to. So I'm willing to spend a little extra $$$ to be prepared for that.

Everything else on the current generation of Motherboards is overkill, typically relating to Video Performance.

Cost is an issue, but I'll pay extra for more features/performance if it's worth it.

So here's what I'm considering:

Case: Fractal Design Define R3 USB3.0 Titanium Grey

http://www.fractal-design.com/?view=product&category=2&...

Powersupply: I just assume I'd get one from Fractal, but they list 10 different models. NO idea how to pick. Quiet is very important.

Motherboard: Asus P9X79 Deluxe - 2011 Socket, has 8 DIMM sockets, 6 gb/sec SATA, USB 3, but no SAS, and not sure if the 1 RealTek Ethernet will support ESX. The Intel Ethernet will, although I have no idea why Asus put 2 different manufacturer's Ethernet chips on-board . . .

http://usa.asus.com/Motherboards/Intel_Socket_2011/P9X7...

CPU: I'm an Intel guy, but willing to listen . . . but at half the price, are the only differences between the X and K .1 ghz of clockspeed and 3 MB Cache? The K seems like a no-brainer . . .
Intel Core i7-3960X: 6 Core; 3.3 ghz, 15MB Cache; $1150
Intel Core i7-3960K: 6 Core; 3.2 ghz, 12MB Cache; $600

The CPUs are sold without a heatsink, and I'm getting a cooling unit, so do I need one? If so, what do you recommend?

Do I want a Liquid Cooling Unit? Water in the CPU seems fraught with peril . . . but the following seems to be well thought of . . . how to decide?

H100 Extreme Performance Liquid CPU Cooler
http://www.corsair.com/cpu-cooling-kits/hydro-series-wa...

Although this has its fans as well:

Noctua NH-D14 SE2011
http://noctua.at/main.php?show=productview&products_id=...

RAM: How do you pick between manufacturers? And how important is the RAM Speed if I want to overclock down the road?
A single 4x4GB kit for 16 GB total will be fine, leaving 4 slot open for either 16 GB or 32 GB additional RAM down the road. The Asus compatibility guides for the P9X79 typically list the Crucial and G.skill RipJaws in all the various speed settings . . . and I'll pay a little more for "faster" RAM, but the 2400 are 6 times as expensive. Where's the best cutoff on speed/price?

Corsair Memory Vengeance 16 Dual Channel Kit DDR3 1600 MHz 240-Pin DDR3 SDRAM CMZ16GX3M4A1600C9
G.skill Ripjaws X Series 16gb (4 X 4gb) 240-pin Ddr3 Sdram Ddr3 1600 (Pc3 12800) Desktop Memory Model F3-12800cl9q-16gbxl

SSD Drive: Never bought one, don't know how to pick . . . VMware ESX doesn't need much space, but if I have to go to Windows HyperV, I'll need at least 64 GB.

Case, Motherboard, Processor, Heat-Sink (if needed), cooling fan, RAM, SSD. Anything else I'm forgetting?

Thanks in advance . . .

Doug MacFarlane
madmac63@gmail.com

P.S. for those who are interested, my VMs will include:

2 Core, 2 GB RAM - Win2K8 R2 - Active Directory and VCenter
2 Core, 4 GB RAM - Win2K8 R2 - my Product's "Command and Control" application and primary data mover
2 Core, 4 GB RAM - Win2K8 R2 - a small Exchange Server
2 Core, 4 GB RAM - Win2K8 R2 - a small MS SQL Server
2 Core, 4 GB RAM - Win2K8 R2 - a small SharePoint Server
1 or more Linux Server Distros, not typically running unless specifically needed
2 VMs, not typically running, of old desktops with apps I might need (old versions of Turbo Tax, for example)
2 Core, 4 GB RAM - Win2K8 R2 - a 2nd small Exchange Server to show DAG replication, not typically running
February 3, 2012 8:29:46 PM

K-series CPU for sure.

I really like the H100 if the case supports it. It might be a little more to install, but it keeps the area of the board itself so much more free and easy to work on.

No on the Fractal PSUs. I won't make a specific req until I know what GPU you need.

Get a 2x8GB kit, not 4x4GB RAM.

On the SSD, probably a 120GB Intel 510.

Are you going to have the driver support you need on this cutting edge hardware?

If your GPU is not important, you will not need a huge PSU. A Seasonic X-series 560W perhaps.

Intel overclocking insurance
http://click.intel.com/tuningplan/
February 3, 2012 9:23:33 PM

Proximon,

Thanks for taking the time to read and respond. Agreed on the K-series CPU and 2x8 GB RAM kit. Intel SSD also makes sense.

Drivers are going to be an issue, and if I have to, I can start out with Windows and Hyper-V, and then convert the system to ESX and convert the VMs to VMware format .vmdk/.vmx . . .

I'm not planning on putting a GPU in - the on-board video will be fine, so the Seasonic X-series you recommended will be sufficient.

What about a heatsink? The processor ships without one. From what I've read, heat sink and thermal paste are issues people have strong feelings on . . .

Doug
Related resources
February 3, 2012 9:43:17 PM

I'm a serious water type of guy, so I come from a different direction than those panicky types that think any water is bad, based not on experience but just anxiety.

The enclosed H100 units are not like serious water. They are fairly safe, fully contained solutions. The coolant is, I think, a type that has low electrical conductivity. The advantage is temperatures... because the rad is mounted right on an exhaust port the heat is being transferred outside the case. Also you have much easier access to RAM slots and other things near the CPU, that a large air cooler would normally block.

Obviously you value quiet, based on your case choice. Using two good PWM fans on the H100, such as Scythe Gentle Typhoons, would give you excellent cooling at a low db.

Much is made of TIM, but truly any good ceramique-type is fine, the differences being minor. you will get some old-school reqs for Arctic Silver, but it's no better. AS5 has a long cure time (250 hours) and short life span. Ceramique has better temp performance up front and about the same after the AS5 cure time.

Plus ceramique based compounds are non-conductive (electrically) and so safer.

http://www.newegg.com/Product/Product.aspx?Item=N82E168...

You need a GPU :)  P9X79 Deluxe does not have onboard video. Any little Radeon will do
http://www.newegg.com/Product/Product.aspx?Item=N82E168...
February 3, 2012 9:46:58 PM

Oh, and on the PSU... 560W is a bit overkill, but will give you more connectors for HDDs and such. You'll never break 300W probably, so don't be surprised if you never see the PSU fan actually spin.
February 5, 2012 5:57:35 PM

Proximon said:
Oh, and on the PSU... 560W is a bit overkill, but will give you more connectors for HDDs and such. You'll never break 300W probably, so don't be surprised if you never see the PSU fan actually spin.



Proximon,

Thanks again for all the guidance. Can you suggest a Heatsink? And what does TIM stand for?

I think that's it - I'm ready to order . . . .

You've been great. I'll take Pics and post the build.

Doug
February 5, 2012 6:21:16 PM

Tim stands for Thermal Interface Material. Thermal paste.

We have been discussing the H100. Do you want a different heatsink?

Here is a picture of someone using the H100 in a Fractal case. Note that you will not need to use 4 huge fans on it :p 
http://forums.nvidia.com/index.php?showtopic=206612
February 5, 2012 6:22:31 PM

Proximon said:


The enclosed H100 units are not like serious water. They are fairly safe, fully contained solutions. The coolant is, I think, a type that has low electrical conductivity. The advantage is temperatures... because the rad is mounted right on an exhaust port the heat is being transferred outside the case. Also you have much easier access to RAM slots and other things near the CPU, that a large air cooler would normally block.

Obviously you value quiet, based on your case choice. Using two good PWM fans on the H100, such as Scythe Gentle Typhoons, would give you excellent cooling at a low db.



Proximon,

OK - let me see if I get this straight.

I don't need a Heatsink if I get the H100 - Thermal Paste and clip the H100 to the 3960K, and then there is a radiator that needs to mount on the Case next to fan openings. The H100 comes with 2 120 mm fans in the Radiator - are you suggesting that I replace them with with the Scythe Gentle Typhoons to minimize noise?

Thanks

Doug
February 7, 2012 2:02:10 AM

you want to run 4 sticks of memory to take advantage of the quad memory controller of the x79 platform
February 7, 2012 4:46:05 AM

That's right, sorry haven't done enough of these builds. Not sure it will make a huge difference, but either get 4x8 up front or go back to the 4x4.
February 7, 2012 4:57:19 AM

4x8 up front since he got the bread and it will be very useful for him too
!