Dual Xeon Nehalem Workhorse Build - Page 2

Last message on previous page:
Well, according to this: http://techreport.com/articles.x/15363/12

The velociraptor alone will be like 2.0 Gbps, the two caviar black 1TBs are about 1.6 Gbps a piece, the caviar se16 is about the same, and then the ssd will also be quite fast

~Lyuokdea

Am I missing something here? Velociraptor = .25 Gb/s

Whooops - yeah 3.0Gb/s = 300 MB/s

Dang!! You'd think those motherboard engineers wouldn't ScrewUp like that, would you? Having a motherboard that would be saturated by just 2 HDDs? What were they THINKING?

So maybe the engineers didn't mess up so bad after all?
Each HDD has its own SATA channel and each channel has the full 3.0Gb/s.

We shall see on monday... i have all the parts now, just need to put them together....the parts I went with ended up being a little bit different than the latest version released here. I'll try to post a full report on them at some point, to help others deal with any problems that i had.

~Lyuokdea

I had the same problem with heatsinks and power supplies and case. I dont know if you saw this reply I posted to your other thread awhile ago:
http://www.tomshardware.com/forum/ [...] w=0&nojs=0

I got the Coolermaster Hyper N520 heatsink but found out about the backplate when I got the board Asus D12x board. Really frustrated I decided to go with the Intel server heatsink designed for the 130W chip (W5500+) http://www.newegg.com/Product/Prod [...] 6835203001

You can go with the 80w (http://www.newegg.com/Product/Product.aspx?Item=N82E16835203002) since ur using the E5520, I just wanted the extra cooling power. They are actually very quiet and work well, but no backplate included. Also there should be settings in the bios to ramp up the fans or keep it whisper quiet. I went with the Generic setting and they work fine/quiet. The xeon chips use much less power and run a lot less hotter than the i7 ships. i7 heatsinks might be overkill.

You can use the Noctua which is a nice heatsink but HUGE 120mm, I found a reviewer here that got it to work with the Asus D12x board : "...2X Noctua NH-U12P SE1366 just buy 8 - 3mil thread screws about 10 mil length and just use the mount on MB, no need to use noctua's supplied under MB mount.., you just use the top parts and use 2 bucks of screws to make it work..."

Since you went with the supermicro I guess you dont need to worry about SSI EEB cases. I still have not offically found a case that works. I've ordered the LianLi A77, a review somewhere said it had a SSI EEB attachment to the tray for compatibility. From the look of it on NewEgg it looks like it has all the holes on the mobo tray the Asus needs. I get it tomorrow so we'll see.

Here is my final config:
ASUS Z8PE-D12X(ASMB4-IKVM) Dual LGA 1366
2x Intel Xeon E5520 Nehalem 2.26GHz LGA 1366
2x Intel BXSTS100C Passive/active heatsink
5x Western Digital RE3 WD5002ABYS-01B1B0 500GB at raid 10 (one spare)
2x WD Blue 320GB at raid 0 (for os)
SAMSUNG Black 1.44MB 3.5" Internal Floppy Drive
Corsair HX1000
ASUS Xonar DX 7.1 Channels PCI Express Interface Sound Card
SYBA IEEE 1394a FireWire PCI Card with Internal 9-pin Header Model SD-VIA-FW1E1H
4x Crucial 6GB (3 x 2GB) DDR3 1333 (PC3 10600) ECC Unbuf CT3KIT25672BA1339 (24GB ram total)
running XP64

Waiting on the case.Right now I'm using this, got it for the hotswap drive bays:
Athena Power CA-SWH01BH8 (http://www.newegg.com/Product/Product.aspx?Item=N82E16811192058)
Says its EATX (SSI EEB 3.x), its not! Right now my Asus board is not attached to all the offsets, missing like 3 towards the center of the board. Hopfully the LianLi works or I'm getting my drill out and drillin holes!!

Pretty happy with performance so far, I'm a 3D/2D vfx artist using XSI, AfterEffects, Nuke and other Adobe ***.

Figured I should post a short review now that I have everything installed and working (for the most part). Thanks again to everybody who helped out on this build, the thing is beastly.

The final build for the main computer components is listed here:

http://secure.newegg.com/WishList/ [...] er=6860429

Some important notes:

I originally bought the Coolermaster Hyper N520 heatsinks...these do not fit with the supermicro motherboard (which already has a backplate in place). The N520 has an odd double twisted screw design, which means that it will not screw into an existing backplate. Instead of removing the backplate, I simply RMAed the heatsink and got the dark knights, which seem to install and work perfectly.

I'm having a very odd error with the motherboard. For some reason, it takes about 50 seconds to post. After it does post, absolutely no problems, however it took me two weeks (and many posts on here, and an RMAed motherboard, etc) to find out that it wasn't actually freezing the whole time (i had always worried the voltages were off or something, and turned it off after 20 seconds of hanging). Another user on newegg has posted the same problem, and Supermicro is looking into it, but be forewarned if you're working with this motherboard.

The bottom right screw on the motherboard won't fit into an EATX case, i just have it out.

For the number of fans and HDs in this thing, it actually runs pretty quietly. I love the SSD (which is really the biggest speed improvement you notice when working on the thing). I am running an SMP'ed folding at home on 4 cores in the background (at ~ 16 Gflops), was running some other processor intensive codes in the background, and couldn't notice any slowdown in my user experience.

The system is running Ubuntu Linux 9.04, and has the following harddrive partitioning table:

30GB SSD:
/ (running on ext4 with noatime enabled) - customizations for ssd taken from http://tombuntu.com/index.php/2008 [...] te-drives/

640 GB drive #1:
/var - 2GB
/tmp - 2GB
/swap - 24 GB
/home - rest of drive

640 GB Drive #2:
/fdata - 70 GB (fast section on the outer end of the drive, for simulations where hard drive speed is limiting)
/data - rest of drive

2x 1TB drives
/sdata - more secure data, and previous runs

Haven't quite got /sdata configured properly as of yet, having some problems with linux on that one, hope to sort it out in the next day or so. You'll note that the hard drive situation is set up to isolate the user experience (/ and /home) from any codes that are running in the background (and potentially locking up a disk). I think this should be really helpful for dealing with disk intensive programs. It is also set up to minimize writes to the ssd (no /var or /tmp on that disk) in order to preserve the lifespan of the ssd.

I'm using 2x 24" 1920x1200 monitors from the soyo refurbs that were on slick deals a month back. All in all, these are pretty crappy monitors (only one dead pixel, but the color is off a bit between the monitors, and seems unable to be fixed). However, it is 3840x1200 of space for about $320, so I'm not complaining at all (I knew i wasn't getting high quality monitors, and for coding work, pixel real estate is really more important than color balance).

If you have any questions, let me know....overall this worked out pretty well. But the post problems were extremely annoying for about 2 weeks.

Thanks again for all your help!

~Lyuokdea

rininio,
It could be one of several things, I dont know the Supermicro board, I got the Asus. Here are 2 things you should check first:

1 - Video card - try pluging the monitor to the onboard video. From my experience most mobos require you to manually set your video output from onboard video to addon video card. There might be jumpers you need to set (check the supermicro manual) or it might be a setting in the bios. If its a setting in the bios you'd need to boot up with onboard video first to access and change that bios setting.

2- It could be the power supply, esp if your had to "twitch" it. I would highly recommend you get a power supply with 2x 8pins. I've only found 2 that I can be 100% positive they have this feature. If you want to stick with corsair brand get the 1000HX thats the one I got, modular, quiet, great psu:
http://www.newegg.com/Product/Prod [...] 6817139007

if 1000w is too much, go with the PC Power and Cooling 860w (great brand, Dell uses them on their builds):
http://www.newegg.com/Product/Prod [...] 6817703013
PDF showing schematic with 2x 8pin: http://www.pcpower.com/downloads/T [...] g_3852.pdf


In my opinion resorting to molex adapter is a last option. If your building a new system there is no reason to take the risk of burning anything out with a molex. There is a good article here about psu power connections:
http://www.playtool.com/pages/psuc [...] ctors.html
Scroll half way down and read the subsection: 8 pin EPS +12 volt power cable
Note:
" If you don't have an 8 pin 12 volt cable then you can use the adapter shown above. It converts a couple of 4 pin peripheral power cables into an 8 pin 12 volt cable. If you use one of these adapters then be sure to plug the 4 pin peripheral connectors into separate cables coming from the power supply. If you plug them both into the same power supply cable then you are drawing all the power of the 8 pin 12 volt connector through a single 18 gauge wire. You can often get away with that but there's no reason to do it."

Hope this helps.

8 bits = 1 Byte but remember parity and/or ECC

I generally just lop off a 0 for conversion.

Watch w/care as sometimes 1k =1024 and sometimes 1000. That scheme used to be used in memory vs disk before the progression of Mb to Gb.

Personally, and professsionally, I have built custom scientific machines since 1999 and some observations:

I've had nothing but trouble with tyan mobos. In once case, a dead mobo, a second one died after 5 hours.

Asus is great and Supermicro even better, as I have tended to use Supermicro in my clusters.

Always consider SAS over SATA, the expandability is much better as you can daisy chain drives.

Consider: if you are going with a *NIX system, make sure /swap is to the SSD as latency is lower.

Remember, too, that in scientific computing, the answer to every question is, it all depends.

If your calculations are parallelizable, remember amdahls law, that overall performance is equal to the sum of the parallel activity + the serial activity/serial activity. If you have heavy loops, then cache size will rule, and a cluster will beat a single system. If it's all serial, then you fall back on compiler opptimization, multithreading, etc. etc.

Remember, too, grant extensions. You may be able to expand later, and that's where the early preparation will byte you if you're not carefull.

Google the top 50 or 500 list of supercomputers, just for fun, and compare configs. It's always nice to dream.

Also, explore sgi, for fun. Their deskside supercomputer systems start at $7k with one xeon, but you can get some ideas. Also, check out using GPUs such as NVIDIA if you have cheap programming labor (grad students).

Along those same lines, if you have extra $s, get a modular power supply. The extra $s will support later expansion.

Also time frame. The longer you wait, the better things will get (in general).

If you want free (professional advice), email me direct as I'm currently job hunting...

Is there similar / equivalent plate to cover both CPU-s on ASUS Z8NA-D6 and ASUS Z8NA-D6C and mount single liquid cooled block. The reson I ask is that a lof of people complained about the tight space, but on the other side the form factor is very attractive (ATX instead of 12x13)