Hello everyone, I thought I'd consult the wisdom of the forum for my new computer I'm going to be building.

I think my requirements might be a lot different then what you are all used to hearing, so hopefully it will be an interesting exercise for all of you.

I'm a software engineer both professionally and for fun, so hardware is a bit out of my expertise. What I want in my next machine is a lot of cores. Basically, I could care less how 'fast' it is. I just need extreme parallelism.

The machine will be used by me for testing a lot of complicated multithreaded software. The more cores and the more simultaneous threads, the better I'll be able to test the software. So an 8 way system running at 1 ghz is preferrable to a 4 way system with each core running at 4 ghz.

I'm a little flexible with my pricing and requirements. 8 way seems doable for a reasonable amount of money. I see that Intel will be releasing Nehalem, which brings back hyperthreading, so a 4 core with 2 threads per core would bring me up to 8, that would be good too. An octa-core with hyperthreading would make me giddy. But it may be cheaper to go with dual quads, as who knows when nehalem will be released...

Anyways, I'd love to hear anyones thoughts on this, thanks in advance for your time.

Seems like you will need a dual socket server mobo. Since it doesnt seem you need the top peformance out of it I would check which system would be cheaper. Im thinking it would the AMD machine since it wouldnt need FBDIMMS. But im sure somebody with more experience on 2 socket systems will help out.

16 Cores // Socket F

Tyan Thunder n4250QE (S4985)
http://www.tyan.com/archive/produc [...] 250qe.html
$1,200

M4985 CPU Expansion Board
$700

8 sticks PC2-5300 (DDR2 667) ECC 512mb
$300

8 Opty Dual Core 8212 2.0GHz Socket F Processor AMD 2 x 1MB L2 Cache
$2,750


32 Cores // Socket F

Opty 8346 HE 1.8GHZ Socket F quads will run you around $800 each and will (hopefully) be rolling out soon. I would contact Tyan and verify (I assume they are testing as we speak) with the S4985.


$6k to $10k anyway you look at it

Woah! I had no idea you could get an 'expansion' board to hold more CPUs!!!


I could always buy the quad core mobo, stick in 4 quads, and then wait for my finances to recharge before I get the other 4 along with the expansion board no?

I do ok financially, but 6k+ is by no means chump change for me....

BUT 32 CORES

8 Cores Socket 771

ASUS DSBV-DX Dual LGA 771 Intel 5000V SSI CEB 1.1 Server Motherboard - Retail $224.99
Intel Xeon E5405 Harpertown 2.0GHz LGA 771 80W Quad-Core Processor Model BX80574E5405A - Retail $229.99 (x2)
Kingston 4GB (2 x 2GB) 240-Pin DDR2 FB-DIMM DDR2 667 (PC2 5300) ECC Fully Buffered Dual Channel Kit Server Memory Model KVR667D2D4F5K2/4G - Retail $179.99

Total: $864.96

This is of course excluding graphics card, PSU, case, optical drive and hard drive but I don't think you could find a cheaper way to 8 cores at the minute. I would go for something like this then upgrade to Nehalem for 16 or 32 cores or whatever.

Is the software you're testing desktop or server side?

I've seen the S4985 mobo as low as $800. You can start with a single proc or two and build your way up. You need at least one stick of ram per proc (each cpu socket has its own DIMM bank)

I've seen OEM Harpertowns as low as $200. Both Tyan and Super have 2p mobos in the $3-400 range for the Xeons. I would anticipate s771 prices to tumble a bit. It's pretty much a dead end until MP Nehalem rolls out (which if you believe the rumahs on the internets will be a while).

OR ...

Can you CUDA?

Thanks for all the replys. The quad motherboard option looks cool, but the 8 core xeon setup that Dan posted is tempting too....and defintely a cheaper way (initially) to get to 8 cores.

I was hoping a quad nehalem to be out this summer and be less then the price of those pricey 8000 series opterons...but I guess not.

I take it in a 4 socket MoBo, the procs have to be homogenous?

And is there an actual technical reason that I can't stick the non-8000 series opterons in the 4 socket motherboard? Is this just something AMD did to get more money from people buying heavy duty computing, or is there actual more neccessary silicon on a proc to support 4 sockets vs. two?

I realize that last question might be getting into the juicy details of how the Opteron is designed, but if anyone knows I'm interested.

I'm going to be testing Server Side software, and while I haven't heard of CUDA (had to wikipedia it), I don't think it would be applicable to the type of programming I'm doing. Thanks again everyone!

You can get a dual 771 setup with the Xeon 5100 (not 5000/5400) chipset and two Harpertown (Xeon 54xx) chips and a dual socket 1207 with Opteron 2300 CPUs for about the same amount of money. The platforms both use the less-expensive, cooler-running, and faster registered DDR2-667 versus the FB-DIMMs that the Xeon 5000 and 5400 motherboards use. The advantage of the Xeon setup is that you can get Xeon 5400s that are much higher-clocked than Opteron 2300s (3.2 GHz vs 2.5 GHz) and *single-threaded* apps are typically a fuzz faster clock-for-clock on Xeon 5400s than on Opteron 2300s. However, on applications that have many threads, the FSB architecture of the Xeons means that they do not scale as well as the Opterons do. With your heavily-threaded application, I'd imagine that the Opterons would be better for your uses than Xeons unless you manage to find a *great* deal on fast Xeon 5400 CPUs.

I bet the Nehalem MPs will be not much if any less expensive than the Opteron 8000 series. Plus, the Nehalen MPs use slow, hot, expensive FB-DIMM RAM.

Yes.

There *is* a reason. The Opteron 2300 series has two HyperTransport links. One is the standard HT link that all AMD CPUs have to talk to the chipset. The other is the NUMA HT link to talk to the other socket and access it's RAM if need be. Opteron 8000 series CPUs have two of these NUMA HT links, so each CPU can talk to two of its neighbors instead of only one. Technically all of the quad-core chips- Phenom, Opteron 2300, Opteron 8300, have the silicon needed for the two NUMA + one standard HT link as they are all made from the same CPU dies, but the model and testing determines what is active.

Hey, thanks for the response. This is interesting, so to access Memory that the CPU doesn't own, it has to use some bandwidth to go through another socket.

Now, does this mean in a 4 or 8 socket setup, you might have to go two or more 'hops' to access certain parts of Memory, since each CPU doesn't have enough HT links to access all other sockets?

I wonder what kind of bottleneck this incurs. Does intel's 4 socket stuff have this problem too? (or is it not a problem...)

The issue is less defined by 'bandwidth' (as opposed to 'bandwidth efficiency') and more by latency.

AMD's on-die memory controllers reduce latency by a significant amount and more effectively use available bandwidth. However ...

In order to to take advantage of the reduced latency and bandwidth efficiency the code in your application must be “NUMA" aware - - don't write to the DIMM bank of cpu0 and access from cpu4.

If your memory access is organized so that each microprocessor operates with data in its DIMM bank, it's got an advantage over a FSB design.

Without going into a great deal of specifics, can you spell out in general terms what you are trying to do?
Language? OS? Compiler?