Dual socket memory install 1111

[garberw]

dual socket memory install 1111


Hello,
My server has 8 sockets,
cpu1=dimm1,2,3,4;
cpu2=dimm5,6,7,8
The following pairs are interleaved:
(1,2)(3,4)(5,6)(7,8)
If I have a pair of 4 gig dimms
and a pair of 2 gig dimms,
can I install
dimm1=4gig,dimm2=4gig,
dimm5=2gig,dimm6=2gig.
This means that cpu1 and cpu2
have a different amount of ram.
If I install instead,
dimm1=4gig,dimm2=2gig,
dimm5=4gig,dimm6=2gig
then they won't be interleaved.
The motherboard can enable/disable numa as well.
What to do?

 

[GhislainG]

What motherboard? 4 + 4 and 2 + 2 should work, but info on all supported configurations should be in the manual.

You can enable NUMA if your OS is Windows Server 2008 R2, Windows 7 or a version of Linux that's NUMA aware. You may want to read http://archive.msdn.microsoft.com/64plusLP and then decide if it should be enabled or not.

 

[garberw]

Motherboard is Dell Poweredge 2970.
It is not an "optimal" configuration according to the manual (that's why I asked),
but maybe it is possible anyway.
The O.S. is Red Hat RHEL 5.1

Thanx.

 

[GhislainG]

It obviously isn't optimal. Why not get another 2 x 4GB kit?

According to the release notes, it should work if you don't use virtualization. To play it safe, I would disable it. http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html/5.1_Release_Notes/RELEASE-NOTES-U1-ia64-en.html

 

[GhislainG]

Edit: I checked the manual and it says:

DIMMs must be installed in matched pairs of identical speed, technology, and size in the following pairs of sockets:
– DIMM 1 and DIMM 2
– DIMM 3 and DIMM 4
– DIMM 5 and DIMM 6
– DIMM 7 and DIMM 8

Therefore you have no choice but to install 4 GB modules in DIMM 1 and DIMM 2 and 2 GB modules in DIMM 5 and DIMM 6.

 

[garberw]

I tried it with 4 GB modules in DIMM 1 and DIMM 2 and 2 GB modules in DIMM 5 and DIMM 6.
It halts on post/boot but if you press F1, it continues with the rest of the boot normally.
bios reports 12GB running at 667 Mhz (the speed of the dimms).
Then RHEL 5.1 command "free -m" reports the full 12GB and no errors in dmesg.

The question is whether the ram is still interleaved so it operates at full speed.
Thanks

 

[GhislainG]

It's easy to check with Memtest86+. SInce you need to press F1 to boot, what message is displayed?

 

[garberw]

it displays "non optimal ram configuration"
"press F1 to continue, press F2 to enter setup"
then halts.
If you press F1 it continues with the rest of the boot process
Incidentally Memtest86+ failed to detect the errors in one of the 2 gb dimms (there were 4 originally)
two of which were replaced by the pair of 4gb dimms.
But Memtest86+ would probably report the memory parameters correctly.

 

[GhislainG]

Since that Dell server requires identical modules in each pair of sockets, I would presume that they operate at full speed.

Unless the server only is for home use and you don't care about the warning at POST, you should install 4 identical modules. Having to press F1 to continue the boot process is not ideal, particularly if there's a power failure and the system is shutdown and powered off by the UPS. It will power up again when the UPS provides power, but it won't boot. Same issue when you need to reboot after installing OS updates, etc.

 

[garberw]

We don't want to pay for the additional 4gb+4gb since it is $200.00.
It is not a problem to reboot manually. We have to do that often anyway.
Sure it would be nice to have the extra ram so we could reboot from
the offices on the other side of the building where we are.
It's like 1/4 mile walk to the server room.

Dells always have a ridiculously simple bios.
The "Node Interleaving" setting in bios could not be changed;
it was DISabled. I suppose this implies that NUMA is ENabled
according to the manual; the virtualization in bios was disabled
and we don't have any need for it.

Memtestx86+ with the old faulty ram (2gb+2gb+2gb+2gb) reported 8gb running at 2GB/s.

Memtestx86+ with the newly installed ram reported 12gb
AMD Opteron(0.09) 2394 MHz
Memory 12gb 2338 MB/s (<---------- same speed, no?)
Settings: RAM: 342 MHz (DDR 684) / CAS: 5-5-5-15/DDR2 (128 bits).

so it's interleaved and all that jazz, and working optimally, no?

Dual-channel: A property of the motherboard where the memory bus is split into two 64-bit channels, effectively doubling the bandwidth in relation to a single-channel setup. Dual-channel is not a property of the memory modules.

I think the "(128 bits)" from Memtestx86+ means it is interleaved / dual channel correctly.

 

[garberw]

It is not a problem to reboot manually. We have to do that often anyway.
Sure it would be nice to reboot from
the offices on the other side of the building where we are.
We don't want to spend $200.00 for more ram.

Dells always have a ridiculously simple bios.
The "Skip Memory Test" setting didn't help.
The "Node Interleaving" setting in bios could not be changed;
it was DISabled. I suppose this implies that NUMA is ENabled
according to the manual; the virtualization in bios was disabled
and we don't have any need for it. You can't do virtualization with NUMA
enabled.
There weren't any other pertinent settings. So much for Dell.


Memtestx86+ with the old faulty ram (2gb+2gb+2gb+2gb) reported 8gb running at 2GB/s.

Memtestx86+ with the newly installed ram reported 12gb
AMD Opteron(0.09) 2394 MHz
Memory 12gb 2338 MB/s (<---------- same speed, no?)
Settings: RAM: 342 MHz (DDR 684) / CAS: 5-5-5-15/DDR2 (128 bits).

so it's interleaved and all that jazz, and working optimally, no?

Dual-channel: A property of the motherboard where the memory bus is split into two 64-bit channels, effectively doubling the bandwidth in relation to a single-channel setup. Dual-channel is not a property of the memory modules.

I think the "(128 bits)" from Memtestx86+ means it is interleaved / dual channel correctly.
no?

 

[GhislainG]

Dell doesn't allow single channel memory; therefore it has to run at optimal settings for the modules that are installed.

 

[garberw]

thank you very much. Case closed!!