Serial Memory Management: FB-DImms
Fully Buffered DIMMs are considered both a blessing and a curse. Server systems often require large amounts of memory, which usually cannot be done with conventional DDR SDRAM due to repidly degrading signal strength. Deploying registered memory usually helps, becuase a little component (the register) takes care of signal integrity.
However, Intel's latest 5000 series professional chipset allows the administrator to deploy four memory modules at once to polulate all four channels for performance reasons. The electrical strain for the memory controller therefore would be high even with registered memory, especially if several modules per channel were to be deployed.
Intel decided to engage with Fully Buffered DIMM technology, which is based on DDR2 memory that is enhanced by a little controller chip called the AMB (Advanced Memory Buffer). It basically converts the parallel design of DRAM modules into a serial transmission protocol, which establishes point-to-point connections between the controller and a module. This enables the deployment of multiple fully buffered memory modules per channel, but the price is the AMB that runs at high clock frequencies (3.2 GHz for DDR2-533 and 4 GHz for DDR2-667) and consumes at least around 6 W per device. So if you need eight memory modules you should consider an attional 50 W power consumption. In exchange you will have the means to assemble a machine with 64 GB RAM easily.
The FB-DIMM technology does not have a direct impact on performance. The 5000 chipset runs DDR2-533 at FSB1066 speed and DDR2-667 at FSB1333 and performs similar to what you may be used from desktop solutions. Although the quad channel memory controller introduces higher bandwidth, it does not scale that well. Expect gross bandwidth increases in the area of 50% when compared to a dual channel controller, as latencies grow as well.
The test system was powered by four FB-DIMMs. As we wanted to analyze their energy consumption, we measured the system energy consumption with four and with two DIMMs (see benchmark section).
