Every Node Gets Its Memory Controllers
Let's talk about the integrated memory controller. If you think 32 cores per chip you have to find a way to implement the memory logic without creating new bottlenecks. A single high-bandwidth, multi-channel DDR controller as a shared resource would not perform well. The other extreme would be a dedicated memory controller per core, which technically doesn't work at such a core count. But a memory controller per node certainly does, which is exactly what Intel is thinking of.
Eight nodes would provide eight 12.8 GB/s FBD2-1066 interfaces, resulting in a total bandwidth of 102.4 GB/s in Intel's current projections. Four cores sharing a memory unit sounds like a reasonable compromise, and the ring interconnect would provide an adequate inter-node communication pathway.
This very modular approach is not only promising from the performance standpoint; it also makes a lot of sense from a business perspective. Processors with defective cores could be turned into models with a smaller node count, or a smaller core count per node. Silicon with defective L3 cache areas could be turned into models with less L3 cache, etc.
Whether all eight memory controllers will actually be used will be the customer's decision, but it is already very obvious that such a single-processor, 32-core server with only eight memory modules would be amazingly inexpensive and breathtakingly fast.
We are not entirely sure how long the Keifer project has been evaluated inside Intel, but it must at least be half a year. At the same time, we've heard rumors that the project might already dead. The documents we received from undisclosed sources are dated March to May 2006. However, the information is very interesting, as it shows which direction Intel may be going in the upcoming years when it's time to replace Core. It also shows that the decision on future processors is usually made analytically, several years in advance, and tightly coupled to available manufacturing technologies.
Also check out the latest TG Daily news about project Keifer.