While the Core architecture was remarkably efficient, certain design details had begun to show their age, first among them the Front Side Bus (FSB) system. This bus connecting the processors to the northbridge was a real anachronism in an otherwise highly modern architecture. The major fault was most noticeable in multiprocessor configurations, where the architecture had a hard time keeping up with increasing loads. The processors had to share this bus not only for access to memory, but also for ensuring the coherence of the data contained in their respective cache memories.
In this kind of situation, the influx of transactions to the bus can lead to its saturation. For a long time, Intel simply worked around the problem by using a faster bus or larger cache memories, but it was high time for them to address the underlying cause by completely overhauling the way its processors communicate with memory and outside components.
The solution Intel chose—called QuickPath Interconnect (QPI)—was nothing new; an integrated memory controller is an extremely fast point-to-point serial bus. The technology was introduced five years ago in AMD processors, but in reality it’s even older than that. These concepts, showing up in AMD and now Intel products, are in fact the result of work done ten years ago by the engineers at DEC during the design of the Alpha 21364 (EV7). Since many former DEC engineers ended up in Santa Clara, it’s not surprising to see these concepts surfacing in the latest Intel architecture.
From a technical point of view, a QPI link is bidirectional and has two 20-bit links—one in each direction—of which 16 are reserved for data; the four others are used for error detection codes or protocol functions. This works out to a maximum of 6.4 GT/s (billion transfers per second), or a usable bandwidth of 12.8 GB/s, both read and write. Just for comparison, the FSB on the most recent Intel processors operates at a maximum clock frequency of 400 MHz, and address transfers need two clock cycles (200 MT/s) whereas data transfers operate in QDR mode, with a bandwidth of 1.6 GT/s. With its 64-bit width, the FSB also has a total bandwidth of 12.8 GB/s, but it’s usable for writing or reading.
So a QPI link has a theoretical bandwidth that’s up to twice as high, provided reads and writes are well balanced. In a theoretical case consisting of reads only or writes only, the bandwidth would be identical to that of the FSB. However, you have to keep in mind that the FSB was used both for memory access and for all transfers of data to peripherals or between processors. With Nehalem, a QPI link will be exclusively dedicated to transfers of data to peripherals, with memory transfers handled by the integrated controller and inter-CPU communications in multi-socket configurations by another QPI link. Even in the worst cases, a QPI link should show significantly better performance than the FSB.

As we’ve seen, Nehalem was designed to be a flexible, scalable architecture, and so the number of QPI links available will vary with the market segment being aimed at—from one link to the chipset for single-socket configurations to as many as four in four-socket server configurations. This enables true fully-connected four-processor systems, meaning that each processor can access any position in memory with a maximum of a single QPI link hop since each CPU is connected directly to the three others.



I regard being late as a quality seal really. No point being first, if your info is only as credible as stuff on inquirer. Better be last, but be sure what you write is correct.
Perhaps, if you count being translated from French.
Nice article, good depth, well written
I don't know french, so no idea if it actually works. But I've tried from english to germany and danish, and viseversa. Also tried from danish to german, and the result is always the same - it's incomplete, and anything that is slighty technical in nature won't be translated properly. In short - want it done right, do it yourself.
You claimed the article on toms was a copy paste from another article. He merely stated that the article here was based on a french version.
I actually read the whole thing.
I just don't get TLP when RAM is cheap and the Nehalem/Vista can address 128gigs. Anyway, things have changed a lot since running Win NT with 16megs RAM and constant memory swapping.
1) How's the loop detection feature know when it is a loop ? The diagrams posted don't show any connection between it and the 'front' of the pipeline, so how can it know that the next operation is the same if it hasn't yet entered the loop?
2) On page 8 there's a diagram with a 4 socket setup showing 2 io hubs. Are they connected to the same pcie bus and whatever else they interface with? or are only 2 of the sockets able to directly access a given resource?
3) With the modular design, would one risk buying a cpu that doesn't work in a motherboard because it is intended for a 2 or 4 socket system? or are they all the same, simply with some qpi's disabled?
4) Am I right assuming that qpi replaces fsb when it has to communicate with an i/o hub only? (as shown in one of the top diagrams on page 8) Or is it used for every one of the 'blue' lines on the lower diagram (10 total in a 4 socket layout). The latter would mean 4 qpi's are barely enough to satisfy bandwidth needs in a server enviroment. I imagine an esx server with 4 processors (32 threads) can easily demand memory from dram pools not linked to the local core the threads are running on, and use 96GB/s (3x32) of the 102GB/s (4x12,8x2) total theoretical bandwidth in addition to some of the local 32GB/s bandwidth from the socket a given core/thread is running on. So if this scenario is correct, is it possible to increase the speed of the qpi (read: oc the link) to increase available bandwidth? And what happends if one would successfully find ddr3-1600 modules that would run within the 1,65v limitation? Wouldn't that mean the qpi was already at its limit? (38,4GB/s per dram pool x 3 sockets not local to the core that runs a thread). I know memory isn't truely the bottleneck in modern computers, but I still find it wierd that they put so much effort into the memory controller if it isn't actually the problem. Simply adding a few qpi links between the sockets and the chipset would've solved the bandwidth issue without limiting usable memory types by choosing a certain cpu. Sure it wouldn't have improved latencies, but honestly, who cares? neither in a gaming pc, netbook or any number of common server configurations is it the memory lantecy that is the bottleneck.
5) How much time should one assume is wasted when a core on conroe flushes the l2 cache? they seem to have solved the issue and as consequence increased cache latency (which should turn into slower overall cache performance). In english : can we expect any gain from this change?
6) Would the immensely increased tlb size improve performance in newer games which precache loads of data? (thinking quicker retrieval of texture data etc)
7) Page 12 mentions unalligned memory access, which I've never heard of before. Appearently compilers already try to avoid this situation, so can we expect the improvement to handling such to be of interest? What's the point of improving a feature to handle a situation that hardly ever arises in the first place?
http://www.xbitlabs.com/articles/cpu/display/replay.html
This was a very good article and is not a copy ... well done Fedy !!
perhaps it will burn out the IMC within the chip since its all done at 45nm, 1.6+v would be deadly, imagine air cooling a 3ghz quad core chip at ~2v? i take it it shares the rail even within the cpu so
depends on how connected that ram is, there might be advantages etc this way, and it also makes you wonder if AMD suffers from this - iv heard of extreme overclockers killing ram channels on AMD's etc
on the other hand who cares about high performance memory - 3 x 1333mhz is going to be better then 2 x 1600+mhz channels etc, along with the fact its an IMC based setup etc and average maximum bandwidths of ~32gb/s vs the current average maximum of ~12.8gb/s etc
as for the memory issue. Who'd want to run 3x1333 if they could run 6x1600 ? any enthusiast will only be satisfied with the best, and 1333 just isn't it. Not even 1600 is. ddr3-1333 is basicly obsolete, and it's not even mainstream yet. It's a disaster really.
A case of perhaps minimising reflected impedence?
Just my theory anyway ... remember ... I am only here for the humour ... not the technology.
AMD4LIFE