Thunderbolt 103: Getting Inside The Controllers
As a technology, Thunderbolt operates similarly, regardless of the controller you're using. For the sake of our technical discussion, we're using Intel's previous-generation two-port Light Ridge chip.
The Thunderbolt controller on your motherboard is always in host mode, with a second-gen PCI Express interface to either a Sandy/Ivy Bridge-based CPU or PCIe-equipped chipset (and one or more available DisplayPort inputs).
Inside the controller, you find a PCI switch and a collection of DMA engines collectively referred to as the Native Host Interface (NHI). The PCI switch enables connectivity for downstream devices, while the NHI is used for software protocols and device discovery (Plug and Play detection). The Thunderbolt switch, marries, if you will, the DisplayPort and PCIe inputs into a single connection.
Recall that each Thunderbolt port requires two channels, one for device I/O and another for display signaling. In the case of Intel's Light Ridge, four channels output to two Thunderbolt ports.
When you have a daisy chain or an endpoint device, Intel's Thunderbolt controller chip provides a PCIe 2.0 x4 downlink. However, the company also enables broader flexibility for attaching multiple components. With four connected, for example, you could configure the downlink as four individual PCIe 2.0 x1 links. According to Intel, Cactus Ridge (2C/4C) can be configured in the following ways:
- 1 * x4: one device of four lanes
- 4 * x1: four devices of one lane each
- 2 * x2: two devices of two lanes each
- 1 * x2 + 2 * x1: One device of two lanes and two devices of one lane
Most of the time you'll see one device attached to one Thunderbolt controller, yielding a 1 * x4 configuration. However, there are situations where a single Thunderbolt controller might control multiple devices.
Apple's 27" Thunderbolt Display is a good example. Its controller is responsible for communicating with a USB hub, a FireWire 800 port, Gigabit Ethernet, and a FaceTime camera. Each device requires an interface to the Light Ridge controller, with its internal PCI switch divided into four single-link lanes when it's in switch mode. Each lane is then mapped to a device controller (USB, FireWire, Ethernet, and the camera). This setup doesn't negatively affect the display itself because, remember, I/O and DisplayPort are on different channels.
In its current form, Thunderbolt employs PCIe fanout mapping. This means that daisy-chained Thunderbolt devices are routed through the internal PCI switch of the controller ahead of it in the sequence. As a result, the first device in the chain always enjoys the lowest latency.
The PCI Express protocol also influences latency. For example, a storage device on a desktop PC might negotiate precedence over a capture card, and you'd assume that Thunderbolt should probably operate the same way since it uses PCI Express signaling. However, each device plays a role in the PCI arbitration of devices connected downstream. Thus, throttling is quite noticeable if every piece of hardware in a Thunderbolt daisy chain operates simultaneously.
A downside of device arbitration is wasted bandwidth due to inefficient management. This is potentially an issue with fanout mapping because the PCI switch located in the preceding controller manages downstream devices. It's possible to circumvent the drawbacks to fanout mapping by using PCI direct mapping, illustrated in the diagram above. This method passes the Thunderbolt signal through each controller's internal switch, completely bypassing the PCI pathway. It'd ultimately impose a greater negotiation burden on the first system's PCI switch, but it delivers the benefit of greater control over bandwidth/resource allocation.
Thunderbolt controller firmware, as it is implemented by Intel and Apple, uses fanout mapping. Direct mapping is possible and is fully compatible with the Thunderbolt standard. But there's no word yet on if or when it might be a user-selectable option.
