Network Interface Cards (NICs) 101

I liked the article. It would have been interesting to see different common controllers on a chart listing different characteristics such as latency versus the other (e.g. PCI NIC v PCIe NIC v on-board NIC v wireless NIC) just to quantify exactly what differences we are dealing with.

Not a very informative article at all. There are plenty of things to mention just to give a few examples: One could discuss the different cat and how that spec affects the cabling and other hardware in detail. Shielded vs unshielded gear. The different features that you find among different NICs, in more technical detail as to what differentiates a server grade NIC from a desktop grade. How different brands of NICs handles more complex issues such as QoS or TSO and what controllers are better than others in different workloads. What differentiates their signal strenghts how good impedance matching one can get from different ports of different brands of hardware and how well this is resolved. How are packet collisions and losses handled? What latencies can we get? How well do these different NICs really offload the CPU? There are many different features out there such as iSCSI, PXE VDEV, SerDes, IPMI, Cable Diagnostics, Crossover detection, pair swap/polarity/skew correction, NDIS5 Checksum Offload (IP, TCP, UDP) and largesend offload support, IEEE 802.1Q VLAN tagging, Transmit/Receive FIFO (8K/64K) support , ... There are concepts such as HMAC+, RMCP, RAKP, SOL Payload, Kg, SIK, BMC, MSI Writes, SKP Ordered Set resets and Training Sequences (TS), RDTR and RADV timer mechanisms, ICR reads, Bit Banging, ...

There are many different features and concepts, and differences among how different manufacturers handle them. Things could be measured and analysed with spectrum analyzers, and dedicated benchmark hardware, both on the performance and CPU utilization. But none of it is discussed in this article that is quite sub-par unfortunately.

And I'm here, with 12 megabits down, 1 megabit up. On ADSL.

+1

Sometimes I don't even get that.

LSO/TSO/GSO are no longer needed for modern NICs.

A bigger issue is the quality of interrupt coalescing. A 100Mb network can recieve up to 144,800 packets per seconds and cheap network cards issue an interrupt for every packet. Normal CPU time slices are in the order of milliseconds, and your CPU may be handling something like 10,000 switches per second. When you cheap 100Mb NIC attempts to issue 144,800 per second, your CPU will crumble. 1Gb is 10x worse.

Most people doing file transfers will be sending large 1500 byte packets and the max packets per second is for 64byte packets. Most home users don't even notice the difference other than they're only getting 800Mb/s, which could very well just be their mechanical harddrive being the bottleneck, but if they see something like 20% cpu usage while copying, that's why.

High end NICs have large onboard buffers and will interrupt the CPU on the first packet, but subsequent packets they will delay. Don't worry bout the delaying being very long, they're still less than a millisecond. Coupled with that, modern NICs support what is called MSI-X, which is part of PCI-Express since 3.0. Don't worry, they back-ported this 3.0 feature to something like 2.1.

This sweet feature allows for "soft interrupts". Once the first interrupt happens and the driver starts processing the packets, the driver can disable the interrupt. This way new packets that come in do not interrupt the CPU or do not need to be delayed until the next interrupt. The NIC will instead flag a location in memory if any new packets come in. Once the driver is done processing the current batch of packets, the driver can check if there were any "soft" interrupts during this time. If there was, process those packets.

At some point the driver will catch up and process all of the packets. Once this happens, the driver will re-enable the interrupt and switch back to what it was doing before the original interrupt. This dramatically reduces CPU usage while still maintaining low latency and high throughput. The best of both worlds!

Don't pay for a premium Killer NIC, you can get a high end Intel i210 server NIC with all of these awesome features for $60 and will work very well with great drivers for all operating systems. If you need something cheaper, there are plenty of Intel NICs in the $15-$30 range that may not have all of these features, but will still be great for any home user.

The major benefit to these cards is that they handle the buffer storage, encoding and decoding of data through the seven network layers.

Um. Network cards don't usually handle offload for anything above TCP (OSI layer 4)... and everything below that is either always done in hardware, or provides negligible overhead.

No one in their right mind is going to offload something like SSH to the NIC.

I would have expected and/or liked to have seen some recommendations in this article. We come to Tom's to see results, and there were not even any charts or data to show how/why this would be important. How many times did the article say "we all know this". It's true, we do know this, so give us the details we've come to expect from your articles. How much latency between the two cards difference can we expect etc. Dissapointing read.