10 Gb ethernet cards are expensive, as well as 10 gb switches (>3000 $ and only 4 ports of 10 GB)
I find a very attractive idea to implement ethernet over SATA, especially for realizing less expensive clusters.
SATA 3 is 6 GB/s. and you can have even 10 SATA ports integrated on a motherboard. Comparatively a dual 10 gigabit adapter like Intel 10GBE Af DA 2 Port Server-PCIE NIC SFP+ costs 600$.