Memory bandwidth and pcie

darkstar845

Distinguished
Mar 29, 2010
230
1
18,680
pcie 2.0 has a maximum of 8GBps (based on wikipedia) and a video card like the gtx 580 has a peak memory bandwidth of 193GBps.

Will the pcie 2.0 bottleneck the GTX 580?
 

nPossible

Distinguished
Nov 20, 2011
1
0
18,510
What the real question here is: What is the bandwidth of the GPU in terms of how much it can process per second to send down those PCI-E lanes back to the CPU?

To give you an idea, here are my results with a HD5870 when running PCI-E Speed Test:

===============================================================

Devices found: 1

===> Testing device 0 <===
Device type: Unknown
Max resource 2D width/height: 16384/16384
Total GPU memory size: 1024 MB
Total CPU cached space size: 2047 MB
Total CPU uncached space size: 2047 MB
GPU engine clock: 900 MHz
GPU memory clock: 1225 MHz
Number of timing loops: 100
[ 16 bytes] CPU->GPU= 220.164 KB/sec, GPU->CPU= 135.958 KB/sec
[ 32 bytes] CPU->GPU= 680.599 KB/sec, GPU->CPU= 558.694 KB/sec
[ 64 bytes] CPU->GPU= 1.297 MB/sec, GPU->CPU= 696.353 KB/sec
[ 128 bytes] CPU->GPU= 2.610 MB/sec, GPU->CPU= 2.947 MB/sec
[ 256 bytes] CPU->GPU= 5.351 MB/sec, GPU->CPU= 4.954 MB/sec
[ 512 bytes] CPU->GPU= 11.133 MB/sec, GPU->CPU= 11.553 MB/sec
[ 1024 bytes] CPU->GPU= 20.219 MB/sec, GPU->CPU= 20.165 MB/sec
[ 2048 bytes] CPU->GPU= 41.596 MB/sec, GPU->CPU= 40.494 MB/sec
[ 4096 bytes] CPU->GPU= 89.187 MB/sec, GPU->CPU= 84.053 MB/sec
[ 8192 bytes] CPU->GPU= 155.075 MB/sec, GPU->CPU= 170.154 MB/sec
[ 16384 bytes] CPU->GPU= 292.882 MB/sec, GPU->CPU= 301.974 MB/sec
[ 32768 bytes] CPU->GPU= 778.494 MB/sec, GPU->CPU= 743.341 MB/sec
[ 65536 bytes] CPU->GPU= 1.234 GB/sec, GPU->CPU= 1.453 GB/sec
[ 131072 bytes] CPU->GPU= 2.844 GB/sec, GPU->CPU= 2.795 GB/sec
[ 262144 bytes] CPU->GPU= 4.321 GB/sec, GPU->CPU= 5.721 GB/sec
[ 524288 bytes] CPU->GPU= 4.973 GB/sec, GPU->CPU= 6.013 GB/sec
[ 1048576 bytes] CPU->GPU= 5.167 GB/sec, GPU->CPU= 6.399 GB/sec
[ 2097152 bytes] CPU->GPU= 5.282 GB/sec, GPU->CPU= 6.533 GB/sec
[ 4194304 bytes] CPU->GPU= 5.303 GB/sec, GPU->CPU= 6.500 GB/sec
[ 8388608 bytes] CPU->GPU= 5.335 GB/sec, GPU->CPU= 6.649 GB/sec
[ 16777216 bytes] CPU->GPU= 5.391 GB/sec, GPU->CPU= 6.675 GB/sec
[ 33554432 bytes] CPU->GPU= 5.384 GB/sec, GPU->CPU= 6.686 GB/sec
[ 67108864 bytes] CPU->GPU= 5.405 GB/sec, GPU->CPU= 6.690 GB/sec
[ 134217728 bytes] CPU->GPU= 5.409 GB/sec, GPU->CPU= 6.688 GB/sec
[ 268435456 bytes] CPU->GPU= 5.482 GB/sec, GPU->CPU= 6.692 GB/sec
[ 536870912 bytes] CPU->GPU= 5.457 GB/sec, GPU->CPU= 6.693 GB/sec
[1073741824 bytes] CPU->GPU= 1.849 GB/sec, GPU->CPU= 1.318 GB/sec
calResAllocRemote2D() returned an error when trying to allocate 1073741824 bytes
(uncached)!
Peak CPU->GPU Bandwidth = 5.482 GB/sec [data size = 268435456 bytes]
Peak GPU->CPU Bandwidth = 6.693 GB/sec [data size = 536870912 bytes]

======================================================================

As you can see from the last two lines, the max bandwidth that the card is able to put across the PCI-E lanes is about 6.7 GB/s. According to the Wikipedia page that is a little more than 1 GB/s less than the max of PCI-E 2.1. These results were achieved with an i3-540 and a HD5870. Both are slightly overclocked but nothing crazy. I only hope that someone running a fancy i7-2600K with a pricey HD6990 can fill in that remaining bandwidth. If anyone has such a rig, please be kind enough to give PCI-E Speed Test a run and post your results.

I am curious about the typical size of objects passed to the GPU while gaming. The larger files post near that bandwidth limitation of PCI-E 2.1, but the smaller files are not even close. I would expect objects being passed in game would be many small objects which would lead to the slower performance like we see in the earlier stage of the test run. Perhaps someone with more knowledge of how objects in real world gaming are referenced could comment on the typical performance we would expect to see in game.

If my suspicions are correct, I wouldn't expect a new line of PCI-E 3.0 GPUs to be all that much of a game changer. Any comments and/or corrections to anything I've said are most certainly welcome as I am no expert in this area. These are only my observations and conclusions.

Cheers