Quadfire scaling issue with 6990 and 2x 6970

[alrobichaud]

Does anyone have any hard evidence in regards to scaling these gpu's together? I thought I had a problem with bottlenecking because of the pci-e lane but after watching the following link, I am not convinced anymore.

http://youtu.be/rSfifE2Domo

My motherboard has 4 pci-e lanes capable of x16 and work with the following configurations :

16/0/0/0
16/0/16/0
8//8/16/0
8/8/8/8

With the 6990 in slot 1(x16) and one 6970 in slot 3(x16) I get a 3dmark11 score of X4888. With the 6990 in slot one and a 6970 in slots two and three, I get a score of X4873 with one of the 6970's disabled. So this should mean that the 6990 is now in a x8 slot with that configuration. As soon as I enable the second 6970 and re-run the benchmark, I get a score of X3916. If it's not due to the lane is it a problem with scaling? I have the latest driver version installed. Does anyone know of a benchmark program that I can use that will test this setup at 5760x1080? I have noticed that in 3dmark11 none of the scores that I browsed for 3 or 4 6970's are much higher than the score I get with 3 6970's(basically a 6990 and 6970 in crossfire) When I have a bit more free time tomorrow, I will load bf3 and use fraps with different configurations to see what kind of results I get.

**edit**

All test were run with the gpu's at 830MHz and 1250MHz(gpu and memory) and my processor and ram were running at 4GHz and 1866MHz

 

[alrobichaud]

BTW, in 3dmark11 there are no scores for 6990's. A single 6990 will benchmark as two 6970's.

 

[alrobichaud]

I found some free time and did some benchmarking with Battlefield3. Firstly, I have spent a few hours browsing the internet looking for some decent info on quadfire 6970 and found little to no info. I watched a few youtube videos with some jackasses showing off there test bench setup but none of them offered any real benchmarking. I found lot's of benchmarks on the 6990 on it's own and crossfired with a 6970 but that was pretty much it.

This is copied and pasted from my FRAPS log:

2012-01-08 18:54:15 - bf3 6990, 6970, 6970 830Mhz 1250Mhz Ultra with AA Off 5760x1080 Frames: 13906 - Time: 180000ms - Avg: 77.256 - Min: 71 - Max: 80

2012-01-08 19:05:28 - bf3 6990, 6970 830Mhz 1250Mhz Ultra with AA Off 5760x1080 Frames: 10393 - Time: 180000ms - Avg: 57.739 - Min: 53 - Max: 63

2012-01-08 19:20:32 - bf3 6990 830Mhz 1250 MHz Ultra with AA Off 5760x1080
Frames: 7348 - Time: 180000ms - Avg: 40.822 - Min: 39 - Max: 42

2012-01-08 19:31:22 - bf3 6990, 6970, 6970 880Mhz 1375Mhz Ultra with AA Off 5760x1080 Frames: 12191 - Time: 180000ms - Avg: 67.728 - Min: 62 - Max: 70

2012-01-08 19:34:53 - bf3 6990, 6970, 6970 880Mhz 1375Mhz Ultra with 4xAA 5760x1080 Frames: 9906 - Time: 180000ms - Avg: 55.033 - Min: 44 - Max: 60

2012-01-08 19:49:34 - bf3 6990, 6970 880Mhz 1375Mhz Ultra with AA Off 5760x1080 Frames: 10145 - Time: 180000ms - Avg: 56.361 - Min: 52 - Max: 60

2012-01-08 19:53:01 - bf3 6990, 6970 880Mhz 1375Mhz Ultra with 4xAA 5760x1080 Frames: 7693 - Time: 180000ms - Avg: 42.739 - Min: 39 - Max: 46

2012-01-08 20:08:54 - bf3 6990, 6970, 6970 830Mhz 1250Mhz Ultra with 4xAA 5760x1080 Frames: 9795 - Time: 180000ms - Avg: 54.417 - Min: 37 - Max: 61

What I did was load a single player mission and basically stood there without doing much except looking around and firing the occaisonal round. It was a wide open level on a rooftop overlooking a city with lot's of smoke, blowing flags, blowing laundry and my squad consisting of 4 guys standing in front of me. Each time I got into the exact same position before starting to record the fps with fraps. With all 3 cards plugged in I disabled the two 6970's in CCC and got a result of 40FPs for the 6990, 57 FPS for the 6990 and 6970 and 77 FPS for the 6990 and both 6970's. This tells me that all 3 cards appear to be working just fine. The two 6970's added approx 93% more FPS. Not bad! These numbers were based on running all cards at stock 6990 speeds of 830MHz(gpu) and 1250MHz(memory). I overclocked the 6990 to run it at stock 6970 speeds which is 880Mhz and 1375MHz and found my FPS went from 77 to 67. This mimics every benchmark that I read for the 6990 which basically said overclocking it resulted in either no gain at all or a slight loss in performance. It also added more heat and noise. At stock 6990 speeds, the noise level was acceptable even with the cards sandwiched together using rubber pads as spacers to allow for a bit of airflow to the blowers. I did not even attempt to switch the 6990 to BIOS 2 which increases the gpu voltage from 1.175 to 1.2 and sets the gpu clock to a stock 880MHz to match the 6970. The reason being is I had been doing all my 3dmark11 runs to this point with an overclock of 940Mhz and 1400MHz and with 2 cards separated by a pci-e slot there was lot's of airflow and not much fan noise but as soon as I added the second 6970 in between the 6990 and the other 6970, things got hot fast and the blowers would hit as high as 60% which sounded like a dyson. Even going from 830Mhz to 880MHz added more noise than I would like to hear so I am kind of glad the performance did no improve with overclocking. The only thing I did not do but I will in the next day or so is to remove the 6970 in slot 2 and run the exact same senario instead of just disabling it to make certain going from a x16 slot to a x8 slot did not make a difference. I think I should also switch to BIOS 2 and re-run the quadfire setup overclocked to make sure that overclocking the 6990 with or without the switch gives the same results.

**edit**

Almost forgot. 3dmark11 sucks ass when trying to benchmark high end crossfire gpu's. Even when I was using a single 6990 and crossfire 6990 with a 6970 it did not accurately report gpu speed, memory speed or the number of gpu's. Not sure if that is due to AMD drivers or simply the limitations of 3dmark11.

 

[iam2thecrowe]

3 things to note without me having to read all that unorganised information:
1. CPU bottleneck
2. crossfire doesnt scale well with 4 cards
3. your psu is not good http://www.hardocp.com/article/2011/07/19/in_win_commander_ii_1200w_power_supply_review/9

 

[FlintIronStagg]

3 things to note without me having to read all that unorganised information:
1. CPU bottleneck
2. crossfire doesnt scale well with 4 cards
3. your psu is not good http://www.hardocp.com/article/2011/07/19/in_win_commander_ii_1200w_power_supply_review/9

^^ agreed

 

[alrobichaud]

3 things to note without me having to read all that unorganised information:
1. CPU bottleneck
2. crossfire doesnt scale well with 4 cards
3. your psu is not good http://www.hardocp.com/article/2011/07/19/in_win_commander_ii_1200w_power_supply_review/9

I have a few things to note in relation to your comment:

First, I expressed my thoughts as they came to me. Forgive me for not writing a Stephen Hawking worthy technical document. Yes, it was a bit drawn out and somewhat unorganized but I did not ask what you thought of the format.



1. CPU bottleneck? In regards to what? 3dmark11 or bf3? Using 3dmark11, my processor maxes out during the physics test but hardly moves during any graphics test. In BF3, none of the cores or threads go above 75%. If you are just talking in general, please provide some sort of technical data to back this up.

2. In regards to crossfire not scaling well, I have read that also but I have not seen any sort of benchmark to back that up. I would say to go from 40FPS to 57FPS to 77 FPS with a 6990 then a 6990+6970 and finally a 6990+6970+6970 is pretty good scaling. I admit, a proper benchmark would have been better than what I did but the point of the OP is that I would like to know what you guys suggest I use to do this. Again, please provide a link to back up your opinion.

3. Thanks for finally providing some useful info in regards to the PSU. Yes, it was the least expensive 1200w. I admit it is not great. A couple of reasons why the reviewer failed the PSU. During the 900W torture test the PSU had an efficiency of 83% but At 1184W the efficiency dropped to 79.72% just below the rated 80plus efficiency. This is a quote from the final thoughts 'The one aspect where the unit does outperform those units, efficiency, is promptly negated for the most part by it not making its advertised and "80 Plus certified" efficiency levels. This is in conjunction with posting less than stellar voltage regulation on the minor rails, really marginal DC output quality, and poor documentation' Anyway, if I were powering a life support system then maybe I would be more concerned with having less than 50mv AC ripple in the extreme worst case. I work in for a cable company and when testing the fibre optic nodes for AC ripple on the DC power supplies we aim for 25mV or less so I am really not too concerned with the numbers. Sure, it won't win any awards but it provides stable power.

 

[FlintIronStagg]

I kind of thought it was common knowledge that trifire sscales better than quadfire. I've read the 6990+1 6970 was the best Scaling for that config.

 

[kitsunestarwind]

Quadfire scaling in 3d mark is still an issue specially on the combined test, Futuremark admits that is the problem but it is upto the vendors to fix the drivers to help solve the problem in the Combined test with the issue occurs, happens way worse in extreme mode

 

[Inferno1217]

Quadfire scaling in 3d mark is still an issue specially on the combined test, Futuremark admits that is the problem but it is upto the vendors to fix the drivers to help solve the problem in the Combined test with the issue occurs, happens way worse in extreme mode

The vendors do not code the drivers in any way. It is up to AMD to fix any driver issues.

 

[alrobichaud]

I kind of thought it was common knowledge that trifire sscales better than quadfire. I've read the 6990+1 6970 was the best Scaling for that config.

In 3dmark11 crossfire and trifire appear to function normally but as soon as you add the fourth gpu into the equation, the score drops drastically. So yes, trifire scales better than quadfire in this situation. With the latest version of bf3 and the latest AMD drivers installed, going from crossfire to trifire to quadfire appears to scale quite well which is why I went through the effort of providing all the relevant data I could in regards to my testing. Back to the OP, I was hoping to get some hardcore benchmark results for quadfire scaling as opposed to just hearing about what people have read somewhere.

 

[alrobichaud]

Quadfire scaling in 3d mark is still an issue specially on the combined test, Futuremark admits that is the problem but it is upto the vendors to fix the drivers to help solve the problem in the Combined test with the issue occurs, happens way worse in extreme mode

I have read both that it is up to AMD and 3dmark11 to fix the issue so, again, until I can find some info released from either company that explains what is happening it is tough to know what to beleive. I have already tried contacting 3dmark11 but their customer support is terrible. I have tried posting in their forum but I kept getting an error reporting that I need to have at least 6 characters in the body of the post in order to proceed. I am leaning towards 3dmarkk11 as being the culprit based on my experience with them so far.

I do agree with scaling being worse in extreme. In performance, going from trifire to quadfire my score stayed at rougly 12600 and in extreme it went from 4873 to 3916.

 

[FlintIronStagg]

In 3dmark11 crossfire and trifire appear to function normally but as soon as you add the fourth gpu into the equation, the score drops drastically. So yes, trifire scales better than quadfire in this situation. With the latest version of bf3 and the latest AMD drivers installed, going from crossfire to trifire to quadfire appears to scale quite well which is why I went through the effort of providing all the relevant data I could in regards to my testing. Back to the OP, I was hoping to get some hardcore benchmark results for quadfire scaling as opposed to just hearing about what people have read somewhere.

Then Google it

 

[alrobichaud]

Then Google it

Wow. I will try and bite my tongue rather than say something totally rude. As I have already stated above, I have read lot's of bechmarks in regards to trifre 6970 but I have not seen a single quadfire benchmark review. I have seen a lot of youtube videos showing off quadfire setups but none provided any benchmark results. Where do you think I got this info from? Of course, I googled it already. I spent many hours of my life I will never get back in the last couple of days googling 4-way crossfire 6970 review and did not get any valid results. If I had the answer then I would not need to come here and ask for assistance.

 

[alrobichaud]

If some of my replies to your posts come across as rude, then I do appoligize. I just get a bit tired of people replying with their opinion and not offering any hard core evidence to back it up.

 

[deadjon]

I would strongly suggest completley ignoring 3dmark 11.

From your results on BF3, it is obvious that the Quadfire Scaling is acceptable @ 5760x1080 with games/benchmarks that are well optimized for Multi-GPU setups.

I suggest downloading and running Haven 2.5 and run the test as you did with 3DMark 11 and BF3 - If it works with Haven and BF3 put it down to 3DMark 11 being crap.

 

[alrobichaud]

I will give haven 2.5 a try. Thanks for the suggestion. After searching Tom's hardware, I found a benchmark titled '6990 crossfire review' but it was similar to most of the ones I found with google in that it is referring to dual gpu's in the same card as opposed to separate cards. Interestingly, they did do one test with two 6990's in crossfire. In both situations, there was no real improvement which could by why everyone has it in their head that quad 6970 or dual 6990's do not scale well at all based on benchmarks like that.

http://www.tomshardware.com/reviews/radeon-hd-6990-antilles-crossfire,2878-14.html

On the other hand, if you look at guru3d crossfire 6990 review they at least have used games with DX11 patches installed so the numbers look a lot better.

http://www.guru3d.com/article/his-radeon-hd-6990-crossfire-review/20

In the end, I am fairly happy with my results. Who wouldn't be hitting 77 FPS in bf3 on ultra at 5760x1080. I would still like to find a good benchmark program whether it is synthetic or an in game benchmark to test out DX 11 games at extreme resolutions which is really the only thing quad 6970's is good for. I will gladly try out any further suggestions in regards to this.

Thanks.

 

[wolfram23]

Have you tried going 6970+6970+6990 to get the 6990 in the 16x slot?

Sorry if you mentioned it, I only skimmed the previous posts.

 

[alrobichaud]

Ha! I don't blame you for skimming through it. I went cross eyed reading through it again. I did try that configuration and found that the two 6970's would crossfire but the 6990 would show in CCC as disabled. I tried a 6970 in slot one and 6990 in slot 3 and the same thing happened so I can only assume that you cannot crossfire a 6990 and 6970 with the 6970 as a primary. The display port on the 6970 is also not capable of outputting to 3 monitors without the use of an active dongle. The 6990's display port has no problem with 3 monitors using the supplied passive dongles.

 

[alrobichaud]

I would strongly suggest completley ignoring 3dmark 11.

From your results on BF3, it is obvious that the Quadfire Scaling is acceptable @ 5760x1080 with games/benchmarks that are well optimized for Multi-GPU setups.

I suggest downloading and running Haven 2.5 and run the test as you did with 3DMark 11 and BF3 - If it works with Haven and BF3 put it down to 3DMark 11 being crap.

Thanks for providing exactly what I was looking for. I had to run the tests at the lowest graphic settings at 5760x1080 because the 6990 did not like running with really low FPS which caused the benchmark to freeze twice. Here is what I ended up with:

6990 score 1018 FPS 40.4
6990+6970 score 1581 FPS 62.8
6990+6970+6970 score 2083 FPS 82.7

BTW, this benchmark looks way cooler than 3dmark11

 

[alrobichaud]

Best answer selected by alrobichaud.