PCIE bus speed - 6950s Xfire

[venkman781]

I'm running into two issues with my new build, and they're probably related. First off, system specs:

i5 2500k 3.3GHz @ stock | ASRock p67 Extreme4 (B3) | 2x 4GB G.Skill RipJaws PC3 1600 | 2x MSI R6950 Twin Frozr III 2GB @ stock | OCZ Agility 3 SSD | OCZ ZX 850w PSU 80+ Gold | Hyper 212+ | LG Flatron w2053 | Win7 x64 | UEFI (bios) v2.0

Issue 1: Display sporadically remains in power saving mode at boot, but machine boots into Windows 7 (confirmed via RDP).

I've been able to replicate this using DVI and DSUB-DVI connections. I can replicate it with both 6950s - both individually and in crossfire. It only happens when the display is connected to the card in the top PCIE slot.

I'm trying to determine if my monitor is starting to go bad, or if my motherboard is borked.

I have been able to resolve this issue by powering off the machine and then turning it back on.

Issue 2: 6950 in top PCIE slot (primary adapter) is running at 4x in Crossfire. Linked adapter runs at 8x.

I haven't been able to get both 6950s to run at 8x/8x in Crossfire. I have updated the motherboard bios, clean install of Win7, fresh install of the latest chipset and Catalyst drivers. I have been unable to find any settings related to PCIE in the ASRock UEFI bios.

When I plug a single 6950 into the top slot, I have been able to get the card to function at x16 according to GPUz 5.5. However, typically, I run into issue 1 after a reboot.

I'm going to the test using a different PSU - but any other suggestions as to what might be causing this? Let me know what other info you need.

 

[gnomio]

the last slot will only provide four PCI-E lanes, rather than the eight lanes the other two slots will receive. So put it another slot.

 

[venkman781]

There's 3 PCIE slots on the board. The 6950's are either in the top or middle slot, both of which run at x16. The bottom slot only runs at x4 and is too close the edge of the board and the front panel audio header input for the card to fit there. I don't have any other PCI devices on the board.

 

[Twoboxer]

If you are using the two blue slots (PCIE_2 and _4) closest to the cpu socket (and I assume you are), and if any required 6-pin power connectors are plugged in (and I assume they are), and the 8-pin power connector is plugged into the mobo . . . you're going about it in the right way. Not much consolation, eh?

Are the two 6950's identical in all respects?

You could try clearing CMOS, loading optimized defaults, etc, and trying each card by itself in slot PCIE_2

If the situation doesn't change with a new psu (ie, on restart with a single 6950 in PCIE_2 you get Issue 1), it would seem to be a mobo issue.

Once each card is proven OK in PCIE_2, you would of course try SLI again.

 

[venkman781]

Yep, that's the setup. I bought the two 6950s at the same time - both MSI Twin Frozr III 2GB. Aside from the bus information, GPUz shows them being identical.

Have a look.

4x card (PCIE2)
http://gpuz.techpowerup.com/11/10/26/9m7.png

8x card (PCIE4)
http://gpuz.techpowerup.com/11/10/26/ew.png

 

[Twoboxer]

I guess if I walked into your room and heard what you had done, I'd go back to ground zero and see if I have two working vid cards, ie,

Uninstall all OCing utilities and graphics drivers. Download fresh WHQL drivers but don't install.

Clear CMOS. Install only Vidcard_A. Load (optimized) defaults. Make only the changes (if any) required for the PC to run correctly. Save and boot into windows. Install graphics drivers.

Test functionality, including what gpu-z says about the card and get a frame rate using Furmark benchmark. Repeat for Vidcard_B and compare.

If everything looks ok - including the restart issue - install Vidcard_A in PCIE_4, attach sli link, and boot into Windows for a test.

You could try a different SLI connector I suppose, and also flip the cards and try SLI again.

But if SLI is the only thing that fails, I guess I'd start with RMAing the mobo - though in an extreme case i suppose it could still be a bad vidcard.

Edit: Forgot to mention it could still be the psu, but you are going to test that.

 

[gnomio]

The board is certified for a 3x gpu Cfx setup. But the one slot is connected to the duplexer and not the cpu. Move the card that's showing x4 to another slot as it looks like you got it in the slot which is connected to the duplexer.

Oh just another thing with which application are you checking the speed?

 

[Twoboxer]

That would occur if he were using PCIE_2 or _4 combined with PCIE_5.

He says he is not.

 

[venkman781]

Thanks - I did do some tests on both cards. I tried swapping them, as well as testing them individually in PCIE2 and PCIE4. But I did not clear CMOS in between and left the existing Cat drivers installed, so now I'm doubting the results, so I'll test again.

 

[venkman781]

That would occur if he were using PCIE_2 or _4 combined with PCIE_5.

He says he is not.

That's right. I'm not using PCIE5. The PLX chip on the Extreme4 is connected to PCIE2, I believe.

 

[gnomio]

what app are you using to check? Your situation rings a bell as I saw somewhere a guy who had the same problem where gpu-z kept on saying x4. In the end it was a problem with the application not reading it right. Have you tried SiSandra and see what it detects as?

 

[Twoboxer]

^ Yeah, I seem to remember that too - I think it was a piece of AMD software and it showed one status while gpu-z reported incorrectly another.

 

[venkman781]

Ok folks, did more testing tonight. This time I was careful to remove the catalyst drivers in between switching cards around.

I found that the blank display issue occurred with both cards individually, but only when in the PCIE2 slot. I also started getting BSODs when either card was in PCIE2. This would happen consistently for both 6950s after installing catalyst drivers and rebooting from Win7. Shutting the machine down and powering it back up would clear this and I could login to Win7. A subsequent reboot would again cause a cycle of BSODs.

I also found that in GPUz, both 6950s ran at 16x v1.1 in PCIE2, whereas in PCIE4 they register at x8 v2.0. I tested both cards individually in PCIE4 and had no issues with blank screens at boot.

I swapped out the OCZ ZX 850w psu for an Antec TruePower 650w for testing. I was able to recreate the BSOD loop in PCIE2 with one card. I still need to test the 2nd.

I notice a subtlely higher amount of play in the way the 6950s each are seated in PCIE2 during testing. The cards are slightly easier to wiggle, making me even more suspicious of the motherboard.

Based on your suggestions, I downloaded Sandra to check the bus speeds against a 2nd source, but wasn't totally sure if I am looking in the right spot. Anyway, with either card in PCIE2, Sandra reported a link speed of x16 (2.5GBs). Is that considered PCIE v2.0 or is that 1.1 as GPUz indicates?

Thanks for the help. My feeling is that issues 1 and 2 from my OP are linked to a faulty PCIE2 slot. It doesn't make any sense that both cards would each run at 8x in PCIE4; it's a x16 slot. Unless if possibly a solder point is damaged on PCIE2, causing the slot to behave as if a card is in it. Hard to say.

Going to see if I can recreate the issue with an older nVidia card I have, as well. Just to be thorough. After that it's time to deal with ASRock to get this replaced under my warranty. That ought to be fun.

 

[Twoboxer]

x16 refers to how many data lanes the slot contains. Slots that can hold a video card must have room for 16 lanes. However, they may operate electrically as x16, x8, x4, or perhaps x1. So a motherboard with a limited, fixed number of available data lanes can spread them across the slots as best it can.

So a card with 2 x16 mechanical slots may operate them electrically as x8-x8 if/when two video cards are present.

PCIE 1.1, 2.0, 3.0, etc refers to how data is sent over those lanes - for our purposes meaning maximum speed. So a PCIE 2.0 x-16 electrical slot will support a higher data rate than a PCIE 1.1 slot.

Your PCIE4 slot is x16 mechanically, but it won't operate above x8 electrically if a video card is also present in PCIE2. So, it may well be permanently locked at x8 or lower electrically . . . after all, you are supposed to put yyour first video card in PCIE2.

The key item to check is does Sandra report the same as GPU-z when BOTH vid cards are installed. in PCIE2 and 4.

 

[venkman781]

Thanks for the reply. I do appreciate the insight into nuances of the PCIE bus - I want to make sure I'm interpreting the data from GPUz and Sandra correctly, and that helps.

Just to restate your question to make sure I understand you correctly, a PCIE x-16 electrical slot will support a higher data rate (i.e. 5GB/s @ 2.0) than a PCIE 1.1 slot (i.e. 2.5GB/s). Simply put: 1.1 = 2.5GB/s and 2.0 = 5GB/s?

Ideally, both vid cards should run at x8 @ 5GB/s (2.0) if installed in PCIE2 and PCIE4, and assuming everything is configured properly in the bios, drivers, etc., right?

 

[Twoboxer]

PCIE 3.0 moves data faster than PCIE 2.0 moves data faster than (yes, twice as fast as) PCIE 1.1.

X16 gives you more paths to run that data through than X8.

So if your PCIE 2.0 x8 slot is really running as a PCIE 1.1 x4 electrical slot, any card in there is restricted in two different ways.

To your last question, yes that's right.

If running PCIE 2.0, most video cards will give nearly the same frame rate in an x8 slot as an x16 slot. This (oversimplification) is why x16-x16 is not necessary for sli, and x16-x8 or x8-x8 work fine. Most cards aren't fast enough to challenge the available bandwith.

OTOH, x4 and PCIE 1.1 wil likely restrict the frame rate you get from most video cards.

 

[venkman781]

Well I did a bit more tinkering this weekend.

When I have both 6950s in Crossfire, Sandra reports the card in PCIE2 as running at PCIE 2.0 at 4x, while the card in PCIE4 is running at PCIE 2.0 at 8x.

I took a closer look at the actual PCIE slots on the motherboard, Twoboxer. To your point about how a slot can run "mechanically" at 4x, 8x or 16x lanes - PCIE4 has pins in the slot that go about half way across the slot. This would make sense - that slot can run a 2nd PCIE video card using 8x lanes at a PCIE 2.0 data rate. PCIE5 has pins that go 1/4 of the way across, so it is mechanically able to do only 4x at a 2.0 data rate.

Just for grins, I tried a nVidia 7800GTX I had kicking around in PCIE2. It booted fine and ran using 16x at PCIE1.1 data rate, this is the same that a single 6950 would register in Sandra in that slot (when it would actually boot).

So I'm at a loss...something is preventing the board from running Crossfire at x8/x8, and causing the blank screen issues during POST. I've put in a 2nd ticket to ASRock support (the first one has gone unanswered since 10/22). I also wrote an email to their warranty information mailbox. ASRock's site directs users to the seller to initiate warranty service, and when I contacted newegg about this, they just referred me back to ASRock. We'll see what happens, I may get some follow up, or I may have to just replace the board out of pocket and eat the $150. Joy.

 

[gnomio]

Well I did a bit more tinkering this weekend.

When I have both 6950s in Crossfire, Sandra reports the card in PCIE2 as running at PCIE 2.0 at 4x, while the card in PCIE4 is running at PCIE 2.0 at 8x.

I took a closer look at the actual PCIE slots on the motherboard, Twoboxer. To your point about how a slot can run "mechanically" at 4x, 8x or 16x lanes - PCIE4 has pins in the slot that go about half way across the slot. This would make sense - that slot can run a 2nd PCIE video card using 8x lanes at a PCIE 2.0 data rate. PCIE5 has pins that go 1/4 of the way across, so it is mechanically able to do only 4x at a 2.0 data rate.

Just for grins, I tried a nVidia 7800GTX I had kicking around in PCIE2. It booted fine and ran using 16x at PCIE1.1 data rate, this is the same that a single 6950 would register in Sandra in that slot (when it would actually boot).

So I'm at a loss...something is preventing the board from running Crossfire at x8/x8, and causing the blank screen issues during POST. I've put in a 2nd ticket to ASRock support (the first one has gone unanswered since 10/22). I also wrote an email to their warranty information mailbox. ASRock's site directs users to the seller to initiate warranty service, and when I contacted newegg about this, they just referred me back to ASRock. We'll see what happens, I may get some follow up, or I may have to just replace the board out of pocket and eat the $150. Joy.

ok first go to you display options and disable all the power savings features mode whatever there is enable. Hibernation, sleep all that disable it. When your done with that go into your bios then disable the power saving features in it as well.

Then go here
http://hwbot.org/forum/showthread.php?t=15952

Just look for your boards bios update and download the latest one you can find on there. Then reflash the bios

 

[Twoboxer]

To your point about how a slot can run "mechanically" at 4x, 8x or 16x lanes - PCIE4 has pins in the slot that go about half way across the slot. This would make sense - that slot can run a 2nd PCIE video card using 8x lanes at a PCIE 2.0 data rate. PCIE5 has pins that go 1/4 of the way across, so it is mechanically able to do only 4x at a 2.0 data rate.

You got the theory right, but the common usage of the terms is reversed.

x16 mechanically (or Physically) refers to a slot that has room for 16 slots. It's physically big enough to allow a vid card's connector to fit in it.

x16 electrically refers to all those slots working . . . having pins actually connected to circuitry. x8 and x4 electrically look just as you described.

Of course, this is no help in solving the problem lol. And I'm getting cinfused, so let's recap what/whether something is proven:

Status as reported:

- Each 6950 shows PCIE 2.0, x16, when run as a single card in slot PCIE2.

- When running the two cards in PCIE2 and 4, both cards are shown as running PCIE 2.0. The card in PCIE2 is shown running x4 and the card in PCIE4 is shown as x8. When the cards are swicthed, the result is the same.

Performance observed:

- When either card is running alone in PCIE2, the two cards perform "identically" in Furmark Benchmark. (Would like to know what that avg. frame rate is.)

- When run in crossfire, the cards show . . . what frame rate? When the cards are switched, is the frame rate "identical"?

 

[venkman781]

Quick update after running Furmark benchies.

Status:

Each 6950 shows PCIE 2.0, x16, when run as a single card in slot PCIE2.

Not quite - each 6950 shows PCIE 1.1, x16, when run as a single card in slot PCIE2.

When running the two cards in PCIE2 and 4, both cards are shown as running PCIE 2.0. The card in PCIE2 is shown running x4 and the card in PCIE4 is shown as x8. When the cards are switched, the result is the same.

Yes. This is correct.

Performance observed:

When either card is running alone in PCIE2, the two cards perform "identically" in Furmark Benchmark. (Would like to know what that avg. frame rate is.)

Yes. Both cards in this scenario performed nearly identically. Card1 1273pts, 21 FPS avg. Card2 1279pts, 21 FPS avg.

When run in crossfire, the cards show . . . what frame rate? When the cards are switched, is the frame rate "identical"?

In Crossfire, avg. FPS for cards is 41. When switched, avg. FPS is literally identical.

 

[Twoboxer]

Corrected Status as reported:

- Each 6950 shows PCIE 1.1, x16, when run as a single card in slot PCIE2.

- When running the two cards in PCIE2 and 4, both cards are shown as running PCIE 2.0. The card in PCIE2 is shown running x4 and the card in PCIE4 is shown as x8. When the cards are swicthed, the result is the same.

Corrected Performance observed:

- When either card is running alone in PCIE2, the two cards perform identically, showing 21 FPS in Furmark Benchmark.

- When run in crossfire, the cards show 41 FPS, 95% scaling. When the cards are switched, the frame rate is identical.

Conclusions:

1) You are getting excellent (95%) scaling when running Furmark Benchmark in crossfire. On the surface, that looks good.

2) But I'm running an i5 750 (lowered to stock) and a 5870 (stock) on this PC. According to this link, the 5870 and 6950 are "comparable":

http://www.anandtech.com/bench/Product/331?vs=294

When I run Furmark at 1920x1080 FS on my 1920x1200 screen, I get 37 FPS, +76% performance vs your 6950. (32 FPS, +52%, at 1920x1200 in case that's what you ran).

Unless I'm missing something here - and that's quite possible lol - your 6950 is significantly underperforming in a single card environment. This could validate the reporting that your cards are running at PCIE 1.1 when in single mode. And it could bw consistent with your 95% scaling with 2x6950 . . . they're running on a PCIE 2.0 bus over narrower bus widths.

I'd be (a) pushing the mobo manufacturer, and (b) testing one of my cards in someone else's PC using Furmark.

 

[venkman781]

1) You are getting excellent (95%) scaling when running Furmark Benchmark in crossfire. On the surface, that looks good.

Indeed! Definitely seems like they play well together.

When I run Furmark at 1920x1080 FS on my 1920x1200 screen, I get 37 FPS, +76% performance vs your 6950. (32 FPS, +52%, at 1920x1200 in case that's what you ran).

Actually, my display currently maxes out at 1600x900. I ran the Furmark bench at FS, 1600x900, 4xMSAA, dynamic background to get the avg. FPS I listed in the previous post. So you're performance % is probably even a bit higher.

I'd be (a) pushing the mobo manufacturer, and (b) testing one of my cards in someone else's PC using Furmark.

Finally got a response from ASRock today via email with an RMA request form. I've not had any experience with an RMA before, let alone one with ASRock, but its a promising start.

 

[Twoboxer]

At 1600x900, Furmark 1.9.0 shows 46 FPS on my 5870.

Best of luck with Asus - if you get a resolution *other than* bad mobo, please post back here. It may help someone else in the future.

 

[venkman781]

Just to close out this thread. I finally got the RMA motherboard back from ASRock. After a clean install of Win7, chipset drivers, and AMD Cat 11.10 - I'm happy to report that the PCIE buses are working as they should in Crossfire. Both are at 8x/8x, PCIE 2.0.

 

[Twoboxer]

Yay - congrats!