I endured playing BF3 from release with my Asus HD4850 for a month or two, so when I upgraded to the Asus HD6950 DCII it felt good to finally use my native resolution of 2048x1152 with 4xMSAA, even though I was only getting mid-30 fps. For a while I had the card flashed/unlocked as an HD6970 DCII running at 980 / 1375 @ 1.218 voltage with full fan speed. Once the novelty of the new performance faded, I wasn't satisfied playing sub 60 fps any longer. I flashed back to stock 6950 with just the shaders unlocked. At this point I began to play with the fps always on display, and I noticed constant fps drops below 20. No matter what combination of clocks (I never tried a higher voltage than about 1.25 v, where I hear up to 1.3 is safe), I couldn't fix it. I uninstalled/cleaned/reinstalled drivers multiple times, with every new release for the last 5 months to no result. At some point I started to play with MSAA turned off, and that allowed the fps to hit 60+ spikes (I run with vsync on) but the fps still drops below 20. I was always assumed it was just the video card under performing but I am now open to it being some bottleneck in general in my system. This lead me to test the game at 640 x 480 on low video settings, vsync off, at stock clocks and voltage and somehow the card was generally hovering in the 50 fps range, which should be utterly impossible with this card at such bare bones settings. Below is a screen shot of this test with a bunch of concurrent hardware info on display. In case it's relevant, I use a manual fan speed profile that is a lot safer than the auto fan profile. Windows 7 Professional runs on my Seagate 500 GB drive while the game runs from my WD 320 GB drive, both of which are Sata 2.0 and regularly defragmented.
Not 100% sure what your dips down to 20 fps is but I can attest to some weird frame limiter in game even though I have Vsync turned off in game and in drivers. Check task manager for how much cpu origin is eating, had a problem a few patches back that had it using 50%, weirdness.
Running it on every setting at its lowest (except res) with a 6870 + athlon 2 x3 (stock) + 4g DDR2, still can't get above 48 fps exactly. Can however change it from low to medium/high and get the exact same fps.
I suspect the CPU to be the culprit. It's a very old CPU too - even though it was a king when it was released - but by today standards, it's slow. BF3 is very CPU bound game, that explains why you were getting terrible performance.