I run a 9800gtx+also by XFX, so your card is better. I have run dual monitors but I haven't tried video and games. What brand was the 3870x2 from? You might try contacting xfx about this, though I don't know how much they can help.
Here's a question how are the two screens configured, meaning when you go into display settings, which is checked as your main monitor and is the desktop extended to the second monitor.
Try setting both screens to the same resolution, just to see what happens, then set it back. Then try setting the game to a higher resolution to a higher resolution.
Try running the game not in full screen. Also what happens when you just run the game and not the video, if the same thing happen, here's my theory, either it's a setting, driver, compatibility issue, or when you load up the game in 1280x1024, it sets the scales the first monitor down to that resolution, and in turn messes with the resolution scaling on the second, but since the actual resolution hasn't changed, not all of the screen is displayed.
You might also try switching which monitor runs the game and video.