That's odd. That would often indicate a CPU bottleneck, but the FX4100 shouldn't be an issue.
Are you using any mods in Skyrim? In particular any config changes (like ugridstoload)?
Also, you should make sure to download the Windows patches/hotfixes for Bulldozer CPUs ( http://support.microsoft.com/kb/2645594
)and install the newest GPU drivers from Nvidia. You might also have to update your motherboard BIOS (check the MSI website).