That's probably because you are looking at it when youre not stressing it at all.
Download MSI Afterburner, run a game, then go back to your desktop and check each gpu's activity levels in afterburner. That will tell you for sure if you have a problem.