TBH, it could be either. The 5400k isn't exactly fantastic, so won't really give you great performance as it is, but combined with a 450, you could be getting bottlenecked anywhere dude.
What kind of games are you playing? What resolution are you playing at? The lower the resolution, the more likely you'd be getting BNd by the CPU, the higher the resolution, the more likely you'll be getting BNd by the graphics card. This is an extremely simple way of describing it, but in most cases it's true.