TBH, it could be either. The 5400k isn't exactly fantastic, so won't really give you great performance as it is, but combined with a 450, you could be getting bottlenecked anywhere dude.
What kind of games are you playing? What resolution are you playing at? The lower the resolution, the more likely you'd be getting BNd by the CPU, the higher the resolution, the more likely you'll be getting BNd by the graphics card. This is an extremely simple way of describing it, but in most cases it's true.
I tried my GPU on my friend's CPU which is i5 and I can play BF4 on medium settings with an average fps of 40.I tried playing the game on my rig on medium settings and the fps is dropping constantly.1600x900 is my resolution.
So probably CPU is the problem right?