GPU, CPU, then RAM in that order. Those are the only devices that directly affect FPS.
Motherboard only really matters if you plan on overclocking, using SLI/Crossfire, or you want features like USB3.0 or SataIII. From a purely gaming perspective you could get a really cheap motherboard and still get great frame rates with a single graphics card. Obviously it needs to support everything you are buying though.
To those saying PSU. Buying a bigger PSU can't increase FPS. No matter how much money you spend on the PSU it makes no difference (for FPS). I bought a PSU for $12 and I'm 100% confident that my CX600V2 Corsair will hold up fine to a ~350W max load. However, you do have to buy a properly sized PSU and you should buy one that isn't garbage, but you can say that about any component in the build.
He said importance for gaming...not FPS. Read what words are typed. If you dont have the power for your system, and a quality unit, you cant run a gaming rig. So therefore, the psu is right up there with the gpu in my book.
Well obviously the parts all play together and contribute. You can't run the system without a proper PSU, but they way you guys put it indeed makes it sound as though the better PSU you buy, the better your experience will be. If it's enough, it's enough - if not, it's not. That's all.