Probably the least botleneck, would be the processor itself, but really you should be updating GPU, mobo, RAM, which would rather force a CPU change, as well. Even if it were suitable, I wouldn't recommend connecting an old PSU, to new components. That's just as likely to fail, and destroy half your new components, with it. Not worth the risk. If you HDD is SATA, it could be re-used. The same would apply to an optical drive. If you case is a reasonable mid-tower ATX it may be worth re-using it, but as you can get a reasonable new case for under $40, it's a question of whether it's worth re-using.
One thing which is important, in cost of new build, is operating system. If you need a new OS, Windows 7 64bit is basically $100. Other than that, you can put something decent together, in the $500 region, especially if you can re-use some parts.