1. Your PSU. It allows for future expansion.
2. Your motherboard. For the processor before anything else.
3.Your CPU/APU . Depends if your going Intel or AMD. Some my ague that the GPU should come first, but you can't properly utilize the it with a crappy processor. (Intergrated graphics for now)
3. Your GPU. Needless to say, this is the "magic" that allows you to play awesome looking games.
4. Your RAM. your current system sounds quite old, so I don't know if it's ddr2 or ddr3. If its ddr2, get it before the processor.
Your 2120 probably wont match the socket of the new mobo, so get them both together.
Your choice if you want to use the stock cooler (if any). Just a reminder to get a CPU/APU cooler if not provided.