Well, it is both physically and electronically possible to upgrade parts. Whether you should or not depends on what you want it to do better. You could, for example, put in an SSD and get snappier system loads. If it's Win7, you could even set up the SSD for the correct AHCI controller mode.
So what do you want it to do better? Crunch spreadsheets? Folding@home? Higher framerates for Quake XVII? Photochop?
It also depends on your pocketbook. After a few years, a whole new system will probably be better, but more expensive, than an upgrade. First thing I would upgrade on that is memory - it came with a base of 1 GB. What OS are you running, and is it 32-bit or 64-bit?
It's got integrated graphics, so almost any video card would improve the framerate of gameplay in modern games. Let me look - does it have expansion slots to put one in? Yes, one PCI Express x16.
So what's your price range and what do you want to accomplish? Or are you just noodling around? In the latter case, if you want gameplay get a graphics card and memory. Otherwise, just get memory.
If money is not a problem (would that that were true for me), buy a new machine that's built with a view to whatever load you want to put on it.