The Pi is essentially a self-contained computer, you just need a display (can be HDMI or composite video), microUSB cable for power, keyboard, mouse, and SD card for the OS. A case is also a decent idea because it's just a bare PCB.
I'm going to buy one of these soon and it is even good for a CHEAP media center since their is videos of it playing 720P just fine. For 35$ its not bad
I had kind of the same idea in mind when I got one, and while it CAN play SOME specific formats at 720p and even 1080p without issue, just running the OS and something like XBMC uses around 50% of the CPU power sitting idle. So it can play 1080p content, but it basically doesn't have the power to do more than one thing at a time, so you can watch a movie or whatever, but if you then try and do anything else with it... You bump into the limits of its abilities.
The new sort of turbo modes developed are promising, but it looks like right now they have a tendency to corrupt your SD card. Guess we'll see what comes of that, since right now the Raspbmc dev is saying you can only go up to about 800MHz (a 100MHz bump) without risking corrupting your SD card, and there's speculation it may be an issue with the Pi's firmware.