I did this a few months ago (for 7.1) The biggest issue is having the proper inputs and outputs.
It works best if you have what they call multi-channel inputs on your receiver. This is a set of 6 (or 8 for 7.1) rca inputs. To use them, you'll need a bunch of mini-jack to rca adapters. The thing is, most new receivers don't have these inputs, they were more common back when component cables were standard for HDTV. You can still get them today if you look for them, but they usually only come on high-end receivers.
As a side note, it's fairly common for the sub not to have a signal when going the multi-channel route. This is easy to fix by looking for an option called "Bass Redirection" on your sound card's menu.
If you decide to go the optical route, you will need a sound card that can encode to either Dobly Digtial Live, or DTS connect (otherwise you just get stereo). The sound quality isn't as good as multi-channel, but it's not bad. Try either a sound blaster x-fi titanium (or better) or an asus xonar (read the fine print.)
I hear mixed reports about using HDMI from a video card. Theoretically, it's the easiest option, so try it first. But you might find the audio is lagged.