I'm trying to back up my sister's files on her MacBook Pro (10.5.8) to my desktop which is Windows 7 Pro, so she can upgrade to Mountain Lion. Whenever I connect through SMB, it disconnects shortly after giving me an error code -41. I've been looking around the web for a solution, but I haven't found anything reliable. Any help would be much appreciated.
You shouldn't have to backup. The mountain lion install should transfer it all i would think. If not, you might want to give an FTP solution a go. Install a server on the mac and a client on the PC. It shouldn't disconnect with FTP.
Never, ever do an upgrade without backing up important data! Who knows what might go wrong. In fact, never run a computer with keeping regular backups.
This is trivial with a Mac. Buy an external hard drive (they are very cheap nowadays) and use Time Machine. This makes it trivial to restore any or all files. And with Mountain Lion it is almost essential to use Time Machine. Time Machine automatically backs up everything (unless you specifically exclude files or directories) and you don't need to worry about what files you should be copying.
Do what Ijack said and buy an external hard drive to back up. If all you want is the files of the macbook I don't see why you would even need to use Time Machine. Also I would do a fresh install instead of an upgrade. An example would be when I upgraded my work laptop up to 10.7 from 10.6, the installation failed and then the hard drive failed shortly after.