How much does the SD card hold? 2GB? 4GB? How much RAM do you have? The swap file size is directly proportional to the amount of RAM you have.
If you're looking to free space, I suggest you use a program like CCleaner (download from www.piriform.com ) to delete unnecessary files that clutter your system. If you don't use the Windows "Hibernate" function, disable it and delete the hiberfil.sys file. This will free up the same amount of space as your current system RAM. You can also trim down on programs and uninstall the ones you don't use often.
I think it's risky to keep the swap file in an SD card. If it accidentally comes lose, or if someone takes it off, you'll have a BSOD.