What OS is this? And the quick answer is yes, drivers could be the culprit. Also, could be the physical slot on your mobo. Creative is know for poopy drivers. One common thread I saw was people with 32 bit OS and 4 gigs of ram. Creative tech support actually told them to remove RAM to make the sound card work, LOL!
I know there is a guy that made aftermarket drivers for x-fis that supposedly work really well, but I forgot his name, sorry. If you search google for x-fi and drivers and dig for a bit, you could find them. Seem to cure everyones problems.