Hi I had this same problem with Nero Recode 14 and my gtx970. After much researching I found it has to do with a missing dll for cuda in the newer nvidia drivers. You can fix this problem usually by either installing old nvidia drivers (which isn't really an option for us gtx970 owners) or installing the missing dll's into system32 and sysWOW64. Google "nvcuvenc_337.94" and you can obtain a zip/rar file with the dll's if you can't locate them on your PC. You then simply extract the zip file and put the dll's in the appropriate directories and reboot. This method works for Nero Recode and Adobe Premiere I am fairly sure it will work for Vegas also.
EDIT Here is a post on Geforce forums describing the problem and the fix...