First, DO NOT USE THE DRIVE, and DO NOT FORMAT IT. When you delete files, you are only removing the indexes to them (not actually deleting them). However, they are at risk of being overwritten. I would try running Recuva
on it (might need a deep scan). Also, recover the files to another drive - this is CRITICAL to avoiding overwriting the drive.
Finally, next time, remember to keep a backup. (FYI, there is in fact a 4GB file size limit on FAT32 drives. exFAT is intended for thumb drives more than NTFS and will do the trick while being fully compatible with Macs, but requires some updates to work - you should already have these.)