There are different file systems that storage devices (SD card, SDDs, HDDs, etc) can be formatted to. I am confused on what machines you formatted to and copied files from. But I can tell you this: Your phone, as do all phones (besides Windows phone) run off of Linux. The file system that Linux normally runs on is either ext2, ext3, or ext4. All Linux systems can read this format (which includes Mac), however, Windows can't. And since Xbox is a Microsoft product, it too cannot read Linux file systems.
Linux can also read and write to Windows file systems (NFTS, FAT32, FAT16, etc). You should format your SD card to FAT 32. That way you can read and write to it on different operating systems.