If he really says "anyone can access your files", ask him to prove it by going over to his house and have him access the files on your computer.
While there are ways (spyware and viruses) that can make your files vulnerable to a few people, not everyone has the knowledge and ability to access your files.