Just about every router has a blacklist or whitelist you can setup to block sites, look in the configuration for that. Probably no need to go though the hassle of setting up a proxy server.
There is software also you can use like this http://www.imfirewall.us/