There may be a Beta or earlier 64 bit ver of FF floating around but nobody I know uses it. Firefox 32 works fine with Vista 64 so I suspect your problem is not related to 64 vs. 32.
Your 'program files' directory is for 64 bit programs, of which there are precious few at the moment. There is a 64 bit version of IE in there but it will not run flash which makes it unpopular in this youtube age.
Program files (x86) is where almost all your apps will install, since most apps are 32 bit still. Vista 64 runs 32 bit software fine under windows on windows (wow). This process is automatic. Just let programs install themselves automatically.
Don't delete any folders - There are separate ones for 32 bit and 64 bit ones because they are mapped differently.
Regarding Firefox - the 32 bit version runs perfectly well, so IMHO the issue should be somewhere else. Have you tested your connection to/from your ISP? If you insist, there is a 64 bit build called 'Minefield'. But please be advised that Adobe have not created a 64 bit version of Flash Player.