Mouse Clicking Problem

I'm having issues with my mouse. Before giving the symptoms let me add that I have done the following:
- tried different mouse
- updated mouse driver
- tried different USB port


The issue that I run into is occasionally my mouse buttons stop working. I can still scroll around and my mouse cursors over everything properly, but when I click on things it's not registering. I have found that opening task manager (CtrlAltDel) and then closing it remedies the problem temporarily. Eventually it happens again though.

My concern is that some hardware piece in my computer is not working properly. I have no problem with replacing something, but it's just that I don't know what to replace.

Any advice would be helpful.

Thanks

It does sound like a OS issue. I had similar problems that I corrected by going into the control panel > Mouse > Hardware > Properties > Advanced settings and changing the polling to 200 per second changing the wheel setting to look for wheel and finally changing the buffer length to 200. This did correct my problem with my mouse. This was done on a system running XP. I believe that the settings can be changed in Vista as well.