Hi, I have a HAF X myself.
I might be stating the obvious here but here goes:
check if the fan is connected to the toggle button, there should be a 2-pin connector to the button, and if it is double check if the connection is good (i.e. doesn't have shorts, etc.)
If you have already done the above, then you might have a bad fan, you should get it replaced.