Irfanview can convert many file formats including .icohttp://www.irfanview.com/
Open it save as then select .ico. You may need to enable transparency for it to look right.
EDIT, do not let irfanview take all image formats(as a default app or file association) unless you want it to. And there may be some kind of toolbar option you will want to uncheck.
iconedit32 is also a good one with options for multi sized(16x16 + 32x32 + 48x48 ect) icons in one file