While it won't hurt it, it maybe difficult to clean it that way. My concern is routers don't usually get that hot. You may have other issues going on with it. Are you loosing connections or is it rebooting?
dont worry about it being hot... most electronics run hot and a router will not suffer any performance decrease by it running hot... although you could clean it, it wont really help much but I like cleaning my electronics inside out lol my computer aswell so go for it blowing air into it will not damage it