Is this with an overclock or at stock settings? If it is at stock settings then that is horribly high for idle. If anything it should be idling around the mid 30s if your ambient temperature in the room is halfway decent.
What kind of case are you using? Do you have good cable management? You could be potentially choking your airflow throughout the case.
ideally cpu idling temps are roughly 20C more than ambient temps..i think ur temp is very high..u can do these-check for dust and clear it..manage cables as much as u can so that they dont interfere with the airflow..have a minimum of 2 case fans on for air intake and one for exhaust..the exhaust fan preferably of a larger diameter..check for the proper attachment of cpu cooler..reapply thermal paste-use a well branded one like arctic silver or arctic ceramique..if none of these work, u might have to change the CPU cooler to a better one...placement and proper attachment of cooler is very important so double or triple check it..