Idle temps are usually @10°C above ambient temps, give or take a few °C. That's in a case with decent airflow. In cases where there's only 1 fan, that can easily be 20°C or more above ambient.
So if your case has just 1 intake or exhaust fan, and it's @20-30°C ambients, then yeah, expect idle of 40-50°C.
Also, ppl have forgotten to remove the safety plastic on the cooler base, that'll screw temps, incorrect mounting of the cooler (it's loose) because they are scared of breaking something, mounted the cooler fan backwards on the cooler, have all case fans as intakes to get Alot of air in the case with 1 or no exhausts, had all case fans as exhausts to get rid of as much heat as possible, the list of things not correct is numerable, all of which mean higher temps at idle and loads.