I'd remount the cooler. It seems to me there is a problem with heat transferrance if your idle temps are that high. I don't know your experience level, but there are many videos on youtube showing thermal paste application and how to mount the Ryzen cooler.