Figure idle temp on a smaller aio is usually somewhere around 10-13°C above your ambient temp, depending also on airflow, just how soon you tested after heating up the case with the gpu, if the radiator is sole exhaust etc, so it's entirely possible to see a 20°C above normal ambient temps. Also must take into consideration that an air cooler will return to minimal temps very fast, whereas liquids don't, so if you test the cpu too soon after running extensive heat, the liquid will be hotter than normal, which lowers its ability to absorb heat energy as fast. And, of course, in Windows 10 there's Always junk running in the background, so a true (less than 5%cpu usage) idle temp is getting quite hard to get. So 40°C ish, isn't uncommon for that particular cpu, which can rub warm anyways. So you are good there.
Yes, 4000± rpm is normal for that pump as it's designed to run flat out 12v constant. Trying to lower or vary speeds can damage the mechanism, and/or even raise temps, either at idle or load.
Personally, I prefer RealTemp over CoreTemp. It's just as accurate and easy to use for Intel cpu's, but was also written in conjunction with the use of Prime95 and has no issues with multiple core readings and gpu, all of which can be stuck in the task bar for easy viewing.