-> Looks good. That's just a 14C rise over the ambient (room) temps and perfectly normal for a slow fan speed at idle.
Do a check during a gaming session. Load temps are much better at confirming you have a good installation.
If you use a program like HWInfo it can track the hi/lo temps for you during a gaming session. www.hwinfo.com/
Is this normal? Have I screwed up the heatsink/thermal paste thingy? Which way is the heatsink meant to be placed anyway?
The 212 +/EVO has very low fan speed at idle (and still very low under load in my setup - only 700RPM) so somewhat high idle temp is to be expected. Heat propagation through pipes to fins to air is also much slower at low temperature difference so as suggested above, try giving it a proper workout.
As for orientation, that is entirely dependent on how your case airflow is setup. If you have a rear exhaust fan, you can have the heatsink perpendicular to the PCIe slot with the HSF blowing towards the rear exhaust or if you have a blowhole exhaust fan, you can mount the HSF parallel with PCIe blowing towards the blowhole, either setup works.