I want to reduce the load temperatures a little bit, and am thinking of trying an offset voltage of minus 0.1-0.2V to reduce my temperatures a bit, would this work/be worth it?
Since it does not cost you anything to try it, there is no harm in trying it. Worst case, you may need to re-install your OS if it gets corrupted by silent errors.
As for aftermarket HSF, no need to look very high price-wise for decent performance. The Hyper 212+ ($20-30) and 212EVO ($30-40) are relatively inexpensive but will perform 12-20C better than stock HSF. The EVO performs 2-5C better than the + under load, which may (or not) be worth $10 to you.
AFAIK, the 212s still have the best bang-per-buck ratios on the market.