So after some tinkering with fan curves and befriending Ryzen Master I got a partial solution. In the BIOS I set the fan's curve so that it is constant (35%) up until 72 degrees celsius, when it shifts to maximum. This way it is silent when idle and only revs up while under pressure. I might still have to adjust it in order to not fluctuate when gaming.
In Ryzen Master i played a bit with overclocking and downclocking and figured that 1.45V and the like (once 1.49V also) are because of Precision Boost and only appear when only 1 or 2 cores are overclocked for background calculations. When all clocks are pushed all cores go up to <4.3GHz and the voltage goes down to ~1.35. According to other internet people this is perfectly normal.
Only other issue remaining is the temperature. It still goes up by itself even when idle, only now it stops at around 50 degrees because of the constant fan curve at that temp.
I would consider that normal in the case i was playing and pressuring the CPU, but this also happens when idle. I didnt dare try out how hot it would get if i capped the fan speed lower.
Ps.: BIOS updated to the latest version without issue, but no changes happened.