I have tried some other scenarios and things are getting weirder and weirder.
If I switch Turbo Boost from Auto to Enabled (and leave the individual settings to Auto), it goes up to 3.4 regardless of how many cores are in use.
If I leave 1 core to Auto, but switch 2, 3, 4 cores to 37, Turbo Boost will work how I originally wanted, it would go to 3.7 regardless of how many cores are in use. The problem is that if I set it to 38, it will still only go to 3.7. If I change the 1 core scenario from Auto to 37 or 38, Turbo Boost simply stops working.