Cpu and gpu turbo kind of work differently and they are independent of each other. If it gets a high load it turbos. So less intensive games, let's say a flash game that doesn't need full gpu or cpu power and turbo won't go on. Even on some weaker games and you have vsync on, it won't need full power and turbo won't go on. Just when you get it near 100% it turbos.
How intel is set up, they have different speeds depending on how many cores are being used. This can be any type of load, whatever the cpu is doing, doesn't need to be a game, as long as it's a high load. For the 4590, it's set to 3.7 on 1 core, 3.6 on 2 cores and 3.5 on 3 or 4. Most aftermarket mobos nowadays let you turbo max speed no matter how many cores are being used.
Gpus are a bit simpler as everything is always divided to all resources so they are just in load or not. Again, doesn't need to be a game, just a gpu load. Turbo in any gpu or cpu will go above the nominal tdp, so yes it does create more heat and use more power. When the cpu or gpu gets too hot or is using too much power (not a concern on desktops), turbo will go off and it will go to the normal speed.
I feel I should also mention idle clocks. Even without turbo enabled or on older hardware where turbo didn't exist, your cpu and gpu will downclock on idle. Gpus also have what we call 2d clocks, like an in between step in speeds for more efficiency. You can see the gpu and cpu speeds changing in realtime with monitoring software. Occt can show both I think but I use cpuz and msi afterburner.