Depends what you're doing really. Judging from the drivers comment, Gaming is your main concern but what do you play primarily? You've got a pretty balanced low-end build so upgrading either significantly (say GPU to 1050 Ti or better, CPU to i5 or better) will then lean on the other being the component holding back performance. I'd personally try find a new CPU first since you could try find a second hand i5 or even i7 if budget permits, to use your current Motherboard and RAM (although i'd personally try to upgrade to 8GB RAM along with this). Also keeping in mind that some games no longer outright allow you to play on Dual Core CPUs as well. Not to mention a stronger CPU will speed up your system & multitasking capabilities as well as Gaming even if at lower settings. That said, if you mostly play heavily GPU bound games where CPU matters little, the second hand GPU market is pretty low right now and you could try looking for around GTX 1050 Ti / RX 470 / RX 570 level GPUs to improve your Gaming performance and handle higher settings.
TL/DR - I'm more leaning towards recommending new CPU (Quad Core i5/i7 from your generation) + 8GB RAM upgrade, you can use your current Motherboard if budget is tight. Hope this helps!