If the 2120 was less than $5-10 higher than I would say yes. At $20, no I wouldn't.
The 7850 is a better GPU than the 560ti. If they are close in price get it.
Even using the more power hungry 560ti you are only around 420 watts, so your PSU is fine.
For an SSD I'd recommend the Crucial M4 or the Mushkin Chronos but for your budget I'd skip the SSD.