Yes-- that card is more than powerful enough to run just Windows. Integrated graphics would run windows fine too. It will get a "Windows Experience Score" of around 3.6, although its performance is actually slightly better than that. (its a biased rating system)
However, the 4350 is not a gaming-class graphic card (on any operating system) and wont provide a lot of gaming performance. If you intend to use it as a temporary measure until you can afford a good card it should work fine for some gaming on lower settings.
Yes it'll run windows fine but you won't be able to do any gaming off it. If you're looking for a gaming card for under $100 I think Tom's had the 4670 or the 9600 GT as the best 2 cards at cheap prices