With a low power, office type desktop (~100-150W etc), it can be ok to use an 'included' PSU*. Not preferred, but unlikely to cause any damage.
When you add a GPU to the equation, or any form of higher power draw components, it's definitely not recommended. Not only are you putting more strain on what is likely a questionable PSU to begin with, chances are good it lacks even basic protection & you're potentially risking higher valued components.
*Unless it's a Logisys or Diablotek - it's never ok to use those.