I think your worst problem is airflow....if you would go 650 watt (YES you do need a new PSU).
Worst case scenario you would have to get a new case, which i do highly recommend so you will be able to upgrade allot better and more.
Problem is, i have never seen such a size PSU??? have you tried contacting Alienware to order a better PSU?
And seriously measure the space required for a diffrent GPU, it must breathe too, so please look out.