Why would the raid cards not work? If this is all put together just do it.. and you could easily run all of your vm's at once I have 5 running on a i7 laptop with 12 gigs or ram. I guess I dont understand exactly what info you are looking for. You have plenty power in the cpus you will need about 2 gigs of ram per VM you want to run at the same time with a min of 8 gigs.
I am wondering what the best approach to combining this set of hardware would be...
Should I poach the xeons from the x3650 and grab a new mobo for the 4u case or should I invest in SAS drives for the x3650?
I think what you're doing is more than reasonable. Personally I'd just use the IBM box. And most likely you can use cheap SATAs in that. SATA drives will physically plug into SAS ports, but not the other way around. And most semi-current systems support both SATA and SAS. Seach IBMs site to confirm.
Don't know how much RAM is in that IBM, but thently's guidance on RAM is a good one. Simply, the more RAM the better. Expanding; you can easily over-commit CPU and other resources without too much penalty. But over committing RAM doesn't work so well. Therefore, if you want 2GB per VM, you should have 2GB per VM physically in the system (+512MB/1GB for the hypervisor). Now for a contradiction; VMWare does an amazing job at letting you overcommit memory. It's not ideal, and they strongly advise against it, but of all the hypervisors, they're the best at controlling RAM overcommits.
ESX is my suggested route too. While I haven't checked the QVL, I'd be surprised if that IBM didn't work.
An alternative (less efficient) idea is to install a Win or Linux host, and install OpenBox. Windows Server or Linux would definitely have drivers for the hardware, in the event that ESX didn't. It's less efficient because the full GUI host of Win/Linux will suck up resources that you'd probably prefer be used for the VMs.
Thanks for the tip. I just ordered a drive tray and I will try an old 2.5" sata and see what happens. I bit the bullet the other day and ordered a 300gb SAS drive, about $200 on ebay, isn't so bad. Now I have to find some RAM for cheap, currently only 4gb.
Re: RAM, being that you're just playing around, vs not looking for a "mission critical" solution, you "may" be able to get generic memory for the system (e.g., non-ECC, unbuffered) which will be cheap as dirt.
But because there are several generations of the x3650, I can't tell for sure what you need. Therefore can't advise in more detail. But if it turns out the system needs Registered memory, there's not as much you can do to cut corners.
Regardless, RAM is as cheap as it has been lately, so you're in a good situation there.
Also, there's always a "try it and see" approach - try the 4GB you have now with the VMs you want to set up and see if it's livable. I am pretty sure that if you have more than 2-3VMs, you're going to want more. But try it out and see... all depends on how you configure the VMs and with what Guest OSes...