Ultimate combines all the features from Home Premium and Business into one OS. Bitlocker encryption, full network domain support, etc.
The difference is one is 32-bit and the other 64-bit... just what it says. Right now, there isn't a lot of 64-bit software available, so any performance difference is going to be negligible. The only thing 64-bit brings to the table right now is larger address space... which means you can address 4GB+ of memory. 4GB is the limit of 32-bit operating systems.