The 5870 will fit if you remove at least one, and maybe two hard drives.
The 5850, although shorter, has the connectors on the end rather then the side. So the shorter length is going to be largely negated by the connectors plus the power wires, depending on how stiff your wires are.
Do you still have to remove a hard drive to fit the 5850 in?