Are you sure the side panel isn't removeable? It should be - looking at pictures of it, all you should have to do is remove the two screws in back, grab the handhold in the middle of the back edge of the panel, and pull it straight back until it pops free.
I think your only other alternative, if there's enough room between the side of the bay and the panel, is to use an angle-head screwdriver and remove the screws securing the HDD to the rack. Obviously you'll have to do this by feel unless you have a small mirror you can use to see back there.