Honestly, you should read the
Overclocking the A64 Sticky to give you an idea on how to OC your system. It sounds like you just made some really big changes all at once without testing each component separately. There's a lot of variables involved, like what's your HT running at, are you running a memory divider, etc.. Just because your memory can run at 433 MHZ, doesn't mean your CPU can.
Return everything to stock, read up on how to OC (lot of info available online), then start making your changes a little bit at a time. Test for stability, make some more small adjustments, test for stability and so on.
Be patient.