Start with something pretty ambitious, going up maybe 30mhz at a time. At each interval, run Afterburner's stress test, which I believe will log artifacts. If you get ANY of those, dial it back down. Go to 10mhz increments when you've determined a 30- or 40mhz interval within which you get no artifacts in 15 minutes or so of stress.
After you've got a reasonable core clock, follow the same procedure for the memory clock.