I would use a heavy 3D app to test it, such as 3D Mark. Then overclock the memory one setting at a time between tests, until it gets artifacts (weird lines, block, patches or pixels on the screen). Once you see artifacts, exit immediately (esc key for 3D-Mark) and go down 5MHz. Then do the same for the video chip. If you do it this way you'll run into minor glitches far before you can cause any damage.
What's the frequency, Kenneth?