Well, I'm debating OCing, but I've never done it before, so I'm going to ask for some info, but first, let me post my specs.
 
DualCore Pentium D 930, 3GHZ Processor.
2GB DDR2 RAM
320GB 7,200 RPM Hard drive.
8800GT (G92) Video Card.
650watt, Single +12v Rail (52A)
 
First off, is my Power Supply strong enough to take the extra load? If so, great! The other thing I'm worried about is of course, cooling. I'm not sure what's safe my my specific parts. I searched around on google for about 30 minutes with no success. So, I decided to be lazy and post here. Anyways, read below for my pre-OC temps.
 
Mobo 30-35C, Idle and Gaming
CPU 45-47C Idle, 52-55C Gaming
GPU 33-35C Idle, 38-40C Gaming.
 
So, my question is: Are my temperatures low enough to even consider OCing? And, if so, how high can they go before I'm in a danger-zone? Thanks!

Pentium D's run much hotter and create heat much more quickly than C2D.  C2D or Penryn are much better canditates for overclocking than Pentium D.  That said, you didn't mention your MB (chipset), but with a stock Intel 775 HSF you would likely get only a mild overclock out of the 930.  Raising the Vcore voltage for a high CPU clock would definately require a high performance HSF. Your CPU is already at 55c with no overclock, so that tells you a lot about where any serious attempt at overclocking your system would go.   My 925 D will clock to 3.2-3.3 GHz with the Zalman 8000 on a 945G micro board before heat becomes a factor and my 805 D will clock to 3.4 - 3.5 GHz with proper BIOS tweaking before heat begins to dominate.  
 
Sure, your idle and load temps are only slightly higher than a well cooled CPU with good case air flow would get.  You should manage a mild overclock depending on your chipset.  Your temps will likely become extreme should you push up the Vcore voltage and run up the FSB with a stock 775 cooler trying to manage a larger overclcok.  HTH.  

Yeah, I noticed my CPU temperature is fairly high and I don't think I'm going to OC it. My GPU on the other hand, seems fairly cool, so that's really what I was debating.

Try raising the FSB a couple of digits at a time without raising the CPU voltage.  You should be fine with a mild overclock.  Watch your temps, but you shouldn't do any damage.  

 
 
Yes, you have plenty of head room to overclock your GPU.
Here is a good read on how to do it.
 
GPUs are designed to handle much more heat than CPUs so try to keep it under 90c at load.