What motherboard are you using?
I've got a Q9400 (overclocked to 3.8GHz) on an Asus P5Q-E and every temperature sensor shows two cores at 34, and 2 at 37 almost all the time.
When running at low to medium usage, the temperatures never move from 34 and 37.
When I put some high usage through them (GTA4 for example) they'll move up to between 40-45 degrees but vary every second as I'd expect.
Putting extreme pressure on them (IntelBurnTest) they will hit late 50's, but again move around a little as normal.
As soon as the load is gone, they plumment straight back down to 34 and 37 respectively and never deviate even 1 degree. This had led me to believe that my motherboard sensor cannot register temperatures on this chip lower than 34 and 37, so simply reports this "minimum" temperature all the time the chip is lower than it.
So long as your temperature sensor shows correct when the chip gets warm, I wouldn't worry too much about it and just be happy you've hit your motherboard sensor's minimum.