In general, yes. But you should look at the compatibility guide for the specific motherboard to see which processors it supports. You may require a firmware update, too, which can be tricky if you only have a non-supported CPU while building.
There are very few boards that are designed for the 3rd generation Intel® Core™ processors that don't support the 2nd generation Intel Core processors out of the box. On the other hand for a "6" series board to support a 3rd generation Intel Core processor the board will need a Bios update in most cases. There are a few "6" boards that will not support a 3rd generation Intel Core processor like Q67 chipset based boards.
Putting a Sandy Bridge CPU in a motherboard designed for Ivy Bridge should not be a problem at all. The only difference is that the PCI-e 3.0 slot will operate in PCI-e 2.0 mode because Sandy Bridge CPUs do not support PCI-e 3.0.
Then again, there's nothing really to worry about since it will probably take 2 years before a high end single core graphics card ($500+) will begin to be bottlenecked by PCI-e 2.0.