Yes, it's normal. There are capacitors on the board.
I have found that you can extinguish these lights by unplugging the power supply and then turning on the power switch. This drains the caps. But don't go licking your motherboard even if it's removed from the case; one of the caps might bite you.