Is it a clear constant beeping tone or more of high pitched whine ?
If its a beep its the board telling you the cpu is getting hot and the bios is giving you a warning alarm, this can be setup in the bios, and the alarm can be changed to trigger when the cpu reaches a defined temp setting in the bios. Setting a higher value will stop this in the alarm section for the cpu in the bios.
But as said it could also be an old capacitor, as they often make the same type of noise as they get older with use, or a loose choke coil on the motherboard.
Maybe the system needs a clean out, and you should have a look at taking the cpu cooler and fan assembly off and cleaning it all then re apply some thermal paste to the cpu fit it all back together and see if the problem goes away as a last resort. One more thing a long beep could also mean that the extra power to the Pci-e graphics card is not connected so check it is, some boards make a long beeping sound to denote this.