So it does make noise when just plugged in? Should probably replace it anyway, it should not be making that sound. Something is not right in it.
When just plugged in charging with battery, yes (I don't know about when the battery is fully charged. Have not had time to test that yet). When plugged in with no battery, no sound.
Could it be that the weak battery is making the charger act strange? Or is the charger probably just spoilt?