Fans can draw more power on startup. I would recommend most users stick to 2 fans as well. I have personally used 4 in some rare cases(I do things I do not recommend
) without issues(like yours VERY low power fans).
If the fans are quiet enough for you at full speed. Please use a molex connector to power them to avoid all risk to your boards fan header.