It's a fixed connection, not WiFi. How does one knows if the IP is from the router or the service provider? BTW the internet connection is via a cable modem, and the wires from the cable modem goes into the router, and then the computer connects to the router. Does that means the IP is from the router? BTW all the other computer that's connected to the router (including the computer I'm writing this on) have no problem connecting to the internet. Strangely, the computer with this problem is the one closest to the router! The router is a Cisco Linksys E2000.
Anyway, the exclamation marks is permanent, it will not go away on its own (I wish it does!)