Linksys WRT54G2 - Trying to figure out if it is dying

Righty, to get straight to the point. I have a WRT54G2 router connected to a Zyxel modem (delivered by ISP). The problem is that over the past few months issues like disconnections, high latency (in games) and overall much lower stability in our connection than what we have been used to. Connecting my computer directly to the Zyxel modem seems to have things running smoothly however, so I am suspecting (or rather quite certain) that the router is to blame here.

I would however like to know if there are things I should try out, settings I should check/change, etc, before I literally pull the plug on this one and replace it. Obviously I'd hate to replace it only to discover the issue persists or that it could easily have been resolved if I wasn't such a newbie when it comes to networks  

I have to apologize for not providing much more information, but as mentioned, I'm not -too- sturdy when it comes to routers and if I started blurting out what I've tried and numbers I've dug up, it would likely be rather useless or outright dumb stuff. It should however be mentioned that the router in question is fairly new (bought it around 1.5 years ago unless I'm mistaken). I have also recently upgraded the firmware on it, but this didn't really change anything in the situation. Aside from this it is handling both computers via cable (2 of them) and laptops via wifi (3).

If anyone have suggestions regarding where I should look for the error, or anything I can use to test the stability of the network and/or router, I'd be very grateful.

Cheers!