Need help with Router unable to access internet

I have a DSL-504 D-link router,

just bought it recently, problem now is that i can't use it to connect 2 of my pcs online,

I've tested my 2 other pcs with just a DSL modem (without router, not the DSL-504) and it works individually

PC1: Sony laptop, winXP pro
PC2: P4 win2k

after setting up the router as the instructions said, on the webpage setting for the router it says that i've connected the device to the internet, but when i try to access the internet from my IE from either one of my comps , both can't,

I tried pinging the router from my comps and it works, but when i try to ping anything outside eg:www.abc.com or something, it doesn't work. It says can't find the site.

Communication between the pcs also work, but just not communication to the internet

anyone could help? been stuck at it for along time and it's getting really fustrating

sounds more like a DNS issue to me. Have you got your DNS server addresses set up?

Spec:
Intel P4 2.4B
MSI 645E Max-U Mobo
512MB DDR333
GF3 ti200 64MB
SB Live 5.1
WD 60GB (Storage)
Maxtor 6GB (System)
Maxtor 120GB
LG 24x24x32 CDR
WIN2K PRo SP4