I want to buy a new gpu on newegg. Would I need to pay american taxes on the items as well as a tariff for going across the border, and THEN also canadian taxes?

Newegg is not setup to do direct shipments to CA.
If your somehow order a gpu and have it sent to a US address, like New York.
Newegg charges NY sales tax, then when you take it across into CA you will need to pay the gst/pst.
Newegg only charges sales tax where it has a physical location, like California, New Jersey, and Tennessee. New York is a special case because of the new law here.
Currently Zipzoomfly does not collect New York sales tax, and I have been buying more from them over the last month.
Often no tariffs are involved, just the normal gst/pst. That does not mean that you won't get that one customs guy that try's to look good and drops a tariff on everything he processes.
There are a couple of web sellers that ship to Canada, and they do have pretty good prices.

Check to see if they have the same card on Zipzoomfly, ive used them several times. Once they got me a power supply I ordered to my door 22 hours after I ordered it. I ordered it at 12PM here in Virginia, ZZF is in California, I used the free fedex overnight they had for that particular item, and I got the package at 10AM the next day, which was Saturday.

I noticed that tigerdirect has a .ca website. Should I buy my stuff there?

I've never used tigerdirect, but it is one place you should price shop at.

I use NCIX.com for the most part because they are so good at price matching online and in stock items.

Shopbot.ca is great for searching manufacturer #s for items you want, it'll send you to the lowest priced item which you can then use to price match at NCIX. DirectCanada.com is a partner of NCIX's and generally always has lower prices as well (but they do not pricematch).

In related news, Newegg seems to actually be coming to Canada later this year! Check it out: Newegg.ca

How shipping will work is unknown, good chance it'll still come from the USA though.

Yes

Same as Jevon.

I used TigerDirect.ca but the shipping is quite expensive.

NCIX.com have better prices overall as well as very good shipping fees. I still look at other websites but find myself back to ncix.com everytime. I've been ordering from them ever since I built my latest rig.

Yep NCIX is the way to go.

You can't ship to Canada via NewEgg, and not only that, you cannot ship to a US address with a Canadian Credit Card, you need a US credit card and a US address in order to get things from NewEgg.

For all the hassle, unless it's a wicked deal (which it sometimes is) or rare / US only (ditto) then for the most part you're better off with the really good Canadian e-tailers.

It'll be interesting to see what NewEgg.ca offers, but if it's anything like tigerdirect, then all the really amazing deals and inventory blowouts will still be south of the border only.