When I ask this, I mean it in the sense of being the same thing every time.
Example - if a gpu is faulty or overheating, is it possible for a game to always show a visual anomaly in the same spot every time you look at it, depending on the angle, even after restarting the game and even the PC? Or would artifacts be random because they are, by nature, mathematical errors?
Well artifacts can happen both in 2D and 3D mode.
Sometimes, its a problem with the VGA drivers which causes color corruption or like you mentioned,overheating may cause artifacts.
Artifacts could always be visible or can occur randomly.
I do not think they are predictable in the sense you are looking for. I've had faulty or excessively overclocked cards that always had the same type of artifacts. My 5870, before it was RMA'ed about 4 times, hit a certain temp and had snow, but could never say "there will be a dot right there". It was more like at 80C I am going to get dots somewhere at some point. I've also had cards that I overclocked and in were not 100% stable, well that I thought were, until I got a certain game in a certain environment. For example I had an old x1900xtx that was OCed to all heck. In World in Conflict I know that every time there were tall trees it would have artifacts that made the trees look like they continued up into the sky.
You can notice patterns in artifacting, but I don't think you can predict exactly. Other artifacts are just totally random.