How does a modern 3d game REALLY work???

[NewToATI]

Hello everyone, I was just wondering does anyone know where i could go to look at the full details of just how the simplest 3d object is rendered on a computer, and to really understand what's going on when running a 3d game? Any particaularly good websites?

The purpose of this quest is to try and understand exactly what everything is doing on my computer, I mean i have Directx 9.0c and a graphics card with 256 megs of ram, and a 3.0 ghz cpu, but hey i don't have a clue as to how the 1s and 0s go from my hard drive to the 3d scenes in any of my games.

All i get from the internet is a superficial understanding. For example: From howstuffworks.com they talk about the fact that a computer has to render the polygons, then do some other stuff on top. What I want to know how the the software talks to directx and how directx talks to the windows kernel and how that tells the gpu to process this, this and this and wrap this texture around this and that texture around that.

I know what i'm asking is knowledge in the realm of a university degree, but i just want to know wtf is really going on behind the plain windows desktop that i see every freaking day.

I have a decent understanding of how a microprocessor works, ie. basics: a little assembly programing experience.

 

[infornography42]

Your best bet is to find a book on 3D programming. What you are looking for is more in depth than you are likely to easily find on the internet.

It also is going to take no small amount of reading and research.

Here is a starting point to get you going.

Good luck.

 

[guruofchem]

Yep - a really good question that would take a lot of time to answer. I have often wondered the same myself, so perhaps I will sample Infornography's list on Amazon - it looked fairly interesting, if a bit challenging in spots.

 

[infornography42]

Well that list is just an unfiltered search on the subject. I scanned through it to make sure it was the kind of stuff I was looking for but I didn't dig too deep. Its just a starting point as said.

Good luck and happy reading ;-).

 

[NewToATI]

Hi all, thanks for the replies.

Hmm, when i asked how a game works i think i gave you guys the wrong idea, the purpose of this question was more to gain a more deep understanding of the inner workings of a computer. My goal is not to learn how to create a computer game, but rather understand how it tells windows : "hey here is a new texture to process, put it through the graphics card" or "hey write this to the graphics memory and don't just page it"

Perhaps i should have asked this on a different forum, cause i think what i'm really asking is how does an operating system work and allow a game to do all the stuff it does.

I have studied basic c programming and microprocessor architecture, i mean i can understand from logic gates how a processor is built up (muxes, registers, instruction decoders etc. ) and i can understand how a "hello world" or a relatively small and simple program interacts with an os, but if i see anything that involves a more complicated interaction with the os (the pinnacle of which is a complex 3d computer game) i'm thinking : i don't have a clue what on earth my computer is doing. I sit there while some program runs on my computer.

I just have so many questions as to how and why computers do such random shit: like why do i need to defrag my hard drive? Why is it fragmented in the first place?

Most ppl who use computers just sit there in front of it and go "wow i can listen to music, play my computer games, and surf the web ... " Alright then, so everyone has one of these things called a computer, has anyone asked what is really going on inside that box? and i dont just mean : "yeah whatever the cpu does a whole bunch of memory transfer operations" I mean what is directx doing right now? or which device drivers are loaded and working right now? and what are they doing? there are 32 processes running on my comp, my question is: why and what are they doin?

 

[infornography42]

The level of understanding you want for how a game interacts with windows and DirectX requires knowledge of how to program a video game. It really is quite involved.

The basics of it is that the game is programed to use an API, in this case DirectX. That API in turn communicates with the drivers and Windows. A programmer can write optimizations for specific video cards/processors, but that really just boils down to using the feature sets within the API that are most flattering to those devices.

As for defragging. Your hard drive gets fragmented because of the way files are written to the hard drive. A file will be placed in the first available space on the drive. If there is not enough contiguous empty space there to hold the whole file then it is fragmented and the rest of it is stored in the next empty space of the drive. If there is not enough contiguous empty space for the rest of the file there, then the file is further fragmented and placed in the next empty space in the drive.

Since things are getting written to and deleted from the hard drive constantly there are lots of little bubbles of space and therefore files get fragmented easily.

Defragging a drive takes those pieces of files and consolidates them into one contiguous section of the hard drive. This cuts down on read times. When using a FAT based file system such as FAT 16 or FAT32, fragmentation causes a lot of problems for efficiency. With NTFS and EXT2/3 file systems, this is not AS MUCH of an issue, but a drive with over 80% fragmentation will still cause problems.

Does that make sense?

 

[morfinx]

I would suggest you have a read at "Fundamentals of Computer Graphics" by Peter Shirley. I think that will answer a lot of your questions. If you are in school, I would look into taking an undergraduate level computer graphics course.Some of the topics you can expect to cover are line equations, rasterization (polygon scan conversion), transformation and projection, hidden surfaces, radiometry (lighting), colors, surface shading (flat, gouraud, phong), shadow maps, ray tracing, texturing, graphics hardware pipeline, pixel/vertex shaders, bezier curves, subdivision surfaces etc.

Each topic is complicated and can take a long time to explain. So I'll give you a basic overview of the graphics pipeline:

1. The host CPU calculates and sends polygons (triangles) to the GPU (OUTPUT: vertex position (3D space), color, normal, texture coordinates)
2. The Vertex Processor (vertex shader) on the GPU processes them and sends screen space triangles to the Rasterizer (OUTPUT: position (2D screen space), color, texture coordinates)
3. The Rasterizer takes them and generates fragments (pixels) which are then fed to the Fragment Processor (OUTPUT: color, texture coordinates)
4. The Fragment Processor (pixel shader) takes the info and massages it; the color of a pixel in screen space is then sent to the frame buffer (OUTPUT: color)
5. Image is displayed from the info in the frame buffer. Two copies of the frame buffer alternate to avoid tearing artifacts (double buffering or page flipping)
Note: Note both Vertex Processors and Fragment processors have access to the texture memory which is usually onboard the GPU


Anyways, feel free to ask if you have any specific questions.

 

[llama_man]

It's all done with magic!


And goblins.

 

[NewToATI]

Sorry, I've not been staying up-to-date recently, was a bit busy, anyway, thanks for all the help guys.

And I really think you hit the nail on the head there morfinx, this is exactly what i was asking. Also, thanks infornography42, that does make sense.

Now as we all know answers only lead to more questions, so here they are:

1. Why is fragmentation not AS MUCH of an issue for NTFS and EXT2/3 ? Is it to do with the structure of the metadata?

2. So the CPU is only responsible for the geometry? ie. If I talk about only one frame then the CPU would be responsible for sending all the necessary geometry data to the GPU? ie. if i turn in an first-person-shooter (fps) game then the cpu will recalculate the angle of the camera's perspective, send the relevant information to the GPU, and the GPU is only responsible for rasterising the Geometry and textures ? If this is the case then the workload is divided as follows: CPU: physics/motion, GPU: Draw stuff real quick.

3. The vertex processor generates a "polygon image" right? ie. if i took out the pixel shader from the pipeline then i would see just polygons right? then is it the job of the pixel shader to "wrap" textures onto the polygons or does something separate do that, cause i thought that the pixel shader simply recieves information about that particular pixel (ie. color, lighting and texture info) and recalculates the RGB value via some sort of averaging algorithm?

4. How do you get effects like fog and water? How do they fall into the vertex and pixel shader jobs?

5. Is lighting done by the Vertex Shader?

6. What was this transform and lighting thing that they introduced in Geforce 3s?

7. What happens if i'm looking through wavy glass in an fps game?

8. So the agp is mostly a one way bus? then how would pci express x16 help ? how would it be better than agp 8x? since the bandwidth is half in one dirn and half in another (for pci express x16)? there should be very little data going from the gpu to the cpu since the the gpu sends stuff to the screen, and recieves stuff from the cpu.

9. How do mipmaps work?, i know they are like a freaky bitmap stored on the gpu but what advantage do they have over a conventional bitmap, ie. if i stored one large bitmap in the texture memory rather than a mipmap of it, and if i used a dedicated part of the gpu to rescale these textures according to distance, wouldn't i (a) save texture memory, and (b) have better image quality? (as opposed to simply selecting the appropriate part of the mipmap for the distance)

10. I still don't get why textures are blurry if a surface is almost parallel to the direction you are looking in a fps? ie what causes the problem that anisotropic filtering fixes?

11. This isn't that related to games but hey i'm gonna ask anyway. What does the north bridge in a computer do ? ie how does it take a whole bunch of stuff from the rest of the computer and commnicate it to the cpu? does it make the rest of the computer look like a whole bunch of memory locations, or is it more complicated than that?

12. How does an operating system like windows just interact with any piece of hardware, how can it run on any computer configuration? On what level does windows see hardware? How does it control the hardware? Eg. If i was the windows kernel (a piece of software) what does the world look like ot me?

 

[morfinx]

Sorry, I've not been staying up-to-date recently, was a bit busy, anyway, thanks for all the help guys.

And I really think you hit the nail on the head there morfinx, this is exactly what i was asking. Also, thanks infornography42, that does make sense.

lol that's the longest question thread I've ever seen. I sat there for a good minute debating whether I should answer it

1. Why is fragmentation not AS MUCH of an issue for NTFS and EXT2/3 ? Is it to do with the structure of the metadata?

I'm not familiar w/ EXT so I'll pass on this one

2. So the CPU is only responsible for the geometry? ie. If I talk about only one frame then the CPU would be responsible for sending all the necessary geometry data to the GPU? ie. if i turn in an first-person-shooter (fps) game then the cpu will recalculate the angle of the camera's perspective, send the relevant information to the GPU, and the GPU is only responsible for rasterising the Geometry and textures ? If this is the case then the workload is divided as follows: CPU: physics/motion, GPU: Draw stuff real quick.

Yes that's correct for the most part, except he GPU converts the camera from 3D space to 2D screen space in a process called "transformation and projection".

3. The vertex processor generates a "polygon image" right? ie. if i took out the pixel shader from the pipeline then i would see just polygons right? then is it the job of the pixel shader to "wrap" textures onto the polygons or does something separate do that, cause i thought that the pixel shader simply recieves information about that particular pixel (ie. color, lighting and texture info) and recalculates the RGB value via some sort of averaging algorithm?

The vertex processor produces the position of a vertex on the 2D screen space, as well as its color and texture coordinates. So if you look at the output of the vertex shader, you see a bunch of vertices that make up polygons. So yes, the pixel shader does the texture mapping. It does other manipulation to the color of the pixel as well if needed.

4. How do you get effects like fog and water? How do they fall into the vertex and pixel shader jobs?

Water effects and such are done by vertex shaders

5. Is lighting done by the Vertex Shader?

Yes. That's why normals are fed to the vertex shader, they are used for lighting calculations.

6. What was this transform and lighting thing that they introduced in Geforce 3s?

Transformation is the manipulation of polygons. Some basic transformations are translation, rotation, and scaling. Lighting is, well, lighting calculations.

7. What happens if i'm looking through wavy glass in an fps game?

lol you see wavy things behind it?

8. So the agp is mostly a one way bus? then how would pci express x16 help ? how would it be better than agp 8x? since the bandwidth is half in one dirn and half in another (for pci express x16)? there should be very little data going from the gpu to the cpu since the the gpu sends stuff to the screen, and recieves stuff from the cpu.

Well when you are doing a very complex scene, the amount of data that has to be sent to the GPU is huge. Granted, a lot of games nowadays don't saturate the bandwidth limit of AGP 8X, so they don't benefit from PCI Express much. In fact, I think THG did a review of AGP version of X1950 Pro not too long ago, and the performance was the same as the PCI Express version. But I expect newer games that are coming out to saturate the AGP bus eventually though.

9. How do mipmaps work?, i know they are like a freaky bitmap stored on the gpu but what advantage do they have over a conventional bitmap, ie. if i stored one large bitmap in the texture memory rather than a mipmap of it, and if i used a dedicated part of the gpu to rescale these textures according to distance, wouldn't i (a) save texture memory, and (b) have better image quality? (as opposed to simply selecting the appropriate part of the mipmap for the distance)

When you have multiple texture map sizes, for example 512x512 and 256x256, and you need something inbetween, you interpolate to get that. Why load the largest texture map when there is only limited video buffer? Bandwidth and memory are both precious. Also, the rendering speed is improved when you only need the smaller mipmaps, as you don't have to do the interpolation from the biggest one.

10. I still don't get why textures are blurry if a surface is almost parallel to the direction you are looking in a fps? ie what causes the problem that anisotropic filtering fixes?

Wikipedia has a great explanation of it, so I'm just gonna direct you that way http://en.wikipedia.org/wiki/Anisotropic_filtering

11. This isn't that related to games but hey i'm gonna ask anyway. What does the north bridge in a computer do ? ie how does it take a whole bunch of stuff from the rest of the computer and commnicate it to the cpu? does it make the rest of the computer look like a whole bunch of memory locations, or is it more complicated than that?

The Northbridge usually takes care of the link between the CPU and Memory/AGP/PCI Express. Just about everything in a computer is accessed by addresses.

12. How does an operating system like windows just interact with any piece of hardware, how can it run on any computer configuration? On what level does windows see hardware? How does it control the hardware? Eg. If i was the windows kernel (a piece of software) what does the world look like ot me?

Windows communicates with hardware via a HAL (Hardware Abstraction Layer). HAL hides the difference in hardware, so software doesn't see the difference at all. In Windows, the HAL is part of the kernel. So you have direct view of all of your hardware there. It's the HAL's job to translate everything into a standard "language" that higher level software can understand.

 

[celewign]

12. How does an operating system like windows just interact with any piece of hardware, how can it run on any computer configuration? On what level does windows see hardware? How does it control the hardware? Eg. If i was the windows kernel (a piece of software) what does the world look like ot me?

Windows communicates with hardware via a HAL (Hardware Abstraction Layer). HAL hides the difference in hardware, so software doesn't see the difference at all. In Windows, the HAL is part of the kernel. So you have direct view of all of your hardware there. It's the HAL's job to translate everything into a standard "language" that higher level software can understand.

I think Windows Vista made a new version called HAL9000. ( nerd joke)
-cm

 

[infornography42]

1. Why is fragmentation not AS MUCH of an issue for NTFS and EXT2/3 ? Is it to do with the structure of the metadata?

Quote from Wikipedia

The Master File Table (MFT) essentially contains metadata about every file and directory on an NTFS file system. It includes parameters such as location, size, and permissions. It is used to aid in minimizing disk fragmentation.

Unfortunately I don't know enough to thoroughly answer this question. I've never had a student who thought to ask it. Not sure if Microsoft has even revealed enough information about it to be more specific. If you find out though, let me know.

 

[NewToATI]

Hey guys,

Here's a question that's been naggin me for some time:

Why are consoles so much better at games than PCs?
A PS2 has 32 MB of RDRAM, if you got a pc with 32 MB of ram and tried to run Final Fantasy X you would not succeed.

WHY? Is windows really that bad? Or is it something else?

 

[Flakes]

was gonna explain the fragmentation issue, but it turns out i need to do some more reading...

i can tell you it has something to do with block sizes on the disk between different file systems, i believe the NTFS uses 4KB blocks and FAT uses a larger 16Kb blocks, so when a 5Kb block is written to a FAT disk 16Kb of space is used up, whereas on a NTFS disk that same file would only take up 8Kbs of disk space therefore decreasing the chances of disk fragmenting.(combine this with the explanation above about HDD)

but im here to point you to some resources you might find handy, the level of understanding you are looking for takes years to accomplish...

good simple explanations of hardware
http://computer.howstuffworks.com/hardware-channel.htm

NTFS
http://en.wikipedia.org/wiki/NTFS#Features

 

[Flakes]

Hey guys,

Here's a question that's been naggin me for some time:

Why are consoles so much better at games than PCs?
A PS2 has 32 MB of RDRAM, if you got a pc with 32 MB of ram and tried to run Final Fantasy X you would not succeed.

WHY? Is windows really that bad? Or is it something else?

are you trying to start a flame war here? please go to another topic....

 

[celewign]

You can't compare a console to a computer. A console doesn't have to work with that origional layer of software that is the computer's OS.
-cm

 

[VBDude]

are you trying to start a flame war here? please go to another topic....

What are you, an idiot? This is his topic. From his other questions, it's very clear this is not flame bait.

You can't compare a console to a computer.

You can, and I will.

Why are consoles so much better at games than PCs? Is windows really that bad? Or is it something else?

There are two parts to this answer:

First, all that extra weight of running an OS and multitasking does have a heavy expense when it comes to performance. The processor and other components have to be shared, thus reducing their effectiveness with a single app. Imagine a person with multiple responsibilities of equal importance vs. a person with one repeated task and you can get an idea. The first person has to do a lot more work to do that one task as effectively as the other person.

The second part has to do with hardware consistency.

Back in the day, if a programmer wanted to use a piece of hardware in a game, such as a sound card, they would have to write specific instructions for that piece of hardware. As more and more hardware became prevalent, it became necessary to develop a middle-man, or API (Application Program Interface), to handle communication between the software and hardware layers.

DirectX is a middle-man package for sound, video, input, and networking. Essentially what happens is hardware is made to conform to the DX specification, while software is written to utilize the API for hardware communication. The API layer is handled in OS software. This doesn't not come without some overhead expense.

Consoles, on the other hand, have no need for middle-man APIs. Though they still use them to a degree, this is more of a direct-link API than a hardware independent alternative.

Once a console is assembled, its hardware does not change. Programmers can optimize their code with low-level instructions speaking directly to the hardware with the assurance that everyone who has that console will get the same level of performance they experience, helping greatly with optimization.

These two parts are greatly beneficial for an underpowered console to do something a similarly equipped PC couldn't. However, all benefits have problems. For the console, these problems come in the form of versatility and modularity/upgradeability.

Consoles are great at running games and other simple entertainment, such as music and movies, but they are horrible for anything like content creation or data services. The PC can handle these tasks with ease.

Console don't adapt with the times with a simple hardware upgrade. That act alone would negate any benefit of consistent hardware, thus making games unplayable. A console would need a complete system replacement to upgrade. PCs are made for upgrading, often requiring a simple driver or nothing at all to use their new part.

 

[celewign]

Going a bit off topic here VBDude <respect still here>...
:?

I know the difference between the two.

WHY? Is windows really that bad? Or is it something else?

FOR GOD'S SAKE LEAVE WINDOWS ALONE! IT TRYS!
-cm

 

[Flakes]

appologies to the OP , just seemed like a random question after all the PC related questions, I didnt even take note of the name, my automatic response went off straight away, trying to protect the thread from flamming.

sorry

 

[VBDude]

No hard feelings. I just hope you know I was playing with you. I don't mean to offend...much.

I know the difference between the two.

I see your :? and I'll match it. :? [/Poker]

I never said you didn't know the difference. I was responding to NewToATI's inquiry. I didn't feel your response was in-depth enough and I was in the mood for using maximum verbosity. [/Zork]

I share your sympathies with Windows. He tries so hard, but he gets no respect. [/Rodney Dangerfield]

 

[celewign]

No harm taken.

Back to the meanies who say cruel things to poor Goll... I mean Windows.
<pets Windows software>
It's ok, baby. Everything's going to be ok. There there...
-cm

 

[NewToATI]

Ok Guys, now that everything is calm again,

I have a few more questions:

(NB: I am new to forums, i have no idea what flamming or any forum like terms mean.)

1. I found this cool tutorial on making a triangle in DX9:

http://www.ultimategameprogramming.com/zips/D3D_Intro.ZIP

I know basic c programming, so i understand some of this.

It contains some C and notes, my question is what's all this random stuff that deals with the windows? (primarily talking about the "LRESULT CALLBACK WndProc" and the "int WINAPI WinMain", this is some random stuff)

(Please read all four (Q2-5) of the next questions, otherwise the next one (Q2) will sound stupid)

2. What language are applications/games made of? ie. if i right click on a executable file and open with notepad, then i look at the bytes, are they x86 opcodes or what ???

I compiled a little "hello world" program i had written in c into assembly language and i saw what was pretty obviously x86 instructions. Now if this is the case what's with MMX and SSE and SSE2,3,4 and any new instructions that are introduced with new architectures, cause the new instructions won't even be used, unless the installer recompiles the files or there is a big chunk of extra and useless code in the executable (little bits for each diff architecture).

3. My point is that when such new instructions and new architectures are introduced how can games and any apps for that matter take advantage of them if the executables are already written in assembly?

4. (Follows from before ... ) How does Directx and device drivers fall into all of this?

5. You can understand my confusion in the following question: How can applications that are running in machine code access device drivers and Directx and any other fancy operating system stuff ?

6. From the perspective of the above I'm now gonna ask: How does a game access the graphics card? (remembering I'm pretty sure that apps like quake3.exe and FEAR.exe aren't stored on ppl's HDDs in c code, but rather as machine code)

Thanks for all the help from b4 guys, it been ... educational.