Sign in with
Sign up | Sign in
Your question

Neural nets and game play

Last response: in Video Games
Share
Anonymous
August 18, 2005 1:18:05 AM

Archived from groups: comp.games.development.programming.misc,comp.ai.games (More info?)

I'm pondering some about training neural nets to play games, and I'm
wondering if what I'm thinking of can even be done.

What I have in mind is to have neural nets play existing games, taking
the graphical output of the game as input and giving output as game
commands.

Firstly, would having a neural net trying to respond to a visual field
this way in realtime to respond with movement commands, etc. be
computationally prohibitive?

Secondly, can this be done, technically? Can the graphical output of
the game even be made available to the input neurons of the nets in a
realtime fashion? I'm guessing that a lot of the graphics data is
being rendered/generated through the 3D graphics card, so it may not be
available for random access from other processes generally.

Or am I thinking about this task entirely wrong? If I want to train
neural nets for movement in 3D environments without entirely
reinventing the wheel where game graphics are involved, what is the
best approach?

Any ideas are appreciated,
Jason

More about : neural nets game play

Anonymous
August 18, 2005 8:35:37 AM

Archived from groups: comp.games.development.programming.misc,comp.ai.games (More info?)

In article <1124338685.733044.194630@f14g2000cwb.googlegroups.com>,
rhesus.monk@gmail.com wrote:

> I'm pondering some about training neural nets to play games, and I'm
> wondering if what I'm thinking of can even be done.

Neural nets learn to recognize patterns, based on the scoring with which
you program them. If you already know the patterns you want, it's
faster to just code that up. If you're not sure, a neural net can help
you out, but only if you "reward" it with the right scoring system. If
your scoring system is wrong, it'll learn the wrong patterns.

Every pixel of a game graphic is a lot of input. It can be set up but,
if you're doing something where you can use fewer inputs, it'll train
(and process!) faster.

--
Please take off your shoes before arriving at my in-box.
I will not, no matter how "good" the deal, patronise any business which sends
unsolicited commercial e-mail or that advertises in discussion newsgroups.
August 18, 2005 10:24:47 AM

Archived from groups: comp.games.development.programming.misc,comp.ai.games (More info?)

rhesus.monk@gmail.com wrote:
> I'm pondering some about training neural nets to play games, and I'm
> wondering if what I'm thinking of can even be done.
>
> What I have in mind is to have neural nets play existing games, taking
> the graphical output of the game as input and giving output as game
> commands.
>
> Firstly, would having a neural net trying to respond to a visual field
> this way in realtime to respond with movement commands, etc. be
> computationally prohibitive?

The answer to that is pretty straightforward. How many patterns does
your NN need to recognize? The "How Stuff Works" folks estimate the
number of possible moves to be about 10^120 (that'd be the same as the
number of possible board configurations):

http://computer.howstuffworks.com/chess1.htm

10^120 sounds computationally prohibitive to me.

>
> Secondly, can this be done, technically? Can the graphical output of
> the game even be made available to the input neurons of the nets in a
> realtime fashion? I'm guessing that a lot of the graphics data is
> being rendered/generated through the 3D graphics card, so it may not be
> available for random access from other processes generally.

I'm certain that efficient recognition of a chessboard's layout is
possible. Let's say you use a simple overhead camera on a chessboard
whose pieces have been specially designed to be easily recognized by a
camera (or some other kind of sensor). I suspect you could easily rig
up a system that could recognize the positions in a second or two --
probably faster, since only one piece changes position after each move.

Can the same be said for a camera image of a staunton chess set from an
angled view? Probably, since the only piece whose appearance changes
based on its orientation is the knight. But I suspect various shapes or
styles of the pieces will make the task easier/harder.

You could encode the entire board in very few bits (4 bits per piece --
3 bits to represent the 6 different kinds of pieces and color). With 64
squares of 3 bits each, (using 1 bit per square to represent the
presence/absence of a piece) you can represent the board very
efficiently: 3 bits X 64 squares, or 192 bits. However, even an
efficient representation of the chessboard won't diminish the size of
that intractable search space...

What's more, I'd also want to somehow represent the temporal aspect of
the game -- the sequence of moves -- since the most recently moved
pieces are more likely to be integral to the opponent's developing
strategy than are the least recently moved pieces. How might you
capture this with a NN? Offhand, I don't know.

>
> Or am I thinking about this task entirely wrong? If I want to train
> neural nets for movement in 3D environments without entirely
> reinventing the wheel where game graphics are involved, what is the
> best approach?

You could always train two NNs -- one to recognize the pieces and
another to choose the next move based on the layout. But all things
considered, I doubt a NN could play chess well AND efficiently. You've
got to diminish the size of that search space somehow.

Could you combine traditional game techniques like minimax with NNs? Or
could you reduce the number of pieces that you model? Or could you
place higher priorities on certain regions of the board? Probably you'd
want to employ numerous tricks somehow to whittle down the size of that
googol of a search space.

Randy

>
> Any ideas are appreciated,
> Jason
>

--
Randy Crawford http://www.ruf.rice.edu/~rand rand AT rice DOT edu
Related resources
Anonymous
August 18, 2005 1:31:11 PM

Archived from groups: comp.games.development.programming.misc,comp.ai.games (More info?)

Actually, what I have in mind is not to have the neural nets play
complex games that require much "intelligence" as such. I'm primarily
interested in training them to respond to the visual environment. The
games they would play would be things like "chase the reward" or "run
from the predator", etc.

I'll be using genetic algorithms for creating the neural architecture,
and this will include the neurons attaching to certain pixel inputs
from the visual field. So I wouldn't expect that they would turn out
to process the entire resolution of the field. More likely I'd expect
to see a highly clustered focal point in the center and less dense
attachment toward the periphery.

The main problem I'm facing now is how to get the 3D rendered visual
field into main memory to make it available for processing by the
neural net. If I could manage to do this I could make use of the vast
game programming resources available to create my training
environments. If not I'm in for a lot of grunt work.
Anonymous
August 18, 2005 3:29:02 PM

Archived from groups: comp.games.development.programming.misc,comp.ai.games (More info?)

In article <1124338685.733044.194630@f14g2000cwb.googlegroups.com>,
rhesus.monk@gmail.com says...
> I'm pondering some about training neural nets to play games, and I'm
> wondering if what I'm thinking of can even be done.
>
> What I have in mind is to have neural nets play existing games, taking
> the graphical output of the game as input and giving output as game
> commands.
>
> Firstly, would having a neural net trying to respond to a visual field
> this way in realtime to respond with movement commands, etc. be
> computationally prohibitive?

Probably, unless the field is low-res or heavily pre-processed. A big
chunk of our brain is needed for this.

> Secondly, can this be done, technically? Can the graphical output of
> the game even be made available to the input neurons of the nets in a
> realtime fashion? I'm guessing that a lot of the graphics data is
> being rendered/generated through the 3D graphics card, so it may not be
> available for random access from other processes generally.

Good point. What you need, really, I think, is a hardware or pseudo-
hardware (e.g. programmable gate arrays) layer that drastically
simplifies the scene before presenting it to the eye. And there almost
certainly needs to be a programmable 'focus' region, i.e. a real-time
programmable position from which hi-res data is returned, corresponding
to saccadic movement of the human eye. (Most of what we think of as the
visual scene is being processed very crudely at any time.)

Of course there's nothing to stop you using lo-res, and if you want to
do this on a PC system you have the option of ignoring the graphics
card and running a software 3D rendering system, which at least means
you will have a surface in memory that the AI can access randomly.

> Or am I thinking about this task entirely wrong? If I want to train
> neural nets for movement in 3D environments without entirely
> reinventing the wheel where game graphics are involved, what is the
> best approach?

When you've achieved it (unless in a very limited fashion), you WILL
have invented the wheel to some extent. However there are vehicles
that have been built to travel automatically and you could read up on
those. A recent competition race was abandoned because no robot
finished, or even got close.

- Gerry Quinn
August 20, 2005 12:45:26 PM

Archived from groups: comp.games.development.programming.misc,comp.ai.games (More info?)

<rhesus.monk@gmail.com> wrote in message
news:1124338685.733044.194630@f14g2000cwb.googlegroups.com...
> I'm pondering some about training neural nets to play games, and I'm
> wondering if what I'm thinking of can even be done.
>
> What I have in mind is to have neural nets play existing games, taking
> the graphical output of the game as input and giving output as game
> commands.
>
> Firstly, would having a neural net trying to respond to a visual field
> this way in realtime to respond with movement commands, etc. be
> computationally prohibitive?
>
> Secondly, can this be done, technically? Can the graphical output of
> the game even be made available to the input neurons of the nets in a
> realtime fashion? I'm guessing that a lot of the graphics data is
> being rendered/generated through the 3D graphics card, so it may not be
> available for random access from other processes generally.
>
> Or am I thinking about this task entirely wrong? If I want to train
> neural nets for movement in 3D environments without entirely
> reinventing the wheel where game graphics are involved, what is the
> best approach?
>
> Any ideas are appreciated,
> Jason
>


It can be done ,and very easily.... Every object in the game can be
presented in three coordinates (xyz) so basicly you're teaching the neural
net how to interact with every object (or should I say one spot presented
with three coordinates)...
Therefore it's not necessary to take as an input the whole graphical view
,only the objects that NN is going to interact....

Hope i solved the riddle...
Anonymous
August 20, 2005 3:58:43 PM

Archived from groups: comp.games.development.programming.misc,comp.ai.games (More info?)

In article <de1r5v$csa$1@joe.rice.edu>, joe@burgershack.com says...

> You could always train two NNs -- one to recognize the pieces and
> another to choose the next move based on the layout. But all things
> considered, I doubt a NN could play chess well AND efficiently. You've
> got to diminish the size of that search space somehow.
>
> Could you combine traditional game techniques like minimax with NNs? Or
> could you reduce the number of pieces that you model? Or could you
> place higher priorities on certain regions of the board? Probably you'd
> want to employ numerous tricks somehow to whittle down the size of that
> googol of a search space.

The simple answer is that an ordinary NN can't play chess even at
beginner level - minimax is an intrinsic requirement of the game.

Where a NN might be useful is in recognising typical patterns and
configurations, e.g. 'typical King's Indian castled formation'. It
could even identify such patterns for itself. However, in chess this
is not really necessary unless you want to play better than any human,
because over the centuries chess masters have helpfully observed and
written down the more useful patterns. It could be useful for other
games, though. Maybe a future version of 'Zillions Of Games' will
incorporate systems for heuristic selection
  • that may or may not use
    NN technology.

  • By which I mean, selecting or modifying the heuristics that are
    used to evaluate positions. Second or higher order heuristics, I
    suppose you could call it.

    - Gerry Quinn
    !