Windows NetHack I/O for a nascent bot

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

All right. I'm a little tired of not having interesting things to
program, so I would like to begin the looong process of writing a
NetHack bot along with some accompanying bot-strategy development
tools.

However, while I'm a decent programmer in general, my home computer
runs Windows, and I know very little about Windows programming. After
several hours I have failed to figure out how to capture Nethack's
screen output and am feeling a little silly.

I don't read enough Perl to see how the Unix pudding-farming bot does
I/O, but it seems to only handle top-line messages. From a quick look
at the NetHack Recorder's syntax, it looks like on Unix one could just
start NetHack through a bot program, collect the standard output, send
to standard input (? Or at worst simulate keypresses), and work from
there. Is this actually true?

Does anyone know if there would be an equivalent for Windows? Does
the Windows NetHack window even know what standard output means? Is
there some other way?

I'm asking here mainly in case anybody else has tried this before or
at least is familiar with the windowing part of the code (Benjamin
Schieder, Mark Johnson, maybe Dylan, I'm looking at you). I'm also
interested to know how simple the I/O problem is on Unix... if all
else fails I've considered putting Linux on my second hard drive,
although this would cause me all sorts of overhead learning problems.

Chris

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

Chris Maloof wrote:
> All right. I'm a little tired of not having interesting things to
> program, so I would like to begin the looong process of writing a
> NetHack bot along with some accompanying bot-strategy development
> tools.

A worthy quest!

> However, while I'm a decent programmer in general, my home computer
> runs Windows, and I know very little about Windows programming.
After
> several hours I have failed to figure out how to capture Nethack's
> screen output and am feeling a little silly.

Hmmm, my suggestion would be to go the route of the Angband borg (as
opposed to the Rog-o-matic route), and write your borg as an addition
to the Nethack source. It'll make things much simpler, and have the
additional benefit of being more likely to work under a variety of
circumstances, graphic systems, platforms, etc. On the other hand,
you'd have to be careful to separate things the player should know,
from that which he should not, and that which he may be able to deduce.
Best of luck on it in any event.

Angband Borg home page: http://www.phial.com/angborg/
A paper on Rog-o-matic:
http://www.cs.princeton.edu/~appel/papers/rogomatic.html

- John H.

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

Capturing console I/O in Win32:

http://homepages.tesco.net/~J.deBoynePollard/FGA/capture-console-win32.html

The Windows console version of NetHack uses Win32 functions like
WriteConsoleOutput() to display its graphics. You need to call NetHack
as a child process of your program, get the handles of NetHack's
console, take a peek at NetHack's console output while it's waiting for
input, decide what input to send in response to that output, send it to
the console, wait a while for the resulting output to finish, then
begin the cycle again.

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

Chris wrote:

> I'm going to avoid this, however, because I don't really
> like writing in C, and one of my goals for the project is to get
> really good at Python.

Ahhh, do the job properly, write your bot in Expect.

http://www.linuxjournal.com/article/3065
http://expect.nist.gov/
http://www.google.com/search?q=expect.for.windows+download

You'll be able to run it on Linux/Unix, MS-Windows, and Mac
OSx, at least, that way.

Beside, that way you are playing fair, only using what the
player has available, screen output and keyboard input.

xanthian.

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

On 29 Mar 2005 19:04:11 -0800, "John H." <JohnWH@gmail.com> wrote:
>
>> However, while I'm a decent programmer in general, my home computer
>> runs Windows, and I know very little about Windows programming.
>After
>> several hours I have failed to figure out how to capture Nethack's
>> screen output and am feeling a little silly.
>
>Hmmm, my suggestion would be to go the route of the Angband borg (as
>opposed to the Rog-o-matic route), and write your borg as an addition
>to the Nethack source. It'll make things much simpler, and have the
>additional benefit of being more likely to work under a variety of
>circumstances, graphic systems, platforms, etc. On the other hand,
>you'd have to be careful to separate things the player should know,
>from that which he should not, and that which he may be able to deduce.
> Best of luck on it in any event.
>
>Angband Borg home page: http://www.phial.com/angborg/
>A paper on Rog-o-matic:
>http://www.cs.princeton.edu/~appel/papers/rogomatic.html
>
>- John H.

Heh, one of the few occasions when working with the Nethack source
might be described as "simpler". I hadn't thought about doing it
that way, and you're right, it may never get platform-independent
otherwise. I'm going to avoid this, however, because I don't really
like writing in C, and one of my goals for the project is to get
really good at Python.

Too bad, as it does sound like the most general solution. Thanks!
Those are neat links, particularly the Rog-O-Matic paper.

Chris

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

Chris Maloof wrote:
> All right. I'm a little tired of not having interesting things to
> program,

you know you want to be careful saying things like that around here:
Quite apart from all the YANIs that ppl are likely to beg you to
implement (despite the fact that you said you don't want to get
involved in the nethack code) people are likely to point you to
nethack-companion ideas like this one, that wouldn't require you to
touch the nethack code at all:

http://groups-beta.google.com/group/rec.games.roguelike.nethack/browse_frm/thread/769632005e9aeae1/00b2e35acb1bced3?q=manager+graveyard&rnum=1#00b2e35acb1bced3

*dogscoff points frantically at the link, with a hopeful puppy-dog
expression on his face.

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

Chris Maloof <cjmaloof@gmail.com> wrote:

>All right. I'm a little tired of not having interesting things to
>program, so I would like to begin the looong process of writing a
>NetHack bot along with some accompanying bot-strategy development
>tools.
>
>However, while I'm a decent programmer in general, my home computer
>runs Windows, and I know very little about Windows programming. After
>several hours I have failed to figure out how to capture Nethack's
>screen output and am feeling a little silly.

Instead of reading and reparsing the screen output, you might want to add
functions to the interface which dump the available information in a
format more friendly to your program. win/tty and sys/win are the places
to look. If you do so, you could specify the means of communication
yourself.

Of course, if you would like to reuse the data structures of NetHack
anyway, add your bot into NetHack itself.

--
Philipp Lucas
phlucas@online-club.de

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

drussell@wotmania.com wrote:
> Capturing console I/O in Win32:
> http://homepages.tesco.net/~J.deBoynePollard/FGA/capture-console-win32.html
This page is a good read. However, it doesn't mention the existence of
little-know obscure Windows "Microsoft Active Availability" API that
will allow you to set up a hook that will let you know when the screen
changes on Windows versions that include MSAA. Such versions are
certain regional editions of Win98, Win2k and all versions of WinXP.
However, even with this API you don't get atomicity and the screen is
often mangled when you read it. When the screen gets mangled this way,
you'll receive another callback very soon, but this alleviates the
problem only for humans, not for bots. You can't really tell parts that
are being drawn from actual valid display.

If you really want this, you can rip relevant parts from my
ttyrec-for-Windows, http://angband.pl/termrec.html

> The Windows console version of NetHack uses Win32 functions like
> WriteConsoleOutput() to display its graphics.
In other words, forget about Expect

> You need to call NetHack as a child process of your program, get the
> handles of NetHack's console, take a peek at NetHack's console output
> while it's waiting for input, decide what input to send in response
> to that output, send it to the console, wait a while for the
> resulting output to finish, then begin the cycle again.
Because of the lack of atomicity, you don't know when the console's
contents are valid. I would say that this is way too complex to be
worth even a longer thought.
It looks like you should follow the advice from other posters, and do it
the way of Angband Borg (that is, "cheat" by looking at the game's
internal structures directly). It would also make your bot portable.

1KB

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

> [ on writing nethack bots ]

Hi.

Even though I did the whole farming bot in Perl (tried bash first, couldn't get
communication flying), I also think that doing what you want to do directly in
the nethack source is the way to go.

First things first:

The Farming Bot (FB) actually captures all output from nethack. From the source:
open2(*NETHACK, *NETCAT, "telnet -8 $server $port");

This way you can write to nethack using the NETHACK filedescriptor and read from
nethack using the NETCAT filedescriptor. It's called NETCAT because I used to
use netcat for this.
This means that the FB gets all output usually directed at the terminal. This
includes the escape sequences for cursor positioning and colors. I daresay that
this is nearly unparsable (with windows for inventory and stuff on the ground
popping up randomly).
And that's why you want to use the nethack source directly.

A nice idea here would be to write a special interface for bot interaction.
These could be:
- a "bot server" that you can connect to via IP during a running game
- a window port that just outputs easily parsable stuff (what's the current
buzzword? still XML?)
- a bot directly implemented in the nethack source

My personal favorite is the "bot server" with the direct implementation being
least favorite.
(Man, would this have made the FB easy )

Please let me know if you need a subversion repository or something. I'm
currently in the process of finally setting up my own, personal, fixed IP,
10MBit connection, always on server and now need an excuse to actually run it



Greetings,
Benjamin

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

Benjamin Schieder <blindcoder@scavenger.homeip.net> wrote:
> > [ on writing nethack bots ]
>
> Hi.
>
> Even though I did the whole farming bot in Perl (tried bash first, couldn't get
> communication flying), I also think that doing what you want to do directly in
> the nethack source is the way to go.
<snip>
> This means that the FB gets all output usually directed at the terminal. This
> includes the escape sequences for cursor positioning and colors. I daresay that
> this is nearly unparsable (with windows for inventory and stuff on the ground
> popping up randomly).
> And that's why you want to use the nethack source directly.

It's hardly unparsable.
The windows always appear in consistent positions.
xterm (or whatever) terminal sequences are hardly secret.
Took me about 3 mins to write a tool to pull the screen position and level
that an object was last seen on from a ttyrec.

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

> It's hardly unparsable.
> The windows always appear in consistent positions.
They hardly do. depending on amount, max name length and settings they appear on
different places on the screen. Sure it's doable, but I don't think it's worth
the effort if the alternative would be to create a _real_ interface.

> xterm (or whatever) terminal sequences are hardly secret.
> Took me about 3 mins to write a tool to pull the screen position and level
> that an object was last seen on from a ttyrec.
Care to share?


Greetings,
Benjamin

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

Benjamin Schieder <blindcoder@scavenger.homeip.net> wrote:
>> It's hardly unparsable.
>> The windows always appear in consistent positions.
> They hardly do. depending on amount, max name length and settings they appear on
> different places on the screen. Sure it's doable, but I don't think it's worth
> the effort if the alternative would be to create a _real_ interface.
>
>> xterm (or whatever) terminal sequences are hardly secret.
>> Took me about 3 mins to write a tool to pull the screen position and level
>> that an object was last seen on from a ttyrec.
> Care to share?

Alas, it got lost in a power cut, as I hadn't saved it, but had it on the command line :/
I'll rewrite soon, and post.

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

On 29 Mar 2005 19:28:35 -0800, drussell@wotmania.com wrote:

>Capturing console I/O in Win32:
>
>http://homepages.tesco.net/~J.deBoynePollard/FGA/capture-console-win32.html
>
>The Windows console version of NetHack uses Win32 functions like
>WriteConsoleOutput() to display its graphics. You need to call NetHack
>as a child process of your program, get the handles of NetHack's
>console, take a peek at NetHack's console output while it's waiting for
>input, decide what input to send in response to that output, send it to
>the console, wait a while for the resulting output to finish, then
>begin the cycle again.

Thank you! That's extremely clear, and I think it's exactly what I
needed to know. The next question is how to access an arbitrary
Windows function in Python, and I *think* the relevant python mailing
list has just answered that, although there's enough pain involved
that it's not quite getting done tonight.

Very much appreciated. I'll post again if I get it to work in the
next few days.

Chris

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

On 29 Mar 2005 21:23:52 -0800, "Kent Paul Dolan" <xanthian@well.com>
wrote:

>Chris wrote:
>
>> I'm going to avoid this, however, because I don't really
>> like writing in C, and one of my goals for the project is to get
>> really good at Python.
>
>Ahhh, do the job properly, write your bot in Expect.
>
>http://www.linuxjournal.com/article/3065
>http://expect.nist.gov/
>http://www.google.com/search?q=expect.for.windows+download
>
>You'll be able to run it on Linux/Unix, MS-Windows, and Mac
>OSx, at least, that way.
>
>Beside, that way you are playing fair, only using what the
>player has available, screen output and keyboard input.
>
>xanthian.

I actually came across this while looking for methods. It sounds like
it could save me a good bit of trouble, with simple ways to deal with
I/O, switch control back and forth between script and user, etc.
There are even Python implementations for the best of both worlds,
although the only one that seems to be both complete and free is on
Unix. But Adam says below that it won't work for NetHack's type of
output, so it's probably out. Oh well, I have a feeling its biggest
advantage is as a time-saver for short scripting projects anyway.

Chris

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

On Wed, 30 Mar 2005 21:36:17 +0200, Benjamin Schieder
<blindcoder@scavenger.homeip.net> wrote:

>
>The Farming Bot (FB) actually captures all output from nethack. From the source:
>open2(*NETHACK, *NETCAT, "telnet -8 $server $port");
>
>This way you can write to nethack using the NETHACK filedescriptor and read from
>nethack using the NETCAT filedescriptor. It's called NETCAT because I used to
>use netcat for this.
>This means that the FB gets all output usually directed at the terminal. This
>includes the escape sequences for cursor positioning and colors. I daresay that
>this is nearly unparsable (with windows for inventory and stuff on the ground
>popping up randomly).
>And that's why you want to use the nethack source directly.
>

Huh. I didn't even know the game could use multiple windows at all.
My NetHack only has the one console window, and the function I think I
have to use to read from it will just get the characters and their
attributes in a specified rectangle of the console, so that part ought
to be OK if I understand you correctly.

Chris

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

On 2005-03-30, Chris Maloof <cjmaloof@gmail.com> wrote:

> All right. I'm a little tired of not having interesting things to
> program, so I would like to begin the looong process of writing a
> NetHack bot along with some accompanying bot-strategy development
> tools.

Great. Can't really help you out here, but I just wanted to go against the
grain and encourage you to write a bot that isn't part of the Nethack source.
Making it part of the Nethack source would be easier, but it somehow feels
like cheating as well. If you can write a bot to play Nethack, surely you can
write one that parses screen output. In fact, I did this a while ago. Turns
out that with a Valkyrie, and an charge-all-monsters, explore-each-level,
go-down-when-explored algorithm it can get down to the oracle level fairly
often. I must admit it's kind of a pain to figure out when Nethack is waiting
for user input. I wrote this in Python interfacing with os.fakepty(), but the
docs tell me that doesn't work in Windows so I won't bother you with its code.

Tim
--
Tim Newsome nuisance.at.casualhacker.net http://www.casualhacker.net/

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

On Thu, 31 Mar 2005 02:26:08 GMT, drz@casualhacker.net wrote:

>On 2005-03-30, Chris Maloof <cjmaloof@gmail.com> wrote:
>
>> All right. I'm a little tired of not having interesting things to
>> program, so I would like to begin the looong process of writing a
>> NetHack bot along with some accompanying bot-strategy development
>> tools.
>
>Great. Can't really help you out here, but I just wanted to go against the
>grain and encourage you to write a bot that isn't part of the Nethack source.
>Making it part of the Nethack source would be easier, but it somehow feels
>like cheating as well. If you can write a bot to play Nethack, surely you can
>write one that parses screen output.
>
>Tim

Thanks! I agree, and I'm not going to modify the NetHack source for
this, although it's an excellent suggestion and would certainly solve
some problems. One reason is that, as you say, it feels a little like
cheating to access program information; I'd rather be forced to use
only the information that a human player receives. Parsing is a
nontrivial part of the challenge of it all. The more important reason
for me is that I really don't like reading and writing C code, and if
I don't enjoy doing the programming then I'll never work on it at all
(and what would be the point anyway?). So basically using C is not
really an option, so no patchbot for me.

I think if my latest attempt to get the output through Python fails,
which I think it shouldn't *crosses fingers*, I may have to give up on
the Windows version. Not that I'm going to have much time for the
next two months anyway. We'll see if I manage to put my money where
my mouth is.

>In fact, I did this a while ago. Turns
>out that with a Valkyrie, and an charge-all-monsters, explore-each-level,
>go-down-when-explored algorithm it can get down to the oracle level fairly
>often. I must admit it's kind of a pain to figure out when Nethack is waiting
>for user input. I wrote this in Python interfacing with os.fakepty(), but the
>docs tell me that doesn't work in Windows so I won't bother you with its code.

Wow, I'm impressed! I didn't think anyone had gotten so far as this
-- between the input challenges and just generally parsing level maps
and so forth into usable forms, it sounds like a hefty piece of
work...

Chris

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

Chris Maloof wrote:

> Huh. I didn't even know the game could use multiple windows at all.
> My NetHack only has the one console window, and the function I think I
> have to use to read from it will just get the characters and their
> attributes in a specified rectangle of the console, so that part ought
> to be OK if I understand you correctly.

I think such a rectangle is called a window in the sources. You might
call it a "virtual window". Perhaps in other OS'es, these appear as
popups.

--
Boudewijn Waijers (kroisos at home.nl).

The garden of happiness is surrounded by a wall so low only children
can look over it. - "the Orphanage of Hits", former Dutch radio show.

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

Quoting <drz@casualhacker.net>:
>like cheating as well. If you can write a bot to play Nethack, surely you can
>write one that parses screen output. In fact, I did this a while ago. Turns
>out that with a Valkyrie, and an charge-all-monsters, explore-each-level,
>go-down-when-explored algorithm it can get down to the oracle level fairly
>often.

Indeed - a human playing Valks with this approach will also get down to
the Oracle fairly often.

I encourage you to release this bot. It would be a considerable help to
the OP (whose endeavour I think is ultimately futile, but...)
--
David Damerell <damerell@chiark.greenend.org.uk> flcl?
Today is First Aponoia, April.

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

On 2005-03-31, Chris Maloof <cjmaloof@gmail.com> wrote:

> I think if my latest attempt to get the output through Python fails,
> which I think it shouldn't *crosses fingers*, I may have to give up on
> the Windows version. Not that I'm going to have much time for the
> next two months anyway. We'll see if I manage to put my money where
> my mouth is.

If it comes to that, you could always write a small program in C that
redirects output/input in a way that is easily accessible from python. My
current Nethack bot (which isn't nearly as smart as the python one yet) in
ocaml uses a small python script for that, since I couldn't find any ocaml
libraries to fake a pty.

> Wow, I'm impressed! I didn't think anyone had gotten so far as this
> -- between the input challenges and just generally parsing level maps
> and so forth into usable forms, it sounds like a hefty piece of
> work...

Not as much as you'd think. A lot of tweaking and testing, though. I'll see
about putting the source up somewhere, as per David's suggestion.

Tim
--
Tim Newsome nuisance.at.casualhacker.net http://www.casualhacker.net/

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

On 2005-03-31, David Damerell <damerell@chiark.greenend.org.uk> wrote:

>>like cheating as well. If you can write a bot to play Nethack, surely you can
>>write one that parses screen output. In fact, I did this a while ago. Turns
>>out that with a Valkyrie, and an charge-all-monsters, explore-each-level,
>>go-down-when-explored algorithm it can get down to the oracle level fairly
>>often.

> Indeed - a human playing Valks with this approach will also get down to
> the Oracle fairly often.

The bot is certainly no smarter than a human, although I must say that it took
me quite a few games before I got to see the Oracle when I started playing.
Oh, I missed that the bot also prays-when-hungry.

> I encourage you to release this bot. It would be a considerable help to
> the OP (whose endeavour I think is ultimately futile, but...)

I'll see about putting this up on my webpage some time this week. Remember to
bug me about it if it doesn't happen. The main thing is I want to make sure it
at least still runs on my machine. There'll also be some ttyrecord files for
the curious.

Tim
--
Tim Newsome nuisance.at.casualhacker.net http://www.casualhacker.net/

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

On 2005-03-31, drz@casualhacker.net <drz@casualhacker.net> wrote:

> The bot is certainly no smarter than a human, although I must say that it took
> me quite a few games before I got to see the Oracle when I started playing.
> Oh, I missed that the bot also prays-when-hungry.

That should be prays-when-weak.

>> I encourage you to release this bot. It would be a considerable help to
>> the OP (whose endeavour I think is ultimately futile, but...)

> I'll see about putting this up on my webpage some time this week. Remember to

Here it is:
http://www.casualhacker.net/data/bottie.tar.gz

Tim
--
Tim Newsome nuisance.at.casualhacker.net http://www.casualhacker.net/

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

On Thu, 31 Mar 2005 00:04:01 -0500, Chris Maloof <cjmaloof@gmail.com>
wrote:

>
>Very much appreciated. I'll post again if I get it to work in the
>next few days.
>

Trickier than I thought, but for the record: yes, I've managed to snag
the output.

Chris

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

> Here it is:
> http://www.casualhacker.net/data/bottie.tar.gz

I've linked to it from FarmBots Wiki:
http://scavenger.homeip.net/farmbot/OtherBots

I hope you don't mind, otherwise I'm going to remove it again.


Greetings,
Benjamin

 

[Guest]

Archived from groups: rec.games.roguelike.nethack (More info?)

On 2005-04-04, Benjamin Schieder <blindcoder@scavenger.homeip.net> wrote:

>> Here it is:
>> http://www.casualhacker.net/data/bottie.tar.gz

> I've linked to it from FarmBots Wiki:
> http://scavenger.homeip.net/farmbot/OtherBots
>
> I hope you don't mind, otherwise I'm going to remove it again.

I don't mind. I'm not guaranteeing that it's going to stay at that URL (but
feel free to mirror). It was just to share this code with some knowledgable
people.

Tim
--
Tim Newsome nuisance.at.casualhacker.net http://www.casualhacker.net/