Serialization and Saved Game OOD

G

Guest

Guest
Archived from groups: rec.games.roguelike.development (More info?)

I've been racking my brain looking for the best way I can think of to
handle object persistence, so I offer here my latest attempt in the
hopes that others may benefit and improve on my thoughts.

In any object oriented roguelike design that I have considered, there
is a large number of objects that are interconnected by a complicated
web of references. This web is not a tree; many objects will be
referenced from several places. From a physical perspective, it seems
that it should be a tree: Each level contains a map, items, and
monsters, then items and monsters contain other items. But somehow it
never seems to work out to a strict tree, so I reject simply
iterating the tree and writing out it's contents. Instead I need
something similar to Java's serialization, a way of writing a data
structure that remembers each object that it writes and instead of
writing any object a second time it writes a reference back to the
first time it wrote that object. In that way the web of references is
preserved exactly.

Another issue is that our complex structure of objects will contain a
mixture of variable data that changes during a game and constant data
that stays the same for each game. We don't want to write the
constant data to our save file, but we still need the save file to
record which bit of constant data was in each of our objects. For
example: each of your monsters will have constant data such as it's
name, perhaps a brief text description of what it is and some numbers
to indicate its abilities. We don't want to put that into the save
file, we just want to put what type of monster it was and then look
up that information somehow when we restore the game.

Hopefully, constant data is kept in separate objects and not mixed
with variable data so that the constant data may be shared and not
duplicated during play. If that's the case, then we have two broad
groups of objects: variable and constant.

Finally, here is my design, expressed in Java interfaces:
/** This is the base class for all objects we might deal with,
constant and variable. Constant objects are free to do nothing for
write() and read().*/
interface Savable
{
/** This is were you write out all of your encapsulated data. Do
not include anything to distinguish your class from any other,
it is assumed that by the time this is read, the object to hold
this data has already been constructed by SaveFileFormat. You
may want to assist SaveFileFormat by including a name() method
for this class, according to taste.*/
public void write(SaveOutput out);
/** Here, a blank object takes the shape of a game object from the
past by reading in exactly what write() wrote. */
public void read(SaveInput in);
/** We will need equals and hashCode to implement a hash table to
find the ID code of each of our objects. */
public boolean equals(Object o);
public int hashCode();
}

/** This is the heart of the design, the registry. It is a one-to-one
mapping from our Savable objects to the ID codes that represent them
in our save file. It will probably be implemented as a hash table. We
could store the ID code in the game objects themselves, but that's
not game data and our game objects are already complicated enough. */
interface Registry
{
/** Create a new registry that has everything in this registry. We
don't actually need to copy this registry: the new registry can
merely refer to this registry for anything that it can't handle
itself. */
public Registry copy();
/** Assign an ID code to an object so that the ID code will be
written to the save file instead of calling write() on this
object. It is vital that we give the same ID to each object
when restoring it that we did when writing it, but we can count
on the objects coming in the same order. We can either use some
kind of naming system or we can use ints that are incremented
with each registration, or a mixture of both.

When we register constant objects, we will need them to get
consistent IDs for every game. If we ID them with incrementing
ints, we will have a fragile save file that will probably be
broken with the slightest change in order of number of constant
objects. */
public void register(Savable object);
/** Check if this registry has the given object so that we don't
have to write it to the save file. */
public boolean has(Savable object);
/** Write the ID code of this object to the save file. Give this
object an ID code if it doesn't have one already. */
public void write(Savable object, SaveOutput out);
/** Read an ID code from the save file and then return the object
that matches it. If the ID code has no object, throw an
exception. We could return null, but we are likely to want to
jump right out of the entire restoration process if the save
file has any errors.*/
public Savable read(SaveInput in);
}

/** This is where we keep knowledge of every class in our game. When
we add new classes the implementation of this will have to
change. Short of using introspection, code that must change with
each new savable class added is inevitable, so I put it here.
We don't have to worry about the given object being constant or
having been writen already, only data that needs writing will be
given to this object.*/
interface SaveFileFormat
{
/** Write a variable object to the save file. The write() method of
Savable will only be called here, so it can be left empty if
our file format doesn't need it to do anything. This method
will probably involve writing out some naming string or ID
number for the class of object being writen, then calling
object.write() */
public void write(Savable object, SaveOutput out);
/** Read in some object writen by write(), create an instance of it
and initialize it according to what you have read. This is the
only place the read() method of Savable will be called. */
public Savable read(SaveInput in);
}

/** This is where we isolate the actual messy IO activity of writing
the file. We will want to include a bunch of methods for writing
primitive data such as ints, floats, chars or whatever our game
might need to save itself. I include only ints here, to be
concise.

An implementation will need to hold a Registry and a
SaveFileFormat, one to write out brief IDs to keep track of
references and the other to write data. We handle constant
objects by pre-registering them and then copying the registry
when we create our SaveOutput.*/
interface SaveOutput
{
/** Actually put something to a file! */
public void writeInt(int value);
/** To keep our SaveFileFormat classes simple, we may be able to
use more than one. With this method, we can switch between
them. */
public void setFormat(SaveFileFormat f);
/** First we check our registry, if the given object is there, we
write a code then use the registry to write the ID of the
object. Otherwise, we write a different code and then use our
SaveFileFormat to write this object, then register this
object.*/
public void write(Savable object);
public void close();
}

/** Parallel to SaveOutput, we will need a copy of the constant
object registry and the same SaveFileFormats that were used for
writing. */
interface SaveInput
{
public int readInt();
public void setFormat(SaveFileFormat f);
/** Here we use the same codes from ArchiveOutput to know if we are
reading an ID or some data, then we use either our Registry or
our SaveFileFormat to read in the object. Each time we use
SaveFileFormat, we register the new object. Since we are
registering in the same order as we did when writing, each
restored object will get the same ID that it had. */
public Savable read();
public void close();
}

The idea behind this design is to take all the effort out of writing
a saved game from your game classes and moving it all into these four
specialized classes. Unfortunately, I have cheated quite blatantly
with SaveFileFormat, where most of the effort goes and for which I
have provided little implementation suggestion, but fortunately it at
least doesn't have to worry that it's pointers might not be handled
correctly or that constant objects might be writen to the file.

You can keep the information about which objects are constant and
which are variable only in the code that generates the constant
objects. Even constant objects do not need to know that about
themselves.

I can't think of how this might be applied to a non-object oriented
design.

I have never actually implemented this design in a roguelike, so try
it at your own risk.
 
G

Guest

Guest
Archived from groups: rec.games.roguelike.development (More info?)

Brendan Guild wrote:
> I've been racking my brain looking for the best way I can think of to
> handle object persistence, so I offer here my latest attempt in the
> hopes that others may benefit and improve on my thoughts.

If you are using Java, why aren't you simply dumping your data
structures with serialization?

[...]
> file, we just want to put what type of monster it was and then look
> up that information somehow when we restore the game.

Beware of version mismatches between the saved game and edited game
data. Moreover, automatically pulling the whole game state (including
definitions) from the savefile is much simpler than having object
recreation depend on preloaded definitions.

> Hopefully, constant data is kept in separate objects and not mixed
> with variable data so that the constant data may be shared and not
> duplicated during play. If that's the case, then we have two broad
> groups of objects: variable and constant.

There is no need to distinguish them explicitly: references from
mutable to constant objects should be shared (many orcs, only one Orc
definition) so that nothing special needs to be done to avoid
duplication.

[...]

> I have never actually implemented this design in a roguelike, so try
> it at your own risk.

Try using standard Java serialization. If you think you can't, either
you didn't try hard enough or your design has serious flaws.
Your suggested approach doesn't seem bad, but I don't see how making a
complex DIY serialization system when two good ones (old style and XML)
come with the Java standard library can be a good use of your time.

Lorenzo Gatti
 
G

Guest

Guest
Archived from groups: rec.games.roguelike.development (More info?)

gatti@dsdata.it wrote in news:1127117645.646831.282700
@g44g2000cwa.googlegroups.com:
> Brendan Guild wrote:
>> I've been racking my brain looking for the best way I can think of
>> to handle object persistence, so I offer here my latest attempt in
>> the hopes that others may benefit and improve on my thoughts.
>
> If you are using Java, why aren't you simply dumping your data
> structures with serialization?

Java serialization is good, but it is annoyingly wasteful, storing
the name of every member and every type that it writes to a file. It
is extremely sensitive to version mismatches: I can't even change the
name of a private member variable without ruining save files.

Most importantly, I'm not in control. If I use the built-in
serialization, it's writing out the data in a way that I would never
choose to write it. Why should I accept that when I have thought of
what I expect is a simple, elegant technique for doing it myself?
(Assuming that it actually works.)

>> file, we just want to put what type of monster it was and then
>> look up that information somehow when we restore the game.
>
> Beware of version mismatches between the saved game and edited game
> data. Moreover, automatically pulling the whole game state
> (including definitions) from the savefile is much simpler than
> having object recreation depend on preloaded definitions.

It is simpler, but I have to think that it may not be much simpler.
With this design that I have given, recreating objects with preloaded
definitions is as close to automatic as I can currently imagine. And
if I include definitions in the savefile, I could end up with almost
half the savefile being devoted to them, especially if those
definitions include descriptive text and behaviour scripts. I'm aware
that savefile size isn't an important optimization for a roguelike,
but I feel silly being so wasteful.

It seems to me to be desirable that definitions should be updated
with the game, even when loading an old savefile. Presumably the new
definition is a correction or an improvement and there is no reason
not to continue playing an old game with the better data.

However, you are right that version mismatches are the most serious
problem with my current design. If the number or order of the
definition objects changes in any way, bugs will be introduced,
unless the objects are identified by something other than the order
in which they are created. It's more complicated and less efficient,
but for version safety, I suppose they have to be identified by some
kind of naming system.

>> Hopefully, constant data is kept in separate objects and not mixed
>> with variable data so that the constant data may be shared and not
>> duplicated during play. If that's the case, then we have two broad
>> groups of objects: variable and constant.
>
> There is no need to distinguish them explicitly: references from
> mutable to constant objects should be shared (many orcs, only one
> Orc definition) so that nothing special needs to be done to avoid
> duplication.

Nothing needs to be done to distinguish them if you are going to
store them both in the savefile, but my system is nearly as good: it
requires only next to nothing to distinguish them and then stores
only data relevant to the current game.

> Try using standard Java serialization. If you think you can't,
> either you didn't try hard enough or your design has serious flaws.

What sort of flaws do you imagine here? I would expect that standard
serialization would always work. You could probably serialize every
single object used in your entire application if you need to. Is
there some design flaw that would absolutely prevent standard
serialization from working? It would probably involve threads or
other unserializable objects.

> Your suggested approach doesn't seem bad, but I don't see how
> making a complex DIY serialization system when two good ones (old
> style and XML) come with the Java standard library can be a good
> use of your time.

I've never heard of Java XML serialization, but it sounds very
interesting! I will look into it. Thanks for the tip! :)
 
G

Guest

Guest
Archived from groups: rec.games.roguelike.development (More info?)

Hi Brendan,

"Brendan Guild" <dont@spam.me> wrote in message
news:Xns96D63168D4A13bguild31425@64.59.144.76...
> gatti@dsdata.it wrote in news:1127117645.646831.282700
> @g44g2000cwa.googlegroups.com:
>
> Java serialization is good, but it is annoyingly wasteful, storing
> the name of every member and every type that it writes to a file. It
> is extremely sensitive to version mismatches: I can't even change the
> name of a private member variable without ruining save files.
>


Things may be less sensitive than you think. Read up on the serialVersionUID
member; with some care you can even add and remove data members across
versions of a serializable class.

The following thread proved helpful to me:
http://tinyurl.com/bq33b

My save games are about 300-500KB apiece, and most of this is to save off
the current area you inhabit, not "character" data. Not compact, maybe, but
the amount of work it has saved me is immense.

Cheers,
Nathan
 
G

Guest

Guest
Archived from groups: rec.games.roguelike.development (More info?)

"Nathan Jerpe" <nathanunderscorejerpe@msn.com> wrote in
news:432ec996$0$249$14726298@news.sunsite.dk:

> "Brendan Guild" <dont@spam.me> wrote in message
> news:Xns96D63168D4A13bguild31425@64.59.144.76...
>> gatti@dsdata.it wrote in news:1127117645.646831.282700
>> @g44g2000cwa.googlegroups.com:
>>
>> Java serialization is good, but it is annoyingly wasteful, storing
>> the name of every member and every type that it writes to a file.
>> It is extremely sensitive to version mismatches: I can't even
>> change the name of a private member variable without ruining save
>> files.
>
> Things may be less sensitive than you think. Read up on the
> serialVersionUID member; with some care you can even add and remove
> data members across versions of a serializable class.
>
> The following thread proved helpful to me:
> http://tinyurl.com/bq33b

That explains how to keep Java from immediately rejecting your
savefile for the slightest trivial change to a class, which is nice.
It works well enough so long as we keep the same fields with the same
names.

But it bothers me that it breaks through my encapsulation in this
way. If I wanted to change the implementation of a class radically, I
couldn't just do it, I would have to work around the serialization. I
would have to keep all the old fields from the old implementation so
that I could still read in old savefiles, then convert those old
fields to the new fields. Of course, I need to do that conversion for
old savefiles with or without standard serialization, there is no way
around that, but at least I don't have to be stuck with a bunch of
essentially useless and misplaced fields.

Worse, those deprecated fields I kept around for backwards
compatibility will be written out with the new ones in the new
savefiles!

> My save games are about 300-500KB apiece, and most of this is to
> save off the current area you inhabit, not "character" data. Not
> compact, maybe, but the amount of work it has saved me is immense.

Hopefully with a system like the one that I have outlined, the amount
of work will be only trivially greater. The key is to make
marshalling your data as easy and intuitive as possible once, then
use it with a smile on your face for all the rest of your massive
rogueline project, without sacrificing control in the way that
standard java serialization does. Theoretically, no matter how badly
your design changes, you will always be able to salvage something
from the old savefiles because you wrote them.

It's pretty easy to find things to complain about in Java standard
serialization, but I think maybe that I shouldn't. It's a great tool
for doing quick and ugly savefiles, which is a good thing. Saving has
never been the point of a roguelike anyway, so people who aren't
interested in it shouldn't throw away the built-in serialization and
grind through doing it themselves. I'd much rather see a working game
come all the sooner and with less effort. But for those of us who
like this kind of thing, I really believe that it is possible to do
better with very little effort.
 
G

Guest

Guest
Archived from groups: rec.games.roguelike.development (More info?)

Brendan Guild wrote:
> It's pretty easy to find things to complain about in Java standard
> serialization, but I think maybe that I shouldn't. It's a great tool
> for doing quick and ugly savefiles, which is a good thing.

It's also a great tool to do proper savefiles, with compatibility
across versions even in the presence of major changes to the internal
structure. You should probably read some tutorials on Java
serialization. Java's built-in serialitation format covers the most
general case, but can be easily adapted to your needs. It is entirely
possible to supply custom read and write functions to be used with
java.io.Serializable. The great thing is that this is not an
all-or-nothing solution. You only need to supply custom functions where
"waste" is really an issue, or where you anticipate future change. For
all other cases, the default implementation works fine.

A good starting point is the official Java tutorial:
http://java.sun.com/docs/books/tutorial/essential/io/providing.html

For a more comprehensive overview of these and other "Java tricks", I
strongly recommend the book "Effective Java" by Joshua Bloch.

--
Flo Häglsperger
 
G

Guest

Guest
Archived from groups: rec.games.roguelike.development (More info?)

In article <1127228547.272741.84320@f14g2000cwb.googlegroups.com>,
Florian Häglsperger <haeglsperger@gmail.com> wrote:
...
>It's also a great tool to do proper savefiles, with compatibility
>across versions even in the presence of major changes to the internal
>structure. You should probably read some tutorials on Java
>serialization. Java's built-in serialitation format covers the most
>general case, but can be easily adapted to your needs. It is entirely
>possible to supply custom read and write functions to be used with
>java.io.Serializable. The great thing is that this is not an
>all-or-nothing solution. You only need to supply custom functions where
>"waste" is really an issue, or where you anticipate future change. For
>all other cases, the default implementation works fine.

That's pretty slick. Last week I was reading a paper on the early history
of Smalltalk VM's. They used a form of object serialization to persist the
heap to disk. As they changed internal representations of integers,
compoiled code, etc., they overrode parts of the serialization mechanism
to do the appropriate conversions at serialization time.

http://www.iam.unibe.ch/~ducasse/FreeBooks/BitsOfHistory/

-Mike
--
http://www.mschaef.com
 
G

Guest

Guest
Archived from groups: rec.games.roguelike.development (More info?)

Brendan Guild wrote:

> I've been racking my brain looking for the best way I can think of to
> handle object persistence, so I offer here my latest attempt in the
> hopes that others may benefit and improve on my thoughts.
>
> In any object oriented roguelike design that I have considered, there
> is a large number of objects that are interconnected by a complicated
> web of references. This web is not a tree; many objects will be
> referenced from several places. From a physical perspective, it seems
> that it should be a tree: Each level contains a map, items, and
> monsters, then items and monsters contain other items. But somehow it
> never seems to work out to a strict tree, so I reject simply
> iterating the tree and writing out it's contents. Instead I need
> something similar to Java's serialization, a way of writing a data
> structure that remembers each object that it writes and instead of
> writing any object a second time it writes a reference back to the
> first time it wrote that object. In that way the web of references is
> preserved exactly.
[snip]

You think you have it bad? I am implementing monster AI with
co-routines (I am writing almost entirely in Scheme). I have *no* idea
how I am going to serialise that! (But it does make continuous
"tickless" time much easier to reconcile with player interaction. I
just do the interaction in the player object's AI co-routine.)

BTW, I am going to be using object IDs for every object, though
references will be just references, so that should help with
serialisation (write out IDs rather than the object itself), and GCed
objects will take their IDs with them.

--
Simon Richard Clarkstone: s.r.cl?rkst?n?@durham.ac.uk/s?m?n.cl?rkst?n?@
hotmail.com ### "I have a spelling chequer / it came with my PC /
it plainly marks for my revue / Mistake's I cannot sea" ...
by: John Brophy (at: http://www.cfwf.ca/farmj/fjjun96/)
 
G

Guest

Guest
Archived from groups: rec.games.roguelike.development (More info?)

In article <dgpm0e$irp$1@heffalump.dur.ac.uk>,
Simon Richard Clarkstone <s.r.clarkstone@durham.ac.uk> wrote:
...
>You think you have it bad? I am implementing monster AI with
>co-routines (I am writing almost entirely in Scheme).

Do you mind if I ask the obvious (to me, at least) questions: (?)

- Which Scheme?
- Are you building your own co-routines atop call/cc, or using another
library?
- What part is _not_ in Scheme?
- What libraries are you using for keyboard and display
- Is the code public?

>I have *no* idea how I am going to serialise that!

Don't some Scheme's have serializable continuations? I thought I had read
something similar in the context of using continuations to implement
webpages.

>BTW, I am going to be using object IDs for every object,

Are you using Fixnums? Symbols? Something else?

-Mike

--
http://www.mschaef.com
 
G

Guest

Guest
Archived from groups: rec.games.roguelike.development (More info?)

Florian Häglsperger wrote in news:1127228547.272741.84320
@f14g2000cwb.googlegroups.com:

> Brendan Guild wrote:
>> It's pretty easy to find things to complain about in Java standard
>> serialization, but I think maybe that I shouldn't. It's a great
>> tool for doing quick and ugly savefiles, which is a good thing.
>
> It's also a great tool to do proper savefiles, with compatibility
> across versions even in the presence of major changes to the
> internal structure. You should probably read some tutorials on Java
> serialization. Java's built-in serialitation format covers the most
> general case, but can be easily adapted to your needs. It is
> entirely possible to supply custom read and write functions to be
> used with java.io.Serializable. The great thing is that this is not
> an all-or-nothing solution. You only need to supply custom
> functions where "waste" is really an issue, or where you anticipate
> future change. For all other cases, the default implementation
> works fine.

You mean using the Externalizable interface, I'm sure. I'm certainly
in favor of that! If I were to use Java standard serialization, there
would be a lot of readExternal and writeExternal. In fact, I would do
it for any moderately complex class to be sure that I could change it
later if I needed to. And for simple classes, I don't really care
what I do: the readExternal and writeExternal would be trivial, so I
might as well use them there too, just to be safe.

If I'm already writing the Externalizable methods for every class,
why not use my own system where I can arrange it so that constant
objects aren't also written? I can't think of a reason and maybe I
could to port it to C++ or similar languages without introspection.

> For a more comprehensive overview of these and other "Java tricks",
> I strongly recommend the book "Effective Java" by Joshua Bloch.

Thanks; I will take a look at it!
 
G

Guest

Guest
Archived from groups: rec.games.roguelike.development (More info?)

Simon Richard Clarkstone wrote:

> Brendan Guild wrote:
>
>> I've been racking my brain looking for the best way I can think of to
>> handle object persistence, so I offer here my latest attempt in the
>> hopes that others may benefit and improve on my thoughts.
>>
>> In any object oriented roguelike design that I have considered, there
>> is a large number of objects that are interconnected by a complicated
>> web of references. This web is not a tree; many objects will be
>> referenced from several places. From a physical perspective, it seems
>> that it should be a tree: Each level contains a map, items, and
>> monsters, then items and monsters contain other items. But somehow it
>> never seems to work out to a strict tree, so I reject simply
>> iterating the tree and writing out it's contents. Instead I need
>> something similar to Java's serialization, a way of writing a data
>> structure that remembers each object that it writes and instead of
>> writing any object a second time it writes a reference back to the
>> first time it wrote that object. In that way the web of references is
>> preserved exactly.
> [snip]
>
> You think you have it bad? I am implementing monster AI with
> co-routines (I am writing almost entirely in Scheme). I have *no* idea
> how I am going to serialise that! (But it does make continuous
> "tickless" time much easier to reconcile with player interaction. I
> just do the interaction in the player object's AI co-routine.)
>
> BTW, I am going to be using object IDs for every object, though
> references will be just references, so that should help with
> serialisation (write out IDs rather than the object itself), and GCed
> objects will take their IDs with them.

Can't Scheme serialize the co-routines itself ? If not you should consider
using a tool that allows that or else you're in for a major headache soon.

I would advise you to do the savegame feature as soon as possible. No point
continuing on a system that can't be used in the final version :)
 
G

Guest

Guest
Archived from groups: rec.games.roguelike.development (More info?)

MSCHAEF.COM wrote:
>>You think you have it bad? I am implementing monster AI with
>>co-routines (I am writing almost entirely in Scheme).
> Do you mind if I ask the obvious (to me, at least) questions: (?)
>
> - Which Scheme?
Guile. The version of Scheme promoted for application extension by GNU.

> - Are you building your own co-routines atop call/cc, or using another
> library?
I am building them myself, using macros or similar, of course. See also
below.

> - What part is _not_ in Scheme?
> - What libraries are you using for keyboard and display
No proper display code yet, but the interface will probably have to be
in C, as it will use ncurses. I can't find an existing library that
actually compiles on my system (Debian).

> - Is the code public?
Not until it can actually be run as a whole program without requiring
tweaking of lots of values every turn. But it will be then probably.

>>I have *no* idea how I am going to serialise that!
> Don't some Scheme's have serializable continuations?
I don't know, except that Guile doesn't.

> I thought I had read
> something similar in the context of using continuations to implement
> webpages.
That would be at: http://lib.store.yahoo.com/lib/paulgraham/bbnexcerpts.txt

I may replace the co-routines in plain LISP with plain routines that
operate on state-machine objects (I am using GOOPS), which are more
easily serialised. The state machine tables will be built from a
game-specific language by macros. The whole point of LISP macros is
that you can extend LISP one way and your problem the other until they
meet in the middle.

>>BTW, I am going to be using object IDs for every object,
> Are you using Fixnums? Symbols? Something else?
Integers, assigned by a generator function returned from a generator
function factory. I may add a shorter (base-64) representation later,
but only if it matters.

--
Simon Richard Clarkstone: s.r.cl?rkst?n?@durham.ac.uk/s?m?n.cl?rkst?n?@
hotmail.com ### "I have a spelling chequer / it came with my PC /
it plainly marks for my revue / Mistake's I cannot sea" ...
by: John Brophy (at: http://www.cfwf.ca/farmj/fjjun96/)