Hi again,

I didn't have much time lately, so I couldn't finish the udpate
of the FF[4] spec as already discussed in this list (TM+OT,
less restrictive handling of gameinfo properties, ...)

Also we talked about adding a page for "private properties".
So that other people don't use their names, or can implement
those properties which they find useful, etc. Shigeru's proposal
could be added to that "private properties" page.

As you know, I'm rather reluctant to add new properties to the
'official' part of FF[4], as every programmer would have to add
them to their programs. Which is a lot of work, for features
which might be used seldomly. Furthermore the SGF standard would
grow more and more complex and harder to maintain.

Reading through Shigeru's mail I remembered an idea I had quite
a time ago: SGF onion skin concept. This means, that we assign
every property a "skin" number. The innermost skin comprises
the core properties which should be implemented by every SGF client.
Such properties are: B,W,AB,AW,AE,GM,SZ,C,PB,PW,BR,WR,DT,KM,RE,TR,MA.

Every other skin extends the functionality, but is less "important"
than the previous skin. Take as an example the board markup:

Skin 0 (core): TR, MA, LB
Skin 1: CR, SQ, AR, TB, TW
Skin 2: DD, LN, SL
Skin 3: private properties

The onion skin concept should be a help for programmers to decide
which properties their program should support.
Furthermore the properties should be divided into functional
categories, like board-markup, timing, game-info, moves, annotations,
etc.

Private properties would be integrated into the spec as skin 3, and
clearly marked as "unofficial" extensions. There should be a note
on which program uses these properties and how common the property
is (if it's supported by more than one program).

That would make it easy to document private properties in the spec.
Everyone could decide if their program should support a certain property
based on which other programs support those properties and the intended
usage.

That way we could keep the core and common properties (skin 0 & skin 1)
small and give those power users (skin 2 & skin 3) a well defined set
of properties for every purpose.  For example, Shigeru's timing properties
would be documented in the spec as skin 3 properties. And if time shows
that they are useful and are widely adopted, then they could move to the
official part at skin 2.

What do you think about it?

/Arno

This onion skin idea got me thinking. In a way I like it, in a way I don't.
Overall, I'm not sure where I stand.

One thing I know that I diskike is numbering the skins. What makes "1" and
different from "2", for example? Can we have a parameter more important than
2, less than 1, and call it skin "1.5"? (I'm being silly but I hope you see
my point.) Why don't we name the skins? I can think of four right now:

    * Skin "Needed-To-Replay-Games": M, PB, PW, etc.
    * Skin "Needed-To-Annotate-Games-And-Problems-And-Stuff": AE/AB/AW, TR,
      SQ, etc.
    * Skin "Needed-To-Print-Games": DD, etc.
    * Skin "Private-Properties": All others.

I like this better because giving each skin a name makes it clear what
belongs in that skin, and also makes it easy for programmers to decide which
skins to implement, based on how they expect their software to be used. For
example, something like NNGS's Gobot, which only replays saved games, would
only know the "Needed-To-Replay-Games" skin. My program (cgoban) which I
intend to be used to annotate and view other's annotations would need the
annotation skin also. A program which is intended to help book publishers,
who want to create go diagrams, would use the "needed-to-print" skin too.

How does this sound for the "skins" idea? (PS - I recommend we change the
name from onion skins to something else. Skins makes me think of a different
outside on the same inside, which isn't what this is about at all).

> From: Arno Hollosi <ahollosi@...>
>
> Hi again,
>
> I didn't have much time lately, so I couldn't finish the udpate
> of the FF[4] spec as already discussed in this list (TM+OT,
> less restrictive handling of gameinfo properties, ...)
>
> Also we talked about adding a page for "private properties".
> So that other people don't use their names, or can implement
> those properties which they find useful, etc. Shigeru's proposal
> could be added to that "private properties" page.
>
> As you know, I'm rather reluctant to add new properties to the
> 'official' part of FF[4], as every programmer would have to add
> them to their programs. Which is a lot of work, for features
> which might be used seldomly. Furthermore the SGF standard would
> grow more and more complex and harder to maintain.
>
> Reading through Shigeru's mail I remembered an idea I had quite
> a time ago: SGF onion skin concept. This means, that we assign
> every property a "skin" number. The innermost skin comprises
> the core properties which should be implemented by every SGF client.
> Such properties are: B,W,AB,AW,AE,GM,SZ,C,PB,PW,BR,WR,DT,KM,RE,TR,MA.
>
> Every other skin extends the functionality, but is less "important"
> than the previous skin. Take as an example the board markup:
>
> Skin 0 (core): TR, MA, LB
> Skin 1: CR, SQ, AR, TB, TW
> Skin 2: DD, LN, SL
> Skin 3: private properties
>
> The onion skin concept should be a help for programmers to decide
> which properties their program should support.
> Furthermore the properties should be divided into functional
> categories, like board-markup, timing, game-info, moves, annotations,
> etc.
>
> Private properties would be integrated into the spec as skin 3, and
> clearly marked as "unofficial" extensions. There should be a note
> on which program uses these properties and how common the property
> is (if it's supported by more than one program).
>
> That would make it easy to document private properties in the spec.
> Everyone could decide if their program should support a certain property
> based on which other programs support those properties and the intended
> usage.
>
> That way we could keep the core and common properties (skin 0 & skin 1)
> small and give those power users (skin 2 & skin 3) a well defined set
> of properties for every purpose.  For example, Shigeru's timing properties
> would be documented in the spec as skin 3 properties. And if time shows
> that they are useful and are widely adopted, then they could move to the
> official part at skin 2.
>
> What do you think about it?
>
> /Arno
>
> ------------------------------------------------------------------------
> To unsubscribe from this mailing list, or to change your subscription
> to digest, go to the ONElist web site, at http://www.onelist.com and
> select the User Center link from the menu bar on the left.
> ------------------------------------------------------------------------
> SGF spec: http://www.sbox.tu-graz.ac.at/home/h/hollosi/sgf/
> Moderator: Arno Hollosi <ahollosi@...>

--
                                       Bill Shubert (wms@...)
                                       http://www.hevanet.com/wms/

Thanks for your response.

Robert Jasiek wrote in [sgf-std] Re: Suggestion on Time Property:

>What about special rules that count no time for certain
>moves? E.g. a so called token-go or even just a normal
>handicap go game might have no time recording for the
>initial passes respectively handicap plays. Also during
>the late stages of the game time can stop and resume in
>different ways. (Not to mention spectecular adjournment
>rules.)
>
>Also amateurs like to experiment with all kinds of time
>systems, e.g. mutually exchanged times.

I didn't consider very special case or local rules used by
amateurs because absolutely there is no chance to standardize
them.

>Such things can be recorded with TM[] if entered manually,
>but this can be a nasty job. IMO, given the rules and the
>move context, a property should detect the resulting times
>automatically.

I guess TM[] still can be used as a simple text property,
if we use it in a form like TM[3600 : whatever ].
For now, we can do nothing with the TM property because
it is defined as Simple Text type and if it almost
behaves as a private property. Some uses TM[1 hour]
and some uses TM[30 minutes] and there is no way to
retrieve numeric info from TM[] if we need to resume a game
>from an SGF file with the clock continues.

And the above sentence is also an answer to Arno's question
on "What do we use those Time System definition properties for?".

A very simply example is, suppose ,just suppose :-), we want to
use a byo-yomi system for the next FOST cup (like 5 seconds
per move, in order to prevent program from losing on time while
leading an overwhelmihng game because her stupid opponent does not know
how to resign, instead of the programmer or the judge demanding for a
resign. If these properties are in effect and the program keeps it,
a game can be resumed correctly if it happened to be adjourned by accident.

Although there are still many go programs don't even care keeping their
clock working correctly or can't even keep a proper SGF record, that
does not mean that no standardization is required, right?
And if you create a good standard, definitely many programmers would
know what to work for, in order to improve their programs. As a
matter of fact, many programmers give up on taking care of this
time record becuase the current SGF format gives no proper guideline.

>>   A very peculiar case is the Ing's Cup, due to the komi-
>> penalty when a player used up his/her time. However, it
>> still can be grouped as
>>     TM[10800] EW[180] EB[180] NW[] NB[]
>
>Then how is the penality komi deducted from EW[] and EB[] ?
>With current Ing rules it would be clear, however, with
>slightly different rules it can vary. So one would need
>some PP[] property in move nodes for penality points of any
>kind. I.e. PP are additional komi collected during a game.

I guess we should use KM[] to keep the final komi for
scoring purpose. Then, perhaps we can use GC[] or C[] in the
related move (when time was extended and komi penalty added)
to keep track of how the komi changed?



Shigeru Mabuchi

======= W ======= I ======= N ======= G ======
        World-wide  InterNet  Gokaisho
==============================================
Shigeru Mabuchi  (mab 2d* on WING)
http://wing.home.ml.org/
WING Server Address:  wing.brlnet.net 1515

Hi, Arno,

Arno Hollosi wrote in [sgf-std] Re: Suggestion on Time Property:

>About your suggestions:
>
>> Therefore, I added a property TimeLapsed on WING, in order to
>> replay a game on the server at the actual speed (or by double speed..)
>> TL[] has an optional value to record the netlag. That is not important
>> and can be deleted if you think unnecessary).
>
>Seems sensible.
>So the definition is TL[client time][netlag time]?
>This property  relies on the order of the values, e.g.
>client time HAS to be first, and netlag time second.
>I think this is dangerous as some programs may change
>the order of the values. Therefore I suggest either
>omitting netlag time, or using a compound type value
>TL[client time:netlag time].

   Actually, I added TL[] before I read the complete document on FF[4].
I thought the FF[4] document in the CGF Journal (Japanese only) was
the newest and complete doc, but realized that it was only a summary
of  the very first FF[4] lately. (OP[] was still introduced as a FF[4]
property there)

    Therefore, I use OP[] for Byo-Time and OM[] for Byo-Move in SGF
file issued by WING, because it is the only possible property I can find.
And, since there wasn't any Property Format guideline introduced in
the CGF journal, TL[] was designed as TL[mm,nn] where mm is the
time lapsed and nn is the net-lag. Of course, now I know it
should be TL[mm:nn] or TL[mm][nn] if I want to pack them together.

    Anyway, to follow the guideline, perhaps we can use

   TL[integer:integer]  or simply
   TL[integer]

    and use a new property (or OV[]?, which seems to have same
  meaning) for the net-lag..

   What do you think?

>OP isn't part of FF[4]. OP was introduced in FF[3] and
>isn't used any longer. Instead we created OT[] which should
>describe the byo-yomi system (moves/time etc.)
>But as pointed out above, we think about abandoning OT and
>move that information into TM as well.

I agree that using TM[text] is the easiest solution, but
meanwhile it also means to give up on searching a digital
solution and put everything in analog, the old fashion.
(Not always bad) :-)

Although it is impossible to illustrate every types of
time systems, especially those minor one used in Amateur
games, at least, if possible and reasonable, why don't
we focus on the pro's game and major amateur events and
build a standard on it?

To be frank, I hate writing long sentences into TM[] for
every pro games I entered, with each TM[] line slightly
different in each SGF file. :-)
In fact, in the SGF browser I am working on, the users
merely need to type the keyboard even if he wishes to
enter a large variety of game infos. Many PC beginners,
includeing high-dan Pro of Nihon-kiin, has been asking
for such a SGF browser so that they can handle easily.
Most infos are stored in database and can be recalled
by a few mouse clicks.

>>  (C) Byo-Moves: Number of moves required to make in each
>>                 Byo-yomi period.
>>      I guess OM[] is for this. (Overtime = Byoyomi?)
>
>OM was defined in FF[3] and is no longer used in FF[4].
>
>>  (D) Length of Extended Thinking Time:
>>  (E) Number of Extended Thinking Period
>
>As Robert pointed out there are even more cases to think about.
>Your extensions seem to cover most systems used in professional
>tournaments.
>I understand your need for the TL property, but why do you
>need a computer-readable form of the byo-yomi system?
>It's not essential for replaying.

Please refer to my last response for my answer.

There is one more reason that I'm sticking on this.
(Actually more to come) :-)

As you know, Nihon Kiin is not using SGF for their game record,
because of the software they use. However, they are interested
in using SGF because the current UGF format isn't as sophisticated
as SGF. I have talked to Sakai 9p of Nihon Kiin a few time and
to keep pro's game into a useful database, we need to
consider the pro's demand. Infos which look unimportant to us
like judges(mediators), timekeeper, recordkeeper, are important
infos for them. I don't want to write everything at a time
so I'll write again later.


Shigeru Mabuchi

======= W ======= I ======= N ======= G ======
        World-wide  InterNet  Gokaisho
==============================================
Shigeru Mabuchi  (mab 2d* on WING)
http://wing.home.ml.org/
WING Server Address:  wing.brlnet.net 1515

> Infos which look unimportant to us
> like judges(mediators), timekeeper, recordkeeper, are important
> infos for them.

I agree. Currently that would be yet more information to be put
in GC[]. Adding more and more properties for such needs is
possible but not too pleasant, either. AYK, I like general
properties. E.g. we seem to need something like IP[] for
involved persons that could read like

IP[black:Beauty Belle][white:Marc Whale][judge:Jane Jerome]
[judge:Bill Benefit][timekeeper:Timothy Trust]

The same would be possible for other classes of properties,
e.g. markups or comments or conditions (incl. time). Thereby
an FF[5] would be well structured.

While SGF is the most powerful go file standard, the
current situation is a little ridiculous. E.g. each
coordinate can have at most one markup, but the standard
includes many properties for assigning one possible markup.

--
robert jasiek

Hi !  First time to be here, I'm Kazuki, Japanese.

>"William M. Shubert" wrote:
> How does this sound for the "skins" idea? (PS - I recommend
> we change the name from onion skins to something else.

(1) How about call it "SHELL" ?

  I think SHELLs are : (revised Williams')

SHELL 0: Neeeded-To-Replay-A Game at least as an anonymous:
   SZ,HA,ADDs==starting conditions,
   and M(moves) keep which side and point or ADDs or pass etc.
SHELL 1: Neeeded-To-Replay-A Game in a Rule:
   ScoringRule, Advantages(KM), TimeRule and setups,
   PenaltyRule(if needed to specify),
   and each M(moves) keep the positions changes according
   to the specified rule.
SHELL 2: Neeeded-To-Replay-A Game identical:(documentations)
   Informations of the game(DT,PW,PB,EV,PC,etc).
SHELL 3: Usefuls: (to record byproducts of the game and
          post game editings)
       3-1: Useful-To-Inform-and-For-HumanReadability:
          such as MN[number], RE[], C[], RM[point](* my private).
       3-2: Useful-For-Annotaion:
SHELL 4: Useful-For-A-Specified-Purpose-But-Not-General:
   private properties for a special application.
   Needed ignorable by other applications.

  We should not promote to create private properties, I guess.
Most of the case, private properties is not necessary to
print out in SGF, if necessary, it should be a standard
property in SGF.

  Above are ordered according to the degree of necessity to
revival a game. In another saying, original revivabilty first,
then human readabilty second, common props for editing the game
such as decorating the board or add annotations, the third, and
others.
  I think if a property's nature is in SHELL 0-2, it should be
a standard property.
  So, TL[]'s nature matches to SHELL 1. And others that Mabuchi
proposed does also. I support they are to be standardized.
  I guess, however, the names and formats are to be discussed.

(2) My proposal of FN[] property
  Adding to that, by the way, I do think SGF should have FN[]
property which describe it is the FiNal move. Now, you cannot
append variations to the end of the game, 'cause the variation
is taken as the extended trunk by most programs and you lost
where the original final move was.
  I know attaching a vacant node to the original final node then
appending variations to the final is a solution to append variations
to the last move. However, not a technique, I guess SGF should
have the property. Even I guess FN[] should be in SHELL 0.
  I guess FN[] is move property, not root or setup. Its value indicates
the reason of the game finishing, as WhiteResigned, BlackTimeovered,
Adjourned, 3-Ko-draw, Entered-scoring, etc.
  Without this property, you cannot record and keep the final move
while editing, right?

  What do you think ?

Hi all,

I like Bill's idea of organizing the 'skins' into shells for a given
purpose. And I'm quite thrilled about the idea that Nihon Kiin might
start using SGF, provided there are some properties to store
information important to them.

This all would need some restructuring of the spec with only slight
modifications to some properties, and maybe introduction of a handful
of new properties for newly created shells (pro gameinfo, timing, ...).

That all leads to the question: should we start discussing FF[5]?
The shell concept itself doesn't require a new fileformat, as it
just rearranges the way the properties are listed in the spec, with
some additional hints and infos. Adding a lot of new properties might
justify calling it FF[5].

Or should there be larger changes in FF[5]? Personally I'm not too
keen about this, as it would mean incompatibilities with older
programs. Take for example Robert's proposal for IP[].
While it's quite a nice idea, moving from PW,PB,BR,WR,... to IP
would be difficult and troublesome to say the least (and in my opinion
not worth it). Having information about referee, scribes, and time
keeper in IP doesn't pose a problem, as this information wasn't
available before.

Based on the shell concept application coders can decide for themselves
if this information is important to them and if their program should
support those properties.


Comments welcome

/Arno

Hi,

this is a question to those subscribers of this list who use SGF
to store comments (or kibitz) in Chinese, Japanese, or Korean.

Could there be a problem with SGF special characters '\' and ']'?
I.e. could some normal text generate these characters by accident?

And, do you use the CA[] (charset) property to indicate which
charset you're using? Are you using a special encoding?


Thanks
/Arno

> While SGF is the most powerful go file standard, the
> current situation is a little ridiculous. E.g. each
> coordinate can have at most one markup, but the standard
> includes many properties for assigning one possible markup.

This is not quite true.
E.g. it's possible to have TR,LB,TW,DD on the same point with
several AR's and LN's starting from that point too.

What sense would it make to have TR, CR, and SQ on the same point?

/Arno

>This is not quite true.
>E.g. it's possible to have TR,LB,TW,DD on the same point with
>several AR's and LN's starting from that point too.
>
>What sense would it make to have TR, CR, and SQ on the same point?
>

To create abstract paintings :)

         Martin

I think the concept of Skins, Shells, (I like 'Packages' better...) has
been inofficially around for a while, even for SGF4 and earlier. We always
allowed private properties, and we moved the 'Computer Go' package out of
the 'standard' package in the transition from FF[3] to FF[4]. And on the
other hand, many programs have implemented only rudimentary SGF
functionality. There could be an official 'upgrade path' for these. For
example, you could start with package 'minimal', only B[] and W[], which
will already allow you to save game records. I'm in favor of defining a few
well thought out groups of properties, some as a round-about improvement of
their predecessors, some as an optional add-on for specific user groups. We
can discuss details here.

         Martin

> Most of the case, private properties is not necessary to
> print out in SGF, if necessary, it should be a standard
> property in SGF.

FF[4] agreement was about to close when printing properties
came up; they are not well covered yet, IMO. Currently, I
could not do without private printing properties. E.g. for
certain diagrams it is essential to have open/closed grid
boundaries on specified edges.

--
robert jasiek

> This is not quite true.
> E.g. it's possible to have TR,LB,TW,DD on the same point with
> several AR's and LN's starting from that point too.

Yes, of course. I have not wanted to include AR and LN in the
single point markup property class.

> What sense would it make to have TR, CR, and SQ on the same point?

None. Therefore my suggestion.

--
robert jasiek

As I posted earlier, I am using UTF-8 to store non-Roman characters in SGF
files. In UTF-8, all "wide" (ie more than 8 bit) characters will have bit 7
set in every byte, so there is no chance of accidentally getting a '\' or a
']' in the middle of a Chinese character. I can't speak for other encoding
systems, though.

Arno Hollosi wrote:

> From: Arno Hollosi <ahollosi@...>
>
> Hi,
>
> this is a question to those subscribers of this list who use SGF
> to store comments (or kibitz) in Chinese, Japanese, or Korean.
>
> Could there be a problem with SGF special characters '\' and ']'?
> I.e. could some normal text generate these characters by accident?
>
> And, do you use the CA[] (charset) property to indicate which
> charset you're using? Are you using a special encoding?
>
> Thanks
> /Arno
>
> ------------------------------------------------------------------------
> To unsubscribe from this mailing list, or to change your subscription
> to digest, go to the ONElist web site, at http://www.onelist.com and
> select the User Center link from the menu bar on the left.
> ------------------------------------------------------------------------
> SGF spec: http://www.sbox.tu-graz.ac.at/home/h/hollosi/sgf/
> Moderator: Arno Hollosi <ahollosi@...>

--
                                      Bill Shubert (wms@...)
                                      http://www.hevanet.com/wms/

A short opinion on Robert's suggestion.

Robert Jasiek wrote in [sgf-std] Re: Suggestion on Time Property:
>I agree. Currently that would be yet more information to be put
>in GC[]. Adding more and more properties for such needs is
>possible but not too pleasant, either. AYK, I like general
>properties. E.g. we seem to need something like IP[] for
>involved persons that could read like
>
>IP[black:Beauty Belle][white:Marc Whale][judge:Jane Jerome]
>[judge:Bill Benefit][timekeeper:Timothy Trust]

I like this idea. It saves us lots of property resources,
and is flexible to accustom to many cases, like additional
players in pair-go or ren-go. However, essential infos like
players should be kept in PW and PB, and we provide guidelines
for primary keywords to be used before the ':', so that the
SGF browser can handle or group them easily. Otherwise, some
progroms might use 'judge' and some use 'referee'.

eg.

    IP[AdditionalBlackPlayer:Beauty Belle]
      [AdditionalWhitePlayer:Marc Whale]
      [Judge:Jane Jerome][Judge:Bill Benefit]
      [Commentator:Timothy Trust]  ...

    (Analyzer and Commentator are different in many cases)



Shigeru Mabuchi

======= W ======= I ======= N ======= G ======
        World-wide  InterNet  Gokaisho
==============================================
Shigeru Mabuchi  (mab 2d* on WING)
http://wing.home.ml.org/
WING Server Address:  wing.brlnet.net 1515

Here are some infos that I would like to let you know, which is
closely related to topics on FF5[]..

There were a few meetings held last year, which involved Sakai
9p, also rep. from WWGo, Sansan (A Japaneset Net Go-club) and
a few major go-related software developers in Japan. I was there,
representing WING (also as a software developer), and our
common concern is to create a standard for data exchange of
go-game records, and our decision is to use SGF as the basis.
However, we had problem in the procedure on implementing new
properties that we think to be essential. Basically, we agreed
that we should consult you all and if possible, have a discussion
on it, whether to make it FF[5] or simply treat it as
additional properties or local properties. Unfortunately,
most of the members cannot read or write English and things
slowed down for a few months, though we had a Japanese SGF-ML.
I could act as a mediator but I was too busy to go back and forth
translating mails between sgf-std and this Japanese SGF-ML.

Therefore, I think it is important for you all to know what
we have discussed so that you can determine whether it should
be a minor change or FF[5]. Below is a summary of the discussion,
not necessarily means what have to be done.

(1) Add TimeLapsed property for each move
(2) Add property for byo-yomi, time-extension
     (Also talked about using SGF as game format for pro's game
      on the net in the future)
(3) Extend RE[] property to record more kind of game result
     (eg. Uncounted game like triple-kos, cycle-kos and other reasons)
(4) Add better descriptive property to record an end-of-game position.
     Definition for TB,TW is vague and cannot record a SEKI position
     correctly.
(5) Better structure for keeping multi-languages text.
     Suggestion like, use an extension of '\' in simple text.
     eg.  PL[\us:Name in English
             \cn:Chinese Name in Chinese code
             \jp:Japanese Name in Japanse code
             ]
     (A text property always starts with \us: as default and switches
      back to \us: at the end of the property )
     There are also suggestion on using Unicode but too many problems
     and unicode is not yet used in any applications except in the OS
     internally.
(6) Use IY[] to record the default 'direction' of the board,
     whether it should be turned upside down or not..
     (Pros are very sensitive on this)
(7) GN[] <-->  EV[] + RO[]  seems to have the same meaning.
     ie. What's GN[] for? Is it really necessary? Wouldn't it
     cause confusion?
(8) Should we add properties to record additional players name
     in pair-go anad ren-go?
(9) KO[] seems to be obsolete.
     Instead, perhaps we need property to record an illegal
     move which was made but overlooked by the players and judges
     due to a missing stones or other special reason.
     eg. A move on a position where there was already a stone.
     (The current Nihon-Kiin rule states that game must continue with
      that illegal position if both players overlooked it)
(10) Similar problem as above. What if a player did not remove the
     stones which are supposed to be captured and removed?
     Or vice versa, stones not captured are removed.
   Notes: (9) and (10) both happened on past pro's games.

   I'll be glad to have more opinions on this matter. Thanks!


Shigeru Mabuchi

======= W ======= I ======= N ======= G ======
        World-wide  InterNet  Gokaisho
==============================================
Shigeru Mabuchi  (mab 2d* on WING)
http://wing.home.ml.org/
WING Server Address:  wing.brlnet.net 1515

>>
  There were a few meetings held last year, which involved Sakai
  9p, also rep. from WWGo, Sansan (A Japaneset Net Go-club) and
  a few major go-related software developers in Japan. I was there,
  representing WING (also as a software developer), and our
  common concern is to create a standard for data exchange of
  go-game records, and our decision is to use SGF as the basis.
  However, we had problem in the procedure on implementing new
  properties that we think to be essential. Basically, we agreed
  that we should consult you all and if possible, have a discussion
  on it, whether to make it FF[5] or simply treat it as
  additional properties or local properties. Unfortunately,
  most of the members cannot read or write English and things
  slowed down for a few months, though we had a Japanese SGF-ML.
  I could act as a mediator but I was too busy to go back and forth
  translating mails between sgf-std and this Japanese SGF-ML.
<<

I am glad to see that discussion involves Asian countries
and that SGF is spread beyond English-speaking countries.
The connection to professionals users of SGF is crucial
and we can all mutually learn and incooperate our ideas
so that one international standard is possible and splitting
is avoided. This is in everybody's interest. I am especially
grateful to you who offers us a great service by translating
essential information. While quick discussion would be nice and
you can attempt to restrict translation to key information, it
is still a time-consuming work and thus we should respect the
speed possible for you.

> Therefore, I think it is important for you all to know what
> we have discussed

Yes.

> so that you can determine whether it should
> be a minor change or FF[5].

Considering the rather wide need for changes, even if only
concerning a lot of details, I am inclined to opt for FF[5].

> (1) Add TimeLapsed property for each move

Currently the difference of times of the actual and the previous
move gives the spent time. So I wonder why an additional property
might have been suggested.

> (2) Add property for byo-yomi, time-extension
>     (Also talked about using SGF as game format for pro's game
>      on the net in the future)

Time, netlag, etc. has to be discussed in more detail.

> (3) Extend RE[] property to record more kind of game result
>     (eg. Uncounted game like triple-kos, cycle-kos and other reasons)

During the FF[4] discussion I have preferred to offer a detailed
choice of possible results. This agrees with Japanese requirements.
E.g. current Japanese rules know a loss of both players due to
failed agreement to end a game. I still may collect a thorough list
of possible result types.

Triple ko, etc. is covered in FF[4] by so called "void games".
Obviously, this has to made more clear in the specification.

> (4) Add better descriptive property to record an end-of-game position.
>     Definition for TB,TW is vague and cannot record a SEKI position
>     correctly.

Game end positions and their evaluation is very different for
different rules. While I would offer to collect a sufficient
list of evaluation features, it must be clear what the purposes
of new properties shall be.

1) distinction of a) scored for black, b) scored for white,
c) scored for none;
2) distinction of a) scored black stone on board, b) scored
black prisoner, c) scored empty point for black, d), e), f)
... for white, e) not scored empty point;
3) distinction of life and death features: a) uncapturable
living stones, b) uncapturable living stone, c) capturable living
stone, d) living stones, e) dead stone, f) empty point, g) eye point,
h) dame, i) point of group, j) point in seki, k) territory point

(3) cannot be analysed automatically, see
http://www.snafu.de/~jasiek/j1989com.html
One could only use (3) to assign what the players (or possibly
a referee) have agreed to be.

I again ask for the purposes here, they are essential for a
distinction more precise than currently given. One has to fit
the needs of various rule sets.

>>
  (5) Better structure for keeping multi-languages text.
      Suggestion like, use an extension of '\' in simple text.
      eg.  PL[\us:Name in English
              \cn:Chinese Name in Chinese code
              \jp:Japanese Name in Japanse code
              ]
      (A text property always starts with \us: as default and switches
       back to \us: at the end of the property )
      There are also suggestion on using Unicode but too many problems
      and unicode is not yet used in any applications except in the OS
      internally.
<<

I am no expert for language sets, but I know that some suitable
solution to cover different languages has to be discussed and
solved.

> (6) Use IY[] to record the default 'direction' of the board,
>     whether it should be turned upside down or not..
>     (Pros are very sensitive on this)

I suspect that more such requirements exist, e.g. an expression
of "A plays B on sen-ai-sen in Japanese style". Detailed
discussion might be necessary.

> (8) Should we add properties to record additional players name
>     in pair-go anad ren-go?

I think, we have already recognized the need for more details
here and have to discuss it.

>>
  (9) KO[] seems to be obsolete.
      Instead, perhaps we need property to record an illegal
      move which was made but overlooked by the players and judges
      due to a missing stones or other special reason.
      eg. A move on a position where there was already a stone.
      (The current Nihon-Kiin rule states that game must continue with
       that illegal position if both players overlooked it)
  (10) Similar problem as above. What if a player did not remove the
      stones which are supposed to be captured and removed?
      Or vice versa, stones not captured are removed.
    Notes: (9) and (10) both happened on past pro's games.
<<

I agree that KO is doubtful and properties about illegal but
performed moves/positions are much more important. I offer to
retrieve details, if generally desired. Please do not neglect
illegal features, they are just too common in practice.

***
Can we agree to launch FF[5] discussion?
Which major discussion topics should be included?

IMO and AFA I can recall immediately, some topics are:

1) root properties,
2) properties about game end,
3) consequences of tournament play (illegal, net, ...),
4) printing,
5) time,
6) language,
7) variation comments and style,
8) obsolete/classes/...

--
robert jasiek
http://www.snafu.de/~jasiek/rules.html

> FF[4] agreement was about to close when printing properties
> came up; they are not well covered yet, IMO. Currently, I
> could not do without private printing properties. E.g. for
> certain diagrams it is essential to have open/closed grid
> boundaries on specified edges.

This is already possible, Robert.
An open grid can only occur, if some part of the board is
displayed instead of the full board. Partial views can be
defined by VW. See my printing examples on the web pages.

/Arno

Hi all,

please send me an email at <ahollosi@...>
if you think that we should start creating FF[5].

Also, tell me if you'd like a minor or a major update.
If you like, you can send me some ideas as well
(just a short list). I will mail a summary to this list.

If I receive at least 5 positive votes, and more positive
than negative votes, we start FF[5].

Thanks
/Arno

> (3) Extend RE[] property to record more kind of game result
>     (eg. Uncounted game like triple-kos, cycle-kos and other reasons)

RE[] has an entry called RE[Void] which covers uncounted games.
It doesn't give a reason, why these games are uncounted, though.

> (4) Add better descriptive property to record an end-of-game position.
>     Definition for TB,TW is vague and cannot record a SEKI position
>     correctly.

Using Japanese rules TB and TW seems good enough, as points in a seki
don't count, and therefore they are not marked as territory for either
side. Or am I mistaken?
Using other rules (I can't remember which) seki points are counted
according to how many stones surround them and then counts like
like 3/4 pts, 1/3 pt, ... may appear. These are not covered by TB and TW.

> (5) Better structure for keeping multi-languages text.
>     Suggestion like, use an extension of '\' in simple text.
>     eg.  PL[\us:Name in English
>             \cn:Chinese Name in Chinese code
>             \jp:Japanese Name in Japanse code
>             ]

Mutli-lingual files are important, I agree.
We have to think about a consistent design for all properties.
One idea could be to have a list of values, where there's
currently only one, e.g.
PB[\us:Name in English][\cn:Chinese Name][\jp:Japanese Name]
Though this might be critical, as older applications might
remove additional values.

> (6) Use IY[] to record the default 'direction' of the board,
>     whether it should be turned upside down or not..
>     (Pros are very sensitive on this)

IY is a game-specific property of Lines-of-Action (LOA).
It's not used in Go files (of course, we could use it).
I understand that pros care about how the board appears on
screen. But I think, instead of adding functionality to each
SGF client, the recorder of the game should be responsible
to record the game correctly. SGF editors might of course
provide a function to flip the board helping the recorder.

> (7) GN[] <-->  EV[] + RO[]  seems to have the same meaning.
>     ie. What's GN[] for? Is it really necessary? Wouldn't it
>     cause confusion?

EV and RO record the event (e.g. Meijin Tournament) and the
Round (e.g. Final, game 2). GN is the game name, e.g. displayed
in the title bar of the board window.
I can think of some games, which have a name that doesn't fit
into EV or RO: Blood vomitting game, Ear reddening game, ...

> (8) Should we add properties to record additional players name
>     in pair-go anad ren-go?

Yes. Jan van der Steen already uses some private properties to
record additional players. If I remember correctly his properties
are called BP and WP and look like BP[player, rank][player, rank] ....

> (9) KO[] seems to be obsolete.

Yes. Actually I never liked it, but some people voted that
it stays in FF[4]. SGF itself doesn't care about if the
move was legal or not. KO[] is mostly used for computer players
so that their algorithms accept an illegal move.

>     Instead, perhaps we need property to record an illegal
>     move which was made but overlooked by the players and judges
>     due to a missing stones or other special reason.

I think you mean that we need an property to MARK an illegal move.
The move itself should still be recorded with B[] or W[], but an
additional property could provide some information, about
why the move was illegal, e.g. IL[played over another stone] or
IL[recapture of ko without making a ko-threat], etc. Of course,
there should be some short defines, instead of the long sentences
in these exmaples.

> (10) Similar problem as above. What if a player did not remove the
>     stones which are supposed to be captured and removed?
>     Or vice versa, stones not captured are removed.

As SGF defines that B[] and W[] always remove stones, it's hard
to come up with a good solution. Furthermore SGF doesn't allow
having B,W and AB,AW,AE in one node. Without this last restriction
the problem could be solved like this:

We have to define that B,W is executed before AB,AW,AE

. # # # .  Dia. 1
O O O # .
# # # # .
. . . . .
;B[aa]AW[ab][bb][cb]IL[captured stones not removed]


. # # # .  Dia. 2
. O O # .
# # # # .
. . . . .
;B[aa]AE[bb][bc]IL[stones removed which are not captured]



/Arno

Arno Hollosi wrote in [sgf-std] Re: Shells, Time, FF[5]:

>> (3) Extend RE[] property to record more kind of game result
>>     (eg. Uncounted game like triple-kos, cycle-kos and other reasons)
>
>RE[] has an entry called RE[Void] which covers uncounted games.
>It doesn't give a reason, why these games are uncounted, though.

Yes, I learnt about it only very recently. Is there a summary
somewhere around there? Anyway, following is a list of possible
result that I collected, with the property content suggested..

1. Win/lose by score    B+nn  W+nn
2. Win/lose by time     B+T   W+T
3. Win/lose by resign   B+R   W+R
4. Win/lose by forfeit  B+F:Detail in simple text,
                         (Illegal move, violation of rule,etc)
5. Jigo                 0
6. Draw                 D:Detail in simple text
7. Win/lose by judge    B+J:Detail in simple text
8. Void Game:           V:Detail in simple text
    a) Triple kos
    b) Quadruple kos
    c) Cycle ko
    d) Repetition of same position (other cases)
    and whatever..
8. Both lose            L:Detail in simple text
9  Win/lose by abort/absent the game. (unlike resign)
                         B+A:Detail in simple text
10.Win/lose by other reasons
                         B+O:Detail in simple text

>> (4) Add better descriptive property to record an end-of-game position.
>>     Definition for TB,TW is vague and cannot record a SEKI position
>>     correctly.
>
>Using Japanese rules TB and TW seems good enough, as points in a seki
>don't count, and therefore they are not marked as territory for either
>side. Or am I mistaken?

You are right. In Japanese rule, logically it can be covered
by TB and TW, because the program can analyze the position
and find out the SEKI stones (those without any territory)..
But here comes the IGS/WING rule where eyes in SEKI stones are
counted as territory and included in TB/TW. In such case, there
is no way to differentiate seki and living stones from TB and TW.
This is OK if you only deal with computers, since seki stones are
living stones and it makes no difference to computers. However,
to human being (especially weak players who stick on the difference)
it is not. The users want to see the final position with seki stones
marked with, say triangles, not just as living stones.
As a matter of fact, there is no way to find out seki stones from
the TB/TW properties in IGS/WING sgf files if seki with eyes are
involved, unless the program has a perfect 'dead-or-alive' algorithm.

>Using other rules (I can't remember which) seki points are counted
>according to how many stones surround them and then counts like
>like 3/4 pts, 1/3 pt, ... may appear. These are not covered by TB and TW.

I think there is no problem in Chinese rule if the 'dame' are properly
filled (usually it should be). All dames neighboured to both B & W can
simply be treated as seki territory.

>> (5) Better structure for keeping multi-languages text.
>>     Suggestion like, use an extension of '\' in simple text.
>>     eg.  PL[\us:Name in English
>>             \cn:Chinese Name in Chinese code
>>             \jp:Japanese Name in Japanse code
>>             ]
>
>Mutli-lingual files are important, I agree.
>We have to think about a consistent design for all properties.
>One idea could be to have a list of values, where there's
>currently only one, e.g.
>PB[\us:Name in English][\cn:Chinese Name][\jp:Japanese Name]
>Though this might be critical, as older applications might
>remove additional values.

Yes, I guess some older program would get trouble if they
assume no additional [] behind PB or PW...

>> (6) Use IY[] to record the default 'direction' of the board,
>>     whether it should be turned upside down or not..
>>     (Pros are very sensitive on this)
>
>IY is a game-specific property of Lines-of-Action (LOA).
>It's not used in Go files (of course, we could use it).
>I understand that pros care about how the board appears on
>screen. But I think, instead of adding functionality to each
>SGF client, the recorder of the game should be responsible
>to record the game correctly. SGF editors might of course
>provide a function to flip the board helping the recorder.

I guess you are aware of the difference of the 'origin' of
the coordinates used in the East and the West. In the East,
usually the 'origin' is on the upper left corner and the
board is

   A B C D E F G H J K L M N O P Q R S T
1
2
3
.
.
.
19

While in IGS (not sure if this is what being used in the
West), it is on the lower left corner:

    A B C D E F G H J K L M N O P Q R S T
19
18
17
.
.
3
2
1

This causes a problem when you use coordinates in comments.
A  Q16 in the first case becomes Q4 in the second case.
What I suggest is, we use IY[] to record this status so
that the SGF browser can detect it and set the cooridinates
correctly automatically. Flipping the board is another
business (the coordinates filpped as well).

Setting IY[] correctly will give the user always a view
he opts for. eg. A pro who wish to look at the board
always as white by default, and a weak kyu player
who always wish to view the board as black by default...

Well, computers don't care because of the symmetries.

>> (7) GN[] <-->  EV[] + RO[]  seems to have the same meaning.
>>     ie. What's GN[] for? Is it really necessary? Wouldn't it
>>     cause confusion?
>
>EV and RO record the event (e.g. Meijin Tournament) and the
>Round (e.g. Final, game 2). GN is the game name, e.g. displayed
>in the title bar of the board window.
>I can think of some games, which have a name that doesn't fit
>into EV or RO: Blood vomitting game, Ear reddening game, ...

I see. This certainly give us a good clue on when to use these
properties. Perhaps we should add some notes like this into
the format details on how to use GN[]?

>> (8) Should we add properties to record additional players name
>>     in pair-go anad ren-go?
>
>Yes. Jan van der Steen already uses some private properties to
>record additional players. If I remember correctly his properties
>are called BP and WP and look like BP[player, rank][player, rank] ....

Seems confusing, computers don't care, though :-)

>> (9) KO[] seems to be obsolete.
>
>Yes. Actually I never liked it, but some people voted that
>it stays in FF[4]. SGF itself doesn't care about if the
>move was legal or not. KO[] is mostly used for computer players
>so that their algorithms accept an illegal move.
>
>>     Instead, perhaps we need property to record an illegal
>>     move which was made but overlooked by the players and judges
>>     due to a missing stones or other special reason.
>
>I think you mean that we need an property to MARK an illegal move.
>The move itself should still be recorded with B[] or W[], but an
>additional property could provide some information, about
>why the move was illegal, e.g. IL[played over another stone] or
>IL[recapture of ko without making a ko-threat], etc. Of course,
>there should be some short defines, instead of the long sentences
>in these exmaples.

I agree. Instead of KO[], IL[] would be appropiate in order to
cover all other illegal moves.

>> (10) Similar problem as above. What if a player did not remove the
>>     stones which are supposed to be captured and removed?
>>     Or vice versa, stones not captured are removed.
>
>As SGF defines that B[] and W[] always remove stones, it's hard
>to come up with a good solution. Furthermore SGF doesn't allow
>having B,W and AB,AW,AE in one node. Without this last restriction
>the problem could be solved like this:
>
>We have to define that B,W is executed before AB,AW,AE
>
>. # # # .  Dia. 1
>O O O # .
># # # # .
>. . . . .
>;B[aa]AW[ab][bb][cb]IL[captured stones not removed]
>
>
>. # # # .  Dia. 2
>. O O # .
># # # # .
>. . . . .
>;B[aa]AE[bb][bc]IL[stones removed which are not captured]

Good idea. Looks like it is difficult to keep backward
compatibility if we introduce this, but no choice..


Shigeru Mabuchi

======= W ======= I ======= N ======= G ======
        World-wide  InterNet  Gokaisho
==============================================
Shigeru Mabuchi  (mab 2d* on WING)
http://wing.home.ml.org/
WING Server Address:  wing.brlnet.net 1515

> following is a list of possible
> result that I collected, with the property content suggested..
> 1. Win/lose by score    B+nn  W+nn
[...]

This is a good start but incomplete. Later I will add more.
E.g. BothPlayersWin is possible.

> 4. Win/lose by forfeit  B+F:Detail in simple text,

I like the idea of a details option. Thereby we avoid an
endless list of parameters.

> >Using other rules (I can't remember which) seki points are counted
> >according to how many stones surround them and then counts like
> >like 3/4 pts, 1/3 pt, ... may appear.

FYI, so far only Ing 1986-1990 rules used fractions.
this may be indicated as RU[Ing 1986], i.e. with an
unspecified text parameter.

> >;B[aa]AE[bb][bc]IL[stones removed which are not captured]
> Good idea. Looks like it is difficult to keep backward
> compatibility if we introduce this, but no choice..

There are other possibilities than IL. E.g. an inherited
property IH[] might set and kill breathless stones illegally
remaining on the board as long as inherited. Thereby removals
of illegal stones are possible and backward compability is achieved.

--
robert jasiek
http://www.snafu.de/~jasiek/

Arno Hollosi wrote:
> And, do you use the CA[] (charset) property to indicate which
> charset you're using? Are you using a special encoding?

  Yet.
  In Japanese, there are 3 encodings: SJIS(used in PC), EUC(in Unix),
JIS(in mainly mail protocol).
  So, for us Japanese, character-encoding is more needed.

  How about introducing Locale property?
  For example,
   LC[ja_JP_SJIS] : LoCale[lang_country_encoding]

Hi,

First a small note to mention that I signed on the mailing list.
While Arno requested me several times in the past to register
I had no time to participate in the list actively. While I still
have not much time to spend I will at least try to read all
messages and from time to time try to give my point of view on
some matters, especially now FF[5] seems to be on its way.

Some matters which I certainly would like to discuss for FF[5]
concern the main game record properties: PB/PW/BR/WR/DT/PC/RE.
For archive and database applications (and to avoid loss of
information) in general more tokens of information needs to
be added. Two options to achive this seem obvious:

     XY[val,val,...]

     or

     XY[val][val][...]

     or more general:

     XY[VAL][VAL][...] where VAL=val,val,...

     Below some suggestions/applications.


     Multiple players per team
     -------------------------
     This matters for pair-go and rengo games.
     I use in my archive:

	 PB[...]
	 PW[...]
	 BP[Player-1,rank-1][Player-2,rank-2][...]
	 WP[Player-1,rank-1][Player-2,rank-2][...]

     It would be nice if a general solution could be implemented.
     Would software break when we use the above list construction
     in the PB[]/PW[] tags (to avoid the introduction of the new
     BP[]/WP[] tags)?

     At the same time we shouldn't forget to define the order of play.
     Out of courtesy to the players the order could be in order of
     ranking but more practical would be to have them in the same order
     as the move order.


     Event information
     -----------------
     I would like to have the EV[] info more tightly defined.
     What I would like to implement in my archive is something like:

	 EV[tournament,edition,country,sponsor]

     So, for the Meijin tournament for example which had edition
     1962 through 1975 sponsored by Yomiuri and all following editions
     sponsored by Asahi:

	 EV[Meijin,14,Japan,Yomiuri]
	 EV[Meijin,1,Japan,Asahi]

     Will this cover all cases?


     Round information
     -----------------
     Very difficult to standardize this property since so many
     different tournament formula are possible, and still new
     formula are invented. Right now in my archive I use:

	 RO[n]              for title matches
	 RO[league,n]       for league games
	 RO[preliminary,n]  for all other preliminary games

     Is this specific enough?


     Player ranking
     --------------
     Things which I want to be able to cover are:

	 pro versus amateur
	 dan versus kyu
	 rating

     So what seems an option is:

	 RO[rank,scale,status,rating]

	 rank   = integer (1,2,...)
	 scale  = string  (kyu|dan)
	 status = string  (pro,amateur)
	 rating = integer (1,2,...)

     Note: this would affect the new BP[]/WP[] syntax as well.


     Date
     ----
     Although the FF[4] DT[] tag is well defined and is satisfying
     in most cases it's a pity that it's impossible right now
     to specify a period of time, for example 1998-12-01 through
     1999-02-05. So, although this would be a serious change of
     the current definition a possible solution could be:

	 DT[period-1][period-2][...]

	 where period is: ISO,ISO

     So, for example:

	 DT[1998-12-01,31][1999-01-31,02-05]

     for a game played in December 1998 and from the 31st of January
     through the 5th of February 1999.


     Place (venue) specification
     ---------------------------
     There are no restrictions on the contents of the PC[] tag right
     now. Why not use the ISO-3166 specs with an optional city spec?

	 PC[ISO-3166,city]

     For example:

	 PC[JP,Tokyo]
	 PC[US,New York]
	 PC[NL,Amsterdam]


     Result
     ------
     Quite satisfactory right now but for special cases one could
     (similar to some of the previous suggestions) allow extra
     specification to the RE value as in:

	 RO[value,spec]

     For example, the 4th game of the 23rd Meijin match:

	 EV[Meijin,23,Japan,Asahi]
	 DT[1998-10-14,15]
	 RE[Void,triple ko]

     Or in one of Go Seigen's Oteai games:

	 EV[Oteai]
	 DT[1934-10-24,25]
	 RE[B+F,white got sick and didn't show up for the second day]

     The spec could be a free string specification or more strictly
     defined (but that will be hard I think).

A final note:
I have converted my complete game archive (10.000 games) to FF[4].
While I don't allow copying/mirroring of the game records those who wish
to purchase a personal copy can contact me at Jan.van.der.Steen@....

best regards,
Jan van der Steen

Hi,

the results of the CFV are: 7 votes for FF[5], 0 votes against.
Therefore we start discussing FF[5]. I'll post an announcement to
rec.games.go as well. The following issues should be discussed
(suggestions by voters):


Main issues for FF[5]
=====================

- well defined, consistent property sets (shells, packages)

- gameinfo properties: support for uncovered cases (e.g. team players,
   period of time), listing of other people involved in game (referees,
   time keeper, ...), allowing SimpleText after the mandatory part
   (to give a better description)

- supporting multiple languages: definitions for charset and encoding,
   way to include multiple languages into property values

- timing properties: computer-readable form of byo-yomi which covers
   all major professional tournaments, TimeLapsed for realtime replay
   of games, rest time (lunch break?), defining measurement unit.

- provide properties necessary for Nihon Kiin


Other suggestions
=================

- Introducing XML (or HTML) into C[] property

- Marking final move of game with property (FN[])

- Orientation of board (upside down, turning clockwise, ...)

- points discussed in this mailing list during the past two weeks.



Note on compatibility
=====================

If absolutely necessary we could introduce some incompatibilities to
previous versions of SGF. However, the most common properties have
to remain the same, i.e. redefining B[], W[], or similar properties
is out of question. Regarding game-info properties this propblem
isn't as severe, as most values are SimpleText anyway, and most
programs don't enforce a strict format.

Basically this means, that we could clean up some seldom used
properties and define well-behaved property sets instead.


Note about mailing list
-----------------------

You can switch to Digest Mode, if you'd like to receive only one
email per day (which contains all list-emails sent the previous day).
For those who don't want to get flooded by SGF mails :o)
Just go to: http://www.onelist.com click on "User Center" on the left,
log in, and click on "Digest", then "Change".



I suggest that those participating in the discussion should go through
the FF[4] specification again, and through the emails sent to this list.


We're moving forward!
Happy discussion :o)

/Arno

Jan van der Steen wrote:
> now FF[5] seems to be on its way.

Before the CFV result will have been reported, I give only
most essential comments. In principle, many properties
seem to need some general text option.

>         BP[Player-1,rank-1][Player-2,rank-2][...]

I wonder why rank is so important here and further things
are neglected. The one and only really crucial information
is the order of the players. [Most generally, each move
could have its type (colour, pass, pass-for-ko, ...) and
player. With a move type property we would also resolve
the pass notation problems.]

>         EV[tournament,edition,country,sponsor]

Editions can be counted differently. Additional information
(e.g. multiple sponsors) might be necessary.

>>
      Very difficult to standardize this property since so many
      different tournament formula are possible, and still new
      formula are invented. Right now in my archive I use:

          RO[n]              for title matches
          RO[league,n]       for league games
          RO[preliminary,n]  for all other preliminary games

      Is this specific enough?
<<

IMO, a property stating the tournament system / rules would be
appropriate. One could also put all in RU[]:

RU[Nihon Kiin 1989. Tournament rules: Nihon Kiin 1949 and
overriding Kisei Tournament rules. Tournament stage: title match
that is a best of 7 where "no result" means......]

I.e. the editor decides how detailed the text is.

For database purposes a standard entry as usual could be permitted.

>>
      Player ranking [...]
          pro versus amateur
          dan versus kyu
          rating

      So what seems an option is:

          RO[rank,scale,status,rating]

          rank   = integer (1,2,...)
          scale  = string  (kyu|dan)
          status = string  (pro,amateur)
          rating = integer (1,2,...)
<<

It is essential to include exact system information like
"national professional Chinese ranking based on tournament
achievements and the ratings of the CGA" or at least
"Chinese". E.g. an information like RA[1,kyu,amateur,UNKNOWN]
can be almost meaningless.

>         where period is: ISO,ISO
>         DT[1998-12-01,31][1999-01-31,02-05]

If at all, something different is necessary to allow
backward-compability.

>     Place (venue) specification
>         PC[ISO-3166,city]

Optional additional information must be allowed for
"Fujiyama Hot Spring Resort Hotel" etc.

[RE]
>     The spec could be a free string specification or more strictly
>     defined (but that will be hard I think).

Just take a fixed entry and a variable text entry.

--
robert jasiek

I'd like to make a very strong plea for backwards compatibility.
SGF was conceived from the start to be extensible in a way that
new files can still be read by old software, and old files read
by new software.

This is incredibly important since there are so many sgf reader
software packages, and so many existing game files.  We don't
want to convert game files to FF[5], and find that all the
older software no longer works.  This is something Microsoft
does to force people to upgrade software that is widely
hated.

In FF[4] there were several proposals that would break backward
compatibility that in the end were not included.  I hope we
can agree from the start that backward compatibility is required,
and find a way to fit any new features we want into the overall
SGF framework.

Can we put a time limit on FF[5]?  Perhaps have a call for
suggestions for a month or so, then work to quickly finish the
changes?  I'll probably have a new Many Faces this summer, and
I'd like to include FF[5] if it's done by then.

David

At 03:30 PM 2/6/99 +0100, Arno Hollosi wrote:
>From: Arno Hollosi <ahollosi@...>
>
>
>Hi,
>
>the results of the CFV are: 7 votes for FF[5], 0 votes against.
>Therefore we start discussing FF[5]. I'll post an announcement to
>rec.games.go as well. The following issues should be discussed
>(suggestions by voters):
>
>
>Main issues for FF[5]
>=====================
>
>- well defined, consistent property sets (shells, packages)
>
>- gameinfo properties: support for uncovered cases (e.g. team players,
>  period of time), listing of other people involved in game (referees,
>  time keeper, ...), allowing SimpleText after the mandatory part
>  (to give a better description)
>
>- supporting multiple languages: definitions for charset and encoding,
>  way to include multiple languages into property values
>
>- timing properties: computer-readable form of byo-yomi which covers
>  all major professional tournaments, TimeLapsed for realtime replay
>  of games, rest time (lunch break?), defining measurement unit.
>
>- provide properties necessary for Nihon Kiin
>
>
>Other suggestions
>=================
>
>- Introducing XML (or HTML) into C[] property
>
>- Marking final move of game with property (FN[])
>
>- Orientation of board (upside down, turning clockwise, ...)
>
>- points discussed in this mailing list during the past two weeks.
>
>
>
>Note on compatibility
>=====================
>
>If absolutely necessary we could introduce some incompatibilities to
>previous versions of SGF. However, the most common properties have
>to remain the same, i.e. redefining B[], W[], or similar properties
>is out of question. Regarding game-info properties this propblem
>isn't as severe, as most values are SimpleText anyway, and most
>programs don't enforce a strict format.
>
>Basically this means, that we could clean up some seldom used
>properties and define well-behaved property sets instead.
>
>
>Note about mailing list
>-----------------------
>
>You can switch to Digest Mode, if you'd like to receive only one
>email per day (which contains all list-emails sent the previous day).
>For those who don't want to get flooded by SGF mails :o)
>Just go to: http://www.onelist.com click on "User Center" on the left,
>log in, and click on "Digest", then "Change".
>
>
>
>I suggest that those participating in the discussion should go through
>the FF[4] specification again, and through the emails sent to this list.
>
>
>We're moving forward!
>Happy discussion :o)
>
>/Arno
>
>------------------------------------------------------------------------
>To unsubscribe from this mailing list, or to change your subscription
>to digest, go to the ONElist web site, at http://www.onelist.com and
>select the User Center link from the menu bar on the left.
>------------------------------------------------------------------------
>SGF spec: http://www.sbox.tu-graz.ac.at/home/h/hollosi/sgf/
>Moderator: Arno Hollosi <ahollosi@...>
>
>

Arno Hollosi wrote:
> Hi,
> the results of the CFV are: 7 votes for FF[5], 0 votes against.
> Therefore we start discussing FF[5]. I'll post an announcement to
> rec.games.go as well. The following issues should be discussed
> (suggestions by voters):
(snipped)

  Hi, dears,

  It's exciting to start SGF[5] discussion.

  How about that we define SGF[5]'s preceding target
"Implementing the ability to record the whole of and
the all kind of, as far as known, Go game" ?
  In another saying, a SGF[5] will be a perfect document
of a played Go game.

  Then our point of view will be more clear, I guess.

  Thanks.

As one who also also uses SGF for games other than Go, I think the current
discussion is somewhat misguided.  Attempting to optimize and make concrete
the ideal set of properties and conventions for Go is laudable, but isn't
really what SGF is all about.

The core "shell" of SGF is a specification that will allow tools
to be written to read, write and manipulate collections of games
in SGF format, with no knowlege at all about what game type is
contained.

The second "shell" of SGF is a more rigorous specification of the
content of certain specific properties, such that *if* those properties
are used, they should be used in a manner consistant with the specification,
and smart tools that deal with more than one specific game can rely
on common format for those properties among all games.

The third "shell" of SGF is a subspecies of SGF devoted
to a particular game.  It is pointless to try to define a generic
standard that will be meaningful for all games and allow meaningful
display of all games.  There is simply too much variability.  Defining
SFG-Go as an official standard is a fine idea, but shouldn't affect
the first two shells of SGF at all.  And it shouldn't be called SGF5.

Hi

I'm new to this forum, and newcomers are supposed to be circumspect. But
in the hope that I can represent at least one section of ordinary users,
may I make the following comments.

1. Even though I favour standardisation, I am dismayed by what I have
seen so far. There seems to be a trend to make the spec fit the needs of
the big fish - the databases, IGS, or Nihon Ki-in. Ordinary users seem
to have different needs. Arno tells me that most SGF files he gets from
other people are incorrect. That tells me that most users want to
prepare their files in ways that the spec won't let them. Human nature
demands that the spec should change to suit them, rather than users be
dragooned into complying with an increasingly complex spec. Is the fact
that most features in FF4 are unused, and the fact that FF4 is going to
have a short life, not a sign that the standardisation is moving too
fast?

2. I am inclined to believe that, rather than make the spec cleverer and
cleverer, the database or other software should be made cleverer. For
example, splitting EV and RO might suit the needs of current software,
but I don't see why the search engine could not be programmed to scan a
single line of free text for whatever the database regards as key words.

3. While I am not up to Jan's level, with Mark Hall I have a database of
about 6,000 games. Most are in .GO format but I am rapidly converting to
FF3 (I rejected FF4 for reasons that may become apparent below.) I come
across so many things that the spec either doesn't allow or consider. In
general it seems to me that the heavy emphasis by some people on modern
games leads to overlooking many important things. For example, the old
(or not so old  - it is still used in the Oteai) handicap system (B-B-W
etc) is VERY important because it affects the result of the game,
relative standing of the players, etc) yet there is nowhere to put it (I
was told to put it in Game Comment because HA won't allow text - I think
that is absurd).

4. I also find the DT date a major problem. Apart from the simple date,
there are lunar dates. I believe in the ISO standard, so I convert such
dates, but most people can't (or, worse, do what Invincible does and
pretend they are Gregorian). Often old dates are given as era years
(e.g. Kaei 1) but conversion would often require two western years
because of the lunar/solar overlap. How does one deal with the (rather
common) types exemplified by: "Early Qing era" or "Spring 1934" or "1894
or 1895" or "Published 1985-01-01~02-03" or or "Played before he became
a pro" or "Played 1845-03-03 and through the night till 9pm the
following day" [nb putting 1845-03-03,04 would omit the interesting
point that they played non-stop].

5. I find EV and RO a problem in various ways. First the split seems
artificial and secondly EV can be defined in different ways by different
people. I prefer both "Meijin Final" and "Meijin League" (and also
simply "Meijin") to be in EV, partly because these are seen as
significant separate items in biographies of players (e.g. the number of
times one has won a Final, or appeared in one, or appeared in a League,
are listed as discrete achievements).

6. An even bigger problem with EV is that tournament names have yet to
be standardised. At the risk of seeming rude, I would say that much of
the energy spent on rather esoteric parts of the spec could more
fruitfully be spent here. (A couple of examples: is it Kisung, Kiseong,
Kisong, Gisung, Korean Kisei, Kisei or what? Is it Women's Honinbo or
Ladies' Honinbo (or Ladies Honinbo, which may throw some search
engines]? Is it just Meijin or Old Meijin and New Meijin? Is it Hayago
Senshuken, Hayago Championship, Lightning/Speed/Fast Go Championship?

7. Players' names are a major problem, especially Korean. For many there
are 5 or 6 current variants. I can claim to have offered a standard for
these in my Names Dictionary (now out of print!) as virtually every name
that can appear in a significant game record is there, but a strategy
for making anyone follow the "standard" is beyond me.

8. I find RE a problem. Many old games are left unfinished. They are not
void. The reason they were left unfinished may be interesting (one
player was losing so broke off, it was a ceremonial game, it was just
meant to try a fuseki, etc etc). I have also recently encountered a
"both players lost" game - is that covered? To me, it is also logical to
include in the RE rubric information such as "No further moves known" or
"White connects the ko" [it is wrong to play these moves out as part of
the record - historical research often hinges on niceties such as these:
e.g. one record says W connects the ko, another says B.

9. TM: Don't forget that players may be given different times (as in
some pro-am games)

10. I have more points but I would be overegging the pudding to go on.
The conclusions I draw from my own experience are (1) that FF5 would be
an advance if it actually went backwards a little bit - in particular it
should allow free-form text where possible; (2) most urgent effort is
needed to standardise tournament names and player names; (3) the needs
of ordinary users - people who enter game records, not programmers or
anyone else with too much of a vested interest - should be given greater
weight. (It would be of interest to know what the commonest "mistakes"
of ordinary users are.)

--
John Fairbairn