Rui Jiang wrote:
> can the group admin block it from sending SPAMs?

My guess is, someone reg'd under their work address, which autoresponds.

Of course, every time they get a response from themselves, they send
another email, autoresponding to that.

Important point: the more messages posted to the list, the more replies
it will start sending.

This is the third email sent today, which means that their autoresponder
will be replying to three emails at a time (and then autoresponding to
the autoresponses and so on).

So please DON'T post any more to the list until the admin has time to
delete the user, or move them to web-only!

I suggest that in the meantime, the rest of us move to "digest" or
web-only mode, which can be done here:

      http://groups.yahoo.com/group/sgf-std/

   - Dewi Morgan.

Dewi Morgan wrote:
> So please DON'T post any more to the list until the admin has time to
> delete the user, or move them to web-only!

I denied the user to post emails and have switched them to web only.

Sorry, I did not see the emails - apparently my SPAM filter caught them.
Thanks for alerting me.

/Arno

Dear Christian,

I'm CCing the sgf-std list, as I guess this is of interest to others as
well.

> I've read http://www.red-bean.com/sgf/properties.html#CA, but it's
> not clear to me what values the and the CA property can have.
>
> I understand that "US-ASCII" and "ISO-8859-1" are legal values
> according to RFC 1345, but my gGo 1.0 generated properties like
> "CA[UTF-8]" and as far as I can see, this isn't defined by RFC 1345,
> so I'd think this would be an illegal value.  Am I missing something?

Technically speaking you are right. De facto, just about all programs
that are able to deal with characters outside Latin1 use UTF-8 encoding.
I guess I should update the spec to make UTF-8 (and maybe other Unicode
encodings such as UTF-16) legal values as well.

> Do you know to what extent SGF parsers actually deal with all the
> character sets RFC 1345 defines?  For example, how is Japanese
> typically dealt with in terms of SGF?

UTF-8.

Thanks for bringing this to my attention. I shall update the spec soonish.

regards,
/Arno

I'd just like to note, that UTF-16 is not possible as a CA[] parameter.
To get to the CA[] property itself, you must be able to parse at least
some text without knowing the character set, so the character set must
be backward compatible with ASCII. ISO-8859-1 and UTF-8 are both
backward compatible, but UTF-16 is not, so the parser will never be able
to find the CA[] property in the first place.

On Fri, 2006-08-18 at 22:07 +0200, Arno Hollosi wrote:
...
> I guess I should update the spec to make UTF-8 (and maybe other Unicode
> encodings such as UTF-16) legal values as well.
...

> backward compatible, but UTF-16 is not, so the parser will never be able
> to find the CA[] property in the first place.

Not necessarily. We could define that the CA[] property is only valid for
text fields *after* the CA[] property occurs in the text.

But there is one more argument against UTF-16: the parser should be able
to find the end of a text property by looking for Latin1 ']' [*] (i.e.
without knowing about the encoding of the text) I am not sure that UTF-16
has no byte value of ']' occuring e.g. as part of some chinese character.

So the argument for Latin1 compatibility is a very strong one.

/Arno

[*] actually: Latin1 ']' that is not preceeded by Latin1 '\'

HI,

In order to promote compatibility and keep testing work sensible, I
think standard should say that UTF-8 is only allowed encoding for
unicode. Other uncode encodings do not give any additional value (or do
they?).

If CA would affect only properties after it, then order of properties in
root node is critical.
This is quite troublesome change, as currently order is irrelevant.

It is possible to parse any character set, even binary data, from text
values. Process goes as follows:
- Input is stream of bytes.
- These bytes are interpreted as latin-1 to find bytes that belongs text
values. Recognition can understand quoting (\), as it is in latin-1, so
there is no need to know actual character set.
- Latin-1 quoting ("bytes") are  removed from text values.
- Now bytes (quoting removed) are interpreted using character set found
in CA property.

While this possible, I do not see any reason to expect applications to
implement this. For example in Java 1.3 this would create significant
overhead.

regards
Lauri Paatero


Arno Hollosi wrote:
>> backward compatible, but UTF-16 is not, so the parser will never be able
>> to find the CA[] property in the first place.
>>
>
> Not necessarily. We could define that the CA[] property is only valid for
> text fields *after* the CA[] property occurs in the text.
>
> But there is one more argument against UTF-16: the parser should be able
> to find the end of a text property by looking for Latin1 ']' [*] (i.e.
> without knowing about the encoding of the text) I am not sure that UTF-16
> has no byte value of ']' occuring e.g. as part of some chinese character.
>
> So the argument for Latin1 compatibility is a very strong one.
>
> /Arno
>
> [*] actually: Latin1 ']' that is not preceeded by Latin1 '\'
>
>
>
> SGF spec: http://www.red-bean.com/sgf/
> Contact: Arno Hollosi <ahollosi@...>
> Yahoo! Groups Links
>
>
>
>
>
>
>
>

Hi,

I am currently working on the perl module Games::Go::SGF.

This is a module that parses sgf files.

I have put in some property validation checks, which are Go specific.

That means people can't use it anymore to parse sgf files for other
games. However I could easily undo this change.

The module comes under Games::Go, which implies it is Go specific. If
it was called Games::SGF, that would be different.

I wonder how many (or what proportion of) people use sgf for recording
games other than Go.

Any ideas?

dan

Dan,

sorry for the late reply. The list email was classified as spam by my
system. Yahoo seems to be adding to many ads these days. If it is not
too late:

--- In sgf-std@yahoogroups.com, "gilderde" <gilderde@...> wrote:
> I wonder how many (or what proportion of) people use sgf for recording
> games other than Go.
> Any ideas?

I don't know about end users or programs in total, but I have contact
with 3-4 developers who use SGF for other programs than Go.

/Arno

I'm probably the most prolific user of SGF format.  I use it
to record all the games at Boardspace.net, currently 10 different
games.

I use SGF as a common wrapper for all the game formats, so the same
code is able to read/write/parse all the games.  The details of how
a move is recorded are specific to the game and no attempt is made
to make them conform to any kind of standard.

I have recently taken Arno's sgfc source and built a program that processes
collections of
SGF files and produces statistics on property and value usage.  I've been
building a goban
in Second Life and working on a SGF viewer feature for it.  I wanted to know if
some of the
more obscure properties were used enough to warrant implementation.

I ran it over about five thousand SGF files from the GoDatabases page over at
Sensei's and
all five thousand problems from goproblems.com with interesting results.  For
example:

* Other than text comments (C and N), move and node annotation is basically
never used.
* The graphic markup AR, LN, and DD, is never used. Nor is SL.
* The character set (CA), if ever specified, is almost always UTF-8.
The source is available:

	 http://www.ozonehouse.com/mark/sgf/stats.cpp

It is a C++ file that includes its own main().  Just link it with all the object
files from sgfc
except main.o and save.o.

The full output from my two 5,000 file runs are at:

	 http://www.ozonehouse.com/mark/sgf/game-stats.txt
	 http://www.ozonehouse.com/mark/sgf/problem-stats.txt

Of course, I'd like to run it over a few other collections to get a wider
representation, but I
though this might be interesting data to anyone working on SGF FF[5] and/or some
future
XML go format.


	 - Mark

Mark,

nice work. Thanks for the information.

> and produces statistics on property and value usage.

It would be interesting to run this program also on e.g. the GTL archive.
It contains game reviews and therefore I would assume that it has more
markup properties than you would find in a collection of professional
games. I don't have time to do this myself during the next two weeks, so
if anyone beats me to it post your results here :o)

> I wanted to know if some of the
> more obscure properties were used enough to warrant implementation.

Heresy! Of course, one could argue that by leaving out support for such
properties, you contribute to the status quo and such properties will
never emerge from obscurity. I for one am planning to integrate AR & LN
into SenseisLibrary's diagrams.

> * The character set (CA), if ever specified, is almost always UTF-8.

Which goes to show that I should nail down UTF-8 usage in the
specification according to our discussion some months ago.

/Arno

Dear sgf-programmers!

How can an additional sgf property be included into the given sgf standard?
I'm the author of Kogo's Joseki Dictionary, and I need links for this file.
Oftenly identical positions of stone result from different orders of
placements of these stones which are to be found in very idfferent
branches of joseki. In such a case, the following joseki variations are
included only once, and after the other order of placements a reference is
given. But this reference is only a description, not a hyperlink, and that
is unnecessarily complicated for me as author and the users and frustrated
to search through the branches instead of simply clicking like in HTML.
I propose the following syntax:
AC[example] Anchor
LI[example] Link
The anchor is hidden to the viewer. The simplextext of the Link is
displayed in like that of C[] and underlined. Clicked, the sgf viewer
jumps to the move in the branch where the Anchor with the same simpletext
is attached.

Best wishes

Andre Ay

Andre,

> How can an additional sgf property be included into the given sgf standard?

If you add it as a private property all you need to do is to write it in
the SGF file. SGF programs ignore unknown properties. As for getting
others to support your property is out of scope of the spec, I guess.

I take it you would like to add it to the core standard though.

> But this reference is only a description, not a hyperlink, and that
> is unnecessarily complicated for me as author and the users and frustrated
> to search through the branches instead of simply clicking like in HTML.
> I propose the following syntax:
> AC[example] Anchor
> LI[example] Link
> The anchor is hidden to the viewer. The simplextext of the Link is
> displayed in like that of C[] and underlined. Clicked, the sgf viewer
> jumps to the move in the branch where the Anchor with the same simpletext
> is attached.

I don't think that this would be the best approach. Having an AC[]
property seems ok, although I'd argue that we could use the N[] (node
name) property for this.

What I'm unhappy about is LI[]. Why not instead define how hyperlinks
should be done inside C[]? E.g.

---------
(;GM[1]FF[4]
;W[dd]
;B[pq]N[first black move]
;W[po]
;B[pl]C[This is black's second move.
See his awesome {{first move|first black move}}]
;W[fq]
---------

What do others think about such an approach?

/Arno

Dear all,

I have been approached (again) with a proposal to add links to the SGF
standard. As I clearly see the need for this, I would like to add it to
the core SGF FF[4] specification. I'd like your input on this issue
before finalizing the syntax.

Here's the proposal from David Bush, author of the Twixt SGF spec:

""
TP[ simplestring ] is a node annotation property. It means this node
represents an identical position by TransPosition of moves to another
node in the tree. The value is the same as the N property of a target
node which must be listed earlier in the file, and must not be an
ancestor of the TP node. This might be useful for an opening library,
or anywhere that alternate move orders are plausible. A TP node should
be a leaf node in the file.

Effectively, TP is like a macro call. The intention is for the software
to copy the subtree of the target node to the location of the TP node.
Strictly speaking, the two positions need not be completely identical,
as long as this copied subtree with all its moves, evaluations, and
every other property is valid in this new location.
""

comparing this with e.g. last years idea from Andre (Kogo's Dictionary)

""
Oftenly identical positions of stone result from different orders of
placements of these stones which are to be found in very idfferent
branches of joseki. In such a case, the following joseki variations are
included only once, and after the other order of placements a reference
is given. But this reference is only a description, not a hyperlink, and
that is unnecessarily complicated for me as author and the users and
frustrated to search through the branches instead of simply clicking
like in HTML.
I propose the following syntax:
AC[example] Anchor
LI[example] Link
The anchor is hidden to the viewer. The simplextext of the Link is
displayed in like that of C[] and underlined. Clicked, the sgf viewer
jumps to the move in the branch where the Anchor with the same
simpletext is attached.
""


My currrent favourite option is to use N[] as the anchor, and LI[]/TP[]
(we have to decide on a name) as single property in a node. Also, we
could think about allowing links inside C[].

I'd like to hear your opinions.

regards,
/Arno

Great idea Anro

Is he plan to extend sgfc to cover this and optionally expand trees?

----

The idea of common subtrees is very common in computer go. I'm not
sure whether any of the computer go programs externalise their search
trees though. May I suggest that his issue is also raised on the
computer go mailing list?

----

I've forgotten what the exact SGF terminology is, but could we
restrict the positions where this can occur so that the main line of
play has no TP in it? That would keep it easy for "simple" programs.
Since the full tree needs to appear somewhere, that doesn't seem
onerous.

cheers
stuart

On Tue, Jul 29, 2008 at 7:53 PM, Arno Hollosi <ahollosi@...> wrote:
> Dear all,
>
> I have been approached (again) with a proposal to add links to the SGF
> standard. As I clearly see the need for this, I would like to add it to the
> core SGF FF[4] specification. I'd like your input on this issue before
> finalizing the syntax.
>
> Here's the proposal from David Bush, author of the Twixt SGF spec:
>
> ""
> TP[ simplestring ] is a node annotation property. It means this node
> represents an identical position by TransPosition of moves to another
> node in the tree. The value is the same as the N property of a target
> node which must be listed earlier in the file, and must not be an
> ancestor of the TP node. This might be useful for an opening library,
> or anywhere that alternate move orders are plausible. A TP node should
> be a leaf node in the file.
>
> Effectively, TP is like a macro call. The intention is for the software
> to copy the subtree of the target node to the location of the TP node.
> Strictly speaking, the two positions need not be completely identical,
> as long as this copied subtree with all its moves, evaluations, and
> every other property is valid in this new location.
> ""
>
> comparing this with e.g. last years idea from Andre (Kogo's Dictionary)
>
> ""
> Oftenly identical positions of stone result from different orders of
> placements of these stones which are to be found in very idfferent
> branches of joseki. In such a case, the following joseki variations are
> included only once, and after the other order of placements a reference is
> given. But this reference is only a description, not a hyperlink, and that
> is unnecessarily complicated for me as author and the users and frustrated
> to search through the branches instead of simply clicking
> like in HTML.
> I propose the following syntax:
> AC[example] Anchor
> LI[example] Link
> The anchor is hidden to the viewer. The simplextext of the Link is
> displayed in like that of C[] and underlined. Clicked, the sgf viewer
> jumps to the move in the branch where the Anchor with the same simpletext is
> attached.
> ""
>
>
> My currrent favourite option is to use N[] as the anchor, and LI[]/TP[] (we
> have to decide on a name) as single property in a node. Also, we could think
> about allowing links inside C[].
>
> I'd like to hear your opinions.
>
> regards,
> /Arno
>
>

Hello, I support this extension vry much, but I believe that the
restriction of the identical position is too restrictive, when I was
making my private sgf extension to solve this, I made the
Label/Pointer tags but the pointer tag also contained information
about transformation (axes, colors), so the dictionary joseki didn't
need to repeat any variations with same logic.

On Tue, Jul 29, 2008 at 10:48 AM, Stuart A. Yeates <syeates@...> wrote:
> Great idea Anro
>
> Is he plan to extend sgfc to cover this and optionally expand trees?
>
> ----
>
> The idea of common subtrees is very common in computer go. I'm not
> sure whether any of the computer go programs externalise their search
> trees though. May I suggest that his issue is also raised on the
> computer go mailing list?
>
> ----
>
> I've forgotten what the exact SGF terminology is, but could we
> restrict the positions where this can occur so that the main line of
> play has no TP in it? That would keep it easy for "simple" programs.
> Since the full tree needs to appear somewhere, that doesn't seem
> onerous.
>
> cheers
> stuart
>
> On Tue, Jul 29, 2008 at 7:53 PM, Arno Hollosi <ahollosi@...> wrote:
>> Dear all,
>>
>> I have been approached (again) with a proposal to add links to the SGF
>> standard. As I clearly see the need for this, I would like to add it to
>> the
>> core SGF FF[4] specification. I'd like your input on this issue before
>> finalizing the syntax.
>>
>> Here's the proposal from David Bush, author of the Twixt SGF spec:
>>
>> ""
>> TP[ simplestring ] is a node annotation property. It means this node
>> represents an identical position by TransPosition of moves to another
>> node in the tree. The value is the same as the N property of a target
>> node which must be listed earlier in the file, and must not be an
>> ancestor of the TP node. This might be useful for an opening library,
>> or anywhere that alternate move orders are plausible. A TP node should
>> be a leaf node in the file.
>>
>> Effectively, TP is like a macro call. The intention is for the software
>> to copy the subtree of the target node to the location of the TP node.
>> Strictly speaking, the two positions need not be completely identical,
>> as long as this copied subtree with all its moves, evaluations, and
>> every other property is valid in this new location.
>> ""
>>
>> comparing this with e.g. last years idea from Andre (Kogo's Dictionary)
>>
>> ""
>> Oftenly identical positions of stone result from different orders of
>> placements of these stones which are to be found in very idfferent
>> branches of joseki. In such a case, the following joseki variations are
>> included only once, and after the other order of placements a reference is
>> given. But this reference is only a description, not a hyperlink, and that
>> is unnecessarily complicated for me as author and the users and frustrated
>> to search through the branches instead of simply clicking
>> like in HTML.
>> I propose the following syntax:
>> AC[example] Anchor
>> LI[example] Link
>> The anchor is hidden to the viewer. The simplextext of the Link is
>> displayed in like that of C[] and underlined. Clicked, the sgf viewer
>> jumps to the move in the branch where the Anchor with the same simpletext
>> is
>> attached.
>> ""
>>
>>
>> My currrent favourite option is to use N[] as the anchor, and LI[]/TP[]
>> (we
>> have to decide on a name) as single property in a node. Also, we could
>> think
>> about allowing links inside C[].
>>
>> I'd like to hear your opinions.
>>
>> regards,
>> /Arno
>>
>>
>



--

Michal Kovařík

> Hello, I support this extension vry much, but I believe that the
> restriction of the identical position is too restrictive, when I was
> making my private sgf extension to solve this, I made the
> Label/Pointer tags but the pointer tag also contained information
> about transformation (axes, colors), so the dictionary joseki didn't
> need to repeat any variations with same logic.

Duplication by reflection and/or rotation is certainly important, but
this could usually be settled after the first two moves of the game.
Maybe a separate property should deal with this, which is restricted
to the beginning of the tree, maybe even in the game info node. You
might even use an existing property such as ON. Besides naming the
opening, the value in ON could indicate how the actual game is
oriented compared to the moves listed in the file. The
TP property should be allowed throughout the file, subject to two
restrictions listed below.

> I've forgotten what the exact SGF terminology is, but could we
> restrict the positions where this can occur so that the main line of
> play has no TP in it? That would keep it easy for "simple" programs.
> Since the full tree needs to appear somewhere, that doesn't seem
> onerous.

Since the first variation listed is the main variation, this request
is covered by my two restrictions:

1. The target node must be listed before the TP node in the file.

2. The target node must not be an ancestor of the TP node.

This implies that the main variation cannot contain a TP property,
because all the nodes previous to a main node are ancestors of that node.

Regarding the issue of whether TP functions as a link or as a macro
call, I suppose that would be up to whoever writes the software that
uses it. The above restrictions are intended to avoid any endless loop
of references. That's not very likely in Go, but SGF is for many games.

I would like to suggest another property to add to FF[4]:

LO[] is also node annotation. It means the subtree which includes this
node is entirely concerned with one LOcal battle. As the endgame
approaches, battles may become independent of each other. The player
with sente might contest the battles in any order. LO serves to avoid
the tedium of duplicating the same responses to the same moves
depending on what order the sente side chooses. If a move is selected
outside the choices for a local battle, the program could back up to
the LO node and search among the siblings of that node (which would
also be labeled LO) for a battle which does contain that move. Here is
an example of how the nodes should be structured.

(;W[km]
  (;LO[]
    (;B[ec] ;W[gc] GW[])
    (;B[gc] ;W[ec] GW[])
  )
  (;LO[]
    (;B[pq] ;W[po] GW[])
    (;B[po] ;W[pq] GW[])
  )
)

An LO node should never contain a move, even if there is only one
initial move possible for that battle. This restriction makes it
easier for software to deal with it. An LO node could have an N
property or a TP property. If it has a TP property, it should be a
leaf node in the file, and its target should be another LO[] node
which is listed earlier in the file and which is not an ancestor of
this TP node. LO itself should have no value. I suppose it could
contain the target name of a TP call, but that would complicate the
definition of TP. It would be simpler to always use N instead.

LO partitions siblings into subsets. When a specific subset battle is
entered, all the initial moves of the as yet uncontested battles
remain available to explore. If the user chooses to leave battle A for
battle B, the software could remember the progress in battle A, and
allow the user to resume battle A later on.

You might wonder, why bother with these moveless LO[] nodes? There are
two reasons. An LO marker would make it easier for a program to look
for responses to a move which is not a child of the current node.
Also, it is important for the program to know when different battles
are truly independent. So, LO tells the program when it can look
elsewhere, and where it should look.

When used in an LO[] node, TP means "this local battle is identical to
an earlier listed local battle." This can greatly reduce the file size
that would be necessary to provide a comprehensive list of responses.

TP[ simplestring ] is a node annotation property. It means this node
represents an identical position by TransPosition of moves to another
node in the tree. The value is the same as the N property of a target
node which must be listed earlier in the file, and must not be an
ancestor of the TP node. This might be useful for an opening library,
or anywhere that alternate move orders are plausible. A TP node should
be a leaf node in the file.

> TP is fine with me, but the requirement that it's target N[] appear
> before the matching TP[] is strange. Why have it? That means that if
> somebody reorders the subtrees in an SGF file, the app will need to
> go through the subtrees, rearranging the N[]/TP[] groups to make
> sure that the N comes first. Let's face it, that won't happen, so
> you'll end up with a lot of invalid files floating around.

You might have missed my post from about an hour ago. It might not
have reached you before you posted.

The purpose of the restrictions is to avoid forward references, which
could result in an endless loop bug, depending on the software which
is using the file. Software generally reads files from beginning to
end. At the moment it encounters a forward reference, it won't be able
to resolve that reference until it reads the target later in the file,
so it will have to be smart enough to remember where the TP call was,
and go back and fill in the required information after it reads the
target. I'm trying to make it easier for programmers to incorporate TP.

If TP is treated as a macro call, then when the file is initially
read, a duplicate subtree or "expanded macro" will replace each TP
call. The user could work with this enlarged tree which will not have
any TP calls in it. Manipulating the branches should not present any
problem.

> On top of that, I don't see any reason why having the N[] first
> makes things easier for anybody. It's not like it's easier to
> search the start of the file than it is to search the whole file.
> Sure, if you resolve the TP links as you read the file then the N
> must come first, but just read in the file, then resolve the links,
> and the problem is solved.

Besides helping programmers, the two restrictions "N first" and "N
must not be an ancestor of the TP" can create a uniform standard which
makes it more likely that files can be used in a variety of software,
which is what SGF is all about.

One user asked that TP should not appear in the main line. As my
previous post explains, these restrictions guarantee that.

Stuart asked:
  > Is the plan to extend sgfc to cover this and optionally expand trees?

I would certainly add such an option to SGFC.

Bill wrote:
>> TP is fine with me, but the requirement that it's target N[] appear
>>  before the matching TP[] is strange. Why have it?

I tend to agree with Bill. I think that most of us agree that we should
keep the SGF game tree a directed, non-cylic graph. However, as Bill
pointed out, as soon as someone reorders variations the TP[] and subtree
denoted by N[] would have to change place as well. That is not exactly
easy on programmers either. Thus I side with no restriction on N[]
appearing before TP[] in the file.

I am undecided about keeping TP[] out of the main branch. It makes some
sense, as SGF's tree order allows programs to ignore variations and
still read the main line.

Michal wrote:
> when I was making my private sgf extension to solve this, I made the
> Label/Pointer tags but the pointer tag also contained information
> about transformation (axes, colors), so the dictionary joseki didn't
> need to repeat any variations with same logic.

I have not thought about this before, but I have no objections (but, see
below.)


The issue I am currently thinking about is the exact semantics of TP[]
and its implications. If we think of it as a macro that actually
copies/expands to the subtree denoted by N[] we "mostly" know how to
display the result.

But can we really copy all properties as is? E.g. if transformations are
allowed (axes, colors) what about comments like "in the upper left
corner" or annotation properties like GB/GW (good for black/white), just
to name a few issues.

Also, how does navigation behave and look like? Is it really identical
to a file that had its TP[]-macros expanded and replaced? Or does the
standard have to mention some special issues? (Recall the mess about
displaying variations as siblings or children.)

Important issue: what about editing/writing such a file? Suppose I don't
want to write the expanded tree, but the compressed macro-version. How
does the user control whether changing a node affects both subtrees
(N[]'s and TP[]'s) or only one of them (if the display looks exactly the
same)? How do I tell the SGF program to create a new TP[] or
remove/split/copy the tree denoted by N[]. I think the GUI issues (and
therefore the usability of this extension) are not to be underestimated.


Your opinions?


Apart from TP[] I really think we should have something like a real
link, where the semantics mean jumping to another node, not copying its
subtree. The semantics for links, the navigation, display, etc. are much
easier for links than for macro-like TP[].

/Arno

On Wed, Jul 30, 2008 at 9:46 AM, Arno Hollosi <ahollosi@...> wrote:

> The issue I am currently thinking about is the exact semantics of TP[] and
> its implications. If we think of it as a macro that actually copies/expands
> to the subtree denoted by N[] we "mostly" know how to display the result.

I'm also interested in the exact semantics of TP[] with respect to ko
calculations. If two subtrees have different previous board positions
(as opposed to the same previous board positions in a different order)
their children may have different legal moves. This is mainly an issue
when considering the endgame rather then the opening book, of course.

> Important issue: what about editing/writing such a file? Suppose I don't
> want to write the expanded tree, but the compressed macro-version. How does
> the user control whether changing a node affects both subtrees (N[]'s and
> TP[]'s) or only one of them (if the display looks exactly the same)? How do
> I tell the SGF program to create a new TP[] or remove/split/copy the tree
> denoted by N[]. I think the GUI issues (and therefore the usability of this
> extension) are not to be underestimated.

The SGF file format has succeeded by being a file format. Specifying
the way a program interacts with a user should be left up the program
author. I hope/imagine that most programs will make the presence or
absence of N[]'s and TP[]'s invisible to the user.

cheers
stuart

> The SGF file format has succeeded by being a file format. Specifying
> the way a program interacts with a user should be left up the program
> author.

Sometimes the GUI or interaction with the user has to be addressed in
the specification. That's the reason why we have a ST[] property. I am
just trying to find out if someone thinks there may be issues.

> I hope/imagine that most programs will make the presence or
> absence of N[]'s and TP[]'s invisible to the user.

I think that depends. When I am editing I'd like to know whether my
change is copied multiple times within the SGF tree or is a local
modification only. Furthermore, when I know that the subtree is copied
(and maybe transformed) I know that I should avoid comments like "upper
right corner" or "B5 should be at C17".

/Arno

> Bill wrote:
> TP is fine with me, but the requirement that it's target N[] appear
> before the matching TP[] is strange. Why have it?
>
> I tend to agree with Bill. I think that most of us agree that we
> should keep the SGF game tree a directed, non-cylic graph. However,
> as Bill pointed out, as soon as someone reorders variations the
> TP[] and subtree denoted by N[] would have to change place as well.

Not only that, but if the target node is pointed to by more than one
TP, you would have to make sure N comes before all of them.

> That is not exactly easy on programmers either.

I guess it depends on what the program does. It wouldn't necessarily
be a problem if the program deals with an expanded tree instead of the
compressed file.

The purpose of TP is to make generation of a file manually with a text
editor a little less tedious. If you have software that can save a
file in SGF format from a game tree generated with a GUI, ***THERE
SHOULD BE NO NEED TO DEAL WITH TP.*** So as far as I can tell, the
problem of reordering a file is based on the assumption that this
reordering is done manually, with a text editor. Just out of
curiosity, what would be gained by doing this?

As was pointed out in another post, GUI issues and file loading and
saving issues should be regarded as independent of each other.

> Michal wrote:
> when I was making my private sgf extension to solve this, I made
> the Label/Pointer tags but the pointer tag also contained
> information about transformation (axes, colors), so the dictionary
> joseki didn't need to repeat any variations with same logic.

Sorry, I missed your point in my earlier post. But I'm not sure what
scope you want this extra information to apply to. Does a joseki
dictionary file concern itself entirely with one corner of a 19x19
board? Or are you talking about referring to another corner of the
board which was explored earlier in the same file? If the full-board
positions are not effectively identical by axis/color transformation,
there could be a problem when software tries to evaluate this
position. Joseki battles are of course not independent of each other.

Also, do you generate this dictionary with a GUI or with a text editor?

> ... If we think of it as a macro that actually
> copies/expands to the subtree denoted by N[] we "mostly" know how
> to display the result.
>
> But can we really copy all properties as is? E.g. if
> transformations are allowed (axes, colors) what about comments like
> "in the upper left corner" or annotation properties like GB/GW
> (good for black/white), just to name a few issues.

Whether TP is a macro call or a link, these same issues would crop up.
GB versus GW could be automatically taken care of. You could just
ignore all comments in an copied subtree, or not copy them in the
first place if there is any kind of axis/color change.

> Important issue: what about editing/writing such a file? Suppose I
> don't want to write the expanded tree, but the compressed macro-
> version. How does the user control whether changing a node affects
> both subtrees  (N[]'s and TP[]'s) or only one of them (if the
> display looks exactly the same)?

Under what circumstances would you want to change one subtree but not
the other? I'm not asking about axis/color changes here. Why should
the user have this option to make the subtrees of the compressed file
different?

> How do I tell the SGF program to create a new TP[] or
> remove/split/copy the tree denoted by N[]. I think the GUI issues
> (and therefore the usability of this extension) are not to be
> underestimated.

Is this SGF program a text editor or a GUI? If it's a GUI which can
save files in SGF format, why would you want to deal with TP calls?

> The semantics for links, the navigation, display, etc. are much
> easier for links than for macro-like TP[].

I don't mean to require programmers to treat TP one way or the other.
The SGF definition should work for any way the programmer wants to use
it. I just used the "macro" analogy as a way of explaining what I was
talking about. Sorry if I wasn't clear about that.

syeates said:
> I'm also interested in the exact semantics of TP[] with respect to
> ko calculations. If two subtrees have different previous board
> positions (as opposed to the same previous board positions in a
> different order) their children may have different legal moves.
> This is mainly an issue when considering the endgame rather then
> the opening book, of course.

My proposal is that the positions need not be identical as long as
every move, position evaluation, and everything else that gets copied
or linked to is equally valid in both situations. If there are
"irreconcilable differences" then they should be different subtrees.
In a ko situation, and perhaps for the most part in Go, different
battles are arguably not truly independent of each other, because one
usually is more important than the others at any given point. So TP
calls might not be appropriate if the subtree in question is large.

Also I recommend you take a look at my LO[] proposal (in a separate
thread) for dealing with LOcal battles.

Hi,

I think idea is good, but there are several problems with initial
proposal that need some resolution:

1. N property

Property N is inteded to be user-visible and editable name for node. So
duplicated exists, and preventing user from giving same node name twice
in file is bad solution.

I think it would be better to define some other property (for example
AC[ int ]), that could be used to reference node in all cases it is
machine referenced. (I have planned such id for other uses, but I would
prefer having single standard one.)

Proposal: define new invisible-to-user property AC[nro]  for node, and
use that to reference to node.

2. Loops

I would like to say loops are not allowed, bu:

If we consider following chain of editing
1. create file which contain new properties;
2. edit file with editor that does not know new properties (they
continue to exists a long time).
3. Open file again with editor understanding new properties.

Now at step 3 we may have loops, as at step 2 there is no way new rules
are followed, even though editor was SGF4-compliant.
So I think we should not add new restrictions to what is valid file.
Loops will exists, and programs need to be able to deal with them.
Also nodes with TP might not be leaf nodes anymore.

So in essence new limitations would invalidate SGF4-compliance of
program, which is exactly what standards are supposed to prevent.

Proposal: No new restrictions to SGF4 file structure.

3. Link interpretation

I think we should have 2 types of links:
- Just jump to location, as user would have moved to location (so TP
does not need to be leaf)
- Continue with referenced subtree, as proposed here.

t.
Lauri Paatero


Arno Hollosi wrote:
>> The SGF file format has succeeded by being a file format. Specifying
>> the way a program interacts with a user should be left up the program
>> author.
>>
>
> Sometimes the GUI or interaction with the user has to be addressed in
> the specification. That's the reason why we have a ST[] property. I am
> just trying to find out if someone thinks there may be issues.
>
>
>> I hope/imagine that most programs will make the presence or
>> absence of N[]'s and TP[]'s invisible to the user.
>>
>
> I think that depends. When I am editing I'd like to know whether my
> change is copied multiple times within the SGF tree or is a local
> modification only. Furthermore, when I know that the subtree is copied
> (and maybe transformed) I know that I should avoid comments like "upper
> right corner" or "B5 should be at C17".
>
> /Arno
>
> ------------------------------------
>
> SGF spec: http://www.red-bean.com/sgf/
> Contact: Arno Hollosi <ahollosi@...>Yahoo! Groups Links
>
>
>
>
>

The big issue I see to resolve is the semantic of the link.  I see
several possibilities discussed in the thread, and I'm going to try to
restate the concisely:

1) Identity
The node referenced is the identical position as the node with the
link.  Presumably they have the same position *after* each has applied
any moves.

2) Duplicate Sub-tree
The nodes that follow the referenced node are equally applicable to
the position at this node.

3) Jump
The referenced node should be offered to the user as a possible follow
on to this node. The position at the referenced node could be wholly
different.

Identity seems to have a number difficulties in definition and
resiliency:
	 - just how "identical" do the positions have to be?
	 - i.e.: same number of prisoners?  or just same delta?  passes?
	 - same comments?  markup?
	 - what about differing in only a tenuki move?

Duplicate Sub-tree has the downside that now each node can no longer
be transformed into a single game position, since the state of the
game will depend on if you took a jump to get there.

Jump gets around these difficulties, but at the expense of being a
different kind of navigation that the existing options.

One way to approach the choice is to look at the use cases and see
which admits the most flexibility.  The use cases I see are:

A) Joseki dictionary desire to not replicate whole sub-trees when a
position can be arrived at via two different move sequences.  This use
case could be broadened to include color and spacial transposition, or
differing in tenuki.

B) Tsumego.  Like joseki, a desire to not replicate whole sub-trees.
Here the game state identity requirement might be more rigorous.
GoProblems.com already does this automatically by identifying
identical positions in the tree: If one has follow on nodes and the
other doesn't -- they are "spliced".

C) General annotation need to have variant lines or main lines
reference other lines within the tree.

It seems to me that the Jump semantic supports all of these.  If there
is a need for identical position at both ends, it is easy enough for
the creation tool to enforce that.

The only thing that then needs to be addressed is the intended user
navigation: Is it displayed as an option? Is it followed
automatically?  One could support both by this rule:  If the LI[]/TP[]
property appears as the only property in a node, then the link should
be immediately followed, replacing the game state with the state at
the target node.  On the other hand, if there are any other
properties, including just a comment (even an empty comment!), then
the user is offered link as a sort of 'remote variation', and can
choose, as they would choose a variant, to continue in that part of
the tree.

	 - Mark

Dear all,

sorry about the recent spam activity on this list. I only noticed a
single email today, as my spam filter got rid of everything else.
If you find that spam is distributed through this list, please do not
hesitate to contact me, as I may be not aware of it.

regards,
/Arno

Hex uses the swap rule, also known as the pie rule or one-move equalization. WRT
implementation, there are two ways to do it. Here are the official explanations
of these two distinct moves:

# swap-sides - the player elects to swap sides with his opponent; if he was
playing Black he now plays White, and vice versa.
# swap-pieces - the player elects to swap pieces with his opponent - all of
Black's pieces are coloured White, and White's pieces are coloured Black. Then
the entire board is reflected in the long diagonal axis. (Example: ;B[c1]
;W[swap-pieces] - a3 is now white; Black to play).

Let's take a closer look at that second option.

;B[c1] ;W[swap-pieces] - a3 is now white; Black to play

The player who started the game as black is still black.
The player who played "swap-pieces" is still white.

So, no one has swapped piecES. Only one PIECE has been swapped. All of black's
pieces are still black, except for that first token. NONE of white's pieces are
colored black.

This move should be re-labeled "swap-firstpiece" or "swap-piece" or something
like that. The explanation should be clarified. At least, that's how I see it.
Any objection?

BTW there is now a Java app which saves games in SGF:

http://benzene.sourceforge.net/ Look for the "HexGui" link.

Hello all,

I have some questions about numbering. May be the answers are
somewhere at http://www.red-bean.com/ but i didn't find them.

1) What happens after a setup position?

Consider the following sample:
(;FF[4]GM[1]SZ[19]
;B[pd]
(;W[qf];B[nc])
(;AB[pj];W[nc]))

What is the number of W[nc]? 1?

It seems natural that numbering restarts after a setup position resumes.

2) What happens after a MN?
(;FF[4]GM[1]SZ[19]
;B[pd];W[qf]MN[1];B[nc])

What is the number of B[nc]? 2 or 3?

It seems natural that numbering of all moves following a MN changes
accordlingly. But the MN spec say nothing at all about the followers!

3) A figure starts when a FG is found. What is the number of the
first move after the FG? 1 or is its number calculated from the
previous move number?
(;FF[4]GM[1]SZ[19]
;B[pd];W[qf]
(;B[nc];W[rd];B[qc];W[qi])
(;FG[0]B[nd]))

B[pd] and W[qf] have no numbering since they are before the FG, but
what is the number of B[nd]? 1 or 3?

Both seem natural to me.

Best regards, François Mizessyn

>

That sounds good. Paul would be key. As I understand it,
Lenart is rapidly absorbing our perspectives about
Whittaker functions and Sara is also natural. However we
could use Ben instead of me. (I'm willing to do it but
I'm pointing out this possibility.)

Dan