newsml : Messages : 1-30 of 2476

While testing a proposed standard DTD against sample stories is a good idea and is educational for all participants in the process, I would strongly caution against using it as the method to develop the DTD. It's akin to pouring the foundation to a 4 room house, knowing full well you're going to build a hotel there someday. Just as soon as you start throwing more complex problems at the DTD, then solve them by a series of patches, you are already too far down a dead-end road. The underlying tag architecture must be designed at the start to be highly extensible (which does not mean "patchable"). Design the DTD, then test it by running both simple and complex content through it, not the other way around. Despite all its warts, NITF has been able to take the 10's of thousands of stories we threw at it when I was at NewsView.

thx

Glenn Cruickshank
kpmg
Digital Asset Management
Communications & Content
(801) 201-9024 (cell)
(435)843-8701 (home office/fax)
(801)237-1471 (Salt Lake City office)

-----Original Message-----
From: Laurent LE MEUR [mailto:laurent.lemeur@...]
Sent: Monday, January 03, 2000 10:57 AM
To: newsml@egroups.com
Subject: [newsml] newsML samples - simple story

(note : sorry for the first mail posted on iptc-2000 newsgroup : seems that some characters aren't accepted by eGroup web posting facility. The newsml newsgroup seems more adequate at this step)

Following Heathrow meeting, and starting from requirements and from AFP experience (an internal XML multimedia format named XAFP), I tried to imagine what could be the newsML structure.

Since the newML recommendation has to be released in march, I only see one way to go :
- focus on structure,
- discuss metadata on a separate thread,
- begin with simple samples,
- discuss those samples until agreement,
- go on with more complicated issues,
- *finish* with the DTD or schema.

Alan Karben NewsML site is very interesting, but gives the impression that the development is done. The XMLNews tutorial (www.xmlnews.org/docs/story-tutorial.html) way of describing tags seems to me more adequate for a discussion.

In the following, lets assume a "newsml" namespace ; the newsitem notation <item> is then equivalent to <newsml:item>. Alternative : use <newsitem>, which is Jo Rabin proposal.

To keep thing simples, xml declaration (<?xml-version...>) is not represented in the samples.

Proposals on Newsitem element :
- It has a 'type' attribute, of values 'text', 'photo', 'graphic' 'audio', video', 'multimedia' ... that reflects the user (or provider) point of vue about the type of the document, not a pure structural information. The list should be left opened.
- It has an 'uno' attribute, a globally unique name equivalent to the IIM V4 UNO.
- It includes a header <head>, that contains metadatas, like editorial informations (essentially taken from IIM Record 1 and 2). Handling metadatas (that have a different lifetime) will be described later, and can be kept in another header (easier to handle in a specific header than in separate tags).
- It includes one or more named parts (I prefer the name 'object') (<text> ...), that represent actual data. Those objets can include textual data (by value) or reference binary (or textual) data (with an URI (URL or URN)). These attributes are structural only : objects don't possess editorial attributes.
-object elements can be <texte>, <photo>, <graphic>, <audio>, <video>, <note> (a simple message), <opaque> (an unknown CLOB or BLOB).

Exemple 1 : a short text

<item uno='AFP_TX_19991203_555' type='text' xml:lang='en-us'>
<head urgency='2'>
<source>AFP</source>
<date-created type="iso" val="19991231T120000Z"/>
<headline>Text headline</headline>
<byline>llm</byline>
<location>
   <city>PARIS</city>
   <state type="REG" val="IDF">IDF</state>
   <country type="iso" val="FRA">FRANCE</country>
</location >
</head>
<text>
<p>Text body</p>
</text>
</item>

Notes :
- use of 'uno' and not 'itemid' (Reuters proposal) keeps alive the historical link with IIM.
- The uno of each item is an ID attribute : it can be used for example in a XPointer selection (to use the couple source/itemid makes it more difficult).
- the dateline, present in J Rabin an A Karben samples, is considered as a presentation of informations found in 'source', 'date-created' and 'location'.
- for dates, iso format could become the NewsML standard.
- precise element names can be choosen during the discussion (for exemple <text> could become <article> or <story>).

Alternatives :
- The <text> object could be represented as <object media='text'>, an so on for other objects types : it would be simpler to add a new object type (if the media list is 'open'), but structural attributes (like 'words' for a word count in a text, or 'height' and 'width' for an image) wouldn't apply to any media, resulting in a multiplicity of attributes for the <object> element in the final DTD, that could be error prone.

Also, adding a new object type can imply a new attribute (for example <text words='200'>), and so imply a DTD modification.

To compare with a J Rabin sample (XN-5) :
-----------------------------------------
Few differences : metadatas are less directely taken from IIM V4 ; 'title' and 'headline' are identical ; the dateline presentation is provider dependent ; there is no information about the type (text, photo) of the document.

<newsitem itemid="19990526000012" date="1999-05-28" publisher="moonlite.tibcofinance.com/newsml" xml:lang="en-us"
parts="1" revision="0">
<title>DTDs for Today's world</title>
<headline>DTDs for Today's world</headline>
<byline>by Ernest Dull</byline>
<dateline>Palo Alto, May 26 (Reuters)</dateline>
<text>
<p>Many people wonder why DTDs are necessary</p>
<p>Blah Blah Blah</p>
</text>
<copyright>(c) 1999 Dull, Ernest Tech Mags Inc.</copyright>
</newsitem>

To compare with a A Karben sample (on his site) :
-----------------------------------
No big difference at this step ; the 'news-envelope' is, if I remember well, a newsitem possessing handling metadata ; 'publisher' doesn't keep the difference between 'source' and 'credit' ; iso date is the standard ; a news-folder is present, but has no meaning in this case ; 'title' and 'article/headline' are usually identical ; no newsitem 'type'.

<news-envelope
id="AFP_TX_19991203_555"
publisher="AFP"
date="19991231T120000"
>
<news-folder relationship="complements">
<title>Text headlines</title>
<article>
<headline>
   <main-headline>Text headline</main-headline>
</headline>

<byline>llm</byline>

<p>Text body</p>
</article>
</news-folder>

</news-envelope>

To compare with a XMLNews sample :
----------------------------------

- XMLNews-Meta uses a 'deliberately simple and flat' RDF vocabulary for metadata description (is RDF really useful in this simple case ?), similar to the other samples :

<xn:Resource xmlns:xn="http://www.xmlnews.org/namespaces/meta#">
<xn:resourceId>AFP_TX_19991203_555</xn:resourceId>
<xn:sourceName>AFP</xn:sourceName>
<xn:publicationTime>19991231T120000</xn:publicationTime>
<xn:language>en</xn:language>
</xn:Resource>

- XMLNews-Story is a subset of NITF : structure isn't perfectly clear (head, body.head are usually identical, hedline/hl1 is a funny syntax), some tags (as byline, location) could be considered as metadata ; powerful features are essentially found in body.content markup, not showed here :

<nitf>
<head>
<title>Text title</title>
</head>
<body>
<body.head>
   <hedline>
    <hl1>Text title</hl1>
   </hedline>
   <byline>
    <bytag>llm</bytag>
   </byline>
   <dateline>
    <location>PARIS, IDF, FRANCE</location>
    <story.date>Friday December 31 1999 12:00</story.date>
   </dateline>
</body.head>
<body.content>
   <p>Text body</p>
</body.content>
</body>
</nitf>

I feel that we should quickly try to reach an agreement on such a simple story.

On the next step, we could describe a simple photo or video, then go on with multimedia newsitems.

To Post a message, send it to: newsml@eGroups.com To Unsubscribe, send a blank message to: newsml-unsubscribe@eGroups.com

eGroups.com Home: http://www.egroups.com/group/newsml
www.egroups.com - Simplifying group communications

#29

From: "Cruickshank, Glenn" <gcruickshank@...>
Date: Tue Jan 4, 2000 4:11 pm
Subject: Re: One Standard for News in the New Millennium

gcruickshank@...

I would still make the argument that other users of a news standard are very
wary of adopting a news markup language which was developed specifically for
one company without peer review.  When we adopted NITF for the NewsView
product line, we weighted both NITF and XML-News, and selected NITF because
of the peer review.  The industry (and I) got burned in the early '90's by a
vendor-created extension to JPEG, the AXS format, for those who remember
that fiasco, only to have the vendor drop out of the business within a year.

And this is not to say that we shouldn't look at some of the features in
XMLNews for adoption into NewsML, in the same way that XMLNews drew on NITF.

best
gc

Glenn Cruickshank
KPMG
Digital Asset Management
Communications & Content
(801) 201-9024 (cell)
(435)843-8701 (home office/fax)
(801)237-1471 (Salt Lake City office)

David

You could equally well argue that there is the need for further development
of
NITF, the IIM and other news standards that are presently used far more
widely
than XMLNews is. However the users of those standards have taken the view
that
this is not to happen and that the needs of the community will be better
served
by contributing their efforts to NewsML.

All best
Jo

Jo Rabin writes:

  > A couple of weeks ago on your list, you asked for comments on what
  > phase 2 of XML News should be and you stated your long-term goal of
  > converging with IPTC's NewsML.
  >
  > But why not make your phase 2 NewsML? You suggest that NewsML will
  > be a long time in the coming, yet as I mentioned some weeks ago the
  > target for an initial agreed cut of NewsML is April next year. It
  > is well beyond the "requirements gathering" phase you suggest. A
  > public draft of the requirements spec is available for comment
  > (http://www.iptc.org/STA9912.pdf) and the NewsML section of the
  > IPTC site (http://www.iptc.org/NMLIntro.htm) has been updated with
  > a number of additional background papers).
  >
  > It would be more productive for the effort that would be commited
  > to a phase 2 of XML News to be committed to NewsML. In this spirit,
  > comments on the public draft requirements are welcome on the NewsML
  > mailing list (NewsML@egroups.com).

That is wonderful news -- I think that everyone will be very happy
when there is a single, final, agreed-upon standard for news
delivery.

In the meantime, though, since there are a number of parties using
XMLNews (both internally and externally), and since it will probably
take at least a few months for NewsML implementations to spool up once
the initial alpha version of NewsML is ready (and major implementors
will probably wait for the final version), I think that there's still
value in improving XMLNews itself to fill in the gap.  At the very
least, our implementation experience will be valuable for the
development of NewsML from the initial cut in April to the final
version some time later.

All the best,

David

--
David Megginson                 david@...

*****************************************************************************
The information in this email is confidential and may be legally privileged.
It is intended solely for the addressee. Access to this email by anyone else
is unauthorized.

If you are not the intended recipient, any disclosure, copying, distribution
or any action taken or omitted to be taken in reliance on it, is prohibited
and may be unlawful. When addressed to our clients any opinions or advice
contained in this email are subject to the terms and conditions expressed in
the governing KPMG client engagement letter.
*****************************************************************************

Reply | Forward | Messages in this Topic (5)

#28

From: David Megginson <david@...>
Date: Mon Jan 3, 2000 8:59 pm
Subject: Re: newsML samples - simple story

david@...

Karben, Alan writes:

  > I agree with David that it's often great to build on the works of
  > others rather than reinventing wheels.
  >
  > That said, XML itself is the main wheel that I think we should
  > build upon.  XML is a success because individuals and industries
  > have found that creating tagsets that are custom-fit to their
  > information will make that data more useful and reusable. So making
  > our own DTD for news content (as opposed to just blessing XHTML) to
  > me seems quite reasonable.

It depends, really, on fit -- NITF itself built on top of HTML to the
point that unmodified NITF documents can often display almost
perfectly in an HTML browser.  NITF, of course, has many shortcomings
(non-extensibility, rather arbitrary and limited inline markup, etc.)
but none of these is a result of its being based on HTML.

So, the question is why are the NAA and IPTC leaving NITF behind as a
news text format?  If it's because the basic structure (head, body,
paragraphs, tables, lists, etc.) is flawed, then it makes sense to
forget about XHTML and start from scratch; if it's because the
industry-specific stuff (codes, metadata, etc.) is flawed, then why
redo the structural stuff as well?

  > More comments interspersed below:

  > > 2. RDF for serializing data.
  >
  > My opinion is that using RDF for NewsML is a very reasonable
  > idea. The work done on this in XML News as well as the efforts
  > presented by IPTC members in Amsterdam should be studied closely as
  > good cases in point.

The XMLNews RDF format is deliberately stupid, because at the time
there was only one RDF library (in Java) and it wasn't up to major
production use, so it had to be possible to process XMLNews-Meta at
the raw XML level rather than at the higher RDF level.  Now that there
are quite a few good RDF libraries out there, it would make sense to
try something a little more sophisticated.

I'd suggest looking at RPMfind, which is heavily used in the Linux
world, as an example of how well RDF can perform under fairly high
demand.  More generally, it's worth watching the EPICS (book industry)
and INDECS (intellectual property) projects, which are planning to do
very sophisticated and large-scale data exchange with RDF.

  > > 3. XHTML as a base document type for text stories.
  >
  > My opinion is that this would be a very bad idea. The escape
  > hatches of XHTML formatting codes will murder NewsMLs attempts at
  > simplicity and structure.

There are different ways to reuse XHTML.  One of them is inheritance:
start with XHTML (the strict version, without all the escape hatches)
and define news-industry-specific markup together with an RDF metadata
vocabulary for metadata in the header; the other is composition: start
with a new base document type but reuse large chunks of the HTML
markup.

In the first way (inheritance), a NewsML textual news story would
simply be an XHTML document with extra element types and attributes,
something like this:

   <html xmlns="http://www.w3.org/1999/xhtml"
         xmlns:nm="http://www.newsml.org/ns/"
         nm:id="http://www.reuters.com/ids/2000010312345">
   <head>
    <title>Man bites Dog</title>
    <rdf:RDF ...>
     [metadata]
    </rdf:RDF>
   </head>

   <body>
    <h1>Man bites Dog</h1>
    <nm:byline>John Smith</nm:byline>

    <p>Today a man bit a dog.  <nm:person code="12345">Bill
     Clinton</nm:person> denied involvement.</p>

    <img src="bite.jpg" nm:format="rgb"/>
   </body>

   </html>

In the second way (composition), a NewsML textual news story would
simply reuse well-known HTML markup where appropriate but impose its
own high-level structure, like this:

   <nm:item xmlns="http://www.w3.org/1999/xhtml"
         xmlns:nm="http://www.newsml.org/ns/"
         nm:id="http://www.reuters.com/ids/2000010312345">
   <nm:header>
    <title>Man bites Dog</title>
    <rdf:RDF ...>
     [metadata]
    </rdf:RDF>
   </nm:header>

   <nm:copy>
    <h1>Man bites Dog</h1>
    <nm:byline>John Smith</nm:byline>

    <p>Today a man bit a dog.  <nm:person code="12345">Bill
     Clinton</nm:person> denied involvement.</p>

    <img src="bite.jpg" nm:format="rgb"/>
   </nm:copy>

   </nm:item>

The latter gives you the advantage of very fine control over the
structure, but with the disadvantage of having to define more markup.
Another, middle path is to use the first example but with a
well-defined, restricted profile of XHTML.

It would even be possible to use RDF itself as the top-level format:

   <rdf:RDF ...>
    <nm:headline>Man bites dog</nm:headline>
    <nm:byline>John Smith</nm:byline>
    <nm:subject>
      <nm:Person rdf:about="http://www.reuters.com/ids/people/12345">
        <nm:name>Bill Clinton</nm:name>
      </nm:Person>
    </nm:subject>
    <nm:content rdf:parseType="Literal">
     <p>Today a man bit a dog. ...</p>
     <img src="bite.jpg/>
    </nm:content>
   </rdf:RDF>

In this approach, the copy (if any) is just another metadata property,
and can also be stored out of line:

   <nm:content rdf:resource="dogbite.xml"/>

  > And the "extensions to HTML to make it suitable for textual news
  > delivery" will do away with the "authoring tools already out there"
  > issue.

Correct -- in fact, simply choosing XML will do away with the ability
to use existing HTML authoring tools, though XML-specific authoring
tools (like XMetaL and WordPerfect) will work just fine with any
XML-based format.  The advantage of reusing XHTML (either by
inheritance or by composition) is that there is no need to redefine,
document, and teach people basic structures like paragraphs, lists,
and tables that HTML already defines just fine.

  > Also, NewsML will be equipped to "carry along" news marked-up in
  > any format.  So if some publishers wants to use XHTML, then they're
  > welcome to, and can still take advantage of NewsML's rich and
  > consistent metadata features. And if they want to add NewsML's
  > small list of textual-markup tags to the XHTML, then Namespaces
  > will obviously let them do so.

I agree in both cases -- as a matter of fact, there would be great
value in layering the NewsML specs, by defining the reusable markup
separately and then building the document type (XHTML or non-XHTML) on
top of that.  In other words, you'd have short, five-page specs with
titles like the following:

1. NewsML: Elements and attributes for inline markup (v.1.0)
2. NewsML: Elements and attributes for structural markup (v.1.0)
3. NewsML: RDF classes and properties for news metadata (v.1.0)
4. NewsML: Container for information exchange (v.1.0)

In this example, 1 and 2 are independent, 3 depends somewhat on 1 and
2, and 4 depends on all three.

  > My experience in marking up news stories, though, is that (assuming
  > you use Content-based, not Formatting-based tags) less is more.

I agree again -- if NewsML did use XHTML, a restricted profile would
be a very good idea.  One reason that I considered NITF unusable
originally is that it was far too permissive, allowing stuff like

   <p>This is a tagged paragraph.</p>

   This is an untagged paragraph.

   <p>This is another tagged paragraph.</p>

Ouch!  Anyone who's tried to process SGML or XML (as Alan and I have)
knows what a nightmare that kind of thing is.

All the best,

David

--
David Megginson                 david@...
            http://www.megginson.com/

#27

From: "Karben, Alan" <Alan.Karben@...>
Date: Mon Jan 3, 2000 7:49 pm
Subject: Re: newsML samples - simple story

Alan.Karben@...

I agree with David that it's often great to build on the works of others
rather than reinventing wheels.

That said, XML itself is the main wheel that I think we should build upon.
XML is a success because individuals and industries have found that creating
tagsets that are custom-fit to their information will make that data more
useful and reusable. So making our own DTD for news content (as opposed to
just blessing XHTML) to me seems quite reasonable. More comments
interspersed below:

> -----Original Message-----
> From: David Megginson [mailto:david@...]
> Sent: Monday, January 03, 2000 2:15 PM
> To: newsml@egroups.com
> Subject: [newsml] newsML samples - simple story
>
> Things can move quickly if NewsML takes advantage of existing
> standards rather than reinventing them.  We have the following already
> available:
>
> 1. Namespaces (as Laurent points out), for clean extensibility.

Certainly the use of Namespaces will allow NewsML to coexist with other
formats, such as RDF.

> 2. RDF for serializing data.

My opinion is that using RDF for NewsML is a very reasonable idea. The work
done on this in XML News as well as the efforts presented by IPTC members in
Amsterdam should be studied closely as good cases in point.

> 3. XHTML as a base document type for text stories.

My opinion is that this would be a very bad idea. The escape hatches of
XHTML formatting codes will murder NewsMLs attempts at simplicity and
structure. And the "extensions to HTML to make it suitable for textual news
delivery" will do away with the "authoring tools already out there" issue.

Also, NewsML will be equipped to "carry along" news marked-up in any format.
So if some publishers wants to use XHTML, then they're welcome to, and can
still take advantage of NewsML's rich and consistent metadata features. And
if they want to add NewsML's small list of textual-markup tags to the XHTML,
then Namespaces will obviously let them do so. My experience in marking up
news stories, though, is that (assuming you use Content-based, not
Formatting-based tags) less is more.

> 4. URIs as globally-unique identifiers.

Agreed.

Also, to add to some of Laurent's comments, I think that the tutorial method
taken at xmlnews.org is indeed a great way to lead someone through a DTD. I
also liked the XML News logo -- it'll be a necessity to "brand" NewsML with
something equally snazzy!

Regards,
Alan.

        Alan Karben
                       Director of Document Architecture
             The Wall Street Journal Interactive Edition

        karben@...   phone: 212 416 2975
        http://wsj.com                 fax: 212 416 3291

#26

From: David Megginson <david@...>
Date: Mon Jan 3, 2000 7:15 pm
Subject: newsML samples - simple story

david@...

Laurent LE MEUR writes:

  > Exemple 1 : a short text
  >
  > <item uno='AFP_TX_19991203_555' type='text' xml:lang='en-us'>
  >  <head urgency='2'>
  >   <source>AFP</source>
  >   <date-created type="iso" val="19991231T120000Z"/>
  >   <headline>Text headline</headline>
  >   <byline>llm</byline>
  >   <location>
  >    <city>PARIS</city>
  >    <state type="REG" val="IDF">IDF</state>
  >    <country type="iso" val="FRA">FRANCE</country>
  >   </location >
  >  </head>
  >  <text>
  >   <p>Text body</p>
  >  </text>
  > </item>

Things can move quickly if NewsML takes advantage of existing
standards rather than reinventing them.  We have the following already
available:

1. Namespaces (as Laurent points out), for clean extensibility.
2. RDF for serializing data.
3. XHTML as a base document type for text stories.
4. URIs as globally-unique identifiers.

Other people have already done the work of documenting these standards
and writing software tools for them, and if NewsML is going to succeed
(remember that almost all standards efforts, even those with
substantial industry backing, ultimately languish or fail), it should
use the same layered model that worked with the Internet.

Essentially, NewsML needs to add the following layers to existing
standards:

a. Define a Namespace containing extensions to XHTML to make it
    suitable for textual news delivery.

b. Define a Namespace containing RDF classes and properties for
    defining news metadata.

c. Create a top-level design document showing how the two work
    together.

This approach will massively simplify the work of implementors as
well, since they can rely heavily both on existing tools (there are
four or five RDF libraries now and dozens of tools with Namespace
support) and, more importantly, on existing training materials and
documentation.

Many standards efforts suffer from the Not Invented Here syndrome and
try to do too much.  Of course there are problems with Namespaces,
RDF, and XHTML, just as there are problems with IP and TCP, but
imagine how unlikely the Web's success would have been if HTTP had
required custom hardware and software for transport rather than
working on top of TCP/IP.

All the best,

David

--
David Megginson                 david@...
            http://www.megginson.com/

#25

From: "Laurent LE MEUR" <laurent.lemeur@...>
Date: Mon Jan 3, 2000 6:04 pm
Subject: newsML samples - simple story - bis

laurent.lemeur@...

(note : sorry for the previous mail badly sent in html format).

Following Heathrow meeting, and starting from requirements and from AFP
experience (an internal XML multimedia format named XAFP), I tried to
imagine what could be the newsML structure.

Since the newML recommendation has to be released in march, I only see one
way to go :
- focus on structure,
- discuss metadata on a separate thread,
- begin with simple samples,
- discuss those samples until agreement,
- go on with more complicated issues,
- *finish* with the DTD or schema.

Alan Karben NewsML site is very interesting, but gives the impression that
the development is done. The XMLNews tutorial
(www.xmlnews.org/docs/story-tutorial.html) way of describing tags seems to
me more adequate for a discussion.

In the following, lets assume a "newsml" namespace ; the newsitem notation
<item> is then equivalent to <newsml:item>. Alternative : use <newsitem>,
which is Jo Rabin proposal.

To keep thing simples, xml declaration (<?xml-version...>) is not
represented in the samples.

Proposals on Newsitem element :
- It has a 'type' attribute, of values 'text', 'photo', 'graphic' 'audio',
video', 'multimedia' ... that reflects the user (or provider) point of vue
about the type of the document, not a pure structural information. The list
should be left opened.
- It has an 'uno' attribute, a globally unique name equivalent to the IIM V4
UNO.
- It includes a header <head>, that contains metadatas, like editorial
informations (essentially taken from IIM Record 1 and 2). Handling metadatas
(that have a different lifetime) will be described later, and can be kept in
another header (easier to handle in a specific header than in separate
tags).
- It includes one or more named parts (I prefer the name 'object') (<text>
...), that represent actual data. Those objets can include textual data (by
value) or reference binary (or textual) data (with an URI (URL or URN)).
These attributes are structural only : objects don't possess editorial
attributes.
-object elements can be <texte>, <photo>, <graphic>, <audio>, <video>,
<note> (a simple message), <opaque> (an unknown CLOB or BLOB).

Exemple 1 : a short text

<item uno='AFP_TX_19991203_555' type='text' xml:lang='en-us'>
  <head urgency='2'>
   <source>AFP</source>
   <date-created type="iso" val="19991231T120000Z"/>
   <headline>Text headline</headline>
   <byline>llm</byline>
   <location>
    <city>PARIS</city>
    <state type="REG" val="IDF">IDF</state>
    <country type="iso" val="FRA">FRANCE</country>
   </location >
  </head>
  <text>
   <p>Text body</p>
  </text>
</item>

Notes :
- use of 'uno' and not 'itemid' (Reuters proposal) keeps alive the
historical link with IIM.
- The uno of each item is an ID attribute : it can be used for example in a
XPointer selection (to use the couple source/itemid makes it more
difficult).
- the dateline, present in J Rabin an A Karben samples, is considered as a
presentation of informations found in 'source', 'date-created' and
'location'.
- for dates, iso format could become the NewsML standard.
- precise element names can be choosen during the discussion (for exemple
<text> could become <article> or <story>).

Alternatives :
- The <text> object could be represented as <object media='text'>, an so on
for other objects types : it would be simpler to add a new object type (if
the media list is 'open'), but structural attributes (like 'words' for a
word count in a text, or 'height' and 'width' for an image) wouldn't apply
to any media, resulting in a multiplicity of attributes for the <object>
element in the final DTD, that could be error prone.
Also, adding a new object type can imply a new attribute (for example <text
words='200'>), and so imply a DTD modification.

To compare with a J Rabin sample (XN-5) :
-----------------------------------------
Few differences : metadatas are less directely taken from IIM V4 ; 'title'
and 'headline' are identical ; the dateline presentation is provider
dependent ; there is no information about the type (text, photo) of the
document.

<newsitem itemid="19990526000012" date="1999-05-28"
publisher="moonlite.tibcofinance.com/newsml" xml:lang="en-us"
parts="1" revision="0">
  <title>DTDs for Today's world</title>
  <headline>DTDs for Today's world</headline>
  <byline>by Ernest Dull</byline>
  <dateline>Palo Alto, May 26 (Reuters)</dateline>
  <text>
   <p>Many people wonder why DTDs are necessary</p>
   <p>Blah Blah Blah</p>
   </text>
  <copyright>(c) 1999 Dull, Ernest Tech Mags Inc.</copyright>
</newsitem>

To compare with a A Karben sample (on his site) :
-----------------------------------
No big difference at this step ; the 'news-envelope' is, if I remember well,
a newsitem possessing handling metadata ; 'publisher' doesn't keep the
difference between 'source' and 'credit' ; iso date is the standard ; a
news-folder is present, but has no meaning in this case  ; 'title' and
'article/headline' are usually identical ; no newsitem 'type'.

<news-envelope
  id="AFP_TX_19991203_555"
  publisher="AFP"
  date="19991231T120000"
  >
<news-folder relationship="complements">
  <title>Text headlines</title>
  <article>
   <headline>
    <main-headline>Text headline</main-headline>
   </headline>

   <byline>llm</byline>

   <p>Text body</p>
  </article>
</news-folder>

</news-envelope>


To compare with a XMLNews sample :
----------------------------------

- XMLNews-Meta uses a 'deliberately simple and flat' RDF vocabulary for
metadata description (is RDF really useful in this simple case ?), similar
to the other samples :

<xn:Resource xmlns:xn="http://www.xmlnews.org/namespaces/meta#">
  <xn:resourceId>AFP_TX_19991203_555</xn:resourceId>
  <xn:sourceName>AFP</xn:sourceName>
  <xn:publicationTime>19991231T120000</xn:publicationTime>
  <xn:language>en</xn:language>
</xn:Resource>

- XMLNews-Story is a subset of NITF : structure isn't perfectly clear (head,
body.head are usually identical, hedline/hl1 is a funny syntax), some tags
(as byline, location) could be considered as metadata ; powerful features
are essentially found in body.content markup, not showed here :

<nitf>
  <head>
   <title>Text title</title>
  </head>
  <body>
   <body.head>
    <hedline>
     <hl1>Text title</hl1>
    </hedline>
    <byline>
     <bytag>llm</bytag>
    </byline>
    <dateline>
     <location>PARIS, IDF, FRANCE</location>
     <story.date>Friday December 31 1999 12:00</story.date>
    </dateline>
   </body.head>
   <body.content>
    <p>Text body</p>
   </body.content>
  </body>
</nitf>

I feel that we should quickly try to reach an agreement on such a simple
story.

On the next step, we could describe a simple photo or video, then go on with
multimedia newsitems.

#24

From: "Laurent LE MEUR" <laurent.lemeur@...>
Date: Mon Jan 3, 2000 5:56 pm
Subject: newsML samples - simple story

laurent.lemeur@...

(note : sorry for the first mail posted on iptc-2000 newsgroup : seems that some characters aren't accepted by eGroup web posting facility. The newsml newsgroup seems more adequate at this step)

Following Heathrow meeting, and starting from requirements and from AFP experience (an internal XML multimedia format named XAFP), I tried to imagine what could be the newsML structure.

Since the newML recommendation has to be released in march, I only see one way to go :
- focus on structure,
- discuss metadata on a separate thread,
- begin with simple samples,
- discuss those samples until agreement,
- go on with more complicated issues,
- *finish* with the DTD or schema.

Alan Karben NewsML site is very interesting, but gives the impression that the development is done. The XMLNews tutorial (www.xmlnews.org/docs/story-tutorial.html) way of describing tags seems to me more adequate for a discussion.

In the following, lets assume a "newsml" namespace ; the newsitem notation <item> is then equivalent to <newsml:item>. Alternative : use <newsitem>, which is Jo Rabin proposal.

To keep thing simples, xml declaration (<?xml-version...>) is not represented in the samples.

Proposals on Newsitem element :
- It has a 'type' attribute, of values 'text', 'photo', 'graphic' 'audio', video', 'multimedia' ... that reflects the user (or provider) point of vue about the type of the document, not a pure structural information. The list should be left opened.
- It has an 'uno' attribute, a globally unique name equivalent to the IIM V4 UNO.
- It includes a header <head>, that contains metadatas, like editorial informations (essentially taken from IIM Record 1 and 2). Handling metadatas (that have a different lifetime) will be described later, and can be kept in another header (easier to handle in a specific header than in separate tags).
- It includes one or more named parts (I prefer the name 'object') (<text> ...), that represent actual data. Those objets can include textual data (by value) or reference binary (or textual) data (with an URI (URL or URN)). These attributes are structural only : objects don't possess editorial attributes.
-object elements can be <texte>, <photo>, <graphic>, <audio>, <video>, <note> (a simple message), <opaque> (an unknown CLOB or BLOB).

Exemple 1 : a short text

<item uno='AFP_TX_19991203_555' type='text' xml:lang='en-us'>
<head urgency='2'>
  <source>AFP</source>
  <date-created type="iso" val="19991231T120000Z"/>
  <headline>Text headline</headline>
  <byline>llm</byline>
  <location>
   <city>PARIS</city>
   <state type="REG" val="IDF">IDF</state>
   <country type="iso" val="FRA">FRANCE</country>
  </location >
</head>
<text>
  <p>Text body</p>
</text>
</item>

Notes :
- use of 'uno' and not 'itemid' (Reuters proposal) keeps alive the historical link with IIM.
- The uno of each item is an ID attribute : it can be used for example in a XPointer selection (to use the couple source/itemid makes it more difficult).
- the dateline, present in J Rabin an A Karben samples, is considered as a presentation of informations found in 'source', 'date-created' and 'location'.
- for dates, iso format could become the NewsML standard.
- precise element names can be choosen during the discussion (for exemple <text> could become <article> or <story>).

Alternatives :
- The <text> object could be represented as <object media='text'>, an so on for other objects types : it would be simpler to add a new object type (if the media list is 'open'), but structural attributes (like 'words' for a word count in a text, or 'height' and 'width' for an image) wouldn't apply to any media, resulting in a multiplicity of attributes for the <object> element in the final DTD, that could be error prone.

Also, adding a new object type can imply a new attribute (for example <text words='200'>), and so imply a DTD modification.

To compare with a J Rabin sample (XN-5) :
-----------------------------------------
Few differences : metadatas are less directely taken from IIM V4 ; 'title' and 'headline' are identical ; the dateline presentation is provider dependent ; there is no information about the type (text, photo) of the document.

<newsitem itemid="19990526000012" date="1999-05-28" publisher="moonlite.tibcofinance.com/newsml" xml:lang="en-us"
parts="1" revision="0">
<title>DTDs for Today's world</title>
<headline>DTDs for Today's world</headline>
<byline>by Ernest Dull</byline>
<dateline>Palo Alto, May 26 (Reuters)</dateline>
<text>
  <p>Many people wonder why DTDs are necessary</p>
  <p>Blah Blah Blah</p>
  </text>
<copyright>(c) 1999 Dull, Ernest Tech Mags Inc.</copyright>
</newsitem>

To compare with a A Karben sample (on his site) :
-----------------------------------
No big difference at this step ; the 'news-envelope' is, if I remember well, a newsitem possessing handling metadata ; 'publisher' doesn't keep the difference between 'source' and 'credit' ; iso date is the standard ; a news-folder is present, but has no meaning in this case ; 'title' and 'article/headline' are usually identical ; no newsitem 'type'.

<news-envelope
id="AFP_TX_19991203_555"
publisher="AFP"
date="19991231T120000"
>
<news-folder relationship="complements">
<title>Text headlines</title>
<article>
  <headline>
   <main-headline>Text headline</main-headline>
  </headline>

  <byline>llm</byline>

  <p>Text body</p>
</article>
</news-folder>

</news-envelope>

To compare with a XMLNews sample :
----------------------------------

- XMLNews-Meta uses a 'deliberately simple and flat' RDF vocabulary for metadata description (is RDF really useful in this simple case ?), similar to the other samples :

<xn:Resource xmlns:xn="http://www.xmlnews.org/namespaces/meta#">
<xn:resourceId>AFP_TX_19991203_555</xn:resourceId>
<xn:sourceName>AFP</xn:sourceName>
<xn:publicationTime>19991231T120000</xn:publicationTime>
<xn:language>en</xn:language>
</xn:Resource>

- XMLNews-Story is a subset of NITF : structure isn't perfectly clear (head, body.head are usually identical, hedline/hl1 is a funny syntax), some tags (as byline, location) could be considered as metadata ; powerful features are essentially found in body.content markup, not showed here :

<nitf>
<head>
  <title>Text title</title>
</head>
<body>
  <body.head>
   <hedline>
    <hl1>Text title</hl1>
   </hedline>
   <byline>
    <bytag>llm</bytag>
   </byline>
   <dateline>
    <location>PARIS, IDF, FRANCE</location>
    <story.date>Friday December 31 1999 12:00</story.date>
   </dateline>
  </body.head>
  <body.content>
   <p>Text body</p>
  </body.content>
</body>
</nitf>

I feel that we should quickly try to reach an agreement on such a simple story.

On the next step, we could describe a simple photo or video, then go on with multimedia newsitems.

#23

From: Jo Rabin <jo.rabin@...>
Date: Sun Jan 2, 2000 11:37 am
Subject: Re: One Standard for News in the New Millennium

jo.rabin@...

David

You could equally well argue that there is the need for further development of
NITF, the IIM and other news standards that are presently used far more widely
than XMLNews is. However the users of those standards have taken the view that
this is not to happen and that the needs of the community will be better served
by contributing their efforts to NewsML.

All best
Jo

Jo Rabin writes:

  > A couple of weeks ago on your list, you asked for comments on what
  > phase 2 of XML News should be and you stated your long-term goal of
  > converging with IPTC's NewsML.
  >
  > But why not make your phase 2 NewsML? You suggest that NewsML will
  > be a long time in the coming, yet as I mentioned some weeks ago the
  > target for an initial agreed cut of NewsML is April next year. It
  > is well beyond the "requirements gathering" phase you suggest. A
  > public draft of the requirements spec is available for comment
  > (http://www.iptc.org/STA9912.pdf) and the NewsML section of the
  > IPTC site (http://www.iptc.org/NMLIntro.htm) has been updated with
  > a number of additional background papers).
  >
  > It would be more productive for the effort that would be commited
  > to a phase 2 of XML News to be committed to NewsML. In this spirit,
  > comments on the public draft requirements are welcome on the NewsML
  > mailing list (NewsML@egroups.com).

That is wonderful news -- I think that everyone will be very happy
when there is a single, final, agreed-upon standard for news
delivery.

In the meantime, though, since there are a number of parties using
XMLNews (both internally and externally), and since it will probably
take at least a few months for NewsML implementations to spool up once
the initial alpha version of NewsML is ready (and major implementors
will probably wait for the final version), I think that there's still
value in improving XMLNews itself to fill in the gap.  At the very
least, our implementation experience will be valuable for the
development of NewsML from the initial cut in April to the final
version some time later.

All the best,

David

--
David Megginson                 david@...
            http://www.megginson.com/
xmlnews-l: discussion of online news and information exchange.
To unsubscribe, send a message to majordomo@... with the
following text in the message body:
unsubscribe xmlnews-l <your address>

-----------------------------------------------------------------
         Visit our Internet site at http://www.reuters.com

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

Reply | Forward | Messages in this Topic (5)

#22

From: Jo Rabin <jo.rabin@...>
Date: Sat Jan 1, 2000 10:52 am
Subject: Re: One Standard for News in the New Millennium

jo.rabin@...

I intended to send this mail to both newsml and xmlnews-l but it seems I did
not - apologies.

Jo
============
David,

A couple of weeks ago on your list, you asked for comments on what phase 2 of
XML News should be and you stated your long-term goal of converging with IPTC's
NewsML.

But why not make your phase 2 NewsML? You suggest that NewsML will be a long
time in the coming, yet as I mentioned some weeks ago the target for an initial
agreed cut of NewsML is April next year. It is well beyond the "requirements
gathering" phase you suggest. A public draft of the requirements spec is
available for comment (http://www.iptc.org/STA9912.pdf) and the NewsML section
of the IPTC site (http://www.iptc.org/NMLIntro.htm) has been updated with a
number of additional background papers).

It would be more productive for the effort that would be commited to a phase 2
of XML News to be committed to NewsML. In this spirit, comments on the public
draft requirements are welcome on the NewsML mailing list (NewsML@egroups.com).

Jo


-----------------------------------------------------------------
         Visit our Internet site at http://www.reuters.com

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

Reply | Forward | Messages in this Topic (5)

#21

From: Jo Rabin <jo.rabin@...>
Date: Fri Dec 31, 1999 2:30 pm
Subject: Public Draft of Requirements for NewsML

jo.rabin@...

A public draft of the requirements spec for NewsML is available for comment on
this list (http://www.iptc.org/STA9912.pdf).

See also the NewsML section of the IPTC site (http://www.iptc.org/NMLIntro.htm)
which has been updated.

Jo

-----------------------------------------------------------------
         Visit our Internet site at http://www.reuters.com

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

#20

From: johan.Lindgren@... (Johan Lindgren)
Date: Sun Dec 26, 1999 8:56 pm
Subject: Re: Introduction & newbie questions

johan.Lindgren@...

steven.noels@...,Nasta skriver:
>- how will NewsML evolve/compete/supplement/complement NITF

The goal of NewsML is to provide an XML framework for an electronical
"envelope" to hold any multimedia object where a text object could be
encoded in NITF.

>- what is the current process of desiging the DTD

Through electronic cooperative work during the winter

>- and if needed/possible, how we could cooperate on this effort

In these newsgroups.

johan

Johan Lindgren, TT, Box 865, S-85124 Sundsvall, Sweden.
Phone: +46 (0)60-176815. Mobil: 070-6528007.
Fax & messages: 060-7001971

#19

From: "Steven Noels" <steven.noels@...>
Date: Thu Dec 23, 1999 11:51 pm
Subject: Introduction & newbie questions

steven.noels@...

Hello,

since this is becoming more & more a habit on these mailing lists,
please let me start by introducing myself. Having an SGML/XML
background, I'm currently working as Product Manager for an XML-based
content management & web publishing system, being built by the company
I work for, Net It Be. This company has been recently bought by Alcatel
from 'De Tijd Uitgeversgroep', publisher of the 'Financieel-Economische
Tijd', based in Antwerp, Belgium. During the years we belonged to the
Tijd, we developed an editorial management system for their website &
CD-ROM publications, based on SGML.

We were already aware of the NITF DTD, and looking at both the latest
version of that DTD, and the draft of NewsML posted on the vault, I am
a bit at loss whether these can be compared against each other or not.
Is it the intention to replace the NITF efforts by NewsML? What about
industry support on both efforts?

Supporting NewsML with our product could be very interesting. Since we
were formerly owned by a publisher of newspapers, we believe we
understand the specifics of this industry sector. My main questions are:

- how will NewsML evolve/compete/supplement/complement NITF
- what is the current process of desiging the DTD
- and if needed/possible, how we could cooperate on this effort

Many regards,

</Steven>

#18

From: Miles Whitehead 7628 <miles.whitehead@...>
Date: Fri Dec 17, 1999 11:33 am
Subject: Re: [Phase 2] XMLNews-Meta, properties or objects?

miles.whitehead@...

>Hello Miles,
>We are interested in metrics also.  Does the history of code
>assignment that your DTD captures include the source
>of the codeset applied to the story?

It does.  I should have included the codes element which contains that
information and goes as follows:

<!ELEMENT metadata (codes|
			 things|
			 dc)*>


<!-- intention is to have only one codes element for any
      class/role combination  -->
<!ELEMENT codes  (code*)>
<!ATTLIST codes  id  ID #IMPLIED
			 class  CDATA #REQUIRED
			 role  CDATA #IMPLIED
			 publisher CDATA #IMPLIED>

The attribute class is supposed to indicate the scheme and its version.
You can now see the experimental DTDs in the vault area on the eGroups web
site at:

http://www.egroups.com/docvault/newsml/

together with an explanatory text document.

Please comment on any aspect at newsml@egroups.com, mailing first
newsml-subscribe@egroups.com to join in.

>Your DTD looks great to me.

Its actually Jo Rabin's work, and it looks great to me too.

Miles

---------------------------------------------------------------------------
   Miles Whitehead       Email: miles.whitehead@...  85 Fleet Street
   Research Assistant    Voice: +44 171 542 7628             London EC4P 4AJ
   Reuters Limited       Fax  : +44 171 542 8314             UK
---------------------------------------------------------------------------



-----------------------------------------------------------------
         Visit our Internet site at http://www.reuters.com

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

#17

From: Nic Fulton <nic.fulton@...>
Date: Fri Dec 17, 1999 10:46 am
Subject: Experimental DTDs from Reuters

nic.fulton@...

Folks,

As much of the discussion here seems to be 'in the dark' I have uploaded some
experimental DTDs from Reuters. These were written by Jo Rabin and were
submitted to the IPTC as a seed for some of the NewsML work.

They can be found in the vault area on the eGroups web site at:

http://www.egroups.com/docvault/newsml/

I hope this acts as a seed for further discussion, although please be aware
that these are for 'information only'.

Best regards,

Nic Fulton
Leader of XML Architecture and Design
Reuters Ltd


-----------------------------------------------------------------
         Visit our Internet site at http://www.reuters.com

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

#16

From: Leonid <capitalsense@...>
Date: Thu Dec 16, 1999 9:37 pm
Subject: Re:<xn:assignor>

capitalsense@...

Dear Miles and Larry,

I liked your approach very much. We need this functionality now and we face the
same challenge of finding a balance between who applies _contextual_ markup,
how, when and why. I wonder how those issues are currently addressed by
Reuters, Wavo (or anyone else). Do you know of any tools that facilitate
managing this process with a high volume of diverse content based on certain
rules? Something like XMLNews-Meta Manager :-)

Thanks in advance.

Leonid



Miles Whitehead 7628 wrote:

> >
> >Similarly, we need <xn:assignor> to indicate who actually "applied" the
> >code, and the reason for so doing.
>
> The Reuters experimental DTD for NewsML has several useful features in the
> metadata section for marking up documents with codes, in particular:
>
> <!ELEMENT code          (name|editdetail)*>
> <!ATTLIST code          id              ID      #IMPLIED
>                         code            CDATA   #REQUIRED
>                         confidence      CDATA   #IMPLIED
>                         present         (true|false)    "true">
>
> <!ELEMENT editdetail    EMPTY>
> <!ATTLIST editdetail    id              ID      #IMPLIED
>                         attribution     CDATA   #IMPLIED
>                         action          (added|
>                                         removed|
>                                         confirmed)      "added"
>                         date            CDATA   #IMPLIED
>                         agent           (human|
>                                         auto|
>                                         map|
>                                         expansion|
>                                         unknown)        "unknown"
>                         expansion       CDATA   #IMPLIED
>                         confidence      CDATA   #IMPLIED
>                         score           CDATA   #IMPLIED>
>
> This enables the documents to be marked up with a complete history of the
> codes that have been assigned to them.  The date and agent of the
> assignment (or deassignment) of each code is recorded together with a
> confidence value which gives an indication of how sure the agent is that
> the code is appropriate.  This is particularly useful for generating
> metrics for a system where a machine is being used to apply codes to
> stories, and then a human corrects them where necessary.  The confidence
> value can also be used for relevance ranking.
>
> The role of the code is also important - its important to show whether the
> code has been applied because it is the subject of the story, or for some
> other reason, such as the location of the story.  It must be possible to
> distinguish between 'occurrence' coding and 'aboutness' coding.
>
> There are also issues to do with the versions of the codesets the codes are
> taken from.  Codesets change over time, but it is still necesary to
> retrieve data from archives about, for instance, companies and countries
> that no longer exist, and to work out whether a price is quoted in a
> currency pre- or post- devaluation.
>
> There is a definite requirement to be able to represent all this
> information for each news item.
>
> Regards
>
> Miles
>
> ---------------------------------------------------------------------------
>  Miles Whitehead       Email: miles.whitehead@...  85 Fleet Street
>  Research Assistant    Voice: +44 171 542 7628             London EC4P 4AJ
>  Reuters Limited       Fax  : +44 171 542 8314             UK
> ---------------------------------------------------------------------------
>
> -----------------------------------------------------------------
>         Visit our Internet site at http://www.reuters.com
>
> Any views expressed in this message are those of  the  individual
> sender,  except  where  the sender specifically states them to be
> the views of Reuters Ltd.
> xmlnews-l: discussion of online news and information exchange.
> To unsubscribe, send a message to majordomo@... with the
> following text in the message body:
> unsubscribe xmlnews-l <your address>

#15

From: Miles Whitehead 7628 <miles.whitehead@...>
Date: Thu Dec 16, 1999 4:36 pm
Subject: Re: [Phase 2] XMLNews-Meta, properties or objects?

miles.whitehead@...

>
>Similarly, we need <xn:assignor> to indicate who actually "applied" the
>code, and the reason for so doing.

The Reuters experimental DTD for NewsML has several useful features in the
metadata section for marking up documents with codes, in particular:

<!ELEMENT code  (name|editdetail)*>
<!ATTLIST code  id  ID #IMPLIED
			 code  CDATA #REQUIRED
			 confidence CDATA #IMPLIED
			 present  (true|false) "true">


<!ELEMENT editdetail EMPTY>
<!ATTLIST editdetail id  ID #IMPLIED
			 attribution CDATA #IMPLIED
			 action  (added|
					 removed|
					 confirmed) "added"
			 date  CDATA #IMPLIED
			 agent  (human|
					 auto|
					 map|
					 expansion|
					 unknown) "unknown"
			 expansion CDATA #IMPLIED
			 confidence CDATA #IMPLIED
			 score  CDATA #IMPLIED>

This enables the documents to be marked up with a complete history of the
codes that have been assigned to them.  The date and agent of the
assignment (or deassignment) of each code is recorded together with a
confidence value which gives an indication of how sure the agent is that
the code is appropriate.  This is particularly useful for generating
metrics for a system where a machine is being used to apply codes to
stories, and then a human corrects them where necessary.  The confidence
value can also be used for relevance ranking.

The role of the code is also important - its important to show whether the
code has been applied because it is the subject of the story, or for some
other reason, such as the location of the story.  It must be possible to
distinguish between 'occurrence' coding and 'aboutness' coding.

There are also issues to do with the versions of the codesets the codes are
taken from.  Codesets change over time, but it is still necesary to
retrieve data from archives about, for instance, companies and countries
that no longer exist, and to work out whether a price is quoted in a
currency pre- or post- devaluation.

There is a definite requirement to be able to represent all this
information for each news item.

Regards

Miles

---------------------------------------------------------------------------
  Miles Whitehead       Email: miles.whitehead@...  85 Fleet Street
  Research Assistant    Voice: +44 171 542 7628             London EC4P 4AJ
  Reuters Limited       Fax  : +44 171 542 8314             UK
---------------------------------------------------------------------------



-----------------------------------------------------------------
         Visit our Internet site at http://www.reuters.com

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

#14

From: Marrara Angelo <marrara@...>
Date: Tue Dec 7, 1999 2:59 pm
Subject: Re: IPTC200 Progress and Actions

marrara@...

Dear All,

please find attached an example of ANSA text news in NITFX 1.2 format.


The dublin core mapping has been specified as comments.
I suggest to map the dublin core into the nitf element
instead to redesign the nitf format.


Regards
Angelo


Attachment:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE nitf SYSTEM "nitf1_2.dtd">
<nitf>
         <!-- Document header information -->
         <head>
                 <!-- Page title -->
                 <title>
                         KOSOVO: OPERATION RAINBOW; TURKEY TO GET 190
LEFT CONTAINER
                 </title>
                 <!-- Environment data -->
                 <meta name="keyword" content="KOSOVO"/>
                 <!-- Author -->
                 <meta name="author" content="HE"/>
                 <!-- Writer (if present) -->
                 <meta name="writer" content=""/>
                 <!-- Format -->
                 <!-- Priority -->
                 <meta name="priority" content="R"/>
                 <!-- Category -->
                 <meta name="category" content="EST"/>
                 <!-- Product ID -->
                 <meta name="product-id" content="QBXB"/>
                 <!-- Document Meta Data -->
                 <docdata>
                         <!-- News Identifier -->
                         <doc-id id-string="WPE10139"/>
                         <!-- Date of issue -->
                         <date.issue norm="1999/09/06 13:25"/>
                         <!-- Date of release (not present in the news
format) -->
                         <date.release norm="1999/09/06 13:28"/>
                 </docdata>
         </head>
         <!-- Document body -->
         <body>
                 <body.head>
                         <!-- Container for head line -->
                         <hedline>
                                 <hl1>
                                 <!-- Title -->
                                 KOSOVO: OPERATION RAINBOW; TURKEY TO GET

190 LEFT CONTAINER
                                 </hl1>
                         </hedline>
                         <!-- Subtitle (if present) -->
                         <byline/>
                         <!-- News distributor Agency -->
                         <distributor>
                                 <org> ANSA </org>
                         </distributor>
                         <!-- Container for date line information -->
                         <dateline>
                                 <!-- Significant location (not present
in the news format) -->
                                 <location>BARI, Italy</location>
                         </dateline>
                 </body.head>
                 <!-- Content -->
                 <body.content>
                         <!-- Language (not present in the news format)
-->
                         <lang lang="en.us"/>
                         <!-- Text -->
                         <block>


<p>(ANSA) - Bari, September 6 -&lt;&lt;
Civil Protection officials here said that 190 of the abandoned
containers full of supplies
originally meant for Kosovo as part of Operation Rainbow, will
be sent to earthquake-stricken areas of Turkey over coming days.
</p>
<p>
Around 250 of a total of 914 containers left behind on the
dock here were inspected yesterday to sort out the useful items
and separate them by category: water, clothing, mattresses, camp
beds, toys, etc.
</p>
<p>
A second inspection is due to start this afternoon on the
part of three non-governmental associations, Intersos (from
Rome), Avsi (from Milan), and Cesvi (from Bergamo), which have
taken on the task for the Civil Protection agency of opening the
remaining containers and sorting the contents.
</p>
<p>
Speaking for these associations, Nino Sergi of Avsi said
most of the supplies would be given to Italian volunteer groups
working in the Balkans or in other regions where aid is need.
''Whenever the material could be used in Italy, it will be
delivered to associations active in our country, even to those
dealing with aid to refugees'', he added. (ANSA)
</p>
                         </block>
                 </body.content>
         </body>
</nitf>


David Allen wrote:


Dear member
I am attaching the DRAFT Minutes from the recent IPTC2000 ad hoc meeting

in London. There were a number of decisions made at this meeting that
will be of interest to all members.

In particular:


The modified requirements documentation (to be circulated soon) needs to

be commented on before 14 January 1999.


The Dublin Core metadata model plus extras need to be considered for
suitability to replace the metadata used in the IIM and NITF.


Users of NITF are asked to provide to me as soon as possible the names
of elements (and their attributes) that are actively used within their
NITF implementations. These will be subjected to the Karben Tests (see
Minutes) for their suitability for inclusion in the NewsML text mark-up.




The next newsML meeting will be in Washington DC 31Jan to 2 Feb 2000.


David Allen

#13

From: Jo Rabin <jo.rabin@...>
Date: Mon Dec 6, 1999 9:53 pm
Subject: Re: NewsML Extensibility without Chaos

jo.rabin@...

Walt

I share your desire to move forward and agree that we have too little time to
let such debates hold us up.

I think we need to bear in mind that each time the standard is up for
reassessment those concerned need to be able to reevaluate the premises of the
earlier go around. Which is why the NewsML requirements spec specifically
states that a documentation/audit trail needs to be left for this go around,
which will not be the last ...

In practice I think this means that a) NewsML needs to offer a flexible
response in this area (viz that it should offer something more general than a
specific set of predefined features that can be marked) - and b) that the
earlier documentation is made available in this context and discussion
progressed in parallel to everything else - as part of the metadata activity.

all best
Jo

>Folks:
>   I can't believe that we are debating issues that I thought were settled
(with the agreement of everyone) more than 5 years ago. The nature of markup
was thoroughly discussed, and we settled on the content markup architecture of
the NITF.
>  If we are going to reopen such debates as "What is a person?" and "What is
an organization?" then we are going to be here 5 years fro  now. The length of
time it took to design NITF was not so much a function of making a mechanically
correct DTD -- although that part took far too long -- as much as it was a
reflection of the debate about what to mark up and what the markup means.
>  To start this discussion again is an inefficient use of the short time that
we have allotted. I suggest that anyone who wants to review the nature of
content markup as agreed by the IPTC should talk to David Allen and obtain the
relevant documents.
>  The danger of scrapping the NITF -- and that is what we are doing -- is that
we doom ourselves to repeat the same discourses ad infinitum. I still believe
that we should take the current NITF content markup (without the hierarchical
baggage, such as BLOCK) and move on to more productive activities.
>       --- walt (in Moscow)
>
>
>
>>Namespaces do seem to be the answer to several tricky problems, but do you
>>think that XML implementations are ready for them?
>>
>>It's interesting that you choose in-line markup of features as your example.
>>There has been quite a lot of debate as to how this should be handled in
>>NewsML. One view says that since NewsML is an expressly multi-media format
you
>>should be able to mark features in any of your media types in a similar way.
>>This forces you to consider denoting the features outside of the object in
>>question and point into the object. How exactly you do the pointing needs to
be
>>media dependent but this approach buys you consistency across media types in
>>identifying things of interest.
>>
>>The contrary view states that present day tools (editors) make it very easy
to
>>do in-line markup (of text) and that pragmatically speaking there are no
tools
>>that support the pointing stuff.
>>
>>At the November NewsML meeting the idea was floated that when used as an
>>interface the markup should be externalized but that for the pupose of
editing
>>it should be put in-line.
>>
>>The second part of the markup disucssion relates to whether or not it makes
>>sense to have a "standard" notion of what to mark. Reuters notion of what a
>>person is may be very different to some other provider.
>>
>>Also when to mark people is an act of editorial judgmement - i.e. should you
>>mark all people, only people for whom you have biographic data, or should you
>>mark only people who play a significant role in the story, and ignore other
>>people? If NewsML contained standard markup for features would it also have
to
>>standardise these kinds of rules?
>>
>>Further, whether something is significant to mark entirely depends on the
>>reason the recipient is receiving the story - i.e. what their interests are.
It
>>is obviously useful to be able to mark sports entities in news that is part
of
>>a sports service, but obviously burdensome to insist that receivers of news
in
>>a financial service have to know that markup of sports features is even
>>possible.
>>
>>You need also to be able to mark features with several interpretations,
firstly
>>because things do have different interpretations (Bill Clinton is a person,
>>he's the president of the USA, his first name is Bill etc.) and secondly
>>because you may have some doubt as to what something is (especially if you
are
>>trying to apply markup automatically). "Britannia" may be an airline, might
be
>>a building society, could be the mythical person who represents the British
>>nation.
>>
>>So in summary, my feeling is that NewsML needs to provide rather basic
>>facilities for identifying features. The debate about what is useful to mark,
>>when to mark it and so on needs to be conducted separately to a discussion on
>>how to represent it when you do. Namespaces may help us in this.
>>
>>Jo
>>
>>
>>David Megginson wrote:
>>>
>>>There are two important criteria for extensibility:
>>>
>>>1. Information providers need to be able to add things that the
>>>   designers did not anticipate.
>>>
>>>2. Information consumers need to be able to recognize and disambiguate
>>>   those additions, and to behave in predictable ways if they do not
>>>   recognize an addition.
>>>
>>> > It would also be great to hear views about how to achieve such
>>> > flexibility in practice.
>>>
>>>Well, Namespaces can help an awful lot with only a little advanced
>>>planning.  To keep things simple, let's say that NewsML identified
>>>three different contexts where extensibility was allowed:
>>>
>>>1. Within the story header.
>>>
>>>2. In a block context (where paragraph-like things are expected).
>>>
>>>3. In a phrasal context (where text, emphasised phrases, etc. are
>>>   expected).
>>>
>>>4. In the attribute list of any element.
>>>
>>>(There are other contexts, but we can talk about them later.)  Now,
>>>you start by defining the default behaviour for unrecognized elements
>>>in any of these contexts:
>>>
>>>1. Within the story header, ignore the unrecognized element and all of
>>>   its descendants, including any text.
>>>
>>>2. In a block context, ignore the unrecognized element and any text
>>>   immediately inside it, but look for descendant elements that can be
>>>   recognized.
>>>
>>>3. In a phrasal context, simply ignore the boundaries of the
>>>   unrecognized element and integrate its contents into the parent
>>>   element.
>>>
>>>4. All unrecognized attributes are ignored.
>>>
>>>This is pretty close to default HTML browser behaviour, so we have
>>>some real-world industry experience to draw on (with the additional
>>>benefit of Namespaces for disambiguation).  You could also define a
>>>special attribute such as "newsml:defaultAction" to change the default
>>>behavior for exceptional situations.
>>>
>>>Now, with these rules it's quite straight-forward for individual
>>>parties to extend NewsML without breaking deployed software.  For
>>>example, let's say that NewsML didn't have an inline element for
>>>identifying people (it probably will, of course).  You could start
>>>with a paragraph like
>>>
>>>  <p>Today Tony Blair met with his cabinet.</p>
>>>
>>>Now, Reuters may want to tag Tony Blair's name, so it can define its
>>>own Namespace and do
>>>
>>>  <p>Today <reuters:person>Tony Blair</reuters:person> met with
>>>   his cabinet.</p>
>>>
>>>Information consumers who know about the Reuters Namespace can use
>>>this extra information; information consumers who don't will simply
>>>apply the default rule for phrasal content and treat the second
>>>example as identical to the first.
>>>
>>>To take a less canned example, now, imagine that NewsML *did* include
>>>a <person> element, but that Reuters wanted to add its own unique
>>>identifier to it (and NewsML didn't include that).  It would be
>>>possible to have
>>>
>>>  <p>Today <person reuters:id="002345">Tony Blair</person> met with
>>>   his cabinet.</p>
>>>
>>>and again, software that didn't know about the reuters:id attribute
>>>would follow the default rule and ignore it.
>>>
>>>This kind of an approach allows for a lot of innovation without the
>>>possibility of confusion, since each party's extensions will be in a
>>>separate Namespace (there's no chance of AP's 'id' attribute and
>>>Reuter's 'id' attribute being confused).
>>>
>>>
>>>All the best,
>>>
>>>
>>>David
>>>
>>>--
>>>David Megginson                 david@...
>>>           http://www.megginson.com/
>>>
>>>------------------------------------------------------------------------
>>>To Post a message, send it to:   newsml@eGroups.com
>>>
>>>To Unsubscribe, send a blank message to: newsml-unsubscribe@eGroups.com
>>>
>>>------------------------------------------------------------------------
>>>-- 20 megs of disk space in your group's Document Vault
>>>-- http://www.egroups.com/docvault/newsml/?m=1
>>>
>>
>>-----------------------------------------------------------------
>>        Visit our Internet site at http://www.reuters.com
>>
>>Any views expressed in this message are those of  the  individual
>>sender,  except  where  the sender specifically states them to be
>>the views of Reuters Ltd.
>>
>>------------------------------------------------------------------------
>>To Post a message, send it to:   newsml@eGroups.com
>>To Unsubscribe, send a blank message to: newsml-unsubscribe@eGroups.com
>>
>>------------------------------------------------------------------------
>>-- Talk to your group with your own voice!
>>-- http://www.egroups.com/VoiceChatPage?listName=newsml&m=1
>
>
>------------------------------------------------------------------------
>To Post a message, send it to:   newsml@eGroups.com
>
>To Unsubscribe, send a blank message to: newsml-unsubscribe@eGroups.com
>
>------------------------------------------------------------------------
>Was the salesman clueless?  Productopia has the answers.
>http://clickhere.egroups.com/click/1702
>
>
>
>-- Create a poll/survey for your group!
>-- http://www.egroups.com/vote?listname=newsml&m=1
>

-----------------------------------------------------------------
         Visit our Internet site at http://www.reuters.com

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

#12

From: Walter Baranger <walt@...>
Date: Mon Dec 6, 1999 2:01 pm
Subject: Re: NewsML Extensibility without Chaos

walt@...

Folks:
    I can't believe that we are debating issues that I thought were settled (with
the agreement of everyone) more than 5 years ago. The nature of markup was
thoroughly discussed, and we settled on the content markup architecture of the
NITF.
   If we are going to reopen such debates as "What is a person?" and "What is an
organization?" then we are going to be here 5 years fro  now. The length of time
it took to design NITF was not so much a function of making a mechanically
correct DTD -- although that part took far too long -- as much as it was a
reflection of the debate about what to mark up and what the markup means.
   To start this discussion again is an inefficient use of the short time that we
have allotted. I suggest that anyone who wants to review the nature of content
markup as agreed by the IPTC should talk to David Allen and obtain the relevant
documents.
   The danger of scrapping the NITF -- and that is what we are doing -- is that
we doom ourselves to repeat the same discourses ad infinitum. I still believe
that we should take the current NITF content markup (without the hierarchical
baggage, such as BLOCK) and move on to more productive activities.
        --- walt (in Moscow)



>Namespaces do seem to be the answer to several tricky problems, but do you
>think that XML implementations are ready for them?
>
>It's interesting that you choose in-line markup of features as your example.
>There has been quite a lot of debate as to how this should be handled in
>NewsML. One view says that since NewsML is an expressly multi-media format you
>should be able to mark features in any of your media types in a similar way.
>This forces you to consider denoting the features outside of the object in
>question and point into the object. How exactly you do the pointing needs to be
>media dependent but this approach buys you consistency across media types in
>identifying things of interest.
>
>The contrary view states that present day tools (editors) make it very easy to
>do in-line markup (of text) and that pragmatically speaking there are no tools
>that support the pointing stuff.
>
>At the November NewsML meeting the idea was floated that when used as an
>interface the markup should be externalized but that for the pupose of editing
>it should be put in-line.
>
>The second part of the markup disucssion relates to whether or not it makes
>sense to have a "standard" notion of what to mark. Reuters notion of what a
>person is may be very different to some other provider.
>
>Also when to mark people is an act of editorial judgmement - i.e. should you
>mark all people, only people for whom you have biographic data, or should you
>mark only people who play a significant role in the story, and ignore other
>people? If NewsML contained standard markup for features would it also have to
>standardise these kinds of rules?
>
>Further, whether something is significant to mark entirely depends on the
>reason the recipient is receiving the story - i.e. what their interests are. It
>is obviously useful to be able to mark sports entities in news that is part of
>a sports service, but obviously burdensome to insist that receivers of news in
>a financial service have to know that markup of sports features is even
>possible.
>
>You need also to be able to mark features with several interpretations, firstly
>because things do have different interpretations (Bill Clinton is a person,
>he's the president of the USA, his first name is Bill etc.) and secondly
>because you may have some doubt as to what something is (especially if you are
>trying to apply markup automatically). "Britannia" may be an airline, might be
>a building society, could be the mythical person who represents the British
>nation.
>
>So in summary, my feeling is that NewsML needs to provide rather basic
>facilities for identifying features. The debate about what is useful to mark,
>when to mark it and so on needs to be conducted separately to a discussion on
>how to represent it when you do. Namespaces may help us in this.
>
>Jo
>
>
>David Megginson wrote:
>>
>>There are two important criteria for extensibility:
>>
>>1. Information providers need to be able to add things that the
>>   designers did not anticipate.
>>
>>2. Information consumers need to be able to recognize and disambiguate
>>   those additions, and to behave in predictable ways if they do not
>>   recognize an addition.
>>
>> > It would also be great to hear views about how to achieve such
>> > flexibility in practice.
>>
>>Well, Namespaces can help an awful lot with only a little advanced
>>planning.  To keep things simple, let's say that NewsML identified
>>three different contexts where extensibility was allowed:
>>
>>1. Within the story header.
>>
>>2. In a block context (where paragraph-like things are expected).
>>
>>3. In a phrasal context (where text, emphasised phrases, etc. are
>>   expected).
>>
>>4. In the attribute list of any element.
>>
>>(There are other contexts, but we can talk about them later.)  Now,
>>you start by defining the default behaviour for unrecognized elements
>>in any of these contexts:
>>
>>1. Within the story header, ignore the unrecognized element and all of
>>   its descendants, including any text.
>>
>>2. In a block context, ignore the unrecognized element and any text
>>   immediately inside it, but look for descendant elements that can be
>>   recognized.
>>
>>3. In a phrasal context, simply ignore the boundaries of the
>>   unrecognized element and integrate its contents into the parent
>>   element.
>>
>>4. All unrecognized attributes are ignored.
>>
>>This is pretty close to default HTML browser behaviour, so we have
>>some real-world industry experience to draw on (with the additional
>>benefit of Namespaces for disambiguation).  You could also define a
>>special attribute such as "newsml:defaultAction" to change the default
>>behavior for exceptional situations.
>>
>>Now, with these rules it's quite straight-forward for individual
>>parties to extend NewsML without breaking deployed software.  For
>>example, let's say that NewsML didn't have an inline element for
>>identifying people (it probably will, of course).  You could start
>>with a paragraph like
>>
>>  <p>Today Tony Blair met with his cabinet.</p>
>>
>>Now, Reuters may want to tag Tony Blair's name, so it can define its
>>own Namespace and do
>>
>>  <p>Today <reuters:person>Tony Blair</reuters:person> met with
>>   his cabinet.</p>
>>
>>Information consumers who know about the Reuters Namespace can use
>>this extra information; information consumers who don't will simply
>>apply the default rule for phrasal content and treat the second
>>example as identical to the first.
>>
>>To take a less canned example, now, imagine that NewsML *did* include
>>a <person> element, but that Reuters wanted to add its own unique
>>identifier to it (and NewsML didn't include that).  It would be
>>possible to have
>>
>>  <p>Today <person reuters:id="002345">Tony Blair</person> met with
>>   his cabinet.</p>
>>
>>and again, software that didn't know about the reuters:id attribute
>>would follow the default rule and ignore it.
>>
>>This kind of an approach allows for a lot of innovation without the
>>possibility of confusion, since each party's extensions will be in a
>>separate Namespace (there's no chance of AP's 'id' attribute and
>>Reuter's 'id' attribute being confused).
>>
>>
>>All the best,
>>
>>
>>David
>>
>>--
>>David Megginson                 david@...
>>           http://www.megginson.com/
>>
>>------------------------------------------------------------------------
>>To Post a message, send it to:   newsml@eGroups.com
>>
>>To Unsubscribe, send a blank message to: newsml-unsubscribe@eGroups.com
>>
>>------------------------------------------------------------------------
>>-- 20 megs of disk space in your group's Document Vault
>>-- http://www.egroups.com/docvault/newsml/?m=1
>>
>
>-----------------------------------------------------------------
>        Visit our Internet site at http://www.reuters.com
>
>Any views expressed in this message are those of  the  individual
>sender,  except  where  the sender specifically states them to be
>the views of Reuters Ltd.
>
>------------------------------------------------------------------------
>To Post a message, send it to:   newsml@eGroups.com
>To Unsubscribe, send a blank message to: newsml-unsubscribe@eGroups.com
>
>------------------------------------------------------------------------
>-- Talk to your group with your own voice!
>-- http://www.egroups.com/VoiceChatPage?listName=newsml&m=1

#11

From: johan.Lindgren@... (Johan Lindgren)
Date: Fri Dec 3, 1999 2:35 pm
Subject: Re: NewsML Extensibility without Chaos

johan.Lindgren@...

david@...,Nasta skriver:
>Something that's optional for the producer isn't optional for the
>consumer, since you have to be able to process both the optional and
>the mandatory parts.  I think that default rules can help a lot in
>NewsML, though -- systems that don't want to support the optional
>stuff can just apply a single, standard rule to all of it for each
>context.

Well, thats right. But even if we (IPTC) in our common NewsML standard
allow for all the things that need to be allowed for, we (TT) as a
provider of news in Sweden must give our customers a documentation of
what parts of the standard we are going to use and which optional
things we will not use. Then we have to follow that and our customers
need not worry about optional stuff coming unexpectadly.

That is more or less what we did in 1995-96 when we defined our SGML
DTD and started our new service. We used the envelope stuff from the
IIM that we needed and gave that an SGML coding and added the parts
from the NITF that we needed for the textual part. Since then we have
added specific table structures for our sports and financial tables.

Radio and TV tables has been handled in a separate system and not been
transmitted in the normal news channel. We do not handle other media
types in this system.

johan

#10

From: David Megginson <david@...>
Date: Fri Dec 3, 1999 2:26 pm
Subject: Re: NewsML Extensibility without Chaos

david@...

Johan Lindgren writes:
  > david@...,Nasta skriver:

  > >That sounds like a very sound business decision, but since NewsML is
  > >meant to be a standard for widespread use, it still makes sense to
  > >provide the ability for those with different business requirements.
  >
  > Of course. I am not opposed to having that in the standard. You can
  > always use the parts you like as long as they are not mandatory. I just
  > wanted to add that view on things to the other various things discussed
  > around the same thing.

Actually, it's a very reasonable concern.  Optional parts of a
standard are not free -- they still have significant costs, including
the following:

1. Cost of documentation.
2. Cost of training.
3. Cost of processing.

Something that's optional for the producer isn't optional for the
consumer, since you have to be able to process both the optional and
the mandatory parts.  I think that default rules can help a lot in
NewsML, though -- systems that don't want to support the optional
stuff can just apply a single, standard rule to all of it for each
context.

All the best,

David

--
David Megginson                 david@...
            http://www.megginson.com/

From: johan.Lindgren@... (Johan Lindgren)
Date: Fri Dec 3, 1999 2:02 pm
Subject: Re: NewsML Extensibility without Chaos

johan.Lindgren@...

david@...,Nasta skriver:
>That sounds like a very sound business decision, but since NewsML is
>meant to be a standard for widespread use, it still makes sense to
>provide the ability for those with different business requirements.

Of course. I am not opposed to having that in the standard. You can
always use the parts you like as long as they are not mandatory. I just
wanted to add that view on things to the other various things discussed
around the same thing.

johan

From: David Megginson <david@...>
Date: Fri Dec 3, 1999 1:54 pm
Subject: Re: NewsML Extensibility without Chaos

david@...

Johan Lindgren writes:
  > jo.rabin@...,Nasta skriver:

  > >At the November NewsML meeting the idea was floated that when used as an
  > >interface the markup should be externalized but that for the pupose of
  > >editing
  > >it should be put in-line.
  >
  > There are several options to this.
  > a) We can decide to support only external content markup to be as media
  > independent as possible.
  > b) We can decide to support only internal content markup for text for
  > ease of use in today tools.
  > c) We can include support for both external and internal content markup
  > to support both possibilities.
  > d) We can produce one dtd for transmission purposes where the content
  > markup is external and one for editing purposes where the content
  > markup is internal.

The best approach, really, is to separate the model from the view.
You should design a syntax-independent model that you want for the
medium- or long-term, and then come up with a view (markup/DTD) that
works now, with the intention of creating other views as technology
changes.

  > When we developed our SGML system some years back we had quite some
  > discussion on how to handle this. The final decision was not to mark up
  > any of these things at all.
  > The reason beeing that we did not feel we could fulfil that markup in
  > all texts and doing so only in some would create more problems since
  > the customers could no rely on persons, for example, beeing marked. And
  > if we could not do it 100% it was better not trying.

That sounds like a very sound business decision, but since NewsML is
meant to be a standard for widespread use, it still makes sense to
provide the ability for those with different business requirements.


All the best,


David

--
David Megginson                 david@...
            http://www.megginson.com/

From: David Megginson <david@...>
Date: Fri Dec 3, 1999 1:44 pm
Subject: Re: NewsML Extensibility without Chaos

david@...

Jo Rabin writes:

  > Namespaces do seem to be the answer to several tricky problems, but
  > do you think that XML implementations are ready for them?

I'd think so: there has been good Namespace support for about a year
in Expat, which is the C-based XML library used for Perl and Mozilla
(and Python?), and Microsoft's Internet Explorer relies heavily on
Namespaces (as will Office 2000).  Some of the Java-based parsers
include native Namespace support, and there are generic SAX filters
available for the others.  In other words, there's solid,
production-grade Namespace support available in C, C++, Perl, Java,
and probably Python, as well as within the two major XML-aware
browsers themselves.

You don't need Namespace support per se in the authoring tools or
repository tools (they can work at the raw XML 1.0 level), and it
looks like the processing side is very well covered.

On the spec side, many of the higher-level specs, such as XSL, XML
Schema, and RDF, rely on Namespaces.  I know that many of the news
providers and consumers I talk to are using or evaluating XSL for
their systems, so Namespaces will be a very comfortable fit.

  > It's interesting that you choose in-line markup of features as your
  > example.  There has been quite a lot of debate as to how this
  > should be handled in NewsML. One view says that since NewsML is an
  > expressly multi-media format you should be able to mark features in
  > any of your media types in a similar way.  This forces you to
  > consider denoting the features outside of the object in question
  > and point into the object. How exactly you do the pointing needs to
  > be media dependent but this approach buys you consistency across
  > media types in identifying things of interest.

Yes, that's the HyTime approach (HyTime actually supports both, but
this is the one that people were most excited about).  It sounds good,
but it hasn't had much success, even in well-funded systems: I'm not
at liberty to give details because I'm under NDA, but I've seen at
least one spectacular failure with this approach.

  > The contrary view states that present day tools (editors) make it
  > very easy to do in-line markup (of text) and that pragmatically
  > speaking there are no tools that support the pointing stuff.

Absolutely correct, and NewsML has to work now, not ten years from
now.  I know that there is a strong desire to design a future-proof
system, but I doubt that there's anyone on this list capable of
predicting accurately what the requirements will be in ten or twenty
years: what are the odds that you could have designed NewsML in 1985
in such a way that it would be suitable for the Web environment today?

  > At the November NewsML meeting the idea was floated that when used
  > as an interface the markup should be externalized but that for the
  > pupose of editing it should be put in-line.

It might help politically if you separate the NewsML model from the
NewsML markup.  From the perspective of the model, the information can
be out-of-line; you can then define a concrete markup based on the
model where the markup is actually inline, with the intention of
designing different concrete markups as technology changes.  Your
model can have a much longer lifespan than any markup standard, and
the model rather than the markup can form the basis of APIs, etc.

  > The second part of the markup disucssion relates to whether or not
  > it makes sense to have a "standard" notion of what to mark. Reuters
  > notion of what a person is may be very different to some other
  > provider.
  >
  > Also when to mark people is an act of editorial judgmement -
  > i.e. should you mark all people, only people for whom you have
  > biographic data, or should you mark only people who play a
  > significant role in the story, and ignore other people? If NewsML
  > contained standard markup for features would it also have to
  > standardise these kinds of rules?

Those are serious problems, but I agree with what you write further
down. The idea is to come up with a small core vocabulary of shared
concepts (person, event, etc.) and then allow extensibility.  You come
back through future iterations (say, every six months) and examine the
extensions that people make, and allow the most popular or needed to
migrate their way into the core vocabulary.

  > Further, whether something is significant to mark entirely depends
  > on the reason the recipient is receiving the story - i.e. what
  > their interests are. It is obviously useful to be able to mark
  > sports entities in news that is part of a sports service, but
  > obviously burdensome to insist that receivers of news in a
  > financial service have to know that markup of sports features is
  > even possible.

That's where the default rules can be useful.  You can define a
separate Namespace for each knowledge domain, and recipients will by
default ignore the ones they don't care about.

  > You need also to be able to mark features with several
  > interpretations, firstly because things do have different
  > interpretations (Bill Clinton is a person, he's the president of
  > the USA, his first name is Bill etc.) and secondly because you may
  > have some doubt as to what something is (especially if you are
  > trying to apply markup automatically). "Britannia" may be an
  > airline, might be a building society, could be the mythical person
  > who represents the British nation.

That can usually be handled with nesting:

   <newsml:person><newsml:name>Bill
    Clinton</newsml:name></newsml:person>

It's also useful to have standard attributes for representing a unique
identifier and a normalized form, and these can be useful even if the
application doesn't know about the element itself:

   President <name newsml:norm="Bill Clinton">Clinton</name>

  > So in summary, my feeling is that NewsML needs to provide rather
  > basic facilities for identifying features. The debate about what is
  > useful to mark, when to mark it and so on needs to be conducted
  > separately to a discussion on how to represent it when you
  > do. Namespaces may help us in this.

Yes, I think so, if only because it helps you to divide the work.  If
you have types of information (inline, tabular, block, meta, etc.) and
standard rules for handling them, then you can hand off the work of
defining the vocabularies to subject-matter experts: the sports people
can define the Namespace for sports-oriented inline markup, the
financial people can define their Namespace, etc.  There will have to
be some central architectural control and style guidelines, but for
the most part this work should be able to run in parallel.

All the best,

David

--
David Megginson                 david@...
            http://www.megginson.com/

From: johan.Lindgren@... (Johan Lindgren)
Date: Fri Dec 3, 1999 12:39 pm
Subject: Re: NewsML Extensibility without Chaos

johan.Lindgren@...

jo.rabin@...,Nasta skriver:
>At the November NewsML meeting the idea was floated that when used as an
>interface the markup should be externalized but that for the pupose of
>editing
>it should be put in-line.

There are several options to this.
a) We can decide to support only external content markup to be as media
independent as possible.
b) We can decide to support only internal content markup for text for
ease of use in today tools.
c) We can include support for both external and internal content markup
to support both possibilities.
d) We can produce one dtd for transmission purposes where the content
markup is external and one for editing purposes where the content
markup is internal.

>The second part of the markup disucssion relates to whether or not it
>makes
>sense to have a "standard" notion of what to mark. Reuters notion of
>what a
>person is may be very different to some other provider.

>Also when to mark people is an act of editorial judgmement - i.e.
>should you
>mark all people, only people for whom you have biographic data, or
>should you
>mark only people who play a significant role in the story, and ignore
>other
>people? If NewsML contained standard markup for features would it also
>have to
>standardise these kinds of rules?

When we developed our SGML system some years back we had quite some
discussion on how to handle this. The final decision was not to mark up
any of these things at all.
The reason beeing that we did not feel we could fulfil that markup in
all texts and doing so only in some would create more problems since
the customers could no rely on persons, for example, beeing marked. And
if we could not do it 100% it was better not trying.

johan

Johan Lindgren, TT, Box 865, S-85124 Sundsvall, Sweden.
Phone: +46 (0)60-176815. Mobil: 070-6528007.
Fax & messages: 060-7001971

From: Jo Rabin <jo.rabin@...>
Date: Fri Dec 3, 1999 10:48 am
Subject: Re: NewsML Extensibility without Chaos

jo.rabin@...

Namespaces do seem to be the answer to several tricky problems, but do you
think that XML implementations are ready for them?

It's interesting that you choose in-line markup of features as your example.
There has been quite a lot of debate as to how this should be handled in
NewsML. One view says that since NewsML is an expressly multi-media format you
should be able to mark features in any of your media types in a similar way.
This forces you to consider denoting the features outside of the object in
question and point into the object. How exactly you do the pointing needs to be
media dependent but this approach buys you consistency across media types in
identifying things of interest.

The contrary view states that present day tools (editors) make it very easy to
do in-line markup (of text) and that pragmatically speaking there are no tools
that support the pointing stuff.

At the November NewsML meeting the idea was floated that when used as an
interface the markup should be externalized but that for the pupose of editing
it should be put in-line.

The second part of the markup disucssion relates to whether or not it makes
sense to have a "standard" notion of what to mark. Reuters notion of what a
person is may be very different to some other provider.

Also when to mark people is an act of editorial judgmement - i.e. should you
mark all people, only people for whom you have biographic data, or should you
mark only people who play a significant role in the story, and ignore other
people? If NewsML contained standard markup for features would it also have to
standardise these kinds of rules?

Further, whether something is significant to mark entirely depends on the
reason the recipient is receiving the story - i.e. what their interests are. It
is obviously useful to be able to mark sports entities in news that is part of
a sports service, but obviously burdensome to insist that receivers of news in
a financial service have to know that markup of sports features is even
possible.

You need also to be able to mark features with several interpretations, firstly
because things do have different interpretations (Bill Clinton is a person,
he's the president of the USA, his first name is Bill etc.) and secondly
because you may have some doubt as to what something is (especially if you are
trying to apply markup automatically). "Britannia" may be an airline, might be
a building society, could be the mythical person who represents the British
nation.

So in summary, my feeling is that NewsML needs to provide rather basic
facilities for identifying features. The debate about what is useful to mark,
when to mark it and so on needs to be conducted separately to a discussion on
how to represent it when you do. Namespaces may help us in this.

Jo

David Megginson wrote:
>
>There are two important criteria for extensibility:
>
>1. Information providers need to be able to add things that the
>   designers did not anticipate.
>
>2. Information consumers need to be able to recognize and disambiguate
>   those additions, and to behave in predictable ways if they do not
>   recognize an addition.
>
> > It would also be great to hear views about how to achieve such
> > flexibility in practice.
>
>Well, Namespaces can help an awful lot with only a little advanced
>planning.  To keep things simple, let's say that NewsML identified
>three different contexts where extensibility was allowed:
>
>1. Within the story header.
>
>2. In a block context (where paragraph-like things are expected).
>
>3. In a phrasal context (where text, emphasised phrases, etc. are
>   expected).
>
>4. In the attribute list of any element.
>
>(There are other contexts, but we can talk about them later.)  Now,
>you start by defining the default behaviour for unrecognized elements
>in any of these contexts:
>
>1. Within the story header, ignore the unrecognized element and all of
>   its descendants, including any text.
>
>2. In a block context, ignore the unrecognized element and any text
>   immediately inside it, but look for descendant elements that can be
>   recognized.
>
>3. In a phrasal context, simply ignore the boundaries of the
>   unrecognized element and integrate its contents into the parent
>   element.
>
>4. All unrecognized attributes are ignored.
>
>This is pretty close to default HTML browser behaviour, so we have
>some real-world industry experience to draw on (with the additional
>benefit of Namespaces for disambiguation).  You could also define a
>special attribute such as "newsml:defaultAction" to change the default
>behavior for exceptional situations.
>
>Now, with these rules it's quite straight-forward for individual
>parties to extend NewsML without breaking deployed software.  For
>example, let's say that NewsML didn't have an inline element for
>identifying people (it probably will, of course).  You could start
>with a paragraph like
>
>  <p>Today Tony Blair met with his cabinet.</p>
>
>Now, Reuters may want to tag Tony Blair's name, so it can define its
>own Namespace and do
>
>  <p>Today <reuters:person>Tony Blair</reuters:person> met with
>   his cabinet.</p>
>
>Information consumers who know about the Reuters Namespace can use
>this extra information; information consumers who don't will simply
>apply the default rule for phrasal content and treat the second
>example as identical to the first.
>
>To take a less canned example, now, imagine that NewsML *did* include
>a <person> element, but that Reuters wanted to add its own unique
>identifier to it (and NewsML didn't include that).  It would be
>possible to have
>
>  <p>Today <person reuters:id="002345">Tony Blair</person> met with
>   his cabinet.</p>
>
>and again, software that didn't know about the reuters:id attribute
>would follow the default rule and ignore it.
>
>This kind of an approach allows for a lot of innovation without the
>possibility of confusion, since each party's extensions will be in a
>separate Namespace (there's no chance of AP's 'id' attribute and
>Reuter's 'id' attribute being confused).
>
>
>All the best,
>
>
>David
>
>--
>David Megginson                 david@...
>           http://www.megginson.com/
>
>------------------------------------------------------------------------
>To Post a message, send it to:   newsml@eGroups.com
>
>To Unsubscribe, send a blank message to: newsml-unsubscribe@eGroups.com
>
>------------------------------------------------------------------------
>-- 20 megs of disk space in your group's Document Vault
>-- http://www.egroups.com/docvault/newsml/?m=1
>

-----------------------------------------------------------------
         Visit our Internet site at http://www.reuters.com

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

From: David Megginson <david@...>
Date: Thu Dec 2, 1999 3:27 pm
Subject: NewsML Extensibility without Chaos

david@...

Jo Rabin writes:

  > A couple of the requirements of NewsML refer to extensibility,
  > specifically:
  >
  > 150 Metadata: NewsML must be capable of representing metadata
  > necessary for the searching, production [storage, archive etc] and
  > delivery of news. It must be capable of representing non-standard
  > metadata and alternative representations of the same metadata
  > (e.g. organised according to different schemes) ...
  >
  > 700 be simply extensible and flexible
  >
  > A new version of the requirements statement is being worked on
  > following the November meeting but in advance of this new version
  > being available it would be great to hear views about what kinds of
  > flexibility are needed.

There are two important criteria for extensibility:

1. Information providers need to be able to add things that the
    designers did not anticipate.

2. Information consumers need to be able to recognize and disambiguate
    those additions, and to behave in predictable ways if they do not
    recognize an addition.

  > It would also be great to hear views about how to achieve such
  > flexibility in practice.

Well, Namespaces can help an awful lot with only a little advanced
planning.  To keep things simple, let's say that NewsML identified
three different contexts where extensibility was allowed:

1. Within the story header.

2. In a block context (where paragraph-like things are expected).

3. In a phrasal context (where text, emphasised phrases, etc. are
    expected).

4. In the attribute list of any element.

(There are other contexts, but we can talk about them later.)  Now,
you start by defining the default behaviour for unrecognized elements
in any of these contexts:

1. Within the story header, ignore the unrecognized element and all of
    its descendants, including any text.

2. In a block context, ignore the unrecognized element and any text
    immediately inside it, but look for descendant elements that can be
    recognized.

3. In a phrasal context, simply ignore the boundaries of the
    unrecognized element and integrate its contents into the parent
    element.

4. All unrecognized attributes are ignored.

This is pretty close to default HTML browser behaviour, so we have
some real-world industry experience to draw on (with the additional
benefit of Namespaces for disambiguation).  You could also define a
special attribute such as "newsml:defaultAction" to change the default
behavior for exceptional situations.

Now, with these rules it's quite straight-forward for individual
parties to extend NewsML without breaking deployed software.  For
example, let's say that NewsML didn't have an inline element for
identifying people (it probably will, of course).  You could start
with a paragraph like

   <p>Today Tony Blair met with his cabinet.</p>

Now, Reuters may want to tag Tony Blair's name, so it can define its
own Namespace and do

   <p>Today <reuters:person>Tony Blair</reuters:person> met with
    his cabinet.</p>

Information consumers who know about the Reuters Namespace can use
this extra information; information consumers who don't will simply
apply the default rule for phrasal content and treat the second
example as identical to the first.

To take a less canned example, now, imagine that NewsML *did* include
a <person> element, but that Reuters wanted to add its own unique
identifier to it (and NewsML didn't include that).  It would be
possible to have

   <p>Today <person reuters:id="002345">Tony Blair</person> met with
    his cabinet.</p>

and again, software that didn't know about the reuters:id attribute
would follow the default rule and ignore it.

This kind of an approach allows for a lot of innovation without the
possibility of confusion, since each party's extensions will be in a
separate Namespace (there's no chance of AP's 'id' attribute and
Reuter's 'id' attribute being confused).

All the best,

David

--
David Megginson                 david@...
            http://www.megginson.com/

From: Jo Rabin <jo.rabin@...>
Date: Thu Dec 2, 1999 2:55 pm
Subject: FWD: Can xmlnews-story and xmlnews-meta be used in the same file?

jo.rabin@...

A couple of the requirements of NewsML refer to extensibility, specifically:

150 Metadata: NewsML must be capable of representing metadata necessary for the
searching, production [storage, archive etc] and delivery of news. It must be
capable of representing non-standard metadata and alternative representations
of the same metadata (e.g. organised according to different schemes) ...

700 be simply extensible and flexible

A new version of the requirements statement is being worked on following the
November meeting but in advance of this new version being available it would be
great to hear views about what kinds of flexibility are needed.

It would also be great to hear views about how to achieve such flexibility in
practice.

Jo

-----------------------------------------------------------------
         Visit our Internet site at http://www.reuters.com

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

Jeff Barczewski writes:

  > I was reviewing the tutorials and specs and one thing was unclear
  > to me can the xmlnews-story and xmlnews-meta data be contained in
  > one xml file or do they have to be separate?

For strict NITF conformance, they have to be separate, but it is clear
that people would very much like to be able to manage both as a single
file.  There's no reason not to merge the two for internal use, as
long as you split them apart again for exchange (if you're claiming
NITF conformance).

Unfortunately, the current NITF spec makes no allowance for
extensibility, so this is the only conformant way to handle extra
metadata.  I hope that the situation with NewsML will be better.

All the best,

David

--
David Megginson                 david@...
            http://www.megginson.com/
xmlnews-l: discussion of online news and information exchange.
To unsubscribe, send a message to majordomo@... with the
following text in the message body:
unsubscribe xmlnews-l <your address>

From: Jo Rabin <jo.rabin@...>
Date: Tue Nov 30, 1999 11:56 am
Subject: Standards for News

jo.rabin@...

To: newsml@egroups.com, xmlnews-l@...

From: Jo Rabin (jo.rabin@...)

Standards for News

Most people would agree that a single industry-wide initiative for news
standardization is desirable. A lot of people would agree that this is best
achieved through a single forum.

News standards are a product of their time and as time moves on so must the
standards. The IPTC (http://www.iptc.org) has been the industry focus for news
standards for many years. A lot of the world's news is moved using IPTC
standards - IPTC7901, the IIM and more recently NITF in both SGML and XML
versions (see http://www.nitf.org).

At the recent IPTC meeting in Amsterdam, Tony Allday and I from Reuters
presented a view on where we think the IPTC should go next - including building
on earlier IPTC work and moving forward to the next generation of XML standards
for news, which we called NewsML. A synopsis of the features that we see NewsML
including can be found at http://www.iptc.org/newsml.htm. We also made a number
of points about the IPTC operating more swiftly, working to include as wide a
range of industry participants as possible and publicizing its efforts more
clearly.

The most recent edition of the IPTC Mirror (http://www.iptc.org/mirror87.pdf)
has a full report of the Amsterdam meeting including Reuters and other
presentations and the resulting IPTC resolutions. In summary, the IPTC decided
to start an initiative called IPTC 2000 to develop NewsML. This decision is
discussed in the press release issued by the IPTC following the Amsterdam
meeting (http://www.iptc.org/newsmlprel.htm).

Following the Amsterdam meeting the IPTC 2000 initiative has got underway.
Various mailing lists have been established and a plenary session of the
working groups was held Nov 22-24 in London. At the plenary session good
progress was made to establish a firm engineering foundation for the work on
NewsML. As a result we expect to be in a position to present a formal statement
of requirements for NewsML to the IPTC membership early in the new year. We
expect also to have a statement of functionality in a similar timeframe.

Also at the London meeting a start was made in discussing NewsML's DTD. This
work will continue on mailing lists over the New Year period and there will be
a further plenary meeting of the IPTC 2000 working groups in Washington in
early February. The target for ratification of the first iteration of NewsML is
the IPTC members meeting at the end of March.

Reuters supports the idea of open standards. Open standards are not owned or
controlled by a single vendor. Open standards evolve by consensus in a forum in
which industry partners - providers, customers and technology suppliers - can
work together. We believe that while there may be differences of emphasis or
opinion on any number of issues it is important to contribute to the resolution
of those differences in such a forum. We believe that the IPTC is the
appropriate industry forum for News.

XMLNews [the elaboration of NITF by David Megginson] has some interesting
features and these should be brought forward into the next generation of news
standards. Equally, I expect there are many people who find that no existing
standard meets their requirements fully. If you have views about the future of
news standards and want to participate in formulating what that future is, we
hope that you will join us in the IPTC's IPTC 2000 initiative.

The IPTC 2000 working group mailing lists are open to members of the IPTC. The
present membership list and how to join the IPTC is attached below.

The newly-established NewsML mailing list is open to non-members of the IPTC
(subject to moderator approval). To join it, send mail to
newsml-subscribe@egroups.com. The list archive (at
http://www.egroups.com/group/newsml/) is open to the public. Those who have
joined the list may contribute and can view list membership.

Jo Rabin
Vice President Development
Reuters Limited
(IPTC News Structure and Management WG Chair)

Membership of IPTC is open to organisations and companies concerned with news
collection, distribution and publishing.

The following are members of the IPTC:-
A S Norsk Telegrambyr�, AFX News Ltd, Agence France-Presse, Agence
T�l�graphique Belge de Presse S.A., Agence T�l�graphique Suisse S.A, Agencia
EFE, Agenzia Nazionale Stampa Associata, Algemeen Nederlands Persbureau,
Alliance Europ�enne des Agences de Presse, Associated Newspapers Limited,
Associated Press, Australian Associated Press, Austria Presse-Agentur,
Bundesverband Deutscher Zeitungsverleger E.V., Canadian Press, CCI Europe,
Cybergraphic Systems Limited, Deutsche Presse-Agentur, Digital Collections
Verlagsgesellschaft MbH, Dow Jones & Company Inc, FingerPost Ltd, HINA The
Croatian News Agency, ICL in Media, IFRA, ITAR-TASS News Agency, Japan
Newspaper Publishers & Editors Association, Keystone Press AG, Kyodo News
Services, Magyar Tavirati Iroda (MTI), Middle East News Agency (MENA), NewsEDGE
Inc, New York Times Company, Newspaper Association of America, Oy Suomen
Tietotoimisto-Finska Notisbyr�m AB, PR Newswire Europe Limited, ProSieben
Digital Media GmbH, Reuters Limited, Ritzaus Bureau I'S, SAXoTECH A/S,
Slovenska Tiskovna Agencija, Tidningarnas Telegrambyr�, United Press
International, World Association of Newspapers

Companies interested in participating should email David Allen at
m_director_iptc@....

The IPTC is based at Royal Albert House, Sheet
Street, Windsor, SL4 1BE . Telephone number +441753705051 or FAX number
+441753831541.

-----------------------------------------------------------------
         Visit our Internet site at http://www.reuters.com

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

From: "Jo Rabin" <jo.rabin@...>
Date: Tue Nov 30, 1999 10:00 am
Subject: Welcome to NewsML

jo.rabin@...

Hi,

For discussion of representation of news in XML

Cheers,

Jo Rabin