Hi folks

I have been discussing with André Radke, Paolo Valdemarin, Marc Barrot,
and Doug Baron the way Radio generates RSS and would now like to throw
the discussion open. We'd appreciate it if you could share your
experience and insight.

Radio already provides an easy way to experiment with new RSS formats by
implementing an writeRssFile() callback. Using this Radio will delegate
generating RSS to your code. It works very well but has some
shortcomings when you start implementing applications, namely:

1) Only the first registered callback is ever invoked.
2) That callback is responsible for generating the entire RSS stream.

What this means is that only 1 application can modify the RSS. But with
the popularity of Radio it won't be long before more and more
applications want to exploit the power of the new namespace support and
start adding their own tags to the RSS feed. Additionally because the
callback isn't just "doing it's thing" it has to take care of how RSS is
generated and ensure compatibility. This is probably impractical in the
long term.

With this in mind we have set about designing an improved RSS generator
that fully supports the new namespaces of RSS2.0 and allows tool
developers to co-exist peacefully with each other. We hope that in this
way we can help propel Radio further forwards.

Here's the context: liveTopics allows bloggers to attach metadata to
each post in their weblog. At the moment that information is used to
build tables of contents and XFML topic-maps. However it would be very
useful if the same topic information could be exported as part of the
RSS feed from each weblog. An example application would be an
aggregator that combined multiple feeds using topic information (e.g. a
Radio feed could be created from a number of feeds using only posts with
the topic 'Radio' attached). To achieve this we need to add new
information to each <item/> and, since we want to protect it in a
namespace, adding an "xmlns:prefix=" to the <rss/> element.

What we are proposing is to modify how the radio.weblog.writeRssFile()
script works so that it support "in-stream" callbacks. These will allow
multiple applications to independently, but harmoniously, augment the
RSS stream without having to worry about the details of how it gets
done.

In order to support this we have a significant change to the way the RSS
generator works in that instead of producing the RSS feed as a long
string concatenation we use XML tables. Each callback will therefore
receive an XML table containing the element to be modified. The logic
is exactly the same as the original writeRssFile() so that compatibility
is guaranteed (and confirmed by testing).

There are two additional benefits to using XML tables:

1. it's much easier to understand & modify the structure of the XML 2.
there's a significant performance improvement since we're leveraging
kernel verbs for encoding XML.

We are proposing using multiple levels of callback so:

<item/> => user.radio.callbacks.rss.item.myToolCallback =>
itemCallback( adrRssTable, adrChannelTable, categoryName, adrItemTable,
adrPost )
<channel/> => user.radio.callbacks.rss.channel.myToolCallback
=> channelCallback( adrRssTable, adrChannelTable, categoryName )
<feed/> => user.radio.callbacks.rss.feed.myToolCallback =>
feedCallback( adrRssTable, categoryName )

Each time an <item/>, <channel/>, or <rss/> is processed the
corresponding callback gets called and modifies the element
appropriately.

liveTopics now includes a reference implementation of this new
writeRssFile() approach which is generating the RSS2.0 feed for my
weblog every day (see also:
http://radio.weblogs.com/0107808/2002/11/27.html#a566 and
http://radio.weblogs.com/0107808/rss.xml) including topic information
protected by namespace. This feed has passed the RSS validator at
Userland and archive.org, works in Radio and has been independently
tested by 3rd party RSS parsers.

It is our hope that, with feedback from the community, Userland takes
this revision and incorporates it into Radio for everyone to use.

Notes:

(1) XML tables have proved to be faster than string concatenation for
this application. According to André performance advantage will
continue in the favour of tables as feed size (and hence string size)
increases.

(2) The reference implementation was created by converting
section-by-section the existing writeRssFile() to generate XML tables
instead of strings. This preserves the original logic intact and thus
where no callbacks are invoked it produces semantically identical RSS
XML, i.e. there will be no breakage with existing clients.

We look forward to your comments.

Regards,

Matt
André
Paolo
Marc
Doug