I've crossed the line from developer to reporter (I'm writing for
Planet IT at http://www.planetit.com ) and am doing a story on the
state of RSS these days. The focus of the story is to expose IT
managers to interesting development with RSS and show them that it
can be used in a corporate environment. The areas I'm covering are:

- RSS .91
- SOAP Integration
- RSS with payloads
- RSS 1.0
- What to use RSS for in a corporate setting?
- Where is RSS going?

While I have talked to the usual suspects (some of the more vocal
members on this list), I want to get a larger view. As a result, I'd
like to talk to anyone on this list who's working for either a big
content provider or a large corporation and using RSS. If you're
interested in chatting with me, could you please drop me email off-
list with your phone number and let's chat! My deadline is Monday
morning so I'd like to talk to you before then.

Thanks,

TNL

As you may recall, a couple of weeks ago at a dinner with Jeff Barr, I
wished we could add the Washington Post, Wall Street Journal and New York
Times to the roster of sources syndicating in RSS. The next day a friend
sent a pointer to a public Web server where XML versions of the Times feeds
were being updated. What I saw was a perfect fit for RSS, but also an
interesting challenge. Their format, while transparently simple, would
require conversion before it could plug into RSS-based aggregators at
UserLand and Netscape and elsewhere.

On this page I describe the conversion, and point to a folder on a public
UserLand-run server where the RSS versions of the feeds can be found,
updated hourly.

http://backend.userland.com/nyTimesRssRouter

There's much more to say. The Times is doing it right, they're pushing links
to articles, following the grain of the Web; not pushing story content. This
builds flow for their site, and ad revenue, and allows them to add
interactive features at a single canonical location for each story. It's a
win for news readers and software developers, because with the Times flow in
the mix, we have news that covers the world and technology, the Times book
and movie reviews, art and education news, sports and editorial pages. This
is by far the broadest availability of syndicated content for a major
newspaper. So the Times is again a leader, not just in print, and on the
Web, but also in the newest XML-based syndication technologies.

Of course this raises the question "Is this appropriate?" and I've debated
this with myself and others over and over. I've attempted to contact the
Times, to let them know what we are doing. While they have not given their
approval, or responded in any way, the XML files are updating and remain
accessible. We are not disclosing their location or format. I see this as a
potential flow-builder for the Times, and a big boost for the developing
medium. While I hope the Times keeps these resources accessible, of course
it's their choice

Dave

______________________________
Dave Winer, UserLand Software
Daily notes: http://www.scripting.com/
"It's even worse than it appears."

I've written a small ASP script that takes an RSS .91 feed and
translates it into an RSS 1.0 feed. For those of you who are
interested, source and installation info are at
http://www.tnl.net/how/asp/RSS/RSSup/

TNL

PerCurrence proudly announces the release of the PerXML Smart
Transformation System Version 1.0.  As

a special introductory offer for the month of February, all PerXML
Server licenses are reduced 50%!

The PerXML Smart Transformation System is the premiere XML data
transformation and development

platform.  PerXML includes a runtime engine that dramatically reduces
the time and cost of building

and maintaining XML applications.  PerXML greatly simplifies integrating
native XML and legacy data

into XML documents and XSLT transformations.  The PerXML Personal
Edition supports major editing

environments such as XML Spy and XMetal Pro.  The PerXML Server supports
dynamic content integration

and delivery in XML, HTML, WML and most other Web environments.  More
information and an online

demonstration of PerXML are at:

http://www.percurrence.com/products
<http://www.percurrence.com/products>

PerXML provides greater flexibility and performance than products
costing 10 times as much.  It is

our goal to be both the price and performance leader in bringing the
power of XML data

transformation and integration to your project.

Please use this URL to download the latest evaluation version of PerXML
(file size is about 4.6 MB):

http://www.percurrence.com/downloads/default.asp
<http://www.percurrence.com/downloads/default.asp>

Please note that as of today the PerXML Personal Edition is available
from our online store for only

$99.00 per user.  The PerXML Server Edition is available for both
commercial and non-profit /

educational users.  PerXML Server licenses are reduced 50% for the month
of February!  Additional

discounts are available for site license purchasers.  You can order
PerXML here:

http://www.percurrence.com/downloads/default.asp
<http://www.percurrence.com/downloads/default.asp>

This email was sent using the PerXML remote object invoked via XML Spy.

Sincerely,

The PerCurrence Team
PerXML Rocks!

Hi All,

My name is Alis Marsden and I work for Purple Pages, which is an Irish
company.

Content Syndication is an interest of mine and I started a directory of
content syndication resources (http://www.purplepages.ie/site/content) a
little over six months ago.  I am hoping some of you may be able to help me
with an article I am writing for a newsletter.

I am interested in comparing syndication formats, primarily RSS and ICE, and
as I am not an expert on either I am really hoping that I can pick up some
comments from people who are familiar with one or both.  I'd be very
grateful if any of you could send any comment you may have to this list or
to me personally.  If anyone would prefer to speak over the telephone, just
mail me your number and I can give you a call.

Thank you very much,

Alis

Purple Pages
http://www.purplepages.ie
e: alis@...
t: + 353 1 4961943
f: + 353 1 4911497

Hello,

I have just started a new site to track and publicize any site
that is supplying a syndicated news feed of any kind. You can
find it at at http://newsfeeds.manilasites.com/ .

I will use this site to point to syndicated content of all
kinds, RSS and otherwise. I already have lots of sites to feed
in, but I also hope that people creating a new syndication will
be kind enough to keep me informed. Just send them my way!

The site includes direct links to XML content (the "XML" icon).

Enjoy,

Jeff;

Jeff Barr - Vertex Development - (mailto:jeff@...)
   Address:  4610 191st Place NE. Redmond, WA 98074;
   Phone:    Office: 425-868-4919 - Home: 425-836-5624
   Homepage: http://www.vertexdev.com/~jeff
   Weblog:   http://jeffbarr.editthispage.com/
   Resume:   http://www.vertexdev.com/~jeff/real_jb_resume.html

On Tuesday I wished to have The New York Times headlines flowing through my
desktop website, and on Wednesday, my wish came true.

http://scriptingnews.userland.com/backissues/2001/01/18#theNewYorkTimesAndXm
l

Dave

______________________________
Dave Winer, UserLand Software
Daily notes: http://www.scripting.com/
"It's even worse than it appears."

-----Original Message-----
From: Ricardo@... [mailto:ricardo.domingues@...]
Sent: vendredi, 12. janvier 2001 21:35
To: 'syndication@egroups.com'
Subject: [syndication] Content syndication companies in Latin America

Does anyone know any content syndication companies in

Latin America? Companies like ScreamingMedia, iSyndicate

that operate mostly in Portuguese and Spanish content.

 

Thanks,

Ricardo

 

------------------------------

Ricardo Domingues

Managing Director

ricardo@...

 

Emarketeer.net

www.emarketeer.net

 

Av.de Berna, nº6-2ºEsq.

Pt-1050-040 Lisboa

PORTUGAL

Tel:+351 21 781 52 60

Fax:+351 21 781 52 69

------------------------------

Version 0.9.5 of Headline Viewer is now available for download at

     http://www.headlineviewer.com

The full feature list can be found at

     http://www.headlineviewer.com/hist_095.html

Newest features:

* 2 new categories (Celebrities and Science).

* 256 new providers. The built-in list now contains 768 entries;
   loading in all of the external lists now results in a collection of
   over 3100 providers.

* Dynamic updating of the provider list from a master list stored at
   the Headline Viewer web site. Each time the product starts up, it
   loads the master list, adds any new providers, deletes any obsolete
   providers,and makes changed needed to keep existing providers up to
   date. The new list will be updated several times per week.

* Automatic invocation of a language translator to translate content
   written in Spanish, French, German, Italian, or Portuguese in to
   English.

* Special drag-drop handling for RSS content from Echo Factor. This
   site providers thousands of syndicated news feeds in 16 categories.

* Bug fixes and performance improvements.

Background Information:

   Headline Viewer displays syndicated news headlines from over 3100
   sources in a clean and organized fashion. Use Headline Viewer as a
   browser accessory, and scan headlines at lightning speed. Double-click
   on any interesting headline to open the full story in your browser.
   The program is fully skinnable and includes many user-settable
   preferences.

Jeff;

Jeff Barr - Vertex Development - (mailto:jeff@...)
   Address:  4610 191st Place NE. Redmond, WA 98074;
   Phone:    Office: 425-868-4919 - Home: 425-836-5624
   Homepage: http://www.vertexdev.com/~jeff
   Weblog:   http://jeffbarr.editthispage.com/
   Resume:   http://www.vertexdev.com/~jeff/real_jb_resume.html

Mark, I'm going to respond on the Syndication list only.

We have a new concept that I haven't seen elsewhere, for lack of a better
term, we're calling it a "content router". I get a stream of items on my
desktop from a multitude of sources. I have an EDIT button on each one which
allows me to pick from an array of categories which in the UI are just
checkboxes. For example I routed the Payloads for RSS story to the RSS
channel and to the Grateful Dead channel (which I think of as an audio
weblog). The categories are entirely up to the user (author). Now here's the
really cool part, each category is an RSS file that can serve as input to
another user-author-content-router-guy. Here are all the RSS files I'm
maintaining, and I'm just one user-author.

http://www.ourfavoritesongs.com/users/dave@userland.com/rss/

I have only an idea of where this going, I think the network effect is going
to be pretty amazing. We have just a handful of people actively doing this
stuff now, but I love the feedback loops that get going. For example when
one of my guys routes something I routed back I know he read it (or at least
I believe he did). This is just one of the nice little deals.

These channels are not synthetic, like the ones that the scraper-aggregators
produce. There are human minds at work here. It's Two-Way at it's core, I'm
not an eyeball, the user-author is a contributor, not a passive participant.

Also, I'm not sure there really are any low bandwidth connections in this
system. I give my agent four hours every night to download stuff. So far
it's only used five minutes of the four hours.

It's a different paradigm. Really.

Dave



----- Original Message -----
From: "Mark Nottingham" <mnot@...>
To: <syndication@egroups.com>
Cc: <radio-userland@egroups.com>; <decentralization@egroups.com>
Sent: Thursday, January 11, 2001 10:58 PM
Subject: Re: [syndication] Re: [radio-userland] Payloads for RSS


>
> Interesting. Good addition, Dave - this is redefining what a channel
> is, which should make RSS better/stronger.
>
> Thinking aloud, channel discrimination might go three ways (not
> mutually exclusive):
>
> * Each channel is atomic; if you want a channel about the Baltimore
>   Orioles for medium bandwidth connections, you subscribe to
>   "Baltimore Orioles for medium bandwidth connections"
>
> * The user builds the channel. Instead of
>     http://www.example.com/channel.rss
>   the user might request
> http://www.example.com/channel.rss?interests=Orioles,Braves,Yankees
>   where the query args let the user build a dynamic channel according
>   to their interests; of course, the URI can be built by a friendly,
>   web-based (or not) UI.
>
> * The channel contains enough metadata to let the user make decisions
>   about what they want to see (RDF or whatever).
>
> All three should play nicely together; ideally, there'd be some
> information somewhere that would give the client hints about this
> (e.g., the first one would have a pointer to a listing of other
> available channels by the same provider, the second a pointer to the
> web interface for building channels, or a WSDL-ish description of a
> SOAP interface (?), and the third would hopefully be self-contained,
> except for pointers to full schemas for the metadata).
>
> The flip side of that would be a standard meta tag or well-known
> location that allows the easy location of a channel/channel directory
> for a particular site (think I've already seen discussion of this
> somewhere).
>
> RE: transport over email - SOAP/XP allows use of other transport
> bindings, there's been a lot of talk about a SMTP binding. Should be
> easy once that's there.
>
> Cheers,
>
>
> On Thu, Jan 11, 2001 at 10:33:58PM -0800, Jeff Barr wrote:
> > I also like the payload idea. I was just thinking that
> > it might be an interesting email format too, since
> > it would avoid the problem of overly large enclosures.
> >
> > I think the expectation here is that the client program
> > (the one reading the RSS) would either make some decisions
> > or ask the user to make decisions about how to proceed.
> > Dave has thoughtfully provided the type and size
> > information needed to make this possible.
> >
> > For example, it could have rules such as "Download all
> > movies each night", or "Play Sound Files Immediately",
> > or "Wait until late at night to download anything
> > bigger than 1 MB". Or, even more cleverly, "Download
> > any movies where the referenced document includes the
> > string 'George Bush'". That could be kind of cool.
> >
> > I would not expect the client to, by default, "deference
> > all of the pointers" and bring down everything.
> >
> > Note that this client could be a kind of Active Napster,
> > accumulating interesting chunks of out of band content
> > as references to them stream by.
> >
> > Classification seems nice in theory, but remember that
> > each additional piece of information that we add into
> > RSS will make the format more complicated and harder to
> > use, while also bloating the file. A great attribute
> > of RSS is that it is clean, simple, and very easy to
> > produce. I was talking to a VP at the Wall St. Journal
> > yesterday, and she told me that a lot of the value
> > that they add to their syndicated content is in the form
> > of tags for geographic location and stock tickers. They
> > employ editors who do this and nothing else. Most people
> > shipping out RSS are not going to have the wherewithal
> > to do this.
> >
> > Jeff;
> >
> > -----Original Message-----
> > From: David Davies [mailto:d.a.davies@...]
> > Sent: Thursday, January 11, 2001 5:55 PM
> > To: radio-userland@egroups.com
> > Cc: decentralization@egroups.com; syndication@egroups.com
> > Subject: [syndication] Re: [radio-userland] Payloads for RSS
> >
> >
> > I think this is a truly inspired addition to RSS and will help us in our
use
> > of syndicated data no end. I can't wait to get going with it.
> >
> > One thought though. I can imagine a danger of super-bloat with the
enclosure
> > field in regular RSS channels. Suppose I subscribe to a hypothetical CNN
> > sports news channel and with this new enclosure element all its sports
> > stories come with a video clip of a sports game. Maybe I'm only
interested
> > in following one team let alone one sport yet when I wake up in the
morning
> > I find my personal aggregator (e.g. MyUserland on the Desktop) has
> > downloaded megabytes (tens, hundreds, more?) of videos of stuff I'm just
not
> > interested in. Wow, there goes my hard disk.
> >
> > What's the answer? More specific RSS channels?
> >
> > On the radio-UserLand list we talked a little about further refinement
of a
> > channel's data using a category or keyword element.
> >
> > This is a difficult area because of standardisation of keyword
> > classification systems. At one level (e.g. some professional areas)
there
> > are many standard classification systems in operation. In my own field
of
> > medicine we have an international keyword system making highly specific
data
> > syndication a lot easier. Granted, there is a lack of standardisation in
> > other areas though there are pointers such as Library of Congress or the
> > Dewey decimal system. There will always be other areas (personal weblogs
> > with automatic syndication are a good example) where no classification
> > system is ever going to emerge though maybe with these channels by
> > definition they're more focussed so you may not mind or even get lots of
> > unwanted enclosure elements.
> >
> > This isn't a criticism, I think enclosures are a major leap forward, for
me
> > and our application of RSS at least. Just trying to keep the discussion
> > moving forward.
> >
> > Cheers,
> >
> > David
> >
> >
> >
> >
> >
> >
> >
>
> --
> Mark Nottingham
> http://www.mnot.net/
>
>
>

Interesting. Good addition, Dave - this is redefining what a channel
is, which should make RSS better/stronger.

Thinking aloud, channel discrimination might go three ways (not
mutually exclusive):

* Each channel is atomic; if you want a channel about the Baltimore
   Orioles for medium bandwidth connections, you subscribe to
   "Baltimore Orioles for medium bandwidth connections"

* The user builds the channel. Instead of
     http://www.example.com/channel.rss
   the user might request
http://www.example.com/channel.rss?interests=Orioles,Braves,Yankees
   where the query args let the user build a dynamic channel according
   to their interests; of course, the URI can be built by a friendly,
   web-based (or not) UI.

* The channel contains enough metadata to let the user make decisions
   about what they want to see (RDF or whatever).

All three should play nicely together; ideally, there'd be some
information somewhere that would give the client hints about this
(e.g., the first one would have a pointer to a listing of other
available channels by the same provider, the second a pointer to the
web interface for building channels, or a WSDL-ish description of a
SOAP interface (?), and the third would hopefully be self-contained,
except for pointers to full schemas for the metadata).

The flip side of that would be a standard meta tag or well-known
location that allows the easy location of a channel/channel directory
for a particular site (think I've already seen discussion of this
somewhere).

RE: transport over email - SOAP/XP allows use of other transport
bindings, there's been a lot of talk about a SMTP binding. Should be
easy once that's there.

Cheers,


On Thu, Jan 11, 2001 at 10:33:58PM -0800, Jeff Barr wrote:
> I also like the payload idea. I was just thinking that
> it might be an interesting email format too, since
> it would avoid the problem of overly large enclosures.
>
> I think the expectation here is that the client program
> (the one reading the RSS) would either make some decisions
> or ask the user to make decisions about how to proceed.
> Dave has thoughtfully provided the type and size
> information needed to make this possible.
>
> For example, it could have rules such as "Download all
> movies each night", or "Play Sound Files Immediately",
> or "Wait until late at night to download anything
> bigger than 1 MB". Or, even more cleverly, "Download
> any movies where the referenced document includes the
> string 'George Bush'". That could be kind of cool.
>
> I would not expect the client to, by default, "deference
> all of the pointers" and bring down everything.
>
> Note that this client could be a kind of Active Napster,
> accumulating interesting chunks of out of band content
> as references to them stream by.
>
> Classification seems nice in theory, but remember that
> each additional piece of information that we add into
> RSS will make the format more complicated and harder to
> use, while also bloating the file. A great attribute
> of RSS is that it is clean, simple, and very easy to
> produce. I was talking to a VP at the Wall St. Journal
> yesterday, and she told me that a lot of the value
> that they add to their syndicated content is in the form
> of tags for geographic location and stock tickers. They
> employ editors who do this and nothing else. Most people
> shipping out RSS are not going to have the wherewithal
> to do this.
>
> Jeff;
>
> -----Original Message-----
> From: David Davies [mailto:d.a.davies@...]
> Sent: Thursday, January 11, 2001 5:55 PM
> To: radio-userland@egroups.com
> Cc: decentralization@egroups.com; syndication@egroups.com
> Subject: [syndication] Re: [radio-userland] Payloads for RSS
>
>
> I think this is a truly inspired addition to RSS and will help us in our use
> of syndicated data no end. I can't wait to get going with it.
>
> One thought though. I can imagine a danger of super-bloat with the enclosure
> field in regular RSS channels. Suppose I subscribe to a hypothetical CNN
> sports news channel and with this new enclosure element all its sports
> stories come with a video clip of a sports game. Maybe I'm only interested
> in following one team let alone one sport yet when I wake up in the morning
> I find my personal aggregator (e.g. MyUserland on the Desktop) has
> downloaded megabytes (tens, hundreds, more?) of videos of stuff I'm just not
> interested in. Wow, there goes my hard disk.
>
> What's the answer? More specific RSS channels?
>
> On the radio-UserLand list we talked a little about further refinement of a
> channel's data using a category or keyword element.
>
> This is a difficult area because of standardisation of keyword
> classification systems. At one level (e.g. some professional areas) there
> are many standard classification systems in operation. In my own field of
> medicine we have an international keyword system making highly specific data
> syndication a lot easier. Granted, there is a lack of standardisation in
> other areas though there are pointers such as Library of Congress or the
> Dewey decimal system. There will always be other areas (personal weblogs
> with automatic syndication are a good example) where no classification
> system is ever going to emerge though maybe with these channels by
> definition they're more focussed so you may not mind or even get lots of
> unwanted enclosure elements.
>
> This isn't a criticism, I think enclosures are a major leap forward, for me
> and our application of RSS at least. Just trying to keep the discussion
> moving forward.
>
> Cheers,
>
> David
>
>
>
>
>
>
>

--
Mark Nottingham
http://www.mnot.net/

I also like the payload idea. I was just thinking that
it might be an interesting email format too, since
it would avoid the problem of overly large enclosures.

I think the expectation here is that the client program
(the one reading the RSS) would either make some decisions
or ask the user to make decisions about how to proceed.
Dave has thoughtfully provided the type and size
information needed to make this possible.

For example, it could have rules such as "Download all
movies each night", or "Play Sound Files Immediately",
or "Wait until late at night to download anything
bigger than 1 MB". Or, even more cleverly, "Download
any movies where the referenced document includes the
string 'George Bush'". That could be kind of cool.

I would not expect the client to, by default, "deference
all of the pointers" and bring down everything.

Note that this client could be a kind of Active Napster,
accumulating interesting chunks of out of band content
as references to them stream by.

Classification seems nice in theory, but remember that
each additional piece of information that we add into
RSS will make the format more complicated and harder to
use, while also bloating the file. A great attribute
of RSS is that it is clean, simple, and very easy to
produce. I was talking to a VP at the Wall St. Journal
yesterday, and she told me that a lot of the value
that they add to their syndicated content is in the form
of tags for geographic location and stock tickers. They
employ editors who do this and nothing else. Most people
shipping out RSS are not going to have the wherewithal
to do this.

Jeff;

-----Original Message-----
From: David Davies [mailto:d.a.davies@...]
Sent: Thursday, January 11, 2001 5:55 PM
To: radio-userland@egroups.com
Cc: decentralization@egroups.com; syndication@egroups.com
Subject: [syndication] Re: [radio-userland] Payloads for RSS


I think this is a truly inspired addition to RSS and will help us in our use
of syndicated data no end. I can't wait to get going with it.

One thought though. I can imagine a danger of super-bloat with the enclosure
field in regular RSS channels. Suppose I subscribe to a hypothetical CNN
sports news channel and with this new enclosure element all its sports
stories come with a video clip of a sports game. Maybe I'm only interested
in following one team let alone one sport yet when I wake up in the morning
I find my personal aggregator (e.g. MyUserland on the Desktop) has
downloaded megabytes (tens, hundreds, more?) of videos of stuff I'm just not
interested in. Wow, there goes my hard disk.

What's the answer? More specific RSS channels?

On the radio-UserLand list we talked a little about further refinement of a
channel's data using a category or keyword element.

This is a difficult area because of standardisation of keyword
classification systems. At one level (e.g. some professional areas) there
are many standard classification systems in operation. In my own field of
medicine we have an international keyword system making highly specific data
syndication a lot easier. Granted, there is a lack of standardisation in
other areas though there are pointers such as Library of Congress or the
Dewey decimal system. There will always be other areas (personal weblogs
with automatic syndication are a good example) where no classification
system is ever going to emerge though maybe with these channels by
definition they're more focussed so you may not mind or even get lots of
unwanted enclosure elements.

This isn't a criticism, I think enclosures are a major leap forward, for me
and our application of RSS at least. Just trying to keep the discussion
moving forward.

Cheers,

David

I think this is a truly inspired addition to RSS and will help us in our use
of syndicated data no end. I can't wait to get going with it.

One thought though. I can imagine a danger of super-bloat with the enclosure
field in regular RSS channels. Suppose I subscribe to a hypothetical CNN
sports news channel and with this new enclosure element all its sports
stories come with a video clip of a sports game. Maybe I'm only interested
in following one team let alone one sport yet when I wake up in the morning
I find my personal aggregator (e.g. MyUserland on the Desktop) has
downloaded megabytes (tens, hundreds, more?) of videos of stuff I'm just not
interested in. Wow, there goes my hard disk.

What's the answer? More specific RSS channels?

On the radio-UserLand list we talked a little about further refinement of a
channel's data using a category or keyword element.

This is a difficult area because of standardisation of keyword
classification systems. At one level (e.g. some professional areas) there
are many standard classification systems in operation. In my own field of
medicine we have an international keyword system making highly specific data
syndication a lot easier. Granted, there is a lack of standardisation in
other areas though there are pointers such as Library of Congress or the
Dewey decimal system. There will always be other areas (personal weblogs
with automatic syndication are a good example) where no classification
system is ever going to emerge though maybe with these channels by
definition they're more focussed so you may not mind or even get lots of
unwanted enclosure elements.

This isn't a criticism, I think enclosures are a major leap forward, for me
and our application of RSS at least. Just trying to keep the discussion
moving forward.

Cheers,

David

When I started talking with Adam Curry late last year, he wanted me to think
about high quality video on the Internet, and I totally didn't want to hear
about it. Like a lot of people I had tried it, and found it unsatisfying and
frankly, exhausting. But Adam persisted and showed me that if I was willing
to change my point of view, it could work, without any waiting and with very
high quality.

http://www.thetwowayweb.com/payloadsForRss

Dave Winer

The application is different, but I think the problem is the same -
there's a need for a way to describe a channel that pushes out
messages that describe, in some way, changes to / metadata about
resources contained in the channel.

I believe that a well-designed solution can encompass both
applications, and that it's beneficial to do so, because of the
difficulty in making such a protocol scale well across the Internet.


On Sun, Jan 07, 2001 at 10:21:01PM -0800, Dave Winer wrote:
> Mark, the problem we're solving is the inverse of the one you are taking on.
>
> You're wanting to make the current browsing experience better.
>
> We're trying to change the browsing experience so that the click-and-wait
> effect goes away.
>
> Both activities happen in the Web browser, but that's all they have in
> common.
>
> Both threads are worth pursuing, imho.
>
> That said, we will have a cache on the local users' hard drive, independent
> of the browser (but accessed through the browser) and that cache can benefit
> from an invalidation protocol/format.
>
> Dave
>
>
> ----- Original Message -----
> From: "Mark Nottingham" <mnot@...>
> To: <syndication@egroups.com>
> Cc: "Radio Userland" <radio-userland@egroups.com>
> Sent: Sunday, January 07, 2001 10:09 PM
> Subject: Re: [syndication] SOAP meets RSS
>
>
> >
> > Hey Dave,
> >
> > You might be interested in one of the proposed work items for the
> > IETF's WEBI working group (aka WREC) - we just had our first meeting
> > in San Diego.
> >
> > Basically, there are a lot of people in the caching industry who are
> > interested in an invalidation protocol. However, as others have
> > mentioned, it can/should be defined as a more general problem -
> > subscribing to a channel on which updates/invalidations/whatever are
> > sent, pertaining to a particular resource or group of resources.
> >
> > I'd encourage you and anyone else interested to have a look at the
> > presentation -
> >   http://www.mnot.net/papers/ietf-49-wrec.{ppt|pdf|htm}
> >
> > and join the mailing list -
> >   webi-request@...
> > (majordomo)
> >
> > We're just about to start discussing the work items and gathering
> > requirements, so it's a perfect time to have a say. We're also
> > looking for document editors, etc.
> >
> > Cheers
> >
> >
> > On Sat, Jan 06, 2001 at 08:50:21AM -0800, Dave Winer wrote:
> > > Thanks Dan..
> > >
> > > Hammering-avoidance is very much on my mind as our servers are getting
> > > pounded by search engine crawlers in vast numbers and always at the
> worst
> > > possible time. At the same time we're experiencing substantial growth in
> our
> > > free Manila hosting service, and money is a big issue (as it is
> everywhere
> > > these days it seems).
> > >
> > > In that context, we designed this system so that:
> > >
> > > 1. Most bits are served statically.
> > >
> > > 2. The reliance on SOAP/XML-RPC is limited to smallish messages, usually
> > > with just a few scalars and possibly an array. These travel over the
> wire,
> > > even for people with relatively slow lines (like me), very quickly.
> > >
> > > 3. As much as possible use the CPU cycles on the recipient side,
> conserve
> > > CPU cycles in the cloud so that it scales. That's the price you pay for
> good
> > > current content, but it's an easy price for most to pay because the
> cycles
> > > on the recipient end usually are being wasted (this is the P2P
> proposition).
> > >
> > > I want other content types to be able to hitch a ride in a RSS channel.
> We
> > > have a design for this, but it isn't written up yet. The goal is to make
> > > exactly the types you mention, audio and video, flow around in higher
> > > fidelity, optimizing the user experience and using the Internet when it
> > > isn't so busy. It's largely a UI problem, the technology is brain-dead
> > > simple. I'll post a note here for sure when it's written up.
> > >
> > > Basically it flips around the Akamai equation. My goal isn't to get the
> bits
> > > to you as fast as possible while you wait for them, but to have the bits
> > > arrive before you even know they're there. ;->
> > >
> > > Dave
> > >
> > >
> > > > Interesting...
> > > >
> > > > How would you distinguish the RSS case from the general problem of
> > > > caching and change notification for Web content? Would you propose a
> > > > SOAP-based solution to the broader case of wanting to know straight
> away
> > > > about changes to arbitrary Web-accessible chunks of data (images,
> audio,
> > > > markup etc). Normally one would try to retrieve all this via HTTP
> caches
> > > > to avoid hammering the original server; do you reckon it would it make
> > > > sense for Web caches to use SOAP or similar to stay more up to date?
> > >
> > >
> > >
> > >
> >
> > --
> > Mark Nottingham
> > http://www.mnot.net/
> >
> >
> >
>
>
>
>

--
Mark Nottingham
http://www.mnot.net/

Mark, the problem we're solving is the inverse of the one you are taking on.

You're wanting to make the current browsing experience better.

We're trying to change the browsing experience so that the click-and-wait
effect goes away.

Both activities happen in the Web browser, but that's all they have in
common.

Both threads are worth pursuing, imho.

That said, we will have a cache on the local users' hard drive, independent
of the browser (but accessed through the browser) and that cache can benefit
from an invalidation protocol/format.

Dave


----- Original Message -----
From: "Mark Nottingham" <mnot@...>
To: <syndication@egroups.com>
Cc: "Radio Userland" <radio-userland@egroups.com>
Sent: Sunday, January 07, 2001 10:09 PM
Subject: Re: [syndication] SOAP meets RSS


>
> Hey Dave,
>
> You might be interested in one of the proposed work items for the
> IETF's WEBI working group (aka WREC) - we just had our first meeting
> in San Diego.
>
> Basically, there are a lot of people in the caching industry who are
> interested in an invalidation protocol. However, as others have
> mentioned, it can/should be defined as a more general problem -
> subscribing to a channel on which updates/invalidations/whatever are
> sent, pertaining to a particular resource or group of resources.
>
> I'd encourage you and anyone else interested to have a look at the
> presentation -
>   http://www.mnot.net/papers/ietf-49-wrec.{ppt|pdf|htm}
>
> and join the mailing list -
>   webi-request@...
> (majordomo)
>
> We're just about to start discussing the work items and gathering
> requirements, so it's a perfect time to have a say. We're also
> looking for document editors, etc.
>
> Cheers
>
>
> On Sat, Jan 06, 2001 at 08:50:21AM -0800, Dave Winer wrote:
> > Thanks Dan..
> >
> > Hammering-avoidance is very much on my mind as our servers are getting
> > pounded by search engine crawlers in vast numbers and always at the
worst
> > possible time. At the same time we're experiencing substantial growth in
our
> > free Manila hosting service, and money is a big issue (as it is
everywhere
> > these days it seems).
> >
> > In that context, we designed this system so that:
> >
> > 1. Most bits are served statically.
> >
> > 2. The reliance on SOAP/XML-RPC is limited to smallish messages, usually
> > with just a few scalars and possibly an array. These travel over the
wire,
> > even for people with relatively slow lines (like me), very quickly.
> >
> > 3. As much as possible use the CPU cycles on the recipient side,
conserve
> > CPU cycles in the cloud so that it scales. That's the price you pay for
good
> > current content, but it's an easy price for most to pay because the
cycles
> > on the recipient end usually are being wasted (this is the P2P
proposition).
> >
> > I want other content types to be able to hitch a ride in a RSS channel.
We
> > have a design for this, but it isn't written up yet. The goal is to make
> > exactly the types you mention, audio and video, flow around in higher
> > fidelity, optimizing the user experience and using the Internet when it
> > isn't so busy. It's largely a UI problem, the technology is brain-dead
> > simple. I'll post a note here for sure when it's written up.
> >
> > Basically it flips around the Akamai equation. My goal isn't to get the
bits
> > to you as fast as possible while you wait for them, but to have the bits
> > arrive before you even know they're there. ;->
> >
> > Dave
> >
> >
> > > Interesting...
> > >
> > > How would you distinguish the RSS case from the general problem of
> > > caching and change notification for Web content? Would you propose a
> > > SOAP-based solution to the broader case of wanting to know straight
away
> > > about changes to arbitrary Web-accessible chunks of data (images,
audio,
> > > markup etc). Normally one would try to retrieve all this via HTTP
caches
> > > to avoid hammering the original server; do you reckon it would it make
> > > sense for Web caches to use SOAP or similar to stay more up to date?
> >
> >
> >
> >
>
> --
> Mark Nottingham
> http://www.mnot.net/
>
>
>

Hey Dave,

You might be interested in one of the proposed work items for the
IETF's WEBI working group (aka WREC) - we just had our first meeting
in San Diego.

Basically, there are a lot of people in the caching industry who are
interested in an invalidation protocol. However, as others have
mentioned, it can/should be defined as a more general problem -
subscribing to a channel on which updates/invalidations/whatever are
sent, pertaining to a particular resource or group of resources.

I'd encourage you and anyone else interested to have a look at the
presentation -
   http://www.mnot.net/papers/ietf-49-wrec.{ppt|pdf|htm}

and join the mailing list -
   webi-request@...
(majordomo)

We're just about to start discussing the work items and gathering
requirements, so it's a perfect time to have a say. We're also
looking for document editors, etc.

Cheers


On Sat, Jan 06, 2001 at 08:50:21AM -0800, Dave Winer wrote:
> Thanks Dan..
>
> Hammering-avoidance is very much on my mind as our servers are getting
> pounded by search engine crawlers in vast numbers and always at the worst
> possible time. At the same time we're experiencing substantial growth in our
> free Manila hosting service, and money is a big issue (as it is everywhere
> these days it seems).
>
> In that context, we designed this system so that:
>
> 1. Most bits are served statically.
>
> 2. The reliance on SOAP/XML-RPC is limited to smallish messages, usually
> with just a few scalars and possibly an array. These travel over the wire,
> even for people with relatively slow lines (like me), very quickly.
>
> 3. As much as possible use the CPU cycles on the recipient side, conserve
> CPU cycles in the cloud so that it scales. That's the price you pay for good
> current content, but it's an easy price for most to pay because the cycles
> on the recipient end usually are being wasted (this is the P2P proposition).
>
> I want other content types to be able to hitch a ride in a RSS channel. We
> have a design for this, but it isn't written up yet. The goal is to make
> exactly the types you mention, audio and video, flow around in higher
> fidelity, optimizing the user experience and using the Internet when it
> isn't so busy. It's largely a UI problem, the technology is brain-dead
> simple. I'll post a note here for sure when it's written up.
>
> Basically it flips around the Akamai equation. My goal isn't to get the bits
> to you as fast as possible while you wait for them, but to have the bits
> arrive before you even know they're there. ;->
>
> Dave
>
>
> > Interesting...
> >
> > How would you distinguish the RSS case from the general problem of
> > caching and change notification for Web content? Would you propose a
> > SOAP-based solution to the broader case of wanting to know straight away
> > about changes to arbitrary Web-accessible chunks of data (images, audio,
> > markup etc). Normally one would try to retrieve all this via HTTP caches
> > to avoid hammering the original server; do you reckon it would it make
> > sense for Web caches to use SOAP or similar to stay more up to date?
>
>
>
>

--
Mark Nottingham
http://www.mnot.net/

Dave Winer <dave@...> wrote:

> Offlist, Aaron and I got it working. I released the new bits to our testers,
> so now http-post is part of the SOAP-Meets-RSS spec. Thanks Aaron for a very
> nice collaboration this morning. Dave

And thanks to Dave for his help and willingness to get this working. It was
wonderful to work with you. Also thanks to Dan Lyke for helping test and
refine the previous version of the spec[1].

I encourage all of you who are having problems with heavy load to take a
look at the changed page spec[2] and think about integrating it into your
RSS files, CMSes, aggregators or search engines. It's easy to use and fully
compatible with both RSS 0.91 and RSS 1.0:

0.91: http://www.thetwowayweb.com/soapMeetsRss
  1.0: http://my.theinfo.org/changed/1.0/rss/

Please let me know if you have any questions.

[1] http://my.theinfo.org/changed/past.adp
[2] http://my.theinfo.org/changed/1.0/

--
[ Aaron Swartz | me@... | http://www.aaronsw.com ]

Offlist, Aaron and I got it working. I released the new bits to our testers,
so now http-post is part of the SOAP-Meets-RSS spec. Thanks Aaron for a very
nice collaboration this morning. Dave

Dave Winer <dave@...> wrote:

> Do you want me to mock up a prototype or bake this into our implementation?
> If the latter, it might take a few hours to do.

It's probably better to bake it in -- take all the time you need.

I'll be here when it's ready. ;-)

--
[ Aaron Swartz | me@... | http://www.aaronsw.com ]

No, not yet.

The URL I sent was bogus.

I have to do a little more thinking.

Do you want me to mock up a prototype or bake this into our implementation?

If the latter, it might take a few hours to do.

Dave


----- Original Message -----
From: "Aaron Swartz" <aswartz@...>
To: <syndication@egroups.com>
Sent: Sunday, January 07, 2001 10:57 AM
Subject: [syndication] Re: SOAP meets RSS


> Dave Winer <dave@...> wrote:
>
> > Yippee.
>
> Looks good on my end too! Are you ready for me to send an update
> notification?
>
> --
> [ Aaron Swartz | me@... | http://www.aaronsw.com ]
>
>
>
>

Dave Winer <dave@...> wrote:

> Yippee.

Looks good on my end too! Are you ready for me to send an update
notification?

--
[ Aaron Swartz | me@... | http://www.aaronsw.com ]

Dave Winer <dave@...> wrote:

> Help me sort it out. I want to send two bits of information to your cloud:
>
> 1. The URL of the channel I want to subscribe to. (target)
>
> 2. The URL of the page to receive notification of a change. (responder)

Sounds perfect. As an example, for my test of the implementation, here's
what I used:

args.responder = "http://my.theinfo.org/changed/email/?aswartz@upclink.com"
args.target = "http://aaronsw.com/school/"

Hope this helps,

--
[ Aaron Swartz | me@... | http://www.aaronsw.com ]

It worked!

http://www.scripting.com/images/successWithAaron.gif

Yippee.

Dave

I'm sure the spec is very clear, after you know how it's supposed to work.

My dense mind gets very confused by all the %3A's and %2Fs.

Too many miles on the tires, esp in the last few weeks.

Help me sort it out. I want to send two bits of information to your cloud:

1. The URL of the channel I want to subscribe to. (target)

2. The URL of the page to receive notification of a change. (responder)

Is that correct?

Dave


----- Original Message -----
From: "Aaron Swartz" <aswartz@...>
To: <syndication@egroups.com>
Sent: Sunday, January 07, 2001 10:36 AM
Subject: [syndication] Re: SOAP meets RSS


> Dave Winer <dave@...> wrote:
>
> > Hmmm, nope, still getting the same error.
>
> Oops, sorry. What was I thinking? You're using the wrong syntax. When you
> want to subscribe to a feed, you use this syntax:
>
>
responder=http%3A%2F%2Fmy.theinfo.org%2Fchanged%2Fdemo%2F&target=http%3A%2F%
> 2Fwww.theinfo.org
>
> That's the syntax you use to send out notifications that a page has
changed.
> Is the spec unclear? Perhaps I need to make it easier to understand.
>
> Sorry about that,
> --
> [ Aaron Swartz | me@... | http://www.aaronsw.com ]
>
>
>
>

Aaron Swartz <aswartz@...> wrote:

> That's the syntax you use to send out notifications that a page has changed.

Sorry, I meant "The syntax you were using is the one to send out
notifications that a page has changed."

I'm just not being clear today...

--
[ Aaron Swartz | me@... | http://www.aaronsw.com ]

Dave Winer <dave@...> wrote:

> Hmmm, nope, still getting the same error.

Oops, sorry. What was I thinking? You're using the wrong syntax. When you
want to subscribe to a feed, you use this syntax:

responder=http%3A%2F%2Fmy.theinfo.org%2Fchanged%2Fdemo%2F&target=http%3A%2F%
2Fwww.theinfo.org

That's the syntax you use to send out notifications that a page has changed.
Is the spec unclear? Perhaps I need to make it easier to understand.

Sorry about that,
--
[ Aaron Swartz | me@... | http://www.aaronsw.com ]

Ken MacLeod <ken@...> wrote:

> A firewall friendly service might allow for responding to
> notifications from a variety of sources, and then forwarding them by
> subscription.

This is a definite use case for changedPage. The way I'd see it being done
is the machine inside the firewall would call up the notifier, then it would
subscribe it's responder to the feed, with a special URL that indicated that
it was interested in subscribing -- something like:

http://firewall.example.org/changedPage/?notify=myMachinesID

When the firewall received notifications, it would pass them on to the
proper machine. The problem with this system is that if many machines inside
the firewall subscribe to the same feed, there's a lot of waste -- the
firewall machine only needs to be notified once. In such a case, the
inside-firewall machine could subscribe the firewall to the service, then
notify the firewall that it was interested in these notifications.

--
[ Aaron Swartz | me@... | http://www.aaronsw.com ]