We were happily reading Red Herring via RSS, but then a few days ago, it
disappeared.

http://www.herring.com/cgi-bin/edit/harvest.pl?id=userland

Is there any way it can come back?

We miss the Herring!!

Dave

______________________________
Dave Winer, UserLand Software
Daily notes: http://www.scripting.com/
"It's even worse than it appears."

>We do the inverse of that. The channel list from UserLand only includes
>sites that have updated in the last week. Dave

Quite too, although I think earlier criticism on the list was that a week
was too short of a time - I've about 2 hours away from having a list of
feeds updated in the past month...


--
Morbus Iff
     _____
    |  (@ \
  __| <\/> |____.         Here we have Head-Wound Morbus who discovers
|  |------|    |          that flesh objects in motion tend to stay
|_ |______|_ __|          in motion until they hit metal and bleed.
   (_)      (_)

-05--- <\/> ---- <http://www.disobey.com/> --- Bad Ascii, Short Notice ----

We do the inverse of that. The channel list from UserLand only includes
sites that have updated in the last week. Dave

> I'd be interested in doing this - count me in. In conjunction, I'd also
> like to see a "Ghost Sites of RSS" - a large list of feeds that are 404'd
> or haven't been updated in years. Cos mainly, people who say that have
> 3000+ newssources are totally accurate, but only about 700 of them are
> usable and updated, and still there.
>
>
> Morbus Iff
> .sig on other machine.
> http://www.disobey.com/
> http://www.gamegrene.com/
>
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>

Morbus Iff wrote:

> I'd be interested in doing this - count me in. In conjunction, I'd also
> like to see a "Ghost Sites of RSS" - a large list of feeds that are 404'd
> or haven't been updated in years
count me in also. See the service list for NewsIsFree at
http://www.newsisfree.com/ocs/directory.xml

I could include data like last update etc, and build a list of new
sources...


Mike Krus

>I really like Julian's idea. If there was a way to truly
  >enforce the "schema" that Julian proposes, then it would be
  >easy to accumulate a day/week/month's worth of feeds into
  >an OCS list. What if we create a simple web page somewhere
  >that prompts for this information, validates it, and then
  >sends it to the mailing list, archives it, and so forth?
  >
  >Anyone interested in doing this? Anyone? Buehler?

I'd be interested in doing this - count me in. In conjunction, I'd also
like to see a "Ghost Sites of RSS" - a large list of feeds that are 404'd
or haven't been updated in years. Cos mainly, people who say that have
3000+ newssources are totally accurate, but only about 700 of them are
usable and updated, and still there.


Morbus Iff
.sig on other machine.
http://www.disobey.com/
http://www.gamegrene.com/

I really like Julian's idea. If there was a way to truly
enforce the "schema" that Julian proposes, then it would be
easy to accumulate a day/week/month's worth of feeds into
an OCS list. What if we create a simple web page somewhere
that prompts for this information, validates it, and then
sends it to the mailing list, archives it, and so forth?

Anyone interested in doing this? Anyone? Buehler?

While I don't list every new feed, there is an RSS feed of my
newsfeeds weblog (http://newsfeeds.manilasites.com). Just
visit the site and use the "XML" image in the Syndication
section on the left.

Jeff;

-----Original Message-----
From: Julian Bond [mailto:julian@...]
Sent: Wednesday, February 28, 2001 1:53 PM
To: syndication@yahoogroups.com
Subject: [syndication] Alternate delivery mechanisms for RSS


There was some talk about delivering RSS via NNTP.

I get one of the Moreover channels as a daily email. This has a nice
side effect that I don't miss any headlines in this particular category,
because I can easily keep a local archive of a week or so. On Friday
mornings I go through the previous week and pick out the most
interesting 15 or so, which I then re-publish[1]. This is almost
impossible with a web interface as if you miss a day, the headlines are
gone.

Over the last few weeks we've discovered Yahoogroups, Nasdaq, I wonder
what's next. Usenet? How about a meta-meta-news channel. An RSS channel
of new RSS channels. Call it RSS-Announce or something. In fact we could
bootstrap this easily with RSS-Announce@Yahoogroups.com Make it an
"Anyone can post" mailing list and start spreading the word that any new
RSS channel should be announced with the <channel> section in the body
and <channel>.<title> as the subject. Yes?

[1] Julian's weekly B2B picks is at
http://www.netmarketseurope.com/nme_weekly_picks.rss

--
Julian Bond eMail: julian@...
HomeURL: http://www.shockwav.demon.co.uk/
WorkURL: http://www.netmarketseurope.com/
WebLog: http://roguemoon.manilasites.com/
M: +44 (0)77 5907 2173  T: +44 (0)20 7420 4363
ICQ:33679668 tag:So many words, so little time



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

In article <159401c0a1ad$2388b5b0$33a1dc40@murphy2>, Dave Winer
<dave@...> writes
>Julian, I think it's people who will make this work. Editors who do the
>sorting. The model weblog for this is Tomalak's Realm, where the author
>rarely speaks, but his selection of articles has a theme and a point of
>view. I want more human front-page-editors who select from all that's
>available on the Web.

Well I don't. Well actually I do, but not for this. I'm concerned that
all we're doing is pumping up the volume. I think, like you, I want
everyone to be an editor, everybody to be a journalist. But I'm
concerned that we'll end up with N people reading N magazines. Or to put
it another way, every channel will have an audience of one.

But I'm struggling here, I have an insatiable appetite for news, but I'm
skipping way more headlines than I'm clicking through. The signal to
noise ratio is just too low. In a way, I'm trying to find ways of
getting a better (for me) selection of raw news.

--
Julian Bond eMail: julian@...
HomeURL: http://www.shockwav.demon.co.uk/
WorkURL: http://www.netmarketseurope.com/
WebLog: http://roguemoon.manilasites.com/
M: +44 (0)77 5907 2173  T: +44 (0)20 7420 4363
ICQ:33679668 tag:So many words, so little time

There was some talk about delivering RSS via NNTP.

I get one of the Moreover channels as a daily email. This has a nice
side effect that I don't miss any headlines in this particular category,
because I can easily keep a local archive of a week or so. On Friday
mornings I go through the previous week and pick out the most
interesting 15 or so, which I then re-publish[1]. This is almost
impossible with a web interface as if you miss a day, the headlines are
gone.

Over the last few weeks we've discovered Yahoogroups, Nasdaq, I wonder
what's next. Usenet? How about a meta-meta-news channel. An RSS channel
of new RSS channels. Call it RSS-Announce or something. In fact we could
bootstrap this easily with RSS-Announce@Yahoogroups.com Make it an
"Anyone can post" mailing list and start spreading the word that any new
RSS channel should be announced with the <channel> section in the body
and <channel>.<title> as the subject. Yes?

[1] Julian's weekly B2B picks is at
http://www.netmarketseurope.com/nme_weekly_picks.rss

--
Julian Bond eMail: julian@...
HomeURL: http://www.shockwav.demon.co.uk/
WorkURL: http://www.netmarketseurope.com/
WebLog: http://roguemoon.manilasites.com/
M: +44 (0)77 5907 2173  T: +44 (0)20 7420 4363
ICQ:33679668 tag:So many words, so little time

Dave Winer wrote:

> Julian, I think it's people who will make this work. Editors who do the
> sorting. The model weblog for this is Tomalak's Realm, where the author
> rarely speaks, but his selection of articles has a theme and a point of
> view. I want more human front-page-editors who select from all that's
> available on the Web. Dave
I aggree! One feature we were thinking of for NewsIsFree is letting
users building their own channel by selecting news items on the site.
The channel could be used on site as a kind of bookmark feature. But,
most interestingly, could be exported to an RSS file which RU could
monitor, so that the user could edit it there and republish it as
needed.

Would this be useful?

Of course, this could also be achieved by direct XML-RPC calls between
NewsIsFree and RU, but RSS if a little more ubiquitous right now (not
to mention all other export formats support by NewsIsFree, such as
JavaScript or plain HTML).


Mike

Julian, I think it's people who will make this work. Editors who do the
sorting. The model weblog for this is Tomalak's Realm, where the author
rarely speaks, but his selection of articles has a theme and a point of
view. I want more human front-page-editors who select from all that's
available on the Web. Dave

Http://roguemoon.manilasites.com/2001/02/28
An essay on syndication, personal publishing and "The Daily Me". It's
this last that I'm interested in here.

The Media Lab had an idea 15 years ago that might result from high-
bandwidth always-on connections. This was a completely personalized
newspaper. Headline news, sports news, some random think pieces, your
favourite journalist along with some meeting reminders and a couple of
SMS urgent messages.

With RSS, Weblogs and some ingenuity this is pretty much buildable now.
I can see a few problems here though and I keep coming back to them. The
focus at the moment seems to be all around specific channels and
presenting the news sorted by channel name and then time. But as I've
said before it's not the channel I'm interested in it's the items. I sub
to slashdot RSS for the occasional article that perks my interest but
I'm still scanning by eye the other 100. SO the really hard part here is
trying to get an aggregating/indexing system to learn what I like and
what I don't.

Do you think it's possible that some sort of Firefly system or semi
intelligent algorithm could watch my behaviour (and "people like me") to
produce a personalized filter of the global RSS items? Or is the filter
definition going to be too hard and complex to use? The point here is
that RSS just solves the delivery problem and makes the raw data machine
readable. But it doesn't make it any easier for me to "Sip from the
firehose".

--
Julian Bond eMail: julian@...
HomeURL: http://www.shockwav.demon.co.uk/
WorkURL: http://www.netmarketseurope.com/
WebLog: http://roguemoon.manilasites.com/
M: +44 (0)77 5907 2173  T: +44 (0)20 7420 4363
ICQ:33679668 tag:So many words, so little time

In article <3A9CE2AF.8000202@...>, Mike Krus
<mkrus@...> writes
>Julian Bond wrote:
>
>> So I'm firmly in the camp of not caring if one channel is the same as
>> another and trying to identify this. Just categorize them so I can find
>> them and aggregate and re-index the content into new channels so it's
>> useful.
>and you promise not to complain if you get duplicates?

If the duplicate is in the underlying story which will have a unique
URL. ie the <link> field is identical. If you did the sort of synthetic
channel production I'm talking about, you could strip the duplicate
<item>s at the end.

I already get duplicate items by subbing Moreover's Vertical Portal and
Online Auction channels.

And if I sub to channels that turn out to be the same, I won't complain,
I'll just unsub one of them.

--
Julian Bond eMail: julian@...
HomeURL: http://www.shockwav.demon.co.uk/
WorkURL: http://www.netmarketseurope.com/
WebLog: http://roguemoon.manilasites.com/
M: +44 (0)77 5907 2173  T: +44 (0)20 7420 4363
ICQ:33679668 tag:So many words, so little time

Hi,

Julian Bond wrote:

> So I'm firmly in the camp of not caring if one channel is the same as
> another and trying to identify this. Just categorize them so I can find
> them and aggregate and re-index the content into new channels so it's
> useful.
and you promise not to complain if you get duplicates?

Mike Krus

I'm working on a rant about this whole area which I'll publish here (and
other places).

ISTM that there is a fixation here with "Channel" that is related to the
fixation with "Brand" in the world generally. As a user, frankly I could
care less exactly which magazine/website/channel published the news.
It's the content I care about. I don't want every news headline from
Line56. I want every news headline about a story which contains the
acronym "B2B". I want every news headline for articles written by Clay
Shirky, wherever they're published. And so on.

So I'm firmly in the camp of not caring if one channel is the same as
another and trying to identify this. Just categorize them so I can find
them and aggregate and re-index the content into new channels so it's
useful. And if a channel disappears (404) or doesn't publish for a
month, knock it out of the list.

--
Julian Bond eMail: julian@...
HomeURL: http://www.shockwav.demon.co.uk/
WorkURL: http://www.netmarketseurope.com/
WebLog: http://roguemoon.manilasites.com/
M: +44 (0)77 5907 2173  T: +44 (0)20 7420 4363
ICQ:33679668 tag:So many words, so little time

The home page of ourfavoritesongs.com now shows the current top 100 channels
being scanned hourly by signed-on Radio users.

http://www.ourfavoritesongs.com/

The purpose of this list is to allow people outside the community to see
what we're tuned into.

Dave

______________________________
Dave Winer, UserLand Software
Daily notes: http://www.scripting.com/
"It's even worse than it appears."

It's great to see people use our scripting interface to add features to
Manila. That's how Manila Express does it, and Radio. Dave


----- Original Message -----
From: "Mike Krus" <mkrus@...>
To: <syndication@yahoogroups.com>
Sent: Tuesday, February 27, 2001 12:59 AM
Subject: [syndication] NewsIsFree: Posting to Manila powered sites


> Hi,
>
> Since Userland refused even acknowledge the problem with using
> ManilaExpress ( http://manilaexpress.userland.com/ ) with Mozilla to
> post news item directly to your Manila site (
> http://manila.userland.com/ ), I decided to use one of their
> better features: the XML-RPC (http://www.xml-rpc.com/ ) interface to
> post to your homepage!
>
> When you view an article in frame mode (this is the default),
> clicking on the (very) little cactus icon will open a popup window
> where you can edit your submission and send it directly to your
> Manila powered home page.
>
> Enter the site host without the http:// (so you currently can only
> post to sites where the RPC2 interface is at the root). User name
> defaults to your email address but can be changed. Values for both
> those fields are stored in a cookie so you don't have to type them
> again. Password is not stored at all!
>
>
> Mike Krus
>
>
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>

>> a Microsoft COM GUID. These are 128 bits long and are, as the docs
>> "unique to a high degree of certainty." It would be cool if this
>> was included somewhere within the RSS <channel> tag, and if there
>
>for the feed at http://www.scripting.com.  Just take an MD5 hash of
>the URL "http://www.scripting.com".  You get back a 128-bit number,
>and you are more likely to get hit by meteors twice in the same spot
>than that someone else's feed will have the same unique ID.  Now if

Hmmm... that's not a bad idea.
How do other people feel about md5-ing the xmlUrl?

--
Morbus Iff
     _____
    |  (@ \
  __| <\/> |____.         Here we have Head-Wound Morbus who discovers
|  |------|    |          that flesh objects in motion tend to stay
|_ |______|_ __|          in motion until they hit metal and bleed.
   (_)      (_)

-05--- <\/> ---- <http://www.disobey.com/> --- Bad Ascii, Short Notice ----

>Ideally, I would like the IDs to be unique across time and space, like
>a Microsoft COM GUID. These are 128 bits long and are, as the docs say,
>"unique to a high degree of certainty." It would be cool if this
>was included somewhere within the RSS <channel> tag, and if there was
>a web service that the site author would use to assign it.

I think any approach that politely asks users to put the GUID in their
channel listing is going to fail. Not enough people realize that the <?xml>
statement has to be the first thing in the channel - I've had to write a a
few bits of code that handle that, as well as encoding errors. If they
can't that right, then asking them about "GUID"'s and "128 big"'s and so
forth is scary.

>For a moment I thought that a SOAP service that accepted a candidate
>URL, checked it against a master list, then either returned an existing
>GUID or created and then returned a new one. But this takes the human
>judgement out of the loop and then we end up needing an alias list again
>to say that "these two GUIDs are the same."

Would it be all that much harder to check it against a number of known
aliases? Say we set up a SOAP server that does exactly what you suggest. It
returns three response codes: -1, 0, 1. 1 is happy go-lucky and means that
"you can add it to your list - I just added it to mine". 0 means it's
already in the current list.

-1 means it was so close to something that a human has to take a look at it.
We could define a series of checks:

   - first 25 characters the same.
   - only the extensions are different in URI.
   - matches a known alias in your chv_aliases.xml
   - etc, etc, etc.

>For me, this all starts and ends as a quality of service issue. I want
>to give my users a unique list of content. If they are regular readers
>of, say, "CNET", and they've done some customization to that channel
>within my program, then I don't want them to be confused if the source
>of that channel changes, say, from scraped data (once available from
>the late Internet Alchemy) to Moreover data, to what could at sometime
>become a direct feed. I've actually got lots of stuff in the works on
>the customization front (none of which I want to talk about yet), and

I'd actually like to work with you on some of this customization you plan.
As per a previous message, I currently don't like how you handle your
chv_providers.xml, because for someone who wants to use your list in
another program, there's all this extra fun happy customization junk (i'll
ignore the "well, don't use my list" / "wasn't meant for public
consumption").

I was following Winer's lead with Radio Userland actually - he's got his
master list (serviceList4.xml) of sites updated, and then he's got
mySubscriptions.opml, which can contain all the customization, page views,
colors and so forth that he wants. This doesn't fit perfectly into CHV
though, since there's not really a concept of subscribing to a channel,
only of active and inactive...

--
Morbus Iff
     _____
    |  (@ \
  __| <\/> |____.         Here we have Head-Wound Morbus who discovers
|  |------|    |          that flesh objects in motion tend to stay
|_ |______|_ __|          in motion until they hit metal and bleed.
   (_)      (_)

-05--- <\/> ---- <http://www.disobey.com/> --- Bad Ascii, Short Notice ----

Hi,

Since Userland refused even acknowledge the problem with using
ManilaExpress ( http://manilaexpress.userland.com/ ) with Mozilla to
post news item directly to your Manila site (
http://manila.userland.com/ ), I decided to use one of their
better features: the XML-RPC (http://www.xml-rpc.com/ ) interface to
post to your homepage!

When you view an article in frame mode (this is the default),
clicking on the (very) little cactus icon will open a popup window
where you can edit your submission and send it directly to your
Manila powered home page.

Enter the site host without the http:// (so you currently can only
post to sites where the RPC2 interface is at the root). User name
defaults to your email address but can be changed. Values for both
those fields are stored in a cookie so you don't have to type them
again. Password is not stored at all!


Mike Krus

I'd like to announce the beta availability of a new web site which
demonstrates our XML/XSLT portal and personalisation software.

The site is www.paperdog.com and is motivated by the desire to create
something fun and useful (without ads! for as long as is possible).

No service guarantees are made at this point however comments would be
gratefully received.


Paul Freeman

Architek Ltd.
Suite D, Sinclair House,
2a The Avenue,
London, W13 8NT

t: +44 (0)20 8758 7543
m: +44 (0)7973 165 360
f: +44 (0)20 8758 7505

I noticed Aaron is moving forward with some metadata-over-NNTP, which
is something I think can work out very nicely.  I've been thinking of
something that could help this work better, but am not sure yet if it
is unworkable, so I'm hoping some people here will have some ideas.

Suppose that we are distributing RSS feeds by posting them to a
newsgroup, something like alt.metadata.rss.  Obviously some RSS
postings would have just a few items, while others would be larger.
Ideally, the newsgroup would become just one huge river of RSS items
describing stories or content that people were submitting, and
indexers could passively monitor the newsgroup and get content of
interest without the extra step of pulling from a site with the
appropriate feed.  Maybe with RSS, sheer volume wouldn't be a
problem, but lets pretend that we have enough authors participating
that we are getting tens of thousands of individual posts (with two
or three items) pers day.  One useful service to be performed at the
NNTP layer would be for a news server to combine multiple small
messages into larger individual messages before forwarding around
USENET.  Assuming that you could never get all of USENET to
cooperate, you would have to use the existing mechanisms to make this
work.  So your own personal "indexing agent" could just read a bunch
of messages, combine, and "re-post" a new message containing the
merged RSS.  The main problem you need to solve is that, when you
combine all of these messages into one, you do not want the "big"
message to propagate back to the servers that you got the little ones
from (you'll piss off lots of USENET admins).  You want to prop
the "combined" message forward, but *stop* propping the little ones
at that point.

One possible idea hinges on the way that NNTP posts have header info
that tells which servers they've been through (separated by !
symbol).  You could aggregate/merge a batch of RSS entries that all
originated form the same source, and tack on the original source
chain to the "combined" message before posting back.  I know this
used to work on USENET, but I am not sure if some of the attempts to
stop spammers or something regard such headers on POST with
suspicion?  And maybe ISPs are disallowing this now?  I have no idea..

The second piece would be to post CANCEL messages for all of the
messages that had been merged, with the headers specifying that the
CANCEL had already been seen by the source systems (not true, but
would prevent the little guys from getting blasted on previous
servers).  Maybe there are some more restrictions on forging cancels
now, too?  (Used to be great fun on Usenet, when you got in a big
flame war with someone, start cancelling all of their posts at the
source and forge posts from that person recanting their evil ways and
apologizing for ever thinking you wrong, just make sure the header
has their home nntp server listed so they never see their apocryphal
concessions ... I wonder if it is still possible? :-))

Jeff, I think the Hotlist feature in Radio UserLand is going to turn out to
be a way of spreading the work around among a lot of users without very much
centralizing. The channels that rise to the top are the ones that are good.
If there are two versions in popular use that will show up too. We only have
about 20 people using it now, but it's doing a good job of showing us where
the good stuff is. Dave


----- Original Message -----
From: "Jeff Barr" <jeff@...>
To: <rss-dev@yahoogroups.com>; <syndication@yahoogroups.com>
Sent: Monday, February 26, 2001 10:26 PM
Subject: [syndication] RE: [RSS-DEV] Cross-Site Unique ID Formats? And ID
Debating...


> Wow, I get on a plane for a few hours and lots happens :-).
>
> Ideally, I would like the IDs to be unique across time and space, like
> a Microsoft COM GUID. These are 128 bits long and are, as the docs say,
> "unique to a high degree of certainty." It would be cool if this
> was included somewhere within the RSS <channel> tag, and if there was
> a web service that the site author would use to assign it.
>
> However, I do not think that there is a totally automated way to make
> sure that the same (leaving out the definition of "same" for the moment)
> GUID is assigned to different URLs that are aliases to the same info.
> Human judgement is needed to look at the info contained and pointed
> to by two or more candidate URLs and to decide if they are the same or
> not. So this service would have to have the ability to say "I'll get
> back to you on that."
>
> For a moment I thought that a SOAP service that accepted a candidate
> URL, checked it against a master list, then either returned an existing
> GUID or created and then returned a new one. But this takes the human
> judgement out of the loop and then we end up needing an alias list again
> to say that "these two GUIDs are the same."
>
> For me, this all starts and ends as a quality of service issue. I want
> to give my users a unique list of content. If they are regular readers
> of, say, "CNET", and they've done some customization to that channel
> within my program, then I don't want them to be confused if the source
> of that channel changes, say, from scraped data (once available from
> the late Internet Alchemy) to Moreover data, to what could at sometime
> become a direct feed. I've actually got lots of stuff in the works on
> the customization front (none of which I want to talk about yet), and
> it is possible that a user could productively spend 10 or 15 minutes
> on customization. The unique ID is what allows me to do this, and to
> not have them lose their work when the data source or the source's
> name changes. I cannot index the contributions by "CNET", and I cannot
> index them to the URL which is supplying the content. Neither is truly
> fixed.
>
> I never defined "same" before. This is where things get hairy. For the
> work I am doing, I look at the content returned and make a judgement
> call. Identical content is two different formats (e.g. <rss> and
<moreover>)
> is the same. Different human languages (English vs. French) are not.
> Parameterized URLs which specify different item counts to return are
> the same. I make this call from the user's point of view.
>
> Jeff;
>
> -----Original Message-----
> From: Morbus Iff [mailto:morbus@...]
> Sent: Monday, February 26, 2001 7:13 PM
> To: rss-dev@yahoogroups.com
> Subject: RE: [RSS-DEV] Cross-Site Unique ID Formats? And ID Debating...
>
>
> >Right - all I meant is that the creator of an xml channel either in terms
> of
> >an aggregated topic from multiple sources (in which case a third party) a
> >channel of commentary (weblog style) or an original source (in which case
> >the ID should relate to the original publisher via a namespace or
someother
> >and not the third party who may be creating the xml channel) should not
> >necessarily be who the namespace relates to. This way you do not get
> >duplicates and can just use a namespace without a timestamp.
>
> I don't think I'm understannding then - "the creator of an xml channel"?
> So, we'd be forcing the end user to decide a id for themselves?
> How do you feel about the "xmlUrl as unique id" idea?
>
> --
> Morbus Iff
>                   Here we have One DimensionalMorbus - Flatter than
>    _____       Brooke Shields, able to to be ignored for days at a time,
>                slower than a Microsoft Slug. Defender of AOL users, Bill
>                 Gates and other one dimensional life forms who mutter
>                                   "I don't get it..."
>
> -03--- <\/> ---- <http://www.disobey.com/> --- Bad Ascii, Short
Notice ----
>
>
> To unsubscribe from this group, send an email to:
> rss-dev-unsubscribe@egroups.com
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>

> a Microsoft COM GUID. These are 128 bits long and are, as the docs
> "unique to a high degree of certainty." It would be cool if this
> was included somewhere within the RSS <channel> tag, and if there

FYI, I am pretty sure that the GUID spec was around before it was
used in COM (OSF DCE maybe?).  Doesn't every operating system on the
planet now have the capability to generate unique GUIDs?

Besides, the new versions of Microsoft OS's use a secure hash as part
of the GUID-generation process.  Think about this: you want a GUID
for the feed at http://www.scripting.com.  Just take an MD5 hash of
the URL "http://www.scripting.com".  You get back a 128-bit number,
and you are more likely to get hit by meteors twice in the same spot
than that someone else's feed will have the same unique ID.  Now if
you move the feed somewhere else, you can choose to keep the previous
unique ID.  Simple, huh?  (although, a 128-bit number is just a tiny
bit smaller than the string "http://www.scripting.com", and if you
represent the ID in hex, as GUIDs are, the GUID is actually *larger*!)

Anyway, I am not sure if that helps much -- I don't quite know what
the unique IDs would be used for, but that is a simple way to
generate a standard-sized unique ID that can be quickly searched or
joined..

Wow, I get on a plane for a few hours and lots happens :-).

Ideally, I would like the IDs to be unique across time and space, like
a Microsoft COM GUID. These are 128 bits long and are, as the docs say,
"unique to a high degree of certainty." It would be cool if this
was included somewhere within the RSS <channel> tag, and if there was
a web service that the site author would use to assign it.

However, I do not think that there is a totally automated way to make
sure that the same (leaving out the definition of "same" for the moment)
GUID is assigned to different URLs that are aliases to the same info.
Human judgement is needed to look at the info contained and pointed
to by two or more candidate URLs and to decide if they are the same or
not. So this service would have to have the ability to say "I'll get
back to you on that."

For a moment I thought that a SOAP service that accepted a candidate
URL, checked it against a master list, then either returned an existing
GUID or created and then returned a new one. But this takes the human
judgement out of the loop and then we end up needing an alias list again
to say that "these two GUIDs are the same."

For me, this all starts and ends as a quality of service issue. I want
to give my users a unique list of content. If they are regular readers
of, say, "CNET", and they've done some customization to that channel
within my program, then I don't want them to be confused if the source
of that channel changes, say, from scraped data (once available from
the late Internet Alchemy) to Moreover data, to what could at sometime
become a direct feed. I've actually got lots of stuff in the works on
the customization front (none of which I want to talk about yet), and
it is possible that a user could productively spend 10 or 15 minutes
on customization. The unique ID is what allows me to do this, and to
not have them lose their work when the data source or the source's
name changes. I cannot index the contributions by "CNET", and I cannot
index them to the URL which is supplying the content. Neither is truly
fixed.

I never defined "same" before. This is where things get hairy. For the
work I am doing, I look at the content returned and make a judgement
call. Identical content is two different formats (e.g. <rss> and <moreover>)
is the same. Different human languages (English vs. French) are not.
Parameterized URLs which specify different item counts to return are
the same. I make this call from the user's point of view.

Jeff;

-----Original Message-----
From: Morbus Iff [mailto:morbus@...]
Sent: Monday, February 26, 2001 7:13 PM
To: rss-dev@yahoogroups.com
Subject: RE: [RSS-DEV] Cross-Site Unique ID Formats? And ID Debating...


>Right - all I meant is that the creator of an xml channel either in terms
of
>an aggregated topic from multiple sources (in which case a third party) a
>channel of commentary (weblog style) or an original source (in which case
>the ID should relate to the original publisher via a namespace or someother
>and not the third party who may be creating the xml channel) should not
>necessarily be who the namespace relates to. This way you do not get
>duplicates and can just use a namespace without a timestamp.

I don't think I'm understannding then - "the creator of an xml channel"?
So, we'd be forcing the end user to decide a id for themselves?
How do you feel about the "xmlUrl as unique id" idea?

--
Morbus Iff
                   Here we have One DimensionalMorbus - Flatter than
    _____       Brooke Shields, able to to be ignored for days at a time,
                slower than a Microsoft Slug. Defender of AOL users, Bill
                 Gates and other one dimensional life forms who mutter
                                   "I don't get it..."

-03--- <\/> ---- <http://www.disobey.com/> --- Bad Ascii, Short Notice ----


To unsubscribe from this group, send an email to:
rss-dev-unsubscribe@egroups.com



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

There already is a pretty good way to do this:

/dierken.com/my_thing/my_id/whatever_i_want

The VALUE of the ID isn't important, its the NAME.
Everybody gets hung up on the URL used to retrieve something as the only ID.
The 'location' value is misleading. Use some other piece of meta-data. Like
newsgroups and e-mail has Message-ID. Most asynch system have message
headers for unique id.

How things are combined into one large list is much more interesting
problem. Take a look at how Ohaha (a p2p thing) does stuff. Imagine a single
DB table. Each column is a 'key' - but the behind-the-scenes index for that
column isn't on one central server. The index is scattered over a set of
machines. Each index is on a different set of machines. You ask the machines
that you know about & they tell you what they know, plus what their
neighbors know - common p2p stuff. The trick - the 'worse is better'
approach here - is that you don't get EVERY record, but you get enough to do
your job. Don't worry about the time to get each and every record. The fact
that you can get a large number of records from an astronomically huge
database is the cool part. For example, in music searches, people don't need
to know every ocurrance of Johny Cash's songs that are online, just enough
to pick one to listen to.

mike

----- Original Message -----
From: "Morbus Iff" <morbus@...>
To: <syndication@yahoogroups.com>; <rss-dev@yahoogroups.com>
Sent: Monday, February 26, 2001 6:10 PM
Subject: [syndication] Cross-Site Unique ID Formats? And ID Debating...


> >> Again, does our current purpose scale well? Can we editorialize one
million
> >> rss feeds, checking for all possible aliases, defects, user idiocies,
and
> >> so forth, while still giving ourselves time to read the damn feeds that
> >> we're programming software for?
> >
> >It certainly may not scale, but even if it doesn't it's something that's
> >doable and useful now, and can hopefully grow into something that _can_
> >scale.
>
> Ok. Say we all agree that "major providers of rss feed listings will use a
unique id architecture", then how should we handle it? Say XMLtree wants to
keep their list going, I want to make a list, and Barr wants to keep his
individual list going.
>
> How, how, how do we combine them all together into one massive list? We
certainly don't need duplication of effort. And what if Winer doesn't want
to jump in? How do we handle those who don't join? Do we?
>
> What if we did unique IDs based off "epoch seconds,list id", such that if
there were three providers numbered 3, 4, and 5 and they each added a
channel at the same exact moment in time, it'd be something like
"91374293,3" and "91374293,4" and "91374293,5"?
>
> I still, to be honest, don't see any compelling reason to use unique ids.
Sure, it may be good *now*, when the world is relatively small and meager,
but why waste the time worrying about them if they're *only good enough* for
this point in time?
>
> The words "hopefully grow into something that can scale" do not put me at
ease. We shouldn't hope something scales, we should make damn well that it
does. 'Member Gnutella? (and I'll ignore any comments about "well, hey, he
said it would only handle 300 hosts")...
>
> --
> Morbus Iff
>
>  Disobey has been mentioned in The Netly News, Internet World, ABC News,
>    Bruce Sterling's Dead Media Notes and many more. Microsoft and 3Com
>   ripped us off also... that's GOTTA mean we're important. And hell, we
>   got a rise out of Playboy! With sections that have nothing to do with
>     the others, you'll like at least one thing. No, really. Go there.
>
> -07--- <\/> ---- <http://www.disobey.com/> ------- Bad Ego, Any
Notice ----
>
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>

I'm not a fan of OCS for a few reasons:

   - it looks ugly as hell.
   - it looks complicated as hell (ie., bad "code shui").
   - it duplicates a lot of data, bloating the lists.

I'm not fond of Headline Viewer's format either, because it adds
application specific elements in the output, being useless to other
applications. To be honest, the format I've found most entertaining is Dave
Winer's format, shown roughly below:

    <service>
       <url>http://www.flutterby.com/main.rdf</url>
       <id>17</id>
       <timeLastChange>Mon, 26 Feb 2001 02:01:48 GMT</timeLastChange>
       <format>RSS</format>
       <title>Flutterby!</title>
       <description>Last updated 2001-02-25 17:34:44-08</description>
       <htmlUrl>http://www.flutterby.com/</htmlUrl>
       <approved>Sun, 21 Mar 1999 15:35:29 GMT</approved>
    </service>

My main gripes with his format:

   - "url" is to ambiguous. if this is just a "service",
     then how do we know that "url" should point to an
     rss feed?
   - "timelastchange" is too wordy.
   - "format" can change at any time, which can be
     pretty annoying. and what is "RSS" anyways? perhaps
     today it means "defaults to 1.0", but tomorrow it
     may be "0.92".

I also find Microsoft's CDF format appealing because of it's scheduling
tags, which instruct clients when the channel should be updated.

So, long story short, this is what I propose. I'd like a lot of comments as
soon as possible, because I'm in the process of making a large list, and
would rather "get it as close to right as possible" before having to redo
it ;) ...

   <service>
     <description>The best little example description around!</description>
     <htmlurl>http://www.superlugnuts.com/example.html</htmlurl>
     <imageurl>http://www.superlugnuts.com/image.jpg</imageurl>
     <xmlurl>http://www.superlugnuts.com/example.xml</xmlurl>
     <lastmodified>Thu, 08 Jun 2000 13:40:05 GMT</lastmodified>
     <language>en</language>
     <title>Super LugNuts and Happy Examples</title>
     <schedule>
        <!-- the cdf crap from microsoft. -->
     </schedule>
   </service>

Primary differences:

  - "xmlurl" replaces "url". there is no longer any explicit "url".
  - no more "format". clients should figure it out based off the
    content of the channel - this provides the most flexibility
    since the format could change at any moment.
  - "lastmodified" as opposed to "timelastchange".
  - "language" added. (perhaps filter by language?)
  - there's no "id" just yet, cos i'm still up in the air.

I've also removed the "approved" element. Theory: Moreover and Headline
Viewer do subjects. But they don't do any sort of hierarchy. Perhaps add a
<subject> separated by colons, that demonstrate the hiearchy?

Perhaps "Geography: US: New Hampshire" - and no limit on number of <subject>'s?

--
Morbus Iff

   OXFORDMORBUS DICKTIONARY:  Ug - pronounced uh-g (hard 'g' sound)

   Verb, New Hampshire slang, meaning: 1) when one shouldn't have to thank
   another for additional services or favors rendered as part of a larger
   service arrangement but does, the proprietor responds with "Ug". Which
   is to "ug" it out, as in a comment tag in HTML code or a REM comment
   in DOS.

-08--- <\/> ---- <http://www.disobey.com/> ------- Webster Goes to Hell ---

Well, since my name was mentioned, we already did that a few years ago. We
give channels registered on my.userland.com an id. But the idea didn't seem
to catch on too well.

The idea of getting on top of what's going on in RSS-land is not likely to
work. It's just an illusion perhaps to some that this is the top of the
pyramid or the center of the universe. New ideas take a lot of selling,
patience and hard work, and then bending to help people do it the way they
want to do it, for whatever reason they want to.

I think Jeff's question is right on point -- where is your XMLization, let's
start there. You can be #1 if you want, but get some good content flowing,
that's what it's about, not organizing it or getting people like me to "jump
in".

BTW, this isn't really a problem in the big picture. If a site stops
updating, for whatever reason, have your script stop reading it so often. If
you have two versions of the same site in your scan list, delete one.
Low-tech is the way of the Web and the Way To Go. I love linkrot. It's why
the Web works.

Dave


----- Original Message -----
From: "Morbus Iff" <morbus@...>
To: <syndication@yahoogroups.com>; <rss-dev@yahoogroups.com>
Sent: Monday, February 26, 2001 6:10 PM
Subject: [syndication] Cross-Site Unique ID Formats? And ID Debating...


> >> Again, does our current purpose scale well? Can we editorialize one
million
> >> rss feeds, checking for all possible aliases, defects, user idiocies,
and
> >> so forth, while still giving ourselves time to read the damn feeds that
> >> we're programming software for?
> >
> >It certainly may not scale, but even if it doesn't it's something that's
> >doable and useful now, and can hopefully grow into something that _can_
> >scale.
>
> Ok. Say we all agree that "major providers of rss feed listings will use a
unique id architecture", then how should we handle it? Say XMLtree wants to
keep their list going, I want to make a list, and Barr wants to keep his
individual list going.
>
> How, how, how do we combine them all together into one massive list? We
certainly don't need duplication of effort. And what if Winer doesn't want
to jump in? How do we handle those who don't join? Do we?
>
> What if we did unique IDs based off "epoch seconds,list id", such that if
there were three providers numbered 3, 4, and 5 and they each added a
channel at the same exact moment in time, it'd be something like
"91374293,3" and "91374293,4" and "91374293,5"?
>
> I still, to be honest, don't see any compelling reason to use unique ids.
Sure, it may be good *now*, when the world is relatively small and meager,
but why waste the time worrying about them if they're *only good enough* for
this point in time?
>
> The words "hopefully grow into something that can scale" do not put me at
ease. We shouldn't hope something scales, we should make damn well that it
does. 'Member Gnutella? (and I'll ignore any comments about "well, hey, he
said it would only handle 300 hosts")...
>
> --
> Morbus Iff
>
>  Disobey has been mentioned in The Netly News, Internet World, ABC News,
>    Bruce Sterling's Dead Media Notes and many more. Microsoft and 3Com
>   ripped us off also... that's GOTTA mean we're important. And hell, we
>   got a rise out of Playboy! With sections that have nothing to do with
>     the others, you'll like at least one thing. No, really. Go there.
>
> -07--- <\/> ---- <http://www.disobey.com/> ------- Bad Ego, Any
Notice ----
>
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>

>> Again, does our current purpose scale well? Can we editorialize one million
>> rss feeds, checking for all possible aliases, defects, user idiocies, and
>> so forth, while still giving ourselves time to read the damn feeds that
>> we're programming software for?
>
>It certainly may not scale, but even if it doesn't it's something that's
>doable and useful now, and can hopefully grow into something that _can_
>scale.

Ok. Say we all agree that "major providers of rss feed listings will use a
unique id architecture", then how should we handle it? Say XMLtree wants to keep
their list going, I want to make a list, and Barr wants to keep his individual
list going.

How, how, how do we combine them all together into one massive list? We
certainly don't need duplication of effort. And what if Winer doesn't want to
jump in? How do we handle those who don't join? Do we?

What if we did unique IDs based off "epoch seconds,list id", such that if there
were three providers numbered 3, 4, and 5 and they each added a channel at the
same exact moment in time, it'd be something like "91374293,3" and "91374293,4"
and "91374293,5"?

I still, to be honest, don't see any compelling reason to use unique ids. Sure,
it may be good *now*, when the world is relatively small and meager, but why
waste the time worrying about them if they're *only good enough* for this point
in time?

The words "hopefully grow into something that can scale" do not put me at ease.
We shouldn't hope something scales, we should make damn well that it does.
'Member Gnutella? (and I'll ignore any comments about "well, hey, he said it
would only handle 300 hosts")...

--
Morbus Iff

  Disobey has been mentioned in The Netly News, Internet World, ABC News,
    Bruce Sterling's Dead Media Notes and many more. Microsoft and 3Com
   ripped us off also... that's GOTTA mean we're important. And hell, we
   got a rise out of Playboy! With sections that have nothing to do with
     the others, you'll like at least one thing. No, really. Go there.

-07--- <\/> ---- <http://www.disobey.com/> ------- Bad Ego, Any Notice ----

Morbus Iff <morbus@...> wrote:

> Again, does our current purpose scale well? Can we editorialize one million
> rss feeds, checking for all possible aliases, defects, user idiocies, and
> so forth, while still giving ourselves time to read the damn feeds that
> we're programming software for?

It certainly may not scale, but even if it doesn't it's something that's
doable and useful now, and can hopefully grow into something that _can_
scale.

--
[ Aaron Swartz | me@... | http://www.aaronsw.com ]

>>  - besides numbering, the unique id has no purpose.
  >
  >Actually, the unique id is the only piece of info about the site that
  >never changes. From the user's point of view, the fact that a site
  >gets a new URL, or is syndicated in a new format is irrelevant. By
  >indexing off of the unique id I can give my users a long-term stable
  >view of the sites that they read.

Let me play devil's advocate - only on a minor scale though, because I
think it's a valid point. RSS feeds is kinda where the Web was years and
years ago. There aren't that many sites, so it's good and handy for us to
be editors of the content, and assign id's and help out the stupid authors
who don't know how to do things right.

Very college Yahoo. But does it scale?

If RSS can be done by every single site on the net, then it'd be insane
that we assign each site/feed a unique id. It'd be insane for us to be the
mother hen of all the RSS feeds that could number in the millions. We
should start looking for venture capital now.

Perhaps we need to launch a dmoz like service?

  >> - if we can assure getting rid of duplicates, and 404's,
  >>   then wouldn't the xmlUrl be the unique id, ie, if a
  >>   channel changes a url, then the old xmlUrl would 404
  >>   and be removed...
  >
  >No, because there is nothing to relate one to the other.

Is it our job to do that relation? Does Yahoo know that
totalnetnh.net/~morbus used to be the home of disobey.com years and years
ago? Or does it just know that /~morbus doesn't exist anymore? Hell, I
didn't know about server side redirects years and years ago, so I didn't
even have the happy net user knowledge to issue a perm redirect.

Again, does our current purpose scale well? Can we editorialize one million
rss feeds, checking for all possible aliases, defects, user idiocies, and
so forth, while still giving ourselves time to read the damn feeds that
we're programming software for?

  >> - We go with a unique id system. A channel dies. Instead
  >>   of removing the channel, it has to be put in a pending
  >>   area so we don't lose the unique id number. The
  >>   author contacts us with the new xmlUrl*. We now
  >>   have to hunt down the unique id, and change the xmlUrl.
  >>   To make sure the rest of the junk is up to date, a script
  >>   downloads xmlUrl and reassign title/description, etc.
  >
  >I do not think we can count on the authors to (with all due respect)
  >know anything. It should be up to us to keep the list clean.

I'm not sure I follow your response for this blurb.
Hurry back from the airport, eh?


Morbus Iff
.sig on other machine.
http://www.disobey.com/
http://www.gamegrene.com/