>
> If you ask me, the hardest problem is untangling the mess sent by
> recent email systems-- instead of text, you get multipart MIME
> messages, often sprinkled with viruses and goofy HTML. HTML email
> could easily mess up the HTML wrapping in an archive (and often does
> from my experience). Ideally, it would be good to clean up HTML
> messages before placing them in an archive web page, but I have never
> looked for such an animal.

http://www.phred.org/~alex/stripmime.html - happened across this

>
> Steve, Maybe you should write up a description of what you want to
> build.

OK - Here it is ... a bit rough, but you have to start somewhere.

Steve


Uber Archive 0.00001 - Notes by Steven Clift


When I have time to write a full program description I will, I have
a very busy plate this fall.

First, why better archives?

E-mail is our tool that brings publishing down to the lowest common
denominator. It works for us - with the right faciliation and
recruitment approach our participants generate lots of content. We
need archives that help bring up the light and bury the heat.

One lesson we have learned is that it is more important who reads your
message than what you have to say. If participants feel that people
with the power/ability to do something about the issue they care about
READ the forum, most tend to put a lot more thought into what they
write and keep a more civil tone.

How do we get more people to read our forums? How do we extend our
online public sphere to a larger public? I would say that among
typical Internet users, half like web boards and half like e-mail -
very very few like both. With Minnesota E-Democracy
<http://www.e-democracy.org> specifically, I
am opposed to the core to any web-centric tool that treats e-mail
users as second class citizens. It fundamentally undermines our
"online public commons" model where people act as citizens from the
one space on the Internet that they own - there e-mail box.
(I am not opposed to special time-limited web-based online civic
events and would love to find ways to integrate them and IM, SMS and
other real-time tools into what we do.) (Until we find a truly
optimized and equitable web-e-mail dual conferencing system (perhaps
never) extending the value of e-mail-centric discussions to web users
is our best bet for increasing access and relevancy of our e-lists.

In order to raise the value of our e-mail generated discussions, we
need to present them in an extremely user friendly and comfortable
way. We can both reach web-centric people and help e-mail folks
better evaluate which lists to join. Imagine a busy elected official
with one minute to scan our page for relevant "political juice." For
our citizen-based discussions to matter, we have to bring the value of
our citizen-generated content to the surface and develop advanced ways
for people to monitor our many forums as a whole.

...

Basically I want to archive e-democracy e-mail lists running on
Mailman <http://www.list.org> in an extremely user friendly way. I'd
like to be able to extend the system such that it could also archive
other Minnesota political/community e-mail lists.

1. What it might look like (do follow the links):

http://marc.theaimsgroup.com/?l=mhonarc&r=1&b=200208&w=2
or
http://aspn.activestate.com/ASPN/Mail/Browse/Threaded/xml-dev

The second has a clever way (too clever??) of displaying threads. And
it allows linking to a specific page that always has the most recent
messages - that is key. I absolute love their "leaders" feature:
http://aspn.activestate.com/ASPN/Mail/Leaders/xml-dev/
The ability to add the list to your personal page (as they do) on the
site is something to consider in phase two.

As a default I want the text of the message to be flush to the left,
in a proportional font (or the option to change fonts) with template
based header and footer links/search. The template could be controlled
by some sort of web-based interface. The option to turn quoted text
grey or some other color and other tweaks to make the message content
stand out and easy to scan is important.

Back in 1998, I actually had someone adapt <http://www.mhonarc.org/>
for a clean look a feel within the sites template. Although you can't
change it mid-stream and with this version you have to scroll down too
far to find out what the content is:
http://www.publicus.net/emfa-event/9805/
http://www.publicus.net/emfa-event/9805/msg00000.html

This look and feel isn't bad (although I believe a minimalist header
is required to ensure that the key content does not have to be
scrolled to to see the start of it:
http://www.redhat.com/mailing-lists/sound-list/index.html

Say, it perhaps adapting eyebrowse makes sense. I have no idea:
http://eyebrowse.tigris.org/servlets/SummarizeList?listName=users


2. Linear Threads?
http://groups.google.com/groups?dq=&hl=en&lr=&ie=UTF-8&safe=off&th=6e828c81cf22a\
52d
I really really like how Google Groups takes posts in the same threads
and allows people to quickly read the discussion. Some sort of
auto-clipping of excess quoting seems essential.


3. Statistical Context/Popular
I'd add statistical context to page like this with the average number
of posts a day/unique poster daily average, unique posters each month:
http://aspn.activestate.com/ASPN/Mail/Archives/xml-dev/
These kind of graphs as a side are nice:
http://murphy.debian.org/lists/debian-powerpc/details.html

I could also imagine some sort "Hot Topics" feature, where the most
active threads (see syndication below) of the moment or the most web
clicked on articles could be highlighted. I am a big fan of anything
that allows user activity to be reflected automatically to other
users. Examples:
http://download.com.com/3101-2001-0-1.html?tag=dir
http://story.news.yahoo.com/news?tmpl=index2&cid=964
http://story.news.yahoo.com/news?tmpl=index2&cid=964
Most active lists this month: http://www.geocrawler.com/
http://usableweb.com/popular/ - different popular options
http://freshmeat.net/faq/view/27/ - Vitality?:
http://freshmeat.net/stats/
Still digging: http://www.christianbbs.com/popular/
Back on that "leaders" idea - perhaps "most active this week":
http://www.christianbbs.com/popular/users.php

I am often interested in threads with lots of different posters that
spark up ... they are often the reason people love our lists.
Something that generates public interest/opinion and often directly
influences the news - that is the "juice" that needs to be
highlighted.


4. Search the full text of one list or multiple/selected lists:
http://marc.theaimsgroup.com/

... with the ability like Google Groups to set the date range and
select the number of messages per index page for the output:
http://www.google.com/advanced_group_search?hl=en

Like Google, some contextual content lines should be part of the
search results page.


5. Member view/directory

Initially I'd like to make all e-mail addresses clickable and bring up
a standard poster page - Something like
http://e-democracy.org/members/clift@publicus.net (we will need some
sort of spam harvesting stopper, perhaps like Sympa (where they make
you click a button to get into the area with addresses).

From the members page you would list all posts from that address
across all the lists in the archive and have a web-based mailto form
to contact the person (perhaps pull in the URL of the message which
prompted the click through). In the future, I'd like to add opt-in
options for members to share information about their issues interests
and political activities. This is where we could also add
"self-governance" accountability features where aggregate style and
substance ratings could be collected. (Slashdot style rating of
individual posts on e-lists, even in the archives, probably wouldn't
work well.)


6. Syndication - I want to use RSS to make the current subject lines
(clickable into the archive) available from the home page on the
E-Democracy site as well as offer a moreover.com like feature that
allows third party sites to integrate a list's recent headlines into
their site.

This site "collapses" the threads, obviously this is a bit messy
because the message/thread IDs (whatever they are) get changed by
people:
http://aspn.activestate.com/ASPN/Mail/Browse/Collapsed/xml-dev/
I'd syndicate the collapsed threads to have higher variety.

Here is some info:
http://www.mail-archive.com/faq.html#RDF
Example: http://www.mail-archive.com/do-wire@tc.umn.edu/maillist.rdf
(save as text on IE, netscape loads as text file)
http://www.rosat.mpe-garching.mpg.de/mailing-lists/mhonarc/1999-10/msg00072.html


7. Share - E-mail to a Friend

An option that e-mails the full text of the message or 10K of it to
and address with contextual links back to the archive. Every e-mail
out should include a one click e-mail subscribe option.


8. Advanced Monitoring

E-mail list digest are fine, but what would really be useful is a
personalization feature that would allow users to monitor new archive
messages for keywords, posters, hot topics, etc.. Remember our forums
are very very public, we want anyone interested in anything to find
it. Daily or frequent reports would be e-mailed out and be available
as part of a future personal view of the archive.

I have always felt that lists should have two digest options - digest
full text and digest subjects with links. Allowing people to e-mailed
daily/weekly subject line digests with links would be a really useful
service in my opinion.

Here are some e-mail customization services I use:
http://www.sosig.ac.uk/help/custom.html#4
http://www.info4local.gov.uk/emailalert.asp
http://www.spyonit.com (under redevelopment)
http://help.yahoo.com/help/us/alerts/alerts-19.html
Others:
http://www.searchengineshowdown.com/alerts/

Check out this "tips" search:
http://lists.evolt.org/index.cfm/a/harvest/b/search/
Members are asked to <tip> code nuggets of truth in their e-mail</tip>
for collection into this search engine.

9. Post from Archive

E-mail lists are like "coffee talk" if you don't get your point in,
you don't come back the next week and insert it. So any form of web
based posting would have to come from registered web readers (archive
phase two) and replies accept only on recent posts - perhap 3 to 7
days maximum.

10. That is it for now ...