[janeellen_carr:]
>
> Although I don't think the days of huge human compiled web
> directories are quite over and we may be stuck with Yahoo and ODP
> for a while yet, your theory about a "customizable meta-searcher
> that spiders the listings contained in smaller, high quality Web
> directories" is very sound. This is the direction that your Source
> Forge project is taking I assume?

At this point, XODP @ SourceForge.Net is little more than a collection of links
to other projects at SourceForge.Net that have developed and/or are developing
open source Web indexing software. Rather than re-inventing the wheel, XODP @
SourceForge.Net will serve as a clearinghouse for evaluating open source Web
indexing applications and assisting open content indexing startups with the
installation of said applications.

[janeellen_carr:]
>
> You'd be interested to learn that SearchKing
> (http://www.searchking.com) (the directory and portal network) has
> taken the same approach, and the main SK engine is now comprised of
> just that - listings and descriptions gleaned from its collection
> of well over 2,000 smaller directories and portal sites. If you
> search for the term games, for example, on their main search
> engine, you will get sites that have been reviewed by other portals
> (the combined database) first, followed by sites entered directly
> into the main engine (unreviewed, but instantly indexed). The
> quality of the individual portal sites is of course up to each
> individual owner, but thought you might be interested to see an
> example of a similar system to what you envision.

I'm vaguely familiar with SearchKing's system, and it seems to be a viable and
ongoing enterprise. Alas, this was not exactly the sort of system I was
referring to when I suggested a customizable meta-search as the most promising
search solution currently available. The closest thing to such a meta-search
currently in existence is Turbo10. (< http://turbo10.com/ >)

[janeellen_carr:]
>
> As far as the future of human edited directories, there are still
> some great big holes in the automated spidering approach - just
> read an article today pointing out some of the more obvious holes
> in Google (Digging for Googleholes -
> http://slate.msn.com/id/2085668/ ) such as giving preference to
> shopping sites, confusing synonyms, and omission of non-spiderable
> text. As huge as the spiders have become, there is still no
> replacement for human edited directories.

I wholeheartedly agree. The problem is not one of kind, but of scale. Even a
small group of editors will have inevitable conflicts over the proper taxonomy
of a directory-style database. The best way to resolve such conflicts is to
give editors the "right to fork," and give end users the option of a
customizable meta-search where they can incorporate only those Web directories
that they consider authoritative.

Humbly Yours,

XODP Moderator netesq