Search the web
Sign In
New User? Sign Up
archive-crawler
  • Members Only
  • Post
  • Files
  • Photos
  • Links
  • Database
  • Polls
  • Members
  • Calendar
  • Promote
  • Groups Labs (Beta)

Group Information

  • Members: 613
  • Category: Cyberculture
  • Founded: Dec 1, 2002
  • Language: English
? Already a member? Sign in to Yahoo!

Yahoo! Groups Tips

Did you know...
Hear how Yahoo! Groups has changed the lives of others. Take me there.

Best of Y! Groups

   Check them out and nominate your group.
Visit the Groups blog for the latest Yahoo! Groups information

Home

  

Activity within 7 days:

3 New Members - 18 New Messages - New Questions

Description

Discussion group for the Heritrix open-source archival web crawler project.

Most Recent Topics

  (View All)
Search:
Advanced
    Start Topic
Topic  (List as Individual Messages) Messages Latest Post
Exporting Edited Heritrix 2
Hi all, This is not a "real" Heritrix issue but maybe somebody has an answer to this: I programmed 2 processors for Heritrix 2 and put my classes in some
 1 sendaman69
Offline Send Email
Recrawling In Heritrix3
Hi, In H3 I am trying to setup crawl jobs that use FetchHistoryProcessor/ PersistStoreProcessor/PersistLoadProcessor to discard duplicate content. I can get
 6 Gordon Mohr
gojomo
Online Now Send Email
IDN support of heritrix
Hi, I want to know about IDN support of heritrix. ("Internationalized domain name" http://en.wikipedia.org/wiki/Internationalized_Domain_Name) I was tried to
 3 takeru sasaki
sasaki.takeru@...
Send Email
SV: [archive-crawler] Nb documents per host
You should most likely use the QuotaEnforcer module that allows you to set number of documents (succesful and total) and number of bytes per queue. If at the
 1 Bjarne Andersen
bjarne_dk2000
Offline Send Email
Nb documents per host
Hi, I used recently heritrix 1.14.3 and I do not understand how to limit the nb of documents per host. The parameter max-document-download limit the total
 1 bourely
Offline Send Email
Add archive-crawler to your personalized My Yahoo! page Add to My Yahoo! XML What's This?

Message History

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
2009 30 51 42 72 51 38 44 54 62 68 35
2008 72 80 60 72 90 89 39 56 64 63 29 33
2007 132 87 140 213 71 118 86 52 41 70 102 129
2006 126 113 46 54 70 104 140 86 152 119 78 64
2005 138 177 81 62 127 114 46 88 71 76 85 106
2004 56 3 20 62 135 63 168 204 130 72 97 82
2003 14 18 20 15 25 41 14 2 9 30 33
2002 1
What is Yahoo! Answers?

Yahoo! Answers, a new Yahoo! community, is a question and answer exchange where the world gathers to share what they know...and make each other's day. People can ask questions on any topic, and help others out by answering their questions.

Hide this tip
What is Yahoo! Answers?

Yahoo! Answers, a new Yahoo! community, is a question and answer exchange where the world gathers to share what they know...and make each other's day. People can ask questions on any topic, and help others out by answering their questions.

Hide this tip

Questions in Computers & Internet

  • Questions are currently unavailable.

Want to help answer other questions? Go to Yahoo! Answers

Group Email Addresses

Related Link: http://crawler.archive.org
Post message: archive-crawler@yahoogroups.com
Subscribe: archive-crawler-subscribe@yahoogroups.com
Unsubscribe: archive-crawler-unsubscribe@yahoogroups.com
List owner: archive-crawler-owner@yahoogroups.com
Copyright © 2009 Yahoo! Inc. All rights reserved.
Privacy Policy - Terms of Service - Guidelines - Help