Search the web
Sign In
New User? Sign Up
archive-crawler
  • Members Only
  • Post
  • Files
  • Photos
  • Links
  • Database
  • Polls
  • Members
  • Calendar
  • Promote
  • Groups Labs (Beta)

Group Information

  • Members: 625
  • Category: Cyberculture
  • Founded: Dec 1, 2002
  • Language: English
? Already a member? Sign in to Yahoo!

Yahoo! Groups Tips

Did you know...
Real people. Real stories. See how Yahoo! Groups impacts members worldwide.

Best of Y! Groups

   Check them out and nominate your group.
Visit the Groups blog for the latest Yahoo! Groups information

Home

  

Activity within 7 days:

5 New Members - 21 New Messages - New Questions

Description

Discussion group for the Heritrix open-source archival web crawler project.

Most Recent Topics

  (View All)
Search:
Advanced
    Start Topic
Topic  (List as Individual Messages) Messages Latest Post
Two issues for crawling
how can i a.crawl a URL at most once ( i do not want a update-checking followed with the 2nd time crawl) b.add seeds after crawl launch
 3 欧阳希修
tony871209
Offline Send Email
H-3.0 and mem-use-KB heap-size-KB
Hello, I am using H-3.0 and I noticed that even after a clean termination of my broad crawl (that consumed the entire 2G allocated for heap), the 2G memory
 1 Pranay Pandey
sspranay
Offline Send Email
tall toes locked and issue 1609
Hi, I've been noticing that a crawl job will just stall out. CPU and RAM are not maxed out or anything but the Toe Thread report makes it look like maybe all
 1 bopalmer69
Online Now Send Email
Individual File types
Hello, I'm trying to crawl & mirror PDF files thare linked/hosted on a domain. The problem I'm having is getting my decide rules setup so that I get more files
 3 collier.aaron@...
collier.aaro...
Offline Send Email
Canonicalizing URIs
I have setup the canonicalization rules to clean up some dynamic URIs a site is producing. This rules i've created seem to work in the sense that duplicate
 1 steven_bower
Offline Send Email
Add archive-crawler to your personalized My Yahoo! page Add to My Yahoo! XML What's This?

Message History

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
2009 30 51 42 72 51 38 44 54 62 68 42 54
2008 72 80 60 72 90 89 39 56 64 63 29 33
2007 132 87 140 213 71 118 86 52 41 70 102 129
2006 126 113 46 54 70 104 140 86 152 119 78 64
2005 138 177 81 62 127 114 46 88 71 76 85 106
2004 56 3 20 62 135 63 168 204 130 72 97 82
2003 14 18 20 15 25 41 14 2 9 30 33
2002 1
What is Yahoo! Answers?

Yahoo! Answers, a new Yahoo! community, is a question and answer exchange where the world gathers to share what they know...and make each other's day. People can ask questions on any topic, and help others out by answering their questions.

Hide this tip
What is Yahoo! Answers?

Yahoo! Answers, a new Yahoo! community, is a question and answer exchange where the world gathers to share what they know...and make each other's day. People can ask questions on any topic, and help others out by answering their questions.

Hide this tip

Questions in Computers & Internet

  • Questions are currently unavailable.

Want to help answer other questions? Go to Yahoo! Answers

Group Email Addresses

Related Link: http://crawler.archive.org
Post message: archive-crawler@yahoogroups.com
Subscribe: archive-crawler-subscribe@yahoogroups.com
Unsubscribe: archive-crawler-unsubscribe@yahoogroups.com
List owner: archive-crawler-owner@yahoogroups.com
Copyright © 2009 Yahoo! Inc. All rights reserved.
Privacy Policy - Terms of Service - Guidelines - Help