> I wonder if I should change Xenu so that it delivers a proper REFERER,
> or at least making it an option. Anyone interested in testing a version?
Yes! Yes! Yes!
I am interested!
Also, I suggest to add possibility to deliver base URL as referer.
For example, if I check http://somesite.com/dir/subdir/file.ext
referer should automatically be set to:
option1) http://somesite.com/
option2) http://somesite.com/dir/subdir/
--
Eugen
A user had trouble accessing her FTP site to check for orphans because
she was running Zonealarm. While I wasn't able to find out a workable
setting for Zonealarm, I was able to modify Xenu so that it used passive
mode for FTP.
I'm wondering if I should set this by default, or does it have any
unwelcome side effects?
Tilman
On Wed, 11 Aug 2004 14:10:43 -0000, Eugeny Sattler the Spam Hater wrote:
>When I check this single URL manually
>http://home.xtra.co.nz/hosts/nickbolton/mailwasherpro/mailwasher_pro4176.exe
>Xenu (version taken from YahooGroups files area) does the job OK
>
>But when I do it from command line, Xenu reports that the
>site requires authorisation. In fact, the site does not require that.
1) I have now deleted the beta version, because the "real" version is
now available. (It's about the same)
2) Xenu always does the job OK for me. I suspect you may have several
xenu.exe versions on your PC.
Tilman
But when I do it from command line, Xenu reports that the
site requires authorisation. In fact, the site does not require that.
--
Note: FROM line of this message contains a throwaway (disposable)
account intended to fool spammers.
If you need to right me offlist use accmailer#yandex.ru (replace # with @)
hi tilman,
whats the advantage of having this install functionality?
why not just work directly with the xenu.exe?
frank
--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@b...>
wrote:
> I have uploaded the latest beta in the yahoo file area at
> http://groups.yahoo.com/group/linksleuthupdates/files/
>
> If you can't access it, e-mail me, I will send the ZIP to you. If
you
> have tested it or a previous beta version (with the setup) from the
last
> few weeks, e-mail me so that I know it was ok.
>
> I will release it this weekend unless there are any problems.
>
> Tilman
I have uploaded the latest beta in the yahoo file area at
http://groups.yahoo.com/group/linksleuthupdates/files/
If you can't access it, e-mail me, I will send the ZIP to you. If you
have tested it or a previous beta version (with the setup) from the last
few weeks, e-mail me so that I know it was ok.
I will release it this weekend unless there are any problems.
Tilman
On 1 Aug 2004 at 10:04, Tilman Hausherr wrote:
> Try set a lower amount of threads, and retry with CTRL-R.
>
> Tilman
=============
Tilman, Andi and Frank,
Thank you for those quick and helpful replies.
I have dropped the number of threads to 12 and that certainly reduces
the number of sites reported as "no such Host" though I still get
quite a few.
I will try again at some unearthly hour when the net might be less
busy, but anyway I can live with the problem!
Frank, thanks for your detailed report. I don't know why one url did
not work for you, it does for me <grin> In any case, thanks because
in checking it I saw my logo was incorrect on that page!
I think the bloke I was linked to must have dropped Metals galore or
moved without telling me. I have deleted that link.
Up to now I have not been checking my site regularly, only when I
update my own site. Perhaps I should check more often. It is easier
now that I have a faster connection.
Thanks again,
Barrie Smith (in sunny Kent UK)
Visit my Virtual Clock Museum at http://www.clock-museum.co.uk/
could not resist spidering your site ;-), got only 4 issues, 2 are
Nedstat related non-isues, one is probaby a typo in an image name,
the metalsgalore one i cannot figure out:
1631 links found:
Broken links, ordered by link:
http://www.clock-museum.co.uk/20c/mvmnts/mb~101.jpg
error code: 12017 (operation cancelled), linked from page(s):
http://www.clock-museum.co.uk/20c/mvmnts/pm~unknwn.htmhttp://www.metalsgalore.com/
error code: 403 (forbidden request), linked from page(s):
http://www.clock-museum.co.uk/links.htmhttp://www.nedstatbasic.net/
error code: 403 (forbidden request), linked from page(s):
http://www.clock-museum.co.uk/index.htmhttp://www.nedstatbasic.net/s?tab=1&link=1&id=3090148
error code: 403 (forbidden request), linked from page(s):
http://www.nedstatbasic.net/stats?
AC8m5ADpk2xU5bs1nk+JzrngT1ng
4 broken link(s) reported
--- In xenu-usergroup@yahoogroups.com, "frank visser"
<f.visser3@c...> wrote:
> barry, tilman,
>
> that tip always works with "operation cancelled" or "time out"
> issues, but never when "host not found" is reported, at least for
> me.
>
> most of the time, that's caused by typos in the URLs.
>
> i have set the number of threads to 10 to avoid these false
alarms.
> when i had it set to 25 or higher, i got loads of them.
>
> frank
>
> --- In xenu-usergroup@yahoogroups.com, Tilman Hausherr
<tilman@b...>
> wrote:
> > Try set a lower amount of threads, and retry with CTRL-R.
> >
> > Tilman
> >
> > On Sun, 01 Aug 2004 09:01:46 +0100, barrie@c... wrote:
> >
> > >Hi,
> > >
> > >I am sure that I am doing something incorrectly <grin>
> > >
> > >When I request a report from my actual web page many of my
> external
> > >links are reported faulty, with a "no such host" comment.
> > >
> > >What I find odd is that each of these links works quite happily
> if I
> > >double click the link in the error report itself, or in the web
> page
> > >itself.
> > >
> > >I have a reasonably fast connection using DSL.
> > >
> > >Not all external links are reported faulty, perhaps about 50%
> > >
> > >The url of my links page is http://www.clock-
> museum.co.uk/links.htm
> > >
> > >Any help will be most gratefully received. I find Xenu
invaluable
> and
> > >as I said above, I feel sure this problem is due to me being
dim.
> > >
> > >
> > >Barrie Smith (in sunny Kent UK)
> > >
> > >Visit my Virtual Clock Museum at http://www.clock-museum.co.uk/
> > >
> > >
> > >
> > >
> > >
> > >Yahoo! Groups Links
> > >
> > >
> > >
> > >
barry, tilman,
that tip always works with "operation cancelled" or "time out"
issues, but never when "host not found" is reported, at least for
me.
most of the time, that's caused by typos in the URLs.
i have set the number of threads to 10 to avoid these false alarms.
when i had it set to 25 or higher, i got loads of them.
frank
--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@b...>
wrote:
> Try set a lower amount of threads, and retry with CTRL-R.
>
> Tilman
>
> On Sun, 01 Aug 2004 09:01:46 +0100, barrie@c... wrote:
>
> >Hi,
> >
> >I am sure that I am doing something incorrectly <grin>
> >
> >When I request a report from my actual web page many of my
external
> >links are reported faulty, with a "no such host" comment.
> >
> >What I find odd is that each of these links works quite happily
if I
> >double click the link in the error report itself, or in the web
page
> >itself.
> >
> >I have a reasonably fast connection using DSL.
> >
> >Not all external links are reported faulty, perhaps about 50%
> >
> >The url of my links page is http://www.clock-
museum.co.uk/links.htm
> >
> >Any help will be most gratefully received. I find Xenu invaluable
and
> >as I said above, I feel sure this problem is due to me being dim.
> >
> >
> >Barrie Smith (in sunny Kent UK)
> >
> >Visit my Virtual Clock Museum at http://www.clock-museum.co.uk/
> >
> >
> >
> >
> >
> >Yahoo! Groups Links
> >
> >
> >
> >
For your reference, I have a Comcast Broadband connection that averages around 2800kbps. I have Xenu set at 30 threads. I usually make up a special page with about 10,000 external links and run it very late at night. The first pass usually yeilds about a third of the links "action cancelled" which is the ISP choking on the load. I save the .XEN file and retry the broken links, Ctrl-R about an hour later and again I get a smaller portion of the remainder "action cancelled." Usually it is good to redo this several times because some percentage of the links themselves will always be temporarily out of service at any given time.
I set a 14 day interval between checks and so over the course of the two weeks I'm confident there are a minimal number of errors due to my own connection. By that time there are usually just 25-30 dead links out of 10K which are easy enough to check manually. I use the iRider http://www.irider.com/ browser which will open 25 links like a bot but presents itself as IE6 for the final final check thus eliminating sites blocking robot user-agents.
Try set a lower amount of threads, and retry with CTRL-R.
Tilman
On Sun, 01 Aug 2004 09:01:46 +0100, barrie@... wrote:
>Hi, > >I am sure that I am doing something incorrectly <grin> > >When I request a report from my actual web page many of my external >links are reported faulty, with a "no such host" comment. > >What I find odd is that each of these links works quite happily if I >double click the link in the error report itself, or in the web page >itself. > >I have a reasonably fast connection using DSL. > >Not all external links are reported faulty, perhaps about 50% > >The url of my links page is http://www.clock-museum.co.uk/links.htm > >Any help will be most gratefully received. I find Xenu invaluable and >as I said above, I feel sure this problem is due to me being dim. > > >Barrie Smith (in sunny Kent UK) > >Visit my Virtual Clock Museum at http://www.clock-museum.co.uk/ > > > > > >Yahoo! Groups Links > > > >
Try set a lower amount of threads, and retry with CTRL-R.
Tilman
On Sun, 01 Aug 2004 09:01:46 +0100, barrie@... wrote:
>Hi,
>
>I am sure that I am doing something incorrectly <grin>
>
>When I request a report from my actual web page many of my external
>links are reported faulty, with a "no such host" comment.
>
>What I find odd is that each of these links works quite happily if I
>double click the link in the error report itself, or in the web page
>itself.
>
>I have a reasonably fast connection using DSL.
>
>Not all external links are reported faulty, perhaps about 50%
>
>The url of my links page is http://www.clock-museum.co.uk/links.htm
>
>Any help will be most gratefully received. I find Xenu invaluable and
>as I said above, I feel sure this problem is due to me being dim.
>
>
>Barrie Smith (in sunny Kent UK)
>
>Visit my Virtual Clock Museum at http://www.clock-museum.co.uk/
>
>
>
>
>
>Yahoo! Groups Links
>
>
>
>
Hi,
I am sure that I am doing something incorrectly <grin>
When I request a report from my actual web page many of my external
links are reported faulty, with a "no such host" comment.
What I find odd is that each of these links works quite happily if I
double click the link in the error report itself, or in the web page
itself.
I have a reasonably fast connection using DSL.
Not all external links are reported faulty, perhaps about 50%
The url of my links page is http://www.clock-museum.co.uk/links.htm
Any help will be most gratefully received. I find Xenu invaluable and
as I said above, I feel sure this problem is due to me being dim.
Barrie Smith (in sunny Kent UK)
Visit my Virtual Clock Museum at http://www.clock-museum.co.uk/
>>Can you explan why you export the reports to a database?
Yes. All my links and their associated comments, descriptions, categories etc. reside in a local MS Access database and I assemble the static pages with (mostly) VB and (some) Perl scripts. This provides me with maximum flexibility and scalability when redesigning the pages and reconfiguring the categories. It also is a precursor to a MySQL dynamic presentation. So when I delete dead links I remove them first from the database with Xenu and then reassemble and upload the pages. The easiest way I have found to match the Xenu results to the database is to copy the link titles of the dead links in the exported Xenu report (I keep the dead link titles and discard the rest). I match them with a simple query and then delete the dead records after copying them to the "dead table."
As for resurrecting old links, I was doing that for a time, but I have since purchased a spider allows me to move so quickly on the web that it doesn't pay to dig through the dregs. Though I still do keep them, it wouldn't take long to find any residual good ones with Xenu and I may eventually get back to that.
Thats a very nice thing to hear from a linkcheck colleague.
I’ve been spending the last 5 years with quality assurance of Intel’s European websites, so that’s about 3000 pages (was even 8000 in the past) and a lot of links!
Xenu just did the job for me, even in a CMS environment that multiplied all links by 5 due to redirects.
Can you explan why you export the reports to a database? I read something on your page about retrying and reviving dead links, will investigate that more.
frank
========================================
Frank Visser, Waterpoortweg 279, 1051 pv, Amsterdam
Author of: Ken Wilber: Thought as Passion SUNY 2003
From: andi [mailto:andi@...] Sent: dinsdag 27 juli 2004 18:33 To: xenu gp Subject: [xenu-usergroup] manual
Frank, where were you with your manual 18 months ago when I needed you! :) I am going to add links to your manual and this group on the page I mentioned in my last message. This Link Checking page actually is the second most popular on my site and gets quite a bit of traffic from Google as well as the Xenu site so this should send you some visitors. I'll have those links online in an hour or so.
I will read your manual more carefully in the coming days but the only suggestion I can make now is that you should mention that in addition to exporting the results to Excel the data can also go to MS Access or any database program. Lotus 123 will also accept tab delimited files, so maybe just adding "or any spreadsheet or database" would be sufficient. I mention this because I'm sure there are others like myself who use the Lotus spreadsheet and other databases. I export Xenu data to Access because I just use the database more than Lotus 123.
Thats a very nice thing
to hear from a linkcheck colleague.
I’ve been spending
the last 5 years with quality assurance of Intel’s European websites, so that’s
about 3000 pages (was even 8000 in the past) and a lot of links!
Xenu just did the job for
me, even in a CMS environment that multiplied all links by 5 due to redirects.
Can you explan why you
export the reports to a database? I read something on your page about retrying
and reviving dead links, will investigate that more.
frank
========================================
Frank Visser,
Waterpoortweg 279, 1051 pv, Amsterdam
Author of: Ken Wilber:
Thought as Passion SUNY 2003
From:
andi [mailto:andi@...] Sent: dinsdag 27 juli 2004 18:33 To: xenu gp Subject: [xenu-usergroup] manual
Frank, where were you with your manual 18 months ago
when I needed you! :) I am going to add links to your manual
and this group on the page I mentioned in my last message. This
Link Checking page actually is the second most popular on my site and gets
quite a bit of traffic from Google as well as the Xenu site so this should send
you some visitors. I'll have those links online in an hour or so.
I will read your manual more carefully in the coming days
but the only suggestion I can make now is that you should mention that in
addition to exporting the results to Excel the data can also go to MS Access or
any database program. Lotus 123 will also accept tab delimited
files, so maybe just adding "or any spreadsheet or database" would be
sufficient. I mention this because I'm sure there are others like myself
who use the Lotus spreadsheet and other databases. I export Xenu data to
Access because I just use the database more than Lotus 123.
Tilman,
Thanks for the info. Maybe I just have too specific a need.
We restrict our external links through a Links Policy so we only have
260 external links on our whole site right now. I was just looking
for a column on the External Links report that would show what page
(s) that external link was found on, so if it went against the policy
I would know which department webmaster to contact to have it
removed/changed.
--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@b...>
wrote:
> On Wed, 28 Jul 2004 19:38:20 -0000, webdotnet03 wrote:
>
> >Now, could someone let me know if it is possible to get printed
> >report of External Links that shows the Pages linking to the
external
> >link. So far, I've only been able to see the pages linking to an
> >external link by right clicking on the link in the Xenu window and
> >choosing Properties. I would like to be able to see the "pages
> >linking to this one" on a report of all external links.
>
> This is not possible... such a report would be enormous, since you
ask
> for details about *working* links. Technically, it wouldn't be
difficult
> for me to implement. But you would possibly get a 50 MB report, and
then
> never try this again.
On Wed, 28 Jul 2004 03:41:19 -0000, bastrouse wrote:
>Does anyone know how to automate Xenu checks with a script?
Some people don't know that there is a drop-down combo box that shows
all the URLs they tried before in the normal version.
You could use the $300 corporate version (which I almost never sell) to
use the command line feature... but even *I* never use it :) It allows
also to save the report, or to email it.
A free trial of the corporate version is available, it will have the
feature until the end of september 2004. I've put it up in the yahoo
file area at
http://groups.yahoo.com/group/linksleuthupdates/files/
it has also all the current "beta" features.
Syntax:
xenu -url http://www.hp.com -report blah.html
or
xenu -url http://www.hp.com -export export.tab
it will use the "include" / "exclude" / mail settings that it had for
the last manual run of http://www.hp.com . It's that simple :-)
Tilman
On Wed, 28 Jul 2004 19:38:20 -0000, webdotnet03 wrote:
>Now, could someone let me know if it is possible to get printed
>report of External Links that shows the Pages linking to the external
>link. So far, I've only been able to see the pages linking to an
>external link by right clicking on the link in the Xenu window and
>choosing Properties. I would like to be able to see the "pages
>linking to this one" on a report of all external links.
This is not possible... such a report would be enormous, since you ask
for details about *working* links. Technically, it wouldn't be difficult
for me to implement. But you would possibly get a 50 MB report, and then
never try this again.
First, I thank you all for making this product available and the
great documentation. In IE was able to File > Print > Options > Print
all Linked Documents and print the whole manual out at once!
Now, could someone let me know if it is possible to get printed
report of External Links that shows the Pages linking to the external
link. So far, I've only been able to see the pages linking to an
external link by right clicking on the link in the Xenu window and
choosing Properties. I would like to be able to see the "pages
linking to this one" on a report of all external links.
Can anyone help me?
thanks
On Wed, 28 Jul 2004 09:12:39 -0000, frank visser <f.visser3@...> wrote:
> in some xenu documentation i found reference to the option to email a
> report to somebody.
>
> where can i find that in the program? what menu?
Press "email" in the initial dialog box
Tilman
>>This Link Checking page actually is the second most popular on my site >>This is because I link to you :-)
Indeed, since 4/01/03 that page has been viewed 11,691 times. 4,253 were referrals from snafu.de... Many of those 11k were no doubt return visits by those you referred. In that same time the entire site received 161,566 visitors. The usual visitor count caveats apply. :)
Those 161 k visitors were spared the disappointment of clicking on 809 dead links. For this I owe you a debt of gratitude Tillman.
>Frank, where were you with your manual 18 months ago when I needed you! :) I am going to add links to your manual and this group on the page I mentioned in my last message. This Link Checking page actually is the second most popular on my site
This is because I link to you :-)
I am now linking to Frank's manual too.
Tilman
>and gets quite a bit of traffic from Google as well as the Xenu site so this should send you some visitors. I'll have those links online in an hour or so. > >I will read your manual more carefully in the coming days but the only suggestion I can make now is that you should mention that in addition to exporting the results to Excel the data can also go to MS Access or any database program. Lotus 123 will also accept tab delimited files, so maybe just adding "or any spreadsheet or database" would be sufficient. I mention this because I'm sure there are others like myself who use the Lotus spreadsheet and other databases. I export Xenu data to Access because I just use the database more than Lotus 123.
On Tue, 27 Jul 2004 09:32:59 -0700, andi wrote:
>Frank, where were you with your manual 18 months ago when I needed you! :) I
am going to add links to your manual and this group on the page I mentioned in
my last message. This Link Checking page actually is the second most popular on
my site
This is because I link to you :-)
I am now linking to Frank's manual too.
Tilman
>and gets quite a bit of traffic from Google as well as the Xenu site so this
should send you some visitors. I'll have those links online in an hour or so.
>
>I will read your manual more carefully in the coming days but the only
suggestion I can make now is that you should mention that in addition to
exporting the results to Excel the data can also go to MS Access or any database
program. Lotus 123 will also accept tab delimited files, so maybe just adding
"or any spreadsheet or database" would be sufficient. I mention this because
I'm sure there are others like myself who use the Lotus spreadsheet and other
databases. I export Xenu data to Access because I just use the database more
than Lotus 123.
Frank, where were you with your manual 18 months ago when I needed you! :) I am going to add links to your manual and this group on the page I mentioned in my last message. This Link Checking page actually is the second most popular on my site and gets quite a bit of traffic from Google as well as the Xenu site so this should send you some visitors. I'll have those links online in an hour or so.
I will read your manual more carefully in the coming days but the only suggestion I can make now is that you should mention that in addition to exporting the results to Excel the data can also go to MS Access or any database program. Lotus 123 will also accept tab delimited files, so maybe just adding "or any spreadsheet or database" would be sufficient. I mention this because I'm sure there are others like myself who use the Lotus spreadsheet and other databases. I export Xenu data to Access because I just use the database more than Lotus 123.
hi all,
i created an online Xenu manual, have a look at:
http://members.ams.chello.nl/f.visser3/xenu/
it covers the following areas:
Getting started with Xenu
Checking a single link
Analysing the results
Checking a set of links
Checking a complete site
Handling javascript links
Checking a local site
Checking a password protected site
Checking a site with redirects
Finding orphaned files
Appendices
Xenu shortcuts
Xenu update history
Server error messages
The competition
Xenu Yahoo! user group
let me know if you have any improvements, additions,
recommendations, tips, tricks, etc. !
please forward this mail to as many of your web colleagues as might
be interested in the use of Xenu.
frank
hi,
i have activated this:
Email Address Display : Your group allows members to hide their
email address from the group.
Your group cannot use the Polls feature.
Members who hide their email address may not post messages via
email.
This setting cannot be changed.
looks like each member has to hide his/her email adress now?
frank
--- In xenu-usergroup@yahoogroups.com, "nj_skylander"
<jbn8e5693p001@s...> wrote:
> Burried in the options for Yahoo! Groups, you can hide email
addreses
> so that they are not harvestable by members. I plan to leave the
> group by tomorrow if you do not set this option. If you do set
the
> option to hide EMail, thank you and I hope we can hold some useful
> discussions here.
Burried in the options for Yahoo! Groups, you can hide email addreses
so that they are not harvestable by members. I plan to leave the
group by tomorrow if you do not set this option. If you do set the
option to hide EMail, thank you and I hope we can hold some useful
discussions here.