From time to time people contact me because Xenu does not detect broken
links as such. The reason is that their webserver redirects to an error
page, instead of returning a proper 404.

Could anyone make a very short description for popular webservers
(Apache, IIS, ...) how NOT to redirect, and instead maybe deliver a
custom HTML page.

This is an example of a server who does it right:
http://www.ibm.com/blah

This is an example of a server who doesn't do it right:
http://www.ktb.net/blah
(redirects to http://www.ktb.net/WebServ/missing.html )

Xenu's Link Sleuth banner:
http://home.snafu.de/tilman/xenubanner.jpg

Done from the existing graphics, by Bruno Zacke:
http://www.ufocomes.de/files/flsauce.html

I hope someone can help me.  I just joined the user group and have not
been approved yet.  But I hope someone can answer this after I am
approved.

  I have been using Xenu since 2001, when I worked for WORLDCOM.  It has
always worked wonderfully.  However, I just had it installed at my new
job, American Society for Industrial Security.  Our IT people have done
all they could, but I keep getting an ERROR 307 message for all links
requiring a member password and User ID.  It works fine for the public
pages.

The main url for our site is http://www.asisonline.org

If you run this, you will see the links with member-secure in them are
red and say error 307.  What should I do?  IT has installed the XENU.INI
file to enable cookies, however we have version 1.2f, and it says we
need 1.2g, not available on the download page.

Any help would be greatly appreciated.  We plan to use this software
for many departments and uses.  It worked great at WORLDCOM.

Thanks so much

Cathy Mahe

hi cathy,

scanning your site with version 1.2g i found no special issues with
secure links (btw, 307 should mean something like "temporary
redirect", see:
http://members.chello.nl/f.visser3/xenu/appendix3-server-error-
messages.html

here's the report:


ok 9131 URLs 97.72%
skip type 81 URLs 0.87%
not found 73 URLs 0.78%
no such host 39 URLs 0.42%
timeout 10 URLs 0.11%
server error 2 URLs 0.02%
forbidden request 5 URLs 0.05%
ambiguous 1 URLs 0.01%
auth required 1 URLs 0.01%
busy 1 URLs 0.01%
Total 9344 URLs 100.00%

i will mail you the 1.2g version separately.

frank

--- In xenu-usergroup@yahoogroups.com, "Catherine Mahe" <cmahe@a...>
wrote:
> I hope someone can help me.  I just joined the user group and have
not
> been approved yet.  But I hope someone can answer this after I am
> approved.
>
>  I have been using Xenu since 2001, when I worked for WORLDCOM.
It has
> always worked wonderfully.  However, I just had it installed at my
new
> job, American Society for Industrial Security.  Our IT people have
done
> all they could, but I keep getting an ERROR 307 message for all
links
> requiring a member password and User ID.  It works fine for the
public
> pages.
>
> The main url for our site is http://www.asisonline.org
>
> If you run this, you will see the links with member-secure in them
are
> red and say error 307.  What should I do?  IT has installed the
XENU.INI
> file to enable cookies, however we have version 1.2f, and it says
we
> need 1.2g, not available on the download page.
>
> Any help would be greatly appreciated.  We plan to use this
software
> for many departments and uses.  It worked great at WORLDCOM.
>
> Thanks so much
>
> Cathy Mahe

I don't know when she sent her message before you approved it - possibly
long ago? Because she also e-mailed me directly. The 307 problem has
been solved (I hope so!) in the 1.2g version, which I sent her.

Tilman


On Tue, 15 Mar 2005 08:14:49 -0000, frank visser wrote:

>hi cathy,
>
>scanning your site with version 1.2g i found no special issues with
>secure links (btw, 307 should mean something like "temporary
>redirect", see:
>http://members.chello.nl/f.visser3/xenu/appendix3-server-error-
>messages.html
>
>here's the report:
>
>
>ok 9131 URLs 97.72%
>skip type 81 URLs 0.87%
>not found 73 URLs 0.78%
>no such host 39 URLs 0.42%
>timeout 10 URLs 0.11%
>server error 2 URLs 0.02%
>forbidden request 5 URLs 0.05%
>ambiguous 1 URLs 0.01%
>auth required 1 URLs 0.01%
>busy 1 URLs 0.01%
>Total 9344 URLs 100.00%
>
>i will mail you the 1.2g version separately.
>
>frank
>
>--- In xenu-usergroup@yahoogroups.com, "Catherine Mahe" <cmahe@a...>
>wrote:
>> I hope someone can help me.  I just joined the user group and have
>not
>> been approved yet.  But I hope someone can answer this after I am
>> approved.
>>
>>  I have been using Xenu since 2001, when I worked for WORLDCOM.
>It has
>> always worked wonderfully.  However, I just had it installed at my
>new
>> job, American Society for Industrial Security.  Our IT people have
>done
>> all they could, but I keep getting an ERROR 307 message for all
>links
>> requiring a member password and User ID.  It works fine for the
>public
>> pages.
>>
>> The main url for our site is http://www.asisonline.org
>>
>> If you run this, you will see the links with member-secure in them
>are
>> red and say error 307.  What should I do?  IT has installed the
>XENU.INI
>> file to enable cookies, however we have version 1.2f, and it says
>we
>> need 1.2g, not available on the download page.
>>
>> Any help would be greatly appreciated.  We plan to use this
>software
>> for many departments and uses.  It worked great at WORLDCOM.
>>
>> Thanks so much
>>
>> Cathy Mahe
>
>
>
>
>
>
>Yahoo! Groups Links
>
>
>
>
>
>

As a Yahoo! store I tried using Xenu for broken links, but ended up
finding more value in the timeouts and connections troubles.

Please consider adding 5 options; timeout variable, retry variable,
maximum hits per second, auto restart and an auto cycle. With those
features I would have found the problem faster and continue using
Xenu as a website/network monitor.

The problem was long download times. No timeout or connection error
was a hard failure. The first full run using Xenu took over 4 days
and nights of restarting to complete one full pass.

FYI; other stores used NetMechanic and got clean runs until I
convince to use a realistic time frame for an ecommerce site. We
have 8 seconds to capture a surfers attention. How can you download
20 linked files if 1 or more take 30 seconds?

On Tue, 15 Mar 2005 09:43:49 -0500, Catherine Mahe wrote:

>Thanks to everyone for your helpful advice.  I still have not received
>the 1.2g version.  I hope you can send it soon, since it has to be
>installed by our IT department.  I feel hopeful that it will solve all
>my problems.  By the way, I am spreading the word about Xenu.  Everyone
>in my company is anxious to use it.

Catherine,

Here's what I wrote you:

======

To: "Catherine Mahe" <cmahe@...>
Subject: Re: LinkSleuthBug
From: Tilman Hausherr <tilman@...>
Date: Wed, 09 Mar 2005 19:47:47 +0100

Try again with this version (1.2g), I did something for 307. (Yes, the
1.2g isn't released, but the documentation mentions it already)

======

I guess your ISP blocks mails with attachments. You can download the
1.2g version here:
http://home.snafu.de/tilman/tmp/xenubeta.zip

Please answer me so that I know my mail isn't blocked as well.

I'll release the 1.2g version around next week.

Tilman

On Tue, 15 Mar 2005 09:13:37 -0000, eenidagent@... wrote:

>As a Yahoo! store I tried using Xenu for broken links, but ended up
>finding more value in the timeouts and connections troubles.
>
>Please consider adding 5 options; timeout variable, retry variable,
>maximum hits per second, auto restart and an auto cycle. With those
>features I would have found the problem faster and continue using
>Xenu as a website/network monitor.
>
>The problem was long download times. No timeout or connection error
>was a hard failure. The first full run using Xenu took over 4 days
>and nights of restarting to complete one full pass.
>
>FYI; other stores used NetMechanic and got clean runs until I
>convince to use a realistic time frame for an ecommerce site. We
>have 8 seconds to capture a surfers attention. How can you download
>20 linked files if 1 or more take 30 seconds?

Hi,

A timeout variable is already available and described in the FAQ (it's
worthless, because when something times out, it will always time out,
even if you wait forever, because the real cause is a dropped connection
within the TCP/IP protocol. It's like you call someone, and the line
goes "dead", i.e. you hear nothing - it doesn't help if you wait). There
is no retry variable because my experience is that when something times
out, an immediate retry does not help most of the time, which is why I
do only one single automatic retry.

If you get lots of timeouts, the best solution is usually to use less
threads.

Because your server is open for business while you check, it is even
more important to set very few threads.

The way I work on my site is to concentrate on links that are obviously
broken ("not found") while Xenu runs. When I see that it has lots of
timeouts, I press CTRL-R, maybe set less threads (I work with 5-10) and
go on with correcting links.

Tilman

Thank you so much. I think having the new version might solve our
problems.

Thanks again,

Cathy Mahe

>>> f.visser3@... 03/15/05 03:14AM >>>

hi cathy,

scanning your site with version 1.2g i found no special issues with
secure links (btw, 307 should mean something like "temporary
redirect", see:
http://members.chello.nl/f.visser3/xenu/appendix3-server-error-
messages.html

here's the report:


ok      9131 URLs      97.72%
skip type      81 URLs      0.87%
not found      73 URLs      0.78%
no such host      39 URLs      0.42%
timeout      10 URLs      0.11%
server error      2 URLs      0.02%
forbidden request      5 URLs      0.05%
ambiguous      1 URLs      0.01%
auth required      1 URLs      0.01%
busy      1 URLs      0.01%
Total      9344 URLs      100.00%

i will mail you the 1.2g version separately.

frank

--- In xenu-usergroup@yahoogroups.com, "Catherine Mahe" <cmahe@a...>
wrote:
> I hope someone can help me.  I just joined the user group and have
not
> been approved yet.  But I hope someone can answer this after I am
> approved.
>
>  I have been using Xenu since 2001, when I worked for WORLDCOM.
It has
> always worked wonderfully.  However, I just had it installed at my
new
> job, American Society for Industrial Security.  Our IT people have
done
> all they could, but I keep getting an ERROR 307 message for all
links
> requiring a member password and User ID.  It works fine for the
public
> pages.
>
> The main url for our site is http://www.asisonline.org
>
> If you run this, you will see the links with member-secure in them
are
> red and say error 307.  What should I do?  IT has installed the
XENU.INI
> file to enable cookies, however we have version 1.2f, and it says
we
> need 1.2g, not available on the download page.
>
> Any help would be greatly appreciated.  We plan to use this
software
> for many departments and uses.  It worked great at WORLDCOM.
>
> Thanks so much
>
> Cathy Mahe




Yahoo! Groups SponsorADVERTISEMENT

Yahoo! Groups Links
To visit your group on the web, go to:
http://groups.yahoo.com/group/xenu-usergroup/
   To unsubscribe from this group, send an email to:
xenu-usergroup-unsubscribe@yahoogroups.com
   Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.

Thanks to everyone for your helpful advice.  I still have not received
the 1.2g version.  I hope you can send it soon, since it has to be
installed by our IT department.  I feel hopeful that it will solve all
my problems.  By the way, I am spreading the word about Xenu.  Everyone
in my company is anxious to use it.

Thanks again,

Cathy Mahe

>>> f.visser3@... 03/15/05 03:14AM >>>

hi cathy,

scanning your site with version 1.2g i found no special issues with
secure links (btw, 307 should mean something like "temporary
redirect", see:
http://members.chello.nl/f.visser3/xenu/appendix3-server-error-
messages.html

here's the report:


ok      9131 URLs      97.72%
skip type      81 URLs      0.87%
not found      73 URLs      0.78%
no such host      39 URLs      0.42%
timeout      10 URLs      0.11%
server error      2 URLs      0.02%
forbidden request      5 URLs      0.05%
ambiguous      1 URLs      0.01%
auth required      1 URLs      0.01%
busy      1 URLs      0.01%
Total      9344 URLs      100.00%

i will mail you the 1.2g version separately.

frank

--- In xenu-usergroup@yahoogroups.com, "Catherine Mahe" <cmahe@a...>
wrote:
> I hope someone can help me.  I just joined the user group and have
not
> been approved yet.  But I hope someone can answer this after I am
> approved.
>
>  I have been using Xenu since 2001, when I worked for WORLDCOM.
It has
> always worked wonderfully.  However, I just had it installed at my
new
> job, American Society for Industrial Security.  Our IT people have
done
> all they could, but I keep getting an ERROR 307 message for all
links
> requiring a member password and User ID.  It works fine for the
public
> pages.
>
> The main url for our site is http://www.asisonline.org
>
> If you run this, you will see the links with member-secure in them
are
> red and say error 307.  What should I do?  IT has installed the
XENU.INI
> file to enable cookies, however we have version 1.2f, and it says
we
> need 1.2g, not available on the download page.
>
> Any help would be greatly appreciated.  We plan to use this
software
> for many departments and uses.  It worked great at WORLDCOM.
>
> Thanks so much
>
> Cathy Mahe




Yahoo! Groups SponsorADVERTISEMENT

Yahoo! Groups Links
To visit your group on the web, go to:
http://groups.yahoo.com/group/xenu-usergroup/
   To unsubscribe from this group, send an email to:
xenu-usergroup-unsubscribe@yahoogroups.com
   Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.

Consider this applet tag

<APPLET
  WIDTH="400"
  HEIGHT="300"
  CODE="schedule.round.robin.tournament.class"
  ARCHIVE="rrtv.jar"
  STYLE="border: solid thin black"
>

Xeno reports
...foobar/schedule.round.robin.tournament.class
error code: 404 (not found), linked from page(s):
	 ...foobar/myApplet.html

However, the ARCHIVE rrtv.jar contains the java class
schedule/round/robin/tournament.class

Would it be possible to make Xenu look in all the jars listed in the ARCHIVE=
for the class named in the CODE= ?

At present it seems Xenu is only examining the CODE= relative to the directory
of the page.

On Tue, 15 Mar 2005 14:12:32 -0000, Richard A. DeVenezia wrote:

>
>
>
>Consider this applet tag
>
><APPLET
> WIDTH="400"
> HEIGHT="300"
> CODE="schedule.round.robin.tournament.class"
> ARCHIVE="rrtv.jar"
> STYLE="border: solid thin black"
>>
>
>Xeno reports
>...foobar/schedule.round.robin.tournament.class
>error code: 404 (not found), linked from page(s):
> ...foobar/myApplet.html
>
>However, the ARCHIVE rrtv.jar contains the java class
>schedule/round/robin/tournament.class
>
>Would it be possible to make Xenu look in all the jars listed in the ARCHIVE=
for the class named in the CODE= ?
>
>At present it seems Xenu is only examining the CODE= relative to the directory
of the page.

Indeed. Xenu can't search the jar file. Now if someone has MFC code to
search jar archives, I might think about it :)

Tilman Hausherr wrote:
> On Tue, 15 Mar 2005 14:12:32 -0000, Richard A. DeVenezia wrote:
>
>>
>>
>>
>> Consider this applet tag
>>
>> <APPLET
>> WIDTH="400"
>> HEIGHT="300"
>> CODE="schedule.round.robin.tournament.class"
>> ARCHIVE="rrtv.jar"
>> STYLE="border: solid thin black"
>>>
>>
>> Xeno reports
>> ...foobar/schedule.round.robin.tournament.class
>> error code: 404 (not found), linked from page(s):
>> ...foobar/myApplet.html
>>
>> However, the ARCHIVE rrtv.jar contains the java class
>> schedule/round/robin/tournament.class
>>
>> Would it be possible to make Xenu look in all the jars listed in the
>> ARCHIVE= for the class named in the CODE= ?
>>
>> At present it seems Xenu is only examining the CODE= relative to the
>> directory of the page.
>
> Indeed. Xenu can't search the jar file. Now if someone has MFC code to
> search jar archives, I might think about it :)
>


Since a jar file is a zip file with a .jar extension instead of .zip, The
ZipArchive library  might work.
http://www.artpol-software.com/

The licensing page, http://www.artpol-software.com/zipdoc/pageLic.html,
states

" If you want to distribute software in a commercial context that
incorporates ZipArchive and you do not want to meet the conditions of GNU
GPL (e.g. an obligation to provide a customer with the source code of your
software that uses the ZipArchive library), you need to acquire a commercial
license."

But goes on further to say

" The commercial license is available free of charge for using ZipArchive
library in:
- freeware software (freeware is software that is available free of charge,
but which is copyrighted by the developer, who retains the right to control
its redistribution and to sell it in the future)
"

Hope that helps!


Richard A. DeVenezia

On Tue, 15 Mar 2005 13:20:26 -0500, Richard A. DeVenezia wrote:

>Since a jar file is a zip file with a .jar extension instead of .zip, The
>ZipArchive library  might work.
>http://www.artpol-software.com/
>
>The licensing page, http://www.artpol-software.com/zipdoc/pageLic.html,
>states
>
>" If you want to distribute software in a commercial context that
>incorporates ZipArchive and you do not want to meet the conditions of GNU
>GPL (e.g. an obligation to provide a customer with the source code of your
>software that uses the ZipArchive library), you need to acquire a commercial
>license."
>
>But goes on further to say
>
>" The commercial license is available free of charge for using ZipArchive
>library in:
>- freeware software (freeware is software that is available free of charge,
>but which is copyrighted by the developer, who retains the right to control
>its redistribution and to sell it in the future)
>"

But as soon as I sell it, I would have to publish the source code :-(

Tilman Hausherr wrote:
> On Tue, 15 Mar 2005 13:20:26 -0500, Richard A. DeVenezia wrote:
...
>> http://www.artpol-software.com/
...
>> " The commercial license is available free of charge for using
>> ZipArchive library in:
>> - freeware software (freeware is software that is available free of
>> charge, but which is copyrighted by the developer, who retains the
>> right to control its redistribution and to sell it in the future)
>> "
>
> But as soon as I sell it, I would have to publish the source code :-(

There is always something else, mayhap http://www.zlib.net, maybe not MFC
and directly .zip ready, but how far off could it be?
Per zlib.h

<quote>
* zlib.h -- interface of the 'zlib' general purpose compression library
   version 1.0.4, Jul 24th, 1996.

   Copyright (C) 1995-1996 Jean-loup Gailly and Mark Adler

   This software is provided 'as-is', without any express or implied
   warranty.  In no event will the authors be held liable for any damages
   arising from the use of this software.

   Permission is granted to anyone to use this software for any purpose,
   including commercial applications, and to alter it and redistribute it
   freely, subject to the following restrictions:

   1. The origin of this software must not be misrepresented; you must not
      claim that you wrote the original software. If you use this software
      in a product, an acknowledgment in the product documentation would be
      appreciated but is not required.
   2. Altered source versions must be plainly marked as such, and must not be
      misrepresented as being the original software.
   3. This notice may not be removed or altered from any source distribution.

   Jean-loup Gailly        Mark Adler
   gzip@...    madler@...
</quote>

Regardless,
Thank you for Xenu, its a great piece of software.

Richard

Here is an idea for future versions:

   Have a checkbox to tell Xenu to honor directives of the robots name in a
META tag

For example, some of my pages are just 'link aggregators' (which come in
handy when viewing the site in HotMetalPro)
In this case, if the "Honor META robots directive" was checked, Xenu would
_not_ try to follow all the HREFS on the page, because the META content has
NOFOLLOW.

<HTML>
   <HEAD>
   <TITLE>Requires of Mary Lupardi</TITLE>
   <META NAME="robots" CONTENT="noindex,nofollow">
   <LINK HREF=".htaccess">
   <LINK HREF="../index.html">
   <LINK HREF="../memoirs.php">
   <LINK HREF="../images/navdot.png">
   <LINK HREF="php/load_nav_array.inc">
   <LINK HREF="php/nav_array.inc">
   <LINK HREF="php/nav.php">
   <LINK HREF="php/section.php">
   <LINK HREF="php/annotate.php">
   <LINK HREF="php/counter.inc">
   <LINK HREF="html/guestbook.htm">
   <LINK HREF="html/mem00.htm"> <LINK HREF="html/mem01.htm">
   <LINK HREF="html/mem02.htm"> <LINK HREF="html/mem03.htm">
   <LINK HREF="html/mem04.htm"> <LINK HREF="html/mem05.htm">
   </HEAD>
   <BODY> </BODY>
</HTML>


Likewise, honoring the directives of a "robots.txt" that is found in a
folder would sometimes be handy.

Richard A. DeVenezia

I found chasing broken links was secondary to retesting links that
were timing out. I started with 50% timeouts using a 20 second
timeout and have gotten it down to below 2% using a 11 second
timeout out of Yahoo!'s Santa Clara, CA center. I stopped the test,
then immediately restarted and the failing links did not fail again.

Other Yahoo! stores have since found the same problem and Yahoo! is
fixing it. What I need now is a method to monitor by walking slowly
through the website testing for broken links.

I had not thought to check for non-published variable. I found the
how to add the variable and have started testing using a 12 second
timeout. 12 seconds is 150% longer then 33% of all surfers are
willing to wait.

John Tangney

I changed my conf file.  SuppressColumnSorting makes things so Apache will not
show directory listings with extra sorting links.

<IfModule mod_autoindex.c>
...
     IndexOptions FancyIndexing SuppressColumnSorting
....

My site has quite a few folders that are 'open'.  This means the apache web
server presents a folder listing, and listing has some links I want to
avoid.

For example

http://127.0.0.1:81/papers/sugi-29/examples/example10/
http://127.0.0.1:81/papers/sugi-29/examples/example10/?D=A
http://127.0.0.1:81/papers/sugi-29/examples/example10/?D=D
http://127.0.0.1:81/papers/sugi-29/examples/example10/?M=A
http://127.0.0.1:81/papers/sugi-29/examples/example10/?M=D
http://127.0.0.1:81/papers/sugi-29/examples/example10/?N=A
http://127.0.0.1:81/papers/sugi-29/examples/example10/?N=D
http://127.0.0.1:81/papers/sugi-29/examples/example10/?S=A
http://127.0.0.1:81/papers/sugi-29/examples/example10/?S=D

This happens for each of 20 or so folders I have.  I would only want the
first line from this bunch.

Is there a way to avoiding Apache folder listings?
Does the -exclude section accept wildcards or regexs ?

Richard A. DeVenezia

hi all,

i updated my online xenu manual with some useful info on the "Check
redirects" option:

http://members.chello.nl/f.visser3/xenu/9-site-with-redirects.html

it allows you to find the source pages of broken links that
normally "fall off" the link reports when redirects are involved.

frank

Glad to find there's a forum for this great tool. Thanks for being here.



My question is (and I've looked for this in FAQs and in the forum and in the
product documentation) does my traffic measurements include the hits the
site gets when Xenu does a link check? Ideally I get 10 million hits a week
and the few that I do with Xenu don't move that much, but that's not the
case.



Thanks for any advice, pointers in that direction.



Regards,



Scott



[Non-text portions of this message have been removed]

On Sat, 19 Mar 2005 06:58:11 -0600, Scott Farnsworth wrote:

>
>Glad to find there's a forum for this great tool. Thanks for being here.
>
>
>
>My question is (and I've looked for this in FAQs and in the forum and in the
>product documentation) does my traffic measurements include the hits the
>site gets when Xenu does a link check? Ideally I get 10 million hits a week
>and the few that I do with Xenu don't move that much, but that's not the
>case.

You would have to look at the detailed server statistics. If your site
is "external" to the site that is checked with Xenu, the traffic
increase is minimal, because Xenu does not download the page (HEAD). If
you / someone checks your site directly with Xenu, then it's like
someone would download every site page.

Since yesterday, the 1.2g is no longer a "beta", and has been uploaded
to its normal location. I'll make an "official" announcement tonight or
tomorrow.
http://home.snafu.de/tilman/xenulink.html
http://home.snafu.de/tilman/XENU.ZIP


Please save the previous version of XENU.EXE "just in case" :-)


18.3.2004 (1.2g)
Major improvements:
- Attempt at javascript thanks to
   http://www.codeguru.com/Cpp/Cpp/string/regex/article.php/c2779/
   details explained at
   http://home.snafu.de/tilman/xenulink.html#javascript
Minor improvements:
- Show elapsed time in status bar [15.1.2005 changed archiving format]
- TARGET=_blank instead of TARGET=Xenu in report
- New Version 2.44 of CSMTPConnection http://www.naughter.com/smtp.html
- "//" in local files is always an error
- mailed report as "XXXX.htm" instead of "XXXX.tmp.htm"
- Version String in .XEN file
- vTimeoutSimilarHosts() more efficient with huge sites
- Faster local link checking (no copying to %temp% file)
- HTTP_STATUS_REDIRECT_KEEP_VERB (307)
- ERROR_INTERNET_CLIENT_AUTH_CERT_NEEDED (12044) error handling
- passive ftp mode in orphan dialog box
- Send XENU.INI file as mail test instead of CONFIG.SYS
- Orphan check also for https://
- "New" Dialogbox can be used to enter a ftp link (no crawling!)
- Cookies allowed when [Options] AllowCookies=1
   don't use this if you have links that delete or change something!
- (Test / by requests only) MSOEXCLUDE, PARSETEST
Bug fixes:
- better error handling for error 12003 in FTP orphan check
- _findclose in local orphan check (to unlock directory!)
- /> bug fixed in META REFRESH
- &# handling in vReplaceAmpStuff() and in bProcessLink()
- handle redirection target as a possibly relative link
- No empty URLs in URL list
- date, size for file:///
- alexa, google cache and wayback only for http:// and https://
- offset in ParseTag as int instead of short for tags > 64K
- cut off after '?' in remote orphan check
- exclude excluded URLs in Orphan list
- WINVER 0x0400

hi tilman,

i updated the history page of the online manual.

please change 18.3.2004 (1.2g) into 18.3.2005 (1.2g) in your readme
list.

frank

--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@s...>
wrote:
> Since yesterday, the 1.2g is no longer a "beta", and has been
uploaded
> to its normal location. I'll make an "official" announcement
tonight or
> tomorrow.
> http://home.snafu.de/tilman/xenulink.html
> http://home.snafu.de/tilman/XENU.ZIP
>
>
> Please save the previous version of XENU.EXE "just in case" :-)
>
>
> 18.3.2004 (1.2g)
> Major improvements:
> - Attempt at javascript thanks to
>   http://www.codeguru.com/Cpp/Cpp/string/regex/article.php/c2779/
>   details explained at
>   http://home.snafu.de/tilman/xenulink.html#javascript
> Minor improvements:
> - Show elapsed time in status bar [15.1.2005 changed archiving
format]
> - TARGET=_blank instead of TARGET=Xenu in report
> - New Version 2.44 of CSMTPConnection
http://www.naughter.com/smtp.html
> - "//" in local files is always an error
> - mailed report as "XXXX.htm" instead of "XXXX.tmp.htm"
> - Version String in .XEN file
> - vTimeoutSimilarHosts() more efficient with huge sites
> - Faster local link checking (no copying to %temp% file)
> - HTTP_STATUS_REDIRECT_KEEP_VERB (307)
> - ERROR_INTERNET_CLIENT_AUTH_CERT_NEEDED (12044) error handling
> - passive ftp mode in orphan dialog box
> - Send XENU.INI file as mail test instead of CONFIG.SYS
> - Orphan check also for https://
> - "New" Dialogbox can be used to enter a ftp link (no crawling!)
> - Cookies allowed when [Options] AllowCookies=1
>   don't use this if you have links that delete or change something!
> - (Test / by requests only) MSOEXCLUDE, PARSETEST
> Bug fixes:
> - better error handling for error 12003 in FTP orphan check
> - _findclose in local orphan check (to unlock directory!)
> - /> bug fixed in META REFRESH
> - &# handling in vReplaceAmpStuff() and in bProcessLink()
> - handle redirection target as a possibly relative link
> - No empty URLs in URL list
> - date, size for file:///
> - alexa, google cache and wayback only for http:// and https://
> - offset in ParseTag as int instead of short for tags > 64K
> - cut off after '?' in remote orphan check
> - exclude excluded URLs in Orphan list
> - WINVER 0x0400

On Wed, 23 Mar 2005 18:22:28 -0000, frank visser wrote:

>please change 18.3.2004 (1.2g) into 18.3.2005 (1.2g) in your readme
>list.

ooops :-(

I think that the latest version 1.2g has introduced a problem when
checking local files. I select the Local file button select the
correct file for checking and the path "file:///\\lcc-svr-01
\STFCS$\My L2L\Upload\index.html" is entered. This produces a file
not found error message. Exactly the same text for the path is
entered into IE and the web page appears. I guess it is to do with
the way in which the UNC paths are defined. I looked up the
conventions and... "On Microsoft Windows platforms, Tcl supports
both drive-relative and UNC style names. Both / and \ may be used as
directory separators in either type of name. Drive-relative names
consist of an optional drive specifier followed by an absolute or
relative path. UNC paths follow the general form
\\servername\sharename\path\file, but must at the very least contain
the server and share components, i.e. \\servername\sharename." It
would appear that the double back slash is common to all UNC paths.
So is a tweak to 1.2g required for the local file option?

On Wed, 23 Mar 2005 18:40:10 -0000, stfcs wrote:

>I think that the latest version 1.2g has introduced a problem when
>checking local files. I select the Local file button select the
>correct file for checking and the path "file:///\\lcc-svr-01
>\STFCS$\My L2L\Upload\index.html" is entered. This produces a file
>not found error message. Exactly the same text for the path is
>entered into IE and the web page appears. I guess it is to do with
>the way in which the UNC paths are defined. I looked up the
>conventions and... "On Microsoft Windows platforms, Tcl supports
>both drive-relative and UNC style names. Both / and \ may be used as
>directory separators in either type of name. Drive-relative names
>consist of an optional drive specifier followed by an absolute or
>relative path. UNC paths follow the general form
>\\servername\sharename\path\file, but must at the very least contain
>the server and share components, i.e. \\servername\sharename." It
>would appear that the double back slash is common to all UNC paths.
>So is a tweak to 1.2g required for the local file option?

I don't know why it doesn't work for you. Try chosing it from the "Local
file..." button.

Tilman

The original problem was generated using the 'Local file ' button.
A sample of the 'Report'


   Xenu's broken link report

Created on March 24, 2005 at 08:53:37


     Root URL: file://///lcc-svr-01/STFCS$/My%20L2L/Upload/index.html
     <cid:part1.06030508.08010505@...>


       Table of contents

     * Broken links, ordered by link <#bylink>
     * Broken links, ordered by page <#bypage>
     * List of redirected URLs <#redir>
     * List of ftp and gopher URLs <#ftpgopher>
     * List of valid URLs you can submit to a search engine <#valid>
     * Site Map of HTML pages with a Title <#sitemap>
     * Broken page-local links <#targets>
     * Orphan files <#orphans>
     * Statistics for managers <#stats>


     Broken links, ordered by link:

file://///lcc-svr-01/STFCS$/My%20L2L/Upload/index.html
<cid:part1.06030508.08010505@...>
error code: 2 (file not found), linked from page(s):

0 broken link(s) reported

The path "file://///lcc-svr-01/STFCS$/My%20L2L/Upload/index.html
<cid:part1.06030508.08010505@...>"
works Ok in IE.

Can anybody else reproduce the problem?
Clive


Tilman Hausherr wrote:

> On Wed, 23 Mar 2005 18:40:10 -0000, stfcs wrote:
>
> >I think that the latest version 1.2g has introduced a problem when
> >checking local files. I select the Local file button select the
> >correct file for checking and the path "file:///\\lcc-svr-01
> >\STFCS$\My L2L\Upload\index.html" is entered. This produces a file
> >not found error message. Exactly the same text for the path is
> >entered into IE and the web page appears. I guess it is to do with
> >the way in which the UNC paths are defined. I looked up the
> >conventions and... "On Microsoft Windows platforms, Tcl supports
> >both drive-relative and UNC style names. Both / and \ may be used as
> >directory separators in either type of name. Drive-relative names
> >consist of an optional drive specifier followed by an absolute or
> >relative path. UNC paths follow the general form
> >\\servername\sharename\path\file, but must at the very least contain
> >the server and share components, i.e. \\servername\sharename." It
> >would appear that the double back slash is common to all UNC paths.
> >So is a tweak to 1.2g required for the local file option?
>
> I don't know why it doesn't work for you. Try chosing it from the "Local
> file..." button.
>
> Tilman
>
>
>
> *Yahoo! Groups Sponsor*
> ADVERTISEMENT
> click here
>
<http://us.ard.yahoo.com/SIG=129td2ajj/M=298184.6191685.7192823.3001176/D=groups\
/S=1705005512:HM/EXP=1111692841/A=2593423/R=0/SIG=11el9gslf/*http://www.netflix.\
com/Default?mqso=60190075>
>
>
>
> ------------------------------------------------------------------------
> *Yahoo! Groups Links*
>
>     * To visit your group on the web, go to:
>       http://groups.yahoo.com/group/xenu-usergroup/
>
>     * To unsubscribe from this group, send an email to:
>       xenu-usergroup-unsubscribe@yahoogroups.com
>       <mailto:xenu-usergroup-unsubscribe@yahoogroups.com?subject=Unsubscribe>
>
>     * Your use of Yahoo! Groups is subject to the Yahoo! Terms of
>       Service <http://docs.yahoo.com/info/terms/>.
>
>


[Non-text portions of this message have been removed]

hi clive,

i can check local links with the Local File button all right, but my
start URL appears as this in the link report:

file:///G:/FTAF-START.HTM

so it starts with a drive: after file:///

frank


--- In xenu-usergroup@yahoogroups.com, "Clive Sabey [CS]"
<stfcs@l...> wrote:
>
> The original problem was generated using the 'Local file ' button.
> A sample of the 'Report'
>
>
>   Xenu's broken link report
>
> Created on March 24, 2005 at 08:53:37
>
>
>     Root URL: file://///lcc-svr-01/STFCS$/My%20L2L/Upload/index.html
>     <cid:part1.06030508.08010505@l...>
>
>
>       Table of contents
>
>     * Broken links, ordered by link <#bylink>
>     * Broken links, ordered by page <#bypage>
>     * List of redirected URLs <#redir>
>     * List of ftp and gopher URLs <#ftpgopher>
>     * List of valid URLs you can submit to a search engine <#valid>
>     * Site Map of HTML pages with a Title <#sitemap>
>     * Broken page-local links <#targets>
>     * Orphan files <#orphans>
>     * Statistics for managers <#stats>
>
>
>     Broken links, ordered by link:
>
> file://///lcc-svr-01/STFCS$/My%20L2L/Upload/index.html
<cid:part1.06030508.08010505@l...>
> error code: 2 (file not found), linked from page(s):
>
> 0 broken link(s) reported
>
> The path "file://///lcc-svr-01/STFCS$/My%20L2L/Upload/index.html
<cid:part1.06030508.08010505@l...>"
> works Ok in IE.
>
> Can anybody else reproduce the problem?
> Clive
>
>
> Tilman Hausherr wrote:
>
> > On Wed, 23 Mar 2005 18:40:10 -0000, stfcs wrote:
> >
> > >I think that the latest version 1.2g has introduced a problem
when
> > >checking local files. I select the Local file button select the
> > >correct file for checking and the path "file:///\\lcc-svr-01
> > >\STFCS$\My L2L\Upload\index.html" is entered. This produces a
file
> > >not found error message. Exactly the same text for the path is
> > >entered into IE and the web page appears. I guess it is to do
with
> > >the way in which the UNC paths are defined. I looked up the
> > >conventions and... "On Microsoft Windows platforms, Tcl supports
> > >both drive-relative and UNC style names. Both / and \ may be
used as
> > >directory separators in either type of name. Drive-relative names
> > >consist of an optional drive specifier followed by an absolute or
> > >relative path. UNC paths follow the general form
> > >\\servername\sharename\path\file, but must at the very least
contain
> > >the server and share components, i.e. \\servername\sharename." It
> > >would appear that the double back slash is common to all UNC
paths.
> > >So is a tweak to 1.2g required for the local file option?
> >
> > I don't know why it doesn't work for you. Try chosing it from
the "Local
> > file..." button.
> >
> > Tilman
> >
> >
> >
> > *Yahoo! Groups Sponsor*
> > ADVERTISEMENT
> > click here
> >
<http://us.ard.yahoo.com/SIG=129td2ajj/M=298184.6191685.7192823.300117
6/D=groups/S=1705005512:HM/EXP=1111692841/A=2593423/R=0/SIG=11el9gslf/
*http://www.netflix.com/Default?mqso=60190075>
> >
> >
> >
> > ------------------------------------------------------------------
------
> > *Yahoo! Groups Links*
> >
> >     * To visit your group on the web, go to:
> >       http://groups.yahoo.com/group/xenu-usergroup/
> >
> >     * To unsubscribe from this group, send an email to:
> >       xenu-usergroup-unsubscribe@yahoogroups.com
> >       <mailto:xenu-usergroup-unsubscribe@yahoogroups.com?
subject=Unsubscribe>
> >
> >     * Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> >       Service <http://docs.yahoo.com/info/terms/>.
> >
> >
>
>
> [Non-text portions of this message have been removed]

Thanks Frank I guess it is UNC network problem.
Can anyone test a UNC path?

frank visser wrote:

>
> hi clive,
>
> i can check local links with the Local File button all right, but my
> start URL appears as this in the link report:
>
> file:///G:/FTAF-START.HTM
>
> so it starts with a drive: after file:///
>
> frank
>
>
> --- In xenu-usergroup@yahoogroups.com, "Clive Sabey [CS]"
> <stfcs@l...> wrote:
> >
> > The original problem was generated using the 'Local file ' button.
> > A sample of the 'Report'
> >
> >
> >   Xenu's broken link report
> >
> > Created on March 24, 2005 at 08:53:37
> >
> >
> >     Root URL: file://///lcc-svr-01/STFCS$/My%20L2L/Upload/index.html
> >     <cid:part1.06030508.08010505@l...>
> >
> >
> >       Table of contents
> >
> >     * Broken links, ordered by link <#bylink>
> >     * Broken links, ordered by page <#bypage>
> >     * List of redirected URLs <#redir>
> >     * List of ftp and gopher URLs <#ftpgopher>
> >     * List of valid URLs you can submit to a search engine <#valid>
> >     * Site Map of HTML pages with a Title <#sitemap>
> >     * Broken page-local links <#targets>
> >     * Orphan files <#orphans>
> >     * Statistics for managers <#stats>
> >
> >
> >     Broken links, ordered by link:
> >
> > file://///lcc-svr-01/STFCS$/My%20L2L/Upload/index.html
> <cid:part1.06030508.08010505@l...>
> > error code: 2 (file not found), linked from page(s):
> >
> > 0 broken link(s) reported
> >
> > The path "file://///lcc-svr-01/STFCS$/My%20L2L/Upload/index.html
> <cid:part1.06030508.08010505@l...>"
> > works Ok in IE.
> >
> > Can anybody else reproduce the problem?
> > Clive
> >
> >
> > Tilman Hausherr wrote:
> >
> > > On Wed, 23 Mar 2005 18:40:10 -0000, stfcs wrote:
> > >
> > > >I think that the latest version 1.2g has introduced a problem
> when
> > > >checking local files. I select the Local file button select the
> > > >correct file for checking and the path "file:///\\lcc-svr-01
> > > >\STFCS$\My L2L\Upload\index.html" is entered. This produces a
> file
> > > >not found error message. Exactly the same text for the path is
> > > >entered into IE and the web page appears. I guess it is to do
> with
> > > >the way in which the UNC paths are defined. I looked up the
> > > >conventions and... "On Microsoft Windows platforms, Tcl supports
> > > >both drive-relative and UNC style names. Both / and \ may be
> used as
> > > >directory separators in either type of name. Drive-relative names
> > > >consist of an optional drive specifier followed by an absolute or
> > > >relative path. UNC paths follow the general form
> > > >\\servername\sharename\path\file, but must at the very least
> contain
> > > >the server and share components, i.e. \\servername\sharename." It
> > > >would appear that the double back slash is common to all UNC
> paths.
> > > >So is a tweak to 1.2g required for the local file option?
> > >
> > > I don't know why it doesn't work for you. Try chosing it from
> the "Local
> > > file..." button.
> > >
> > > Tilman
> > >
> > >
> > >
> > > *Yahoo! Groups Sponsor*
> > > ADVERTISEMENT
> > > click here
> > >
> <http://us.ard.yahoo.com/SIG=129td2ajj/M=298184.6191685.7192823.300117
> 6/D=groups/S=1705005512:HM/EXP=1111692841/A=2593423/R=0/SIG=11el9gslf/
> *http://www.netflix.com/Default?mqso=60190075>
> > >
> > >
> > >
> > > ------------------------------------------------------------------
> ------
> > > *Yahoo! Groups Links*
> > >
> > >     * To visit your group on the web, go to:
> > >       http://groups.yahoo.com/group/xenu-usergroup/
> > >
> > >     * To unsubscribe from this group, send an email to:
> > >       xenu-usergroup-unsubscribe@yahoogroups.com
> > >       <mailto:xenu-usergroup-unsubscribe@yahoogroups.com?
> subject=Unsubscribe>
> > >
> > >     * Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> > >       Service <http://docs.yahoo.com/info/terms/>.
> > >
> > >
> >
> >
> > [Non-text portions of this message have been removed]
>
>
>
>
> *Yahoo! Groups Sponsor*
> ADVERTISEMENT
> click here
>
<http://us.ard.yahoo.com/SIG=1294dh7ik/M=298184.6191685.7192823.3001176/D=groups\
/S=1705005512:HM/EXP=1111751818/A=2593423/R=0/SIG=11el9gslf/*http://www.netflix.\
com/Default?mqso=60190075>
>
>
>
> ------------------------------------------------------------------------
> *Yahoo! Groups Links*
>
>     * To visit your group on the web, go to:
>       http://groups.yahoo.com/group/xenu-usergroup/
>
>     * To unsubscribe from this group, send an email to:
>       xenu-usergroup-unsubscribe@yahoogroups.com
>       <mailto:xenu-usergroup-unsubscribe@yahoogroups.com?subject=Unsubscribe>
>
>     * Your use of Yahoo! Groups is subject to the Yahoo! Terms of
>       Service <http://docs.yahoo.com/info/terms/>.
>
>


[Non-text portions of this message have been removed]