Hi,

I've got a problem although it may not actually be a problem per se.

1 site I monitor with Xenu reports a 12002 timeout error on about a
dozen pages.

Having used Xenu for 5 yrs or so, never seen this before.  Further a
standard HTTP header viewer shows 200 codes for these pages.

I can't seem to find anything about this error code other than it is a
"timeout".

Is this something I should be concerned about?

Thanks

Geof Day
Cambridge, MA

On Tue, 16 Jan 2007 02:56:40 -0000, geoffday2000 wrote:

>Hi,
>
>I've got a problem although it may not actually be a problem per se.
>
>1 site I monitor with Xenu reports a 12002 timeout error on about a
>dozen pages.
>
>Having used Xenu for 5 yrs or so, never seen this before.  Further a
>standard HTTP header viewer shows 200 codes for these pages.

Maybe Xenu does it at the 2nd attempt (CTRL-R) ?

Tilman

>
>I can't seem to find anything about this error code other than it is a
>"timeout".
>
>Is this something I should be concerned about?
>
>Thanks
>
>Geof Day
>Cambridge, MA
>
>
>
>
>Yahoo! Groups Links
>
>
>

Could you make the "ftp host:" field wider in the "Remote Orphan
Check: Ftp Parameters" dialog display?  The current width does not
display enough characters to show the address in its entirety.

There isn't enough space to display the entire application layer and
octets of a standard IPv4 address without scrolling.  Using a fully
qualified domain name plus a user directory on a shared host usually
won't fit either.  (When IPv6 support comes into wider use, this
field will become more difficult to use for reference.)

I typically need Xenu most when someone asks me to help them clean up
their website. And, since most of those people use shared hosting
services, it takes quite a few attempts to set up the "Remote Orphan
Check: Ftp Parameters" dialog so that it works with the host's
idiosyncrasies.  I haven't figured out why the values that work
successfully in WS-FTP won't also work in Xenu.

(If I don't have time to mess with these settings, I just copy the
whole website via FTP and use Xenu on that local mirror.  Then the
FTP configuration becomes moot for a local orphan check.  That works
so much more simply.)

It would be great if there was a "validate" button that verified that
the settings would provide access.  Even better if it could display a
directory listing (or validated the expected subdirectories existed)
to make sure it was hitting the right subdirectory.  I get a lot
of "550 No such directory" error messages in the orphan report
section until I find the expected subdirectory.)

Thanks for reading,
Brian

We have a calendar application that uses dynamic queries, in the form

http://cmsprod.pacific.edu/x424.xml?category=&view=&month=7&year=2010&date=13

How can I prevent Xenu from checking these links?

Thanks.

Brian Phelps
webmaster
University of the Pacific

I have a link on a webpage with a format like:

<a
href="javascript:new_openStoryWindow('../../Fresh/fresh_story/2007/01/2007-01-17\
-3251875472.html','_blank','blankName','status=no,scrollbars=yes,resizable=yes,w\
idth=500,height=500,left=100,top=50')">Story
title</a>

I have put the javascript option in the .ini file, and I'm running
Xenu 1.2i


[Options]
javascript=javascript: *[_a-zA-Z0-9]+ *\(['"]([^'"]* +)['"]

When I run Xenu, I still get the 'skip type' message on the javascript
links.

I'd appreaciate any ideas I could try to get them checked without
creating new files for them.

Barb

On Tue, 16 Jan 2007 20:09:53 -0000, Brian wrote:

>We have a calendar application that uses dynamic queries, in the form
>
>http://cmsprod.pacific.edu/x424.xml?category=&view=&month=7&year=2010&date=13
>
>How can I prevent Xenu from checking these links?

Enter
http://cmsprod.pacific.edu/x424.xml?category=

in the exclude list at the beginning

Tilman

>
>Thanks.
>
>Brian Phelps
>webmaster
>University of the Pacific
>
>
>
>
>Yahoo! Groups Links
>
>
>

Please mention also an URL that uses that link, so that "friendly
people" can test it. (unless they do already see trouble in the JS
string)

Tilman

On Wed, 17 Jan 2007 19:23:36 -0000, Barb Schmid wrote:

>I have a link on a webpage with a format like:
>
><a
>href="javascript:new_openStoryWindow('../../Fresh/fresh_story/2007/01/2007-01-1\
7-3251875472.html','_blank','blankName','status=no,scrollbars=yes,resizable=yes,\
width=500,height=500,left=100,top=50')">Story
>title</a>
>
>I have put the javascript option in the .ini file, and I'm running
>Xenu 1.2i
>
>
>[Options]
>javascript=javascript: *[_a-zA-Z0-9]+ *\(['"]([^'"]* +)['"]
>
>When I run Xenu, I still get the 'skip type' message on the javascript
>links.
>
>I'd appreaciate any ideas I could try to get them checked without
>creating new files for them.
>
>Barb
>
>
>
>
>
>Yahoo! Groups Links
>
>
>

----- Original Message -----

From: Tilman Hausherr

To: xenu-usergroup@yahoogroups.com

Sent: Wednesday, January 17, 2007 9:23 PM

Subject: Re: [xenu-usergroup] javascript link checking

Please mention also an URL that uses that link, so that "friendly
people" can test it. (unless they do already see trouble in the JS
string)

Tilman

On Wed, 17 Jan 2007 19:23:36 -0000, Barb Schmid wrote:

>I have a link on a webpage with a format like:
>
><a
>href="javascript:new_openStoryWindow('../../Fresh/fresh_story/2007/01/2007-01-17-3251875472.html','_blank','blankName','status=no,scrollbars=yes,resizable=yes,width=500,height=500,left=100,top=50')">Story
>title</a>
>
>I have put the javascript option in the .ini file, and I'm running
>Xenu 1.2i
>
>
>[Options]
>javascript=javascript: *[_a-zA-Z0-9]+ *\(['"]([^'"]* +)['"]
>
>When I run Xenu, I still get the 'skip type' message on the javascript
>links.
>
>I'd appreaciate any ideas I could try to get them checked without
>creating new files for them.
>
>Barb
>
>
>
>
>
>Yahoo! Groups Links
>
>
>

The dialog will be wider in the next version.
The "validate" button sounds like a good idea, but I don't have the time
right now. But I'll try to do it some day.

Tilman

On Tue, 16 Jan 2007 21:53:13 -0000, bamaustin2000 wrote:

>Could you make the "ftp host:" field wider in the "Remote Orphan
>Check: Ftp Parameters" dialog display?  The current width does not
>display enough characters to show the address in its entirety.
>
>There isn't enough space to display the entire application layer and
>octets of a standard IPv4 address without scrolling.  Using a fully
>qualified domain name plus a user directory on a shared host usually
>won't fit either.  (When IPv6 support comes into wider use, this
>field will become more difficult to use for reference.)
>
>I typically need Xenu most when someone asks me to help them clean up
>their website. And, since most of those people use shared hosting
>services, it takes quite a few attempts to set up the "Remote Orphan
>Check: Ftp Parameters" dialog so that it works with the host's
>idiosyncrasies.  I haven't figured out why the values that work
>successfully in WS-FTP won't also work in Xenu.
>
>(If I don't have time to mess with these settings, I just copy the
>whole website via FTP and use Xenu on that local mirror.  Then the
>FTP configuration becomes moot for a local orphan check.  That works
>so much more simply.)
>
>It would be great if there was a "validate" button that verified that
>the settings would provide access.  Even better if it could display a
>directory listing (or validated the expected subdirectories existed)
>to make sure it was hitting the right subdirectory.  I get a lot
>of "550 No such directory" error messages in the orphan report
>section until I find the expected subdirectory.)
>
>Thanks for reading,
>Brian
>
>
>
>
>Yahoo! Groups Links
>
>
>

On Tue, 16 Jan 2007 21:53:13 -0000, bamaustin2000 wrote:

>Could you make the "ftp host:" field wider in the "Remote Orphan
>Check: Ftp Parameters" dialog display?  The current width does not
>display enough characters to show the address in its entirety.

Try this new version:
http://home.snafu.de/tilman/tmp/xenubeta.zip

This also has a new feature:

[Options]
MakeLowerCase=1

to convert all to lower case (for some users)

Thanks, Tono. I still haven't got it working, but I'll try to get
something together and post the URL to see if anyone can see what I'm
doing wrong.

Barb

--- In xenu-usergroup@yahoogroups.com, "Tono" <tonospa@...> wrote:
>
> I think the option must be:
>
>
> [Options]
> javascript=javascript: *[_a-zA-Z0-9]+ *\(['"]([^'"]* *)['"]
>
> At the end there must be an * instead of +. If you put + should be
one or more blank. With * it should be cero or one or more blank.
>
>
>
>
>   ----- Original Message -----
>   From: Tilman Hausherr
>   To: xenu-usergroup@yahoogroups.com
>   Sent: Wednesday, January 17, 2007 9:23 PM
>   Subject: Re: [xenu-usergroup] javascript link checking
>
>
>   Please mention also an URL that uses that link, so that "friendly
>   people" can test it. (unless they do already see trouble in the JS
>   string)
>
>   Tilman

Hi, with Xenu, is it possible to check external javascript files in
the format? <SCRIPT language="JavaScript"
SRC="http://www.pageresource.com/jscript/jxt1.js"></SCRIPT>

or <SCRIPT language="JavaScript" SRC="../jscript/jxt1.js"></SCRIPT>


Many thanks, Clive

Hi,

First, let me say that Xenu is truly a great program. It's come in
very handy for me.

Anyway, I read some older posts talking about a version of Xenu that
supports excluding filenames using wildcards. This would be a
tremendously helpful feature for me.

For instance, I might wish to exclude all "print friendly" pages on my
site by excluding the following:
*display=print*

The link I found to that version was unfortunately dead. Can anyone help?

Thanks a lot!

Some questions I would like to raise about the results I get from
Xenu.

Firstly, more and more sites seem to be giving out a 404 not found
error when the page actually is there, and looks fine when visited.
Same story with the 'no such host' message, which sometimes turns
out not to be true. Why does this happen? I take it that we're dealing
with some behaviour of web hosts here, as much as the Xenu
software.

My second question is about links that appear at the end of the 'By
Page' list in the Xenu report. Some of these are not links on my site
at all. This seems to arise from memberships at Webring.com - Xenu
seems to follow up links that are outside my site but may be in the
web rings.

a little enlightenment on these points would be very good. Is there
any setting I can change to address either of these questions?

Apart from these queries, I have to say Xenu is indispensable on my
system, and must have saved me hundreds of hours of grief when
checking links!

Vince.

That version is now here:
http://home.snafu.de/tilman/tmp/xenuwild1.zip

The old version/ file name seemed to have a problem, it didn't have the
wildcard feature at all (apparently), so I uploaded it under a new name,
and downloaded it myself to check.

Tilman

On Wed, 14 Feb 2007 15:24:34 -0000, maizegoblue wrote:

>Hi,
>
>First, let me say that Xenu is truly a great program. It's come in
>very handy for me.
>
>Anyway, I read some older posts talking about a version of Xenu that
>supports excluding filenames using wildcards. This would be a
>tremendously helpful feature for me.
>
>For instance, I might wish to exclude all "print friendly" pages on my
>site by excluding the following:
>*display=print*
>
>The link I found to that version was unfortunately dead. Can anyone help?
>
>Thanks a lot!

On Wed, 14 Feb 2007 16:06:01 -0000, Vince wrote:

>Some questions I would like to raise about the results I get from
>Xenu.
>
>Firstly, more and more sites seem to be giving out a 404 not found
>error when the page actually is there, and looks fine when visited.

See FAQ - some websites "hate" the Xenu user agent

>Same story with the 'no such host' message, which sometimes turns
>out not to be true. Why does this happen? I take it that we're dealing

This could be a network problem. Try CTRL-R. Also, be sure to have the
latest version.

>with some behaviour of web hosts here, as much as the Xenu
>software.
>
>My second question is about links that appear at the end of the 'By
>Page' list in the Xenu report. Some of these are not links on my site
>at all. This seems to arise from memberships at Webring.com - Xenu
>seems to follow up links that are outside my site but may be in the
>web rings.

These are redirections. Solution: exclude the webring from the spidering
in the initial dialogbox.

Tilman

>
>a little enlightenment on these points would be very good. Is there
>any setting I can change to address either of these questions?
>
>Apart from these queries, I have to say Xenu is indispensable on my
>system, and must have saved me hundreds of hours of grief when
>checking links!
>
>Vince.
>
>
>
>
>
>Yahoo! Groups Links
>
>
>

Thanks a lot, Tilman.

I have a problem with the new version of Xenu, though. It seems that
when I check a number of sites (or even just one) using the "Check URL
list" function, many links do not get checked.

I uninstalled the old version and installed the version that supports
wildcards. I did NOT try that feature yet.

The problem seems to be the way Xenu handles "Check URL list" function
in the new version. If I check a single site using the "Check URL"
dialog, I get 75 links checked. If I put that site by itself in a text
file and use "Check URL list," Xenu only finds 45 links. In the latter
case, Xenu doesn't check some images that appear even on the home page
of the site (though it does check some of them). Some external links
also get skipped, yet some still get checked. Something is not quite
right with that version. I can't determine a pattern, so I'm not sure
what is happening.

Thanks for any insight.

--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@...> wrote:
>
> That version is now here:
> http://home.snafu.de/tilman/tmp/xenuwild1.zip
>
> The old version/ file name seemed to have a problem, it didn't have the
> wildcard feature at all (apparently), so I uploaded it under a new name,
> and downloaded it myself to check.
>
> Tilman
>
> On Wed, 14 Feb 2007 15:24:34 -0000, maizegoblue wrote:
>
> >Hi,
> >
> >First, let me say that Xenu is truly a great program. It's come in
> >very handy for me.
> >
> >Anyway, I read some older posts talking about a version of Xenu that
> >supports excluding filenames using wildcards. This would be a
> >tremendously helpful feature for me.
> >
> >For instance, I might wish to exclude all "print friendly" pages on my
> >site by excluding the following:
> >*display=print*
> >
> >The link I found to that version was unfortunately dead. Can anyone
help?
> >
> >Thanks a lot!
>

The "Check URL list" has many logic problems, i.e. the behaviour is hard
to describe/understand, which is why I hate it. I should delete it.

Check URL list is normally for domains only. It spiders all the URLs,
but accepts only URLs that start with the ones in the txt file. I.e. all
URls are added to the include list.

The best would be to send me
1) the html file
2) the txt file
3) a few URLs that you claim are not checked with that list feature

Tilman

On Wed, 14 Feb 2007 20:18:21 -0000, maizegoblue wrote:

>Thanks a lot, Tilman.
>
>I have a problem with the new version of Xenu, though. It seems that
>when I check a number of sites (or even just one) using the "Check URL
>list" function, many links do not get checked.
>
>I uninstalled the old version and installed the version that supports
>wildcards. I did NOT try that feature yet.
>
>The problem seems to be the way Xenu handles "Check URL list" function
>in the new version. If I check a single site using the "Check URL"
>dialog, I get 75 links checked. If I put that site by itself in a text
>file and use "Check URL list," Xenu only finds 45 links. In the latter
>case, Xenu doesn't check some images that appear even on the home page
>of the site (though it does check some of them). Some external links
>also get skipped, yet some still get checked. Something is not quite
>right with that version. I can't determine a pattern, so I'm not sure
>what is happening.
>
>Thanks for any insight.
>
>--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@...> wrote:
>>
>> That version is now here:
>> http://home.snafu.de/tilman/tmp/xenuwild1.zip
>>
>> The old version/ file name seemed to have a problem, it didn't have the
>> wildcard feature at all (apparently), so I uploaded it under a new name,
>> and downloaded it myself to check.
>>
>> Tilman
>>
>> On Wed, 14 Feb 2007 15:24:34 -0000, maizegoblue wrote:
>>
>> >Hi,
>> >
>> >First, let me say that Xenu is truly a great program. It's come in
>> >very handy for me.
>> >
>> >Anyway, I read some older posts talking about a version of Xenu that
>> >supports excluding filenames using wildcards. This would be a
>> >tremendously helpful feature for me.
>> >
>> >For instance, I might wish to exclude all "print friendly" pages on my
>> >site by excluding the following:
>> >*display=print*
>> >
>> >The link I found to that version was unfortunately dead. Can anyone
>help?
>> >
>> >Thanks a lot!
>>
>
>
>
>
>
>Yahoo! Groups Links
>
>
>

Tilman,

I hope you don't delete the "Check URL list function"--I love it! It
works fine in the release version. It's just not working right in the
beta version.

If you scan the following site with the beta, then try doing it from a
text file, you should see a big difference.
http://dustyfeet.com/index.html

If I use "Check URL," Xenu beta finds 178 URLs. If I put that URL in a
file and use "Check URL list," Xenu only finds 9 URLs.

Again, the release version works fine, and I really like this feature.
I would go back to the release version, but I would like to use the
wildcard functionality.




--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@...> wrote:
>
> The "Check URL list" has many logic problems, i.e. the behaviour is hard
> to describe/understand, which is why I hate it. I should delete it.
>
> Check URL list is normally for domains only. It spiders all the URLs,
> but accepts only URLs that start with the ones in the txt file. I.e. all
> URls are added to the include list.
>
> The best would be to send me
> 1) the html file
> 2) the txt file
> 3) a few URLs that you claim are not checked with that list feature
>
> Tilman
>
> On Wed, 14 Feb 2007 20:18:21 -0000, maizegoblue wrote:
>
> >Thanks a lot, Tilman.
> >
> >I have a problem with the new version of Xenu, though. It seems that
> >when I check a number of sites (or even just one) using the "Check URL
> >list" function, many links do not get checked.
> >
> >I uninstalled the old version and installed the version that supports
> >wildcards. I did NOT try that feature yet.
> >
> >The problem seems to be the way Xenu handles "Check URL list" function
> >in the new version. If I check a single site using the "Check URL"
> >dialog, I get 75 links checked. If I put that site by itself in a text
> >file and use "Check URL list," Xenu only finds 45 links. In the latter
> >case, Xenu doesn't check some images that appear even on the home page
> >of the site (though it does check some of them). Some external links
> >also get skipped, yet some still get checked. Something is not quite
> >right with that version. I can't determine a pattern, so I'm not sure
> >what is happening.
> >
> >Thanks for any insight.
> >
> >--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@> wrote:
> >>
> >> That version is now here:
> >> http://home.snafu.de/tilman/tmp/xenuwild1.zip
> >>
> >> The old version/ file name seemed to have a problem, it didn't
have the
> >> wildcard feature at all (apparently), so I uploaded it under a
new name,
> >> and downloaded it myself to check.
> >>
> >> Tilman
> >>
> >> On Wed, 14 Feb 2007 15:24:34 -0000, maizegoblue wrote:
> >>
> >> >Hi,
> >> >
> >> >First, let me say that Xenu is truly a great program. It's come in
> >> >very handy for me.
> >> >
> >> >Anyway, I read some older posts talking about a version of Xenu that
> >> >supports excluding filenames using wildcards. This would be a
> >> >tremendously helpful feature for me.
> >> >
> >> >For instance, I might wish to exclude all "print friendly" pages
on my
> >> >site by excluding the following:
> >> >*display=print*
> >> >
> >> >The link I found to that version was unfortunately dead. Can anyone
> >help?
> >> >
> >> >Thanks a lot!
> >>
> >
> >
> >
> >
> >
> >Yahoo! Groups Links
> >
> >
> >
>

On Thu, 15 Feb 2007 03:27:06 -0000, maizegoblue wrote:

>Tilman,
>
>I hope you don't delete the "Check URL list function"--I love it! It
>works fine in the release version. It's just not working right in the
>beta version.

This hasn't changed for years...

>
>If you scan the following site with the beta, then try doing it from a
>text file, you should see a big difference.
>http://dustyfeet.com/index.html
>
>If I use "Check URL," Xenu beta finds 178 URLs. If I put that URL in a
>file and use "Check URL list," Xenu only finds 9 URLs.

Yes - as I said, it will spider only URls that start with
"http://dustyfeet.com/index.html".

So an URL with "http://dustyfeet.com/toys.html" would not be spidered,
it is considered as "external".

Solution: put

"http://dustyfeet.com" in the txt file instead

Tilman

>
>Again, the release version works fine, and I really like this feature.
>I would go back to the release version, but I would like to use the
>wildcard functionality.
>
>
>
>
>--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@...> wrote:
>>
>> The "Check URL list" has many logic problems, i.e. the behaviour is hard
>> to describe/understand, which is why I hate it. I should delete it.
>>
>> Check URL list is normally for domains only. It spiders all the URLs,
>> but accepts only URLs that start with the ones in the txt file. I.e. all
>> URls are added to the include list.
>>
>> The best would be to send me
>> 1) the html file
>> 2) the txt file
>> 3) a few URLs that you claim are not checked with that list feature
>>
>> Tilman
>>
>> On Wed, 14 Feb 2007 20:18:21 -0000, maizegoblue wrote:
>>
>> >Thanks a lot, Tilman.
>> >
>> >I have a problem with the new version of Xenu, though. It seems that
>> >when I check a number of sites (or even just one) using the "Check URL
>> >list" function, many links do not get checked.
>> >
>> >I uninstalled the old version and installed the version that supports
>> >wildcards. I did NOT try that feature yet.
>> >
>> >The problem seems to be the way Xenu handles "Check URL list" function
>> >in the new version. If I check a single site using the "Check URL"
>> >dialog, I get 75 links checked. If I put that site by itself in a text
>> >file and use "Check URL list," Xenu only finds 45 links. In the latter
>> >case, Xenu doesn't check some images that appear even on the home page
>> >of the site (though it does check some of them). Some external links
>> >also get skipped, yet some still get checked. Something is not quite
>> >right with that version. I can't determine a pattern, so I'm not sure
>> >what is happening.
>> >
>> >Thanks for any insight.
>> >
>> >--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@> wrote:
>> >>
>> >> That version is now here:
>> >> http://home.snafu.de/tilman/tmp/xenuwild1.zip
>> >>
>> >> The old version/ file name seemed to have a problem, it didn't
>have the
>> >> wildcard feature at all (apparently), so I uploaded it under a
>new name,
>> >> and downloaded it myself to check.
>> >>
>> >> Tilman
>> >>
>> >> On Wed, 14 Feb 2007 15:24:34 -0000, maizegoblue wrote:
>> >>
>> >> >Hi,
>> >> >
>> >> >First, let me say that Xenu is truly a great program. It's come in
>> >> >very handy for me.
>> >> >
>> >> >Anyway, I read some older posts talking about a version of Xenu that
>> >> >supports excluding filenames using wildcards. This would be a
>> >> >tremendously helpful feature for me.
>> >> >
>> >> >For instance, I might wish to exclude all "print friendly" pages
>on my
>> >> >site by excluding the following:
>> >> >*display=print*
>> >> >
>> >> >The link I found to that version was unfortunately dead. Can anyone
>> >help?
>> >> >
>> >> >Thanks a lot!
>> >>
>> >
>> >
>> >
>> >
>> >
>> >Yahoo! Groups Links
>> >
>> >
>> >
>>
>
>
>
>
>
>Yahoo! Groups Links
>
>
>

Tilman,

This time I put "http://dustyfeet.com" in a text file and got the same
result: 9 URLs found.

If I use "Check URL" for "http://dustyfeet.com" I get 178 URLs.

--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@...> wrote:
>
> On Thu, 15 Feb 2007 03:27:06 -0000, maizegoblue wrote:
>
> >Tilman,
> >
> >I hope you don't delete the "Check URL list function"--I love it! It
> >works fine in the release version. It's just not working right in the
> >beta version.
>
> This hasn't changed for years...
>
> >
> >If you scan the following site with the beta, then try doing it from a
> >text file, you should see a big difference.
> >http://dustyfeet.com/index.html
> >
> >If I use "Check URL," Xenu beta finds 178 URLs. If I put that URL in a
> >file and use "Check URL list," Xenu only finds 9 URLs.
>
> Yes - as I said, it will spider only URls that start with
> "http://dustyfeet.com/index.html".
>
> So an URL with "http://dustyfeet.com/toys.html" would not be spidered,
> it is considered as "external".
>
> Solution: put
>
> "http://dustyfeet.com" in the txt file instead
>
> Tilman
>
> >
> >Again, the release version works fine, and I really like this feature.
> >I would go back to the release version, but I would like to use the
> >wildcard functionality.
> >
> >
> >
> >
> >--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@> wrote:
> >>
> >> The "Check URL list" has many logic problems, i.e. the behaviour
is hard
> >> to describe/understand, which is why I hate it. I should delete it.
> >>
> >> Check URL list is normally for domains only. It spiders all the URLs,
> >> but accepts only URLs that start with the ones in the txt file.
I.e. all
> >> URls are added to the include list.
> >>
> >> The best would be to send me
> >> 1) the html file
> >> 2) the txt file
> >> 3) a few URLs that you claim are not checked with that list feature
> >>
> >> Tilman
> >>
> >> On Wed, 14 Feb 2007 20:18:21 -0000, maizegoblue wrote:
> >>
> >> >Thanks a lot, Tilman.
> >> >
> >> >I have a problem with the new version of Xenu, though. It seems that
> >> >when I check a number of sites (or even just one) using the
"Check URL
> >> >list" function, many links do not get checked.
> >> >
> >> >I uninstalled the old version and installed the version that
supports
> >> >wildcards. I did NOT try that feature yet.
> >> >
> >> >The problem seems to be the way Xenu handles "Check URL list"
function
> >> >in the new version. If I check a single site using the "Check URL"
> >> >dialog, I get 75 links checked. If I put that site by itself in
a text
> >> >file and use "Check URL list," Xenu only finds 45 links. In the
latter
> >> >case, Xenu doesn't check some images that appear even on the
home page
> >> >of the site (though it does check some of them). Some external links
> >> >also get skipped, yet some still get checked. Something is not quite
> >> >right with that version. I can't determine a pattern, so I'm not
sure
> >> >what is happening.
> >> >
> >> >Thanks for any insight.
> >> >
> >> >--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@>
wrote:
> >> >>
> >> >> That version is now here:
> >> >> http://home.snafu.de/tilman/tmp/xenuwild1.zip
> >> >>
> >> >> The old version/ file name seemed to have a problem, it didn't
> >have the
> >> >> wildcard feature at all (apparently), so I uploaded it under a
> >new name,
> >> >> and downloaded it myself to check.
> >> >>
> >> >> Tilman
> >> >>
> >> >> On Wed, 14 Feb 2007 15:24:34 -0000, maizegoblue wrote:
> >> >>
> >> >> >Hi,
> >> >> >
> >> >> >First, let me say that Xenu is truly a great program. It's
come in
> >> >> >very handy for me.
> >> >> >
> >> >> >Anyway, I read some older posts talking about a version of
Xenu that
> >> >> >supports excluding filenames using wildcards. This would be a
> >> >> >tremendously helpful feature for me.
> >> >> >
> >> >> >For instance, I might wish to exclude all "print friendly" pages
> >on my
> >> >> >site by excluding the following:
> >> >> >*display=print*
> >> >> >
> >> >> >The link I found to that version was unfortunately dead. Can
anyone
> >> >help?
> >> >> >
> >> >> >Thanks a lot!
> >> >>
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >Yahoo! Groups Links
> >> >
> >> >
> >> >
> >>
> >
> >
> >
> >
> >
> >Yahoo! Groups Links
> >
> >
> >
>

I see now what happened:

As I said, the the URLs in "Check URL list" are used as an include list.

Now remember that in the wildcard version, it is important always to add
"*" at the beginning and end - unless you "want" this to be the
beginning or end.

The root URL in the "Check URL list" version is
file:///c:/dokumente und einstellungen/......../test.txt

So only URLs that start with this, or that "match" whats in the wildcard
list, are internal (and spidered).


What we learn:
Don't use use "Check URL list" with the wildcard version.


Alternatively, I could simply add "*" at the end of every entry for the
include table when using that feature, and it might work.

Tilman


On Thu, 15 Feb 2007 12:47:57 -0000, maizegoblue wrote:

>Tilman,
>
>This time I put "http://dustyfeet.com" in a text file and got the same
>result: 9 URLs found.
>
>If I use "Check URL" for "http://dustyfeet.com" I get 178 URLs.
>
>--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@...> wrote:
>>
>> On Thu, 15 Feb 2007 03:27:06 -0000, maizegoblue wrote:
>>
>> >Tilman,
>> >
>> >I hope you don't delete the "Check URL list function"--I love it! It
>> >works fine in the release version. It's just not working right in the
>> >beta version.
>>
>> This hasn't changed for years...
>>
>> >
>> >If you scan the following site with the beta, then try doing it from a
>> >text file, you should see a big difference.
>> >http://dustyfeet.com/index.html
>> >
>> >If I use "Check URL," Xenu beta finds 178 URLs. If I put that URL in a
>> >file and use "Check URL list," Xenu only finds 9 URLs.
>>
>> Yes - as I said, it will spider only URls that start with
>> "http://dustyfeet.com/index.html".
>>
>> So an URL with "http://dustyfeet.com/toys.html" would not be spidered,
>> it is considered as "external".
>>
>> Solution: put
>>
>> "http://dustyfeet.com" in the txt file instead
>>
>> Tilman
>>
>> >
>> >Again, the release version works fine, and I really like this feature.
>> >I would go back to the release version, but I would like to use the
>> >wildcard functionality.
>> >
>> >
>> >
>> >
>> >--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@> wrote:
>> >>
>> >> The "Check URL list" has many logic problems, i.e. the behaviour
>is hard
>> >> to describe/understand, which is why I hate it. I should delete it.
>> >>
>> >> Check URL list is normally for domains only. It spiders all the URLs,
>> >> but accepts only URLs that start with the ones in the txt file.
>I.e. all
>> >> URls are added to the include list.
>> >>
>> >> The best would be to send me
>> >> 1) the html file
>> >> 2) the txt file
>> >> 3) a few URLs that you claim are not checked with that list feature
>> >>
>> >> Tilman
>> >>
>> >> On Wed, 14 Feb 2007 20:18:21 -0000, maizegoblue wrote:
>> >>
>> >> >Thanks a lot, Tilman.
>> >> >
>> >> >I have a problem with the new version of Xenu, though. It seems that
>> >> >when I check a number of sites (or even just one) using the
>"Check URL
>> >> >list" function, many links do not get checked.
>> >> >
>> >> >I uninstalled the old version and installed the version that
>supports
>> >> >wildcards. I did NOT try that feature yet.
>> >> >
>> >> >The problem seems to be the way Xenu handles "Check URL list"
>function
>> >> >in the new version. If I check a single site using the "Check URL"
>> >> >dialog, I get 75 links checked. If I put that site by itself in
>a text
>> >> >file and use "Check URL list," Xenu only finds 45 links. In the
>latter
>> >> >case, Xenu doesn't check some images that appear even on the
>home page
>> >> >of the site (though it does check some of them). Some external links
>> >> >also get skipped, yet some still get checked. Something is not quite
>> >> >right with that version. I can't determine a pattern, so I'm not
>sure
>> >> >what is happening.
>> >> >
>> >> >Thanks for any insight.
>> >> >
>> >> >--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@>
>wrote:
>> >> >>
>> >> >> That version is now here:
>> >> >> http://home.snafu.de/tilman/tmp/xenuwild1.zip
>> >> >>
>> >> >> The old version/ file name seemed to have a problem, it didn't
>> >have the
>> >> >> wildcard feature at all (apparently), so I uploaded it under a
>> >new name,
>> >> >> and downloaded it myself to check.
>> >> >>
>> >> >> Tilman
>> >> >>
>> >> >> On Wed, 14 Feb 2007 15:24:34 -0000, maizegoblue wrote:
>> >> >>
>> >> >> >Hi,
>> >> >> >
>> >> >> >First, let me say that Xenu is truly a great program. It's
>come in
>> >> >> >very handy for me.
>> >> >> >
>> >> >> >Anyway, I read some older posts talking about a version of
>Xenu that
>> >> >> >supports excluding filenames using wildcards. This would be a
>> >> >> >tremendously helpful feature for me.
>> >> >> >
>> >> >> >For instance, I might wish to exclude all "print friendly" pages
>> >on my
>> >> >> >site by excluding the following:
>> >> >> >*display=print*
>> >> >> >
>> >> >> >The link I found to that version was unfortunately dead. Can
>anyone
>> >> >help?
>> >> >> >
>> >> >> >Thanks a lot!
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >Yahoo! Groups Links
>> >> >
>> >> >
>> >> >
>> >>
>> >
>> >
>> >
>> >
>> >
>> >Yahoo! Groups Links
>> >
>> >
>> >
>>
>
>
>
>
>
>Yahoo! Groups Links
>
>
>

Do you need BOTH the wildcard feature, and the "Check URL list" feature?
If yes, I'll upload the corrected version. This quasi-bug has now been
fixed.

Tilman

====================

I see now what happened:

As I said, the the URLs in "Check URL list" are used as an include list.

Now remember that in the wildcard version, it is important always to add
"*" at the beginning and end - unless you "want" this to be the
beginning or end.

The root URL in the "Check URL list" version is
file:///c:/dokumente und einstellungen/......../test.txt

So only URLs that start with this, or that "match" whats in the wildcard
list, are internal (and spidered).


What we learn:
Don't use use "Check URL list" with the wildcard version.


Alternatively, I could simply add "*" at the end of every entry for the
include table when using that feature, and it might work.

Tilman


On Thu, 15 Feb 2007 12:47:57 -0000, maizegoblue wrote:

>Tilman,
>
>This time I put "http://dustyfeet.com" in a text file and got the same
>result: 9 URLs found.
>
>If I use "Check URL" for "http://dustyfeet.com" I get 178 URLs.
>
>--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@...> wrote:
>>
>> On Thu, 15 Feb 2007 03:27:06 -0000, maizegoblue wrote:
>>
>> >Tilman,
>> >
>> >I hope you don't delete the "Check URL list function"--I love it! It
>> >works fine in the release version. It's just not working right in the
>> >beta version.
>>
>> This hasn't changed for years...
>>
>> >
>> >If you scan the following site with the beta, then try doing it from a
>> >text file, you should see a big difference.
>> >http://dustyfeet.com/index.html
>> >
>> >If I use "Check URL," Xenu beta finds 178 URLs. If I put that URL in a
>> >file and use "Check URL list," Xenu only finds 9 URLs.
>>
>> Yes - as I said, it will spider only URls that start with
>> "http://dustyfeet.com/index.html".
>>
>> So an URL with "http://dustyfeet.com/toys.html" would not be spidered,
>> it is considered as "external".
>>
>> Solution: put
>>
>> "http://dustyfeet.com" in the txt file instead
>>
>> Tilman
>>
>> >
>> >Again, the release version works fine, and I really like this feature.
>> >I would go back to the release version, but I would like to use the
>> >wildcard functionality.
>> >
>> >
>> >
>> >
>> >--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@> wrote:
>> >>
>> >> The "Check URL list" has many logic problems, i.e. the behaviour
>is hard
>> >> to describe/understand, which is why I hate it. I should delete it.
>> >>
>> >> Check URL list is normally for domains only. It spiders all the URLs,
>> >> but accepts only URLs that start with the ones in the txt file.
>I.e. all
>> >> URls are added to the include list.
>> >>
>> >> The best would be to send me
>> >> 1) the html file
>> >> 2) the txt file
>> >> 3) a few URLs that you claim are not checked with that list feature
>> >>
>> >> Tilman
>> >>
>> >> On Wed, 14 Feb 2007 20:18:21 -0000, maizegoblue wrote:
>> >>
>> >> >Thanks a lot, Tilman.
>> >> >
>> >> >I have a problem with the new version of Xenu, though. It seems that
>> >> >when I check a number of sites (or even just one) using the
>"Check URL
>> >> >list" function, many links do not get checked.
>> >> >
>> >> >I uninstalled the old version and installed the version that
>supports
>> >> >wildcards. I did NOT try that feature yet.
>> >> >
>> >> >The problem seems to be the way Xenu handles "Check URL list"
>function
>> >> >in the new version. If I check a single site using the "Check URL"
>> >> >dialog, I get 75 links checked. If I put that site by itself in
>a text
>> >> >file and use "Check URL list," Xenu only finds 45 links. In the
>latter
>> >> >case, Xenu doesn't check some images that appear even on the
>home page
>> >> >of the site (though it does check some of them). Some external links
>> >> >also get skipped, yet some still get checked. Something is not quite
>> >> >right with that version. I can't determine a pattern, so I'm not
>sure
>> >> >what is happening.
>> >> >
>> >> >Thanks for any insight.
>> >> >
>> >> >--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@>
>wrote:
>> >> >>
>> >> >> That version is now here:
>> >> >> http://home.snafu.de/tilman/tmp/xenuwild1.zip
>> >> >>
>> >> >> The old version/ file name seemed to have a problem, it didn't
>> >have the
>> >> >> wildcard feature at all (apparently), so I uploaded it under a
>> >new name,
>> >> >> and downloaded it myself to check.
>> >> >>
>> >> >> Tilman
>> >> >>
>> >> >> On Wed, 14 Feb 2007 15:24:34 -0000, maizegoblue wrote:
>> >> >>
>> >> >> >Hi,
>> >> >> >
>> >> >> >First, let me say that Xenu is truly a great program. It's
>come in
>> >> >> >very handy for me.
>> >> >> >
>> >> >> >Anyway, I read some older posts talking about a version of
>Xenu that
>> >> >> >supports excluding filenames using wildcards. This would be a
>> >> >> >tremendously helpful feature for me.
>> >> >> >
>> >> >> >For instance, I might wish to exclude all "print friendly" pages
>> >on my
>> >> >> >site by excluding the following:
>> >> >> >*display=print*
>> >> >> >
>> >> >> >The link I found to that version was unfortunately dead. Can
>anyone
>> >> >help?
>> >> >> >
>> >> >> >Thanks a lot!
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >Yahoo! Groups Links
>> >> >
>> >> >
>> >> >
>> >>
>> >
>> >
>> >
>> >
>> >
>> >Yahoo! Groups Links
>> >
>> >
>> >
>>
>
>
>
>
>
>Yahoo! Groups Links
>
>
>

Tilman,

Yes, I would very much like to use both of those features at the same
time! I check a lot of sites, but there are some URLs that I need to
exclude using wildcards. Thanks a lot for your hard work!

--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@...> wrote:
>
> Do you need BOTH the wildcard feature, and the "Check URL list" feature?
> If yes, I'll upload the corrected version. This quasi-bug has now been
> fixed.
>
> Tilman
>

Get it here:

http://home.snafu.de/tilman/tmp/xenuwild2.zip


Tilman

On Thu, 15 Feb 2007 16:16:27 -0000, maizegoblue wrote:

>Tilman,
>
>Yes, I would very much like to use both of those features at the same
>time! I check a lot of sites, but there are some URLs that I need to
>exclude using wildcards. Thanks a lot for your hard work!
>
>--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@...> wrote:
>>
>> Do you need BOTH the wildcard feature, and the "Check URL list" feature?
>> If yes, I'll upload the corrected version. This quasi-bug has now been
>> fixed.
>>
>> Tilman
>>
>
>
>
>
>Yahoo! Groups Links
>
>
>

Please, please, please don't delete this feature. I use it a lot in
addition to Xenu's normal function of checking links on a website.

I have found it very helpful when I need to generate a list of links
from a group of specific web pages on a site, but I don't want to
spider all of the pages. I set the maximum level in options to 1 and
then create a list of files I'm interested in. When I run Xenu on the
list of files I get the list of links that I need.

Thanks for a great program. I have been using it for a number of years
now. It does exactly what it claims to do, and what I need. I
appreciate it not having a lot of "bells and whistles" that add
complexity and overhead with little, if any, benefit.

Tilman,

Thanks a lot!! That fixed it. It's beautiful now.

One last question: can I put my "exclusion list" of URLs to not check
in the .ini file?

Thanks again.

--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@...> wrote:
>
> Get it here:
>
> http://home.snafu.de/tilman/tmp/xenuwild2.zip
>
>
> Tilman
>
> On Thu, 15 Feb 2007 16:16:27 -0000, maizegoblue wrote:
>
> >Tilman,
> >
> >Yes, I would very much like to use both of those features at the same
> >time! I check a lot of sites, but there are some URLs that I need to
> >exclude using wildcards. Thanks a lot for your hard work!
> >
> >--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@> wrote:
> >>
> >> Do you need BOTH the wildcard feature, and the "Check URL list"
feature?
> >> If yes, I'll upload the corrected version. This quasi-bug has now
been
> >> fixed.
> >>
> >> Tilman
> >>
> >
> >
> >
> >
> >Yahoo! Groups Links
> >
> >
> >
>

On Thu, 15 Feb 2007 20:40:47 -0000, maizegoblue wrote:

>Tilman,
>
>Thanks a lot!! That fixed it. It's beautiful now.
>
>One last question: can I put my "exclusion list" of URLs to not check
>in the .ini file?

It is already done. (Not in the check URL feature) Look into the INI
file and you'll see it.

Tilman

>
>Thanks again.
>
>--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@...> wrote:
>>
>> Get it here:
>>
>> http://home.snafu.de/tilman/tmp/xenuwild2.zip
>>
>>
>> Tilman
>>
>> On Thu, 15 Feb 2007 16:16:27 -0000, maizegoblue wrote:
>>
>> >Tilman,
>> >
>> >Yes, I would very much like to use both of those features at the same
>> >time! I check a lot of sites, but there are some URLs that I need to
>> >exclude using wildcards. Thanks a lot for your hard work!
>> >
>> >--- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@> wrote:
>> >>
>> >> Do you need BOTH the wildcard feature, and the "Check URL list"
>feature?
>> >> If yes, I'll upload the corrected version. This quasi-bug has now
>been
>> >> fixed.
>> >>
>> >> Tilman
>> >>
>> >
>> >
>> >
>> >
>> >Yahoo! Groups Links
>> >
>> >
>> >
>>
>
>
>
>
>
>Yahoo! Groups Links
>
>
>

----- Original Message -----

From: Tilman Hausherr

To: xenu-usergroup@yahoogroups.com

Sent: Wednesday, February 14, 2007 5:41 PM

Subject: Re: [xenu-usergroup] False negatives and other knotty bits

On Wed, 14 Feb 2007 16:06:01 -0000, Vince wrote:

>Some questions I would like to raise about the results I get from
>Xenu.
>
>Firstly, more and more sites seem to be giving out a 404 not found
>error when the page actually is there, and looks fine when visited.

See FAQ - some websites "hate" the Xenu user agent

>Same story with the 'no such host' message, which sometimes turns
>out not to be true. Why does this happen? I take it that we're dealing

This could be a network problem. Try CTRL-R. Also, be sure to have the
latest version.

>with some behaviour of web hosts here, as much as the Xenu
>software.
>
>My second question is about links that appear at the end of the 'By
>Page' list in the Xenu report. Some of these are not links on my site
>at all. This seems to arise from memberships at Webring.com - Xenu
>seems to follow up links that are outside my site but may be in the
>web rings.

These are redirections. Solution: exclude the webring from the spidering
in the initial dialogbox.

Tilman

>
>a little enlightenment on these points would be very good. Is there
>any setting I can change to address either of these questions?
>
>Apart from these queries, I have to say Xenu is indispensable on my
>system, and must have saved me hundreds of hours of grief when
>checking links!
>
>Vince.
>
>
>
>
>
>Yahoo! Groups Links
>
>
>

I'm protected by SpamBrave