Try using:

DefaultIndex  no


in your config file..

Here is the relevant bit from the README file:

DefaultIndex  This option is used to enable/disable the use of "index." as
                a default index name to be stripped from the end of a URL.
                Most sites should not need to use this option, however some
                may find it useful, particularly those whose default index
                file name is something different, or those sites that use
                'index.php' or similar URLs to generate dynamic content.
                This option does not effect any of the names that may be
                defined using the IndexAlias option, and those names will
                still function as described.  Values may be 'yes' or 'no',
                with 'yes' being the default.

--

On Mon, 15 Aug 2011, tankman67 wrote:

> I'm running into an issue with Webalizer after upgrading to Gallery3.
> It seems Gallery3 places index.php in between the domain and the
> directory structure of the website.  For example,
> www.domain.com/index.php/2011/summervaction/
>
> As a result, Webalizer is stripping the "index." and everything after.
> This makes the stats read as if every single request is for /
>
> I see in the configuration file where I can add additional IndexAlias
> definitions but there doesn't appear to be away to exclude "index."
> Does anyone know if this is possible?
>

--
Bradford L. Barrett                      brad@...
A free electron in a sea of neutrons     DoD#1750 KD4NAW

The only thing Micro$oft has done for society, is make people
believe that computers are inherently unreliable.

Thanks for the reply Bradford.  I'll provide a little more detail.  I installed
Webalizer with apt-get on my Debian system.  This installed version 2.01.  I
checked the README located in /usr/share/doc/webalizer and did some Googling -
which I try to do to prevent asking stupid questions - but didn't see this
option.

I took your advice however, but received the following output:
Warning: Invalid keyword 'DefaultIndex' (/etc/webalizer/webalizer.conf)

I see the README file at ftp://ftp.mrunix.net/pub/webalizer/README (which
appears to be for version 2.23-05) DOES contain this option.

I will try upgrading my Webalizer version and test.  Thanks again for pointing
me in the right direction here.

Dallas

--- In webalizer@yahoogroups.com, "Bradford L. Barrett" <brad@...> wrote:
>
>
> Try using:
>
> DefaultIndex  no
>
>
> in your config file..
>
> Here is the relevant bit from the README file:
>
> DefaultIndex  This option is used to enable/disable the use of "index." as
>                a default index name to be stripped from the end of a URL.
>                Most sites should not need to use this option, however some
>                may find it useful, particularly those whose default index
>                file name is something different, or those sites that use
>                'index.php' or similar URLs to generate dynamic content.
>                This option does not effect any of the names that may be
>                defined using the IndexAlias option, and those names will
>                still function as described.  Values may be 'yes' or 'no',
>                with 'yes' being the default.
>
> --
>
> On Mon, 15 Aug 2011, tankman67 wrote:
>
> > I'm running into an issue with Webalizer after upgrading to Gallery3.
> > It seems Gallery3 places index.php in between the domain and the
> > directory structure of the website.  For example,
> > www.domain.com/index.php/2011/summervaction/
> >
> > As a result, Webalizer is stripping the "index." and everything after.
> > This makes the stats read as if every single request is for /
> >
> > I see in the configuration file where I can add additional IndexAlias
> > definitions but there doesn't appear to be away to exclude "index."
> > Does anyone know if this is possible?
> >
>
> --
> Bradford L. Barrett                      brad@...
> A free electron in a sea of neutrons     DoD#1750 KD4NAW
>
> The only thing Micro$oft has done for society, is make people
> believe that computers are inherently unreliable.
>

All Website access statistics, whether produced by Webalizer or some other
package, suffer from at least one important limitation - they don't distinguish
between access requests from humans and those from Web crawlers (good or bad). 
That's not a problem if you're just looking at capacity management issues, but
it's a huge problem if you want to learn about which parts of your Website are
valuable to your human visitors.

I think I've figured out a way to overcome that limitation (mostly), so I'm
writing to y'all to lay out my ideas, report progress, and ask for comments. 
And the process of writing this will help to clarify what I'm thinking.

The raw data contains no explicit indication of what caused each access attempt,
but it does contain clues from which useful information can be deduced.  After
looking at raw data several different ways, I've come up with what I hope is a
useful approach, based on agent-id.

The basic approach is to use those clues to make a best guess as to which agents
are used by humans and which are bots.  Then use that distinction to split each
access log into two or three parts (human, bot, and maybe indeterminate). 
Finally, use Webalizer to re-analyze whichever part(s) I/we find interesting.

So what are the clues?

1.  Anything that requests "favicon.ico" is most probably a browser used by a
human, while anything that requests "robots.txt" is almost certainly a bot.  A
tiny number of agents ask for both, and a large number ask for neither, so those
are indeterminate at this point.  I've written an analysis program, which
provided the following figures from our first full month of operation:
- nearly 93000 access requests
- from just over 1700 agents
- of which 540 occurred only once
- leaving 1170 reasonably analyzable
- of which over 700 requested favicon.ico
- - - and over 100 requested robots.txt
- - - (though 10 requested both)
- - - leaving over 300 requesting neither

2.  An agent which is only seen once in a month is most probably a bot,
especially if its ID looks like a random string of characters.  (See above for
counts.)

3.  Some access requests identify a referrer [typically misspelled "referer" in
server documentation], but most do not.  It seems likely that those which do
probably come from humans, but this clue requires further investigation.  At the
moment all I can say is that our first month's data shows that about 30% of
requests come from referrals, and about 2/3 of those are from our own Website.

4.  Successful requests for items contained in a password-protected directory
must come from humans.

5.  Requests for items which were never on our Website (e.g., PHP files - YMMV)
must indicate hacker intrusion attempts, which probably would be carried out by
robots.

Further programming is required to investigate these clues more deeply and to
make use of the results.  Meanwhile ...

Questions?  Comments?  Suggestions?

(and does anyone have a Webalizer binary for Mac OS X - any version?)

Carl Scott Zimmerman

Hi,
I joined to see if anyone can help me with a strange problem.
I'm sysadmin at a hosting and web marketing company. Several of our customers
have had their sites moved from one server to a new one to upgrade. The
webalizer stats show some marked differences in data reported between the two,
most notably the Pages and Visits results, which have been reduced by up to 75%.
The config and data have been checked and are identical between the two servers,
so what can it be doing differently?
Can anyone help please?

Thanks very much.
Sam Gyseman.

All:

I have a CentOS 5.2 box running my website, using webalizer to track stats. I am
provisioning a new CentOS 6 box to house the site. Migration of the website,
database, etc. is pretty straight forward. I am a little nervous about moving
webalizer logs over. Has anyone done this before? Any tips you can offer to
insure a smooth transition (and preservation of log data)?

The CentOS 5 box is running webalizer v. 2.01-10
The CentOS 6 box is running webalizer v. 2.21-02

Thanks in advance.
Rich Lohman

I am running Fastream IQ Web Server on Win 7, which claims to output its
log in W3C(extended).
But it uses a 12-hour clock for its "time" field, so the format is one of:
h:mm:ss AM
hh:mm:ss AM
h:mm:ss PM
hh:mm:ss PM

The space in there throws the whole thing out, resulting in all bad records.
Is there any way to work around this ?

Apart from this niggle, IQ is the best server as its separate
administration GUI makes alterations a breeze.

Dave

So I wrote a PHP script to copy the Log file to another file, modifying the
'time' field in each record, and that works OK,
but I still get all records flagged as "skipping bad record", and "no valid
records found!".

From the documentation at Awstats it says the fields in W3C format are:

date
time
c-ip
cs-username
cs-method
cs-uri-stem
cs-uri-query
sc-status
sc-bytes
cs-version
cs(User-Agent)
cs(Referer)

and I have those checked, but there other fields I could have checked for
inclusion,
so the question now becomes:
what fields does Webalizer think are in W3C log format ?
I can't find this specified anywhere in the Webalizer documentation.

>> From the documentation at Awstats it says the fields in W3C format are:
[...]
> and I have those checked, but there other fields I could have checked
> for inclusion, so the question now becomes:
> what fields does Webalizer think are in W3C log format ?
> I can't find this specified anywhere in the Webalizer documentation.

Check: http://www.w3.org/TR/WD-logfile.html

For a complete description of the W3C log format.  Also, note that
the time field is defined as:

<time>  = 2<digit> ":" 2<digit> [":" 2<digit> ["." *<digit>]

The time must have at least the 2 digit hour and minute, in 24
hour format, and specified in GMT.

As for what fields the Webalizer 'thinks' are in the log, that would
be defined in the log file itself; (see the "Format" section of
the document referenced above).

Cheers,
Brad

 

>> From the documentation at Awstats it says the fields in W3C format are:
[...]
> and I have those checked, but there other fields I could have checked
> for inclusion, so the question now becomes:
> what fields does Webalizer think are in W3C log format ?
> I can't find this specified anywhere in the Webalizer documentation.

Check: http://www.w3.org/TR/WD-logfile.html

For a complete description of the W3C log format. Also, note that
the time field is defined as:

<time> = 2<digit> ":" 2<digit> [":" 2<digit> ["." *<digit>]

The time must have at least the 2 digit hour and minute, in 24
hour format, and specified in GMT.

As for what fields the Webalizer 'thinks' are in the log, that would
be defined in the log file itself; (see the "Format" section of
the document referenced above).

Cheers,
Brad

Hi there,

I'm using webalizer V2.21-02 (Linux 2.6.32-131.17.1.el6.x86_64 x86_64) English
on a redhat-server, and trying to configure out why webalizer
is giving me this error message:

'cannot access log file /var/log/nginx/access.log'

I checked the permissions on the file, and they are all fine. Webalizer was
installed using a yum-repository.

here a part of the configuration file (/etc/webalizer.conf):
[code]
LogFile /var/log/nginx/access.log
OutputDir /var/www/usage
[/code]
the permission on /var/www/usage are also fine ;-)
and this is wat 'strace webalizer' gives:
[code]
execve("/usr/bin/webalizer", ["webalizer"], [/* 28 vars */]) = 0
brk(0)                                  = 0x238e000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f113c4de000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=44927, ...}) = 0
mmap(NULL, 44927, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f113c4d3000
close(3)                                = 0
open("/lib64/libbz2.so.1", O_RDONLY)    = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0000\26\0\r7\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=69976, ...}) = 0
mmap(0x370d000000, 2162768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x370d000000
mprotect(0x370d010000, 2093056, PROT_NONE) = 0
mmap(0x370d20f000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xf000) = 0x370d20f000
close(3)                                = 0
open("/lib64/libdb-4.7.so", O_RDONLY)   = 3
read(3,
"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240\206\302\n7\0\0\0"..., 832) =
832
fstat(3, {st_mode=S_IFREG|0755, st_size=1526984, ...}) = 0
mmap(0x370ac00000, 3620704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x370ac00000
mprotect(0x370ad6f000, 2093056, PROT_NONE) = 0
mmap(0x370af6e000, 24576, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16e000) = 0x370af6e000
close(3)                                = 0
open("/usr/lib64/libgd.so.2", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0000\206\240P=\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=275328, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f113c4d2000
mmap(0x3d50a00000, 2384392, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x3d50a00000
mprotect(0x3d50a22000, 2097152, PROT_NONE) = 0
mmap(0x3d50c22000, 135168, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x22000) = 0x3d50c22000
mmap(0x3d50c43000, 12808, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x3d50c43000
close(3)                                = 0
open("/usr/lib64/libpng12.so.0", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0000H`\376;\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=159728, ...}) = 0
mmap(0x3bfe600000, 2252320, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x3bfe600000
mprotect(0x3bfe626000, 2093056, PROT_NONE) = 0
mmap(0x3bfe825000, 4096, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x25000) = 0x3bfe825000
close(3)                                = 0
open("/lib64/libz.so.1", O_RDONLY)      = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\36@\0077\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=88240, ...}) = 0
mmap(0x3707400000, 2181040, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x3707400000
mprotect(0x3707415000, 2093056, PROT_NONE) = 0
mmap(0x3707614000, 4096, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14000) = 0x3707614000
close(3)                                = 0
open("/lib64/libm.so.6", O_RDONLY)      = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240>\300\0067\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=598816, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f113c4d1000
mmap(0x3706c00000, 2633944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x3706c00000
mprotect(0x3706c83000, 2093056, PROT_NONE) = 0
mmap(0x3706e82000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x82000) = 0x3706e82000
close(3)                                = 0
open("/lib64/libc.so.6", O_RDONLY)      = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\r\300\t7\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=13168, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f113c4cd000
mmap(0x3709c00000, 2106112, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x3709c00000
mprotect(0x3709c02000, 2097152, PROT_NONE) = 0
mmap(0x3709e02000, 4096, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x3709e02000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f113c4cc000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f113c4cb000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f113c4ca000
arch_prctl(ARCH_SET_FS, 0x7f113c4cb700) = 0
mprotect(0x3706e82000, 4096, PROT_READ) = 0
mprotect(0x3706385000, 16384, PROT_READ) = 0
mprotect(0x3706616000, 4096, PROT_READ) = 0
mprotect(0x3705a1f000, 4096, PROT_READ) = 0
mprotect(0x3705e02000, 4096, PROT_READ) = 0
munmap(0x7f113c4d3000, 44927)           = 0
set_tid_address(0x7f113c4cb9d0)         = 4866
set_robust_list(0x7f113c4cb9e0, 0x18)   = 0
futex(0x7fffb7f5a03c, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7fffb7f5a03c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, NULL,
7f113c4cb700) = -1 EAGAIN (Resource temporarily unavailable)
rt_sigaction(SIGRTMIN, {0x3706405a90, [], SA_RESTORER|SA_SIGINFO, 0x370640f490},
NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x3706405b20, [], SA_RESTORER|SA_RESTART|SA_SIGINFO,
0x370640f490}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=10240*1024, rlim_max=RLIM_INFINITY}) = 0
brk(0)                                  = 0x238e000
brk(0x23af000)                          = 0x23af000
open("/usr/lib/locale/locale-archive", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=99158752, ...}) = 0
mmap(NULL, 99158752, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f1136639000
close(3)                                = 0
stat("webalizer.conf", {st_mode=S_IFREG|0644, st_size=23946, ...}) = 0
getuid()                                = 505
access("/etc/webalizer.conf", F_OK)     = 0
open("/etc/webalizer.conf", O_RDONLY)   = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=23946, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f113c4dd000
read(3, "#\n# Sample Webalizer configurati"..., 4096) = 4096
read(3, " for the report.  This is used i"..., 4096) = 4096
read(3, "L code to insert at the bottom o"..., 4096) = 4096
read(3, " hideous and ugly, say 'no' here"..., 4096) = 4096
read(3, "l should be seperated from\n# the"..., 4096) = 4096
read(3, "# precidence over the Ignore* ke"..., 4096) = 3466
read(3, "", 4096)                       = 0
close(3)                                = 0
munmap(0x7f113c4dd000, 4096)            = 0
lstat("/var/log/nginx/access.log", {st_mode=S_IFREG|0644, st_size=80760251,
...}) = 0
open("/var/log/nginx/access.log", O_RDONLY) = -1 EACCES (Permission denied)
write(2, "Error: Can't open log file /var/"..., 53) = 53
exit_group(1)
[/code]

thanks in advance

On 1/16/2012 02:36, nicolasfranck.jl12 wrote:
> Hi there,
>
> I'm using webalizer V2.21-02 (Linux 2.6.32-131.17.1.el6.x86_64 x86_64) English
on a redhat-server, and trying to configure out why webalizer
> is giving me this error message:
>
> 'cannot access log file /var/log/nginx/access.log'
>
> I checked the permissions on the file, and they are all fine. Webalizer was
installed using a yum-repository.
>
> here a part of the configuration file (/etc/webalizer.conf):
> [code]
> LogFile /var/log/nginx/access.log
> OutputDir /var/www/usage
> [/code]
> the permission on /var/www/usage are also fine ;-)
> and this is wat 'strace webalizer' gives:

at a guess, i'd say that you're trying to read a live log file... one that the
server has open and locked for all access... it would be nice if it opened the
log file for write and didn't deny read... at least this is what i recall from
years past on my system when i fought a similar problem... i could only process
historical (ie: non-live) logs at that time... i also wanted an easy logrotate
capability... my OS doesn't/didn't have all the capabilites of other OS' so i
use a logfile tool for my server to write my logs with... the tool closes the
file after each write...

if you don't have such a tool and you confirm this is the problem, then you'll
only be able to process historical log (or maybe a copy if you can do that)...

Thanks for your reply

I already tried that. I copied the logfile to my home folder,
renamed it, and changed the configuration file to point
to the custom logfile in my folder. But still the
same error:
'cannot access log file /home/njfrank/test.log'

on my ubuntu webalizer of the same version works fine.


--- In webalizer@yahoogroups.com, waldo kitty <wkitty42@...> wrote:
>
> On 1/16/2012 02:36, nicolasfranck.jl12 wrote:
> > Hi there,
> >
> > I'm using webalizer V2.21-02 (Linux 2.6.32-131.17.1.el6.x86_64 x86_64)
English on a redhat-server, and trying to configure out why webalizer
> > is giving me this error message:
> >
> > 'cannot access log file /var/log/nginx/access.log'
> >
> > I checked the permissions on the file, and they are all fine. Webalizer was
installed using a yum-repository.
> >
> > here a part of the configuration file (/etc/webalizer.conf):
> > [code]
> > LogFile /var/log/nginx/access.log
> > OutputDir /var/www/usage
> > [/code]
> > the permission on /var/www/usage are also fine ;-)
> > and this is wat 'strace webalizer' gives:
>
> at a guess, i'd say that you're trying to read a live log file... one that the
> server has open and locked for all access... it would be nice if it opened the
> log file for write and didn't deny read... at least this is what i recall from
> years past on my system when i fought a similar problem... i could only
process
> historical (ie: non-live) logs at that time... i also wanted an easy logrotate
> capability... my OS doesn't/didn't have all the capabilites of other OS' so i
> use a logfile tool for my server to write my logs with... the tool closes the
> file after each write...
>
> if you don't have such a tool and you confirm this is the problem, then you'll
> only be able to process historical log (or maybe a copy if you can do that)...
>

On 1/18/2012 02:35, nicolasfranck.jl12 wrote:
> Thanks for your reply
>
> I already tried that. I copied the logfile to my home folder,
> renamed it, and changed the configuration file to point
> to the custom logfile in my folder. But still the
> same error:
> 'cannot access log file /home/njfrank/test.log'
>
> on my ubuntu webalizer of the same version works fine.

the only other thing i can think of, at this time, is possibly permissions...
what are the permissions on the file? what user and group is webalizer running
as? is the webalizer user a member of a group that has access to that file?

> --- In webalizer@yahoogroups.com, waldo kitty<wkitty42@...>  wrote:
>>
>> On 1/16/2012 02:36, nicolasfranck.jl12 wrote:
>>> Hi there,
>>>
>>> I'm using webalizer V2.21-02 (Linux 2.6.32-131.17.1.el6.x86_64 x86_64)
English on a redhat-server, and trying to configure out why webalizer
>>> is giving me this error message:
>>>
>>> 'cannot access log file /var/log/nginx/access.log'
>>>
>>> I checked the permissions on the file, and they are all fine. Webalizer was
installed using a yum-repository.
>>>
>>> here a part of the configuration file (/etc/webalizer.conf):
>>> [code]
>>> LogFile /var/log/nginx/access.log
>>> OutputDir /var/www/usage
>>> [/code]
>>> the permission on /var/www/usage are also fine ;-)
>>> and this is wat 'strace webalizer' gives:
>>
>> at a guess, i'd say that you're trying to read a live log file... one that
the
>> server has open and locked for all access... it would be nice if it opened
the
>> log file for write and didn't deny read... at least this is what i recall
from
>> years past on my system when i fought a similar problem... i could only
process
>> historical (ie: non-live) logs at that time... i also wanted an easy
logrotate
>> capability... my OS doesn't/didn't have all the capabilites of other OS' so i
>> use a logfile tool for my server to write my logs with... the tool closes the
>> file after each write...
>>
>> if you don't have such a tool and you confirm this is the problem, then
you'll
>> only be able to process historical log (or maybe a copy if you can do
that)...

All:

I have one remote site that generated an inordinate amount of traffic to my site
over two days this month. It has completely skewed the stats, such that I'm not
comfortable presenting the numbers as they stand.

I went through and removed all of the entries from my access_log, but the stats
still show up. I rather thought they regenerated each morning when I ran
webalizer out of cron, but alas...

So, now that I have the site's traffic removed from the access_log, how can I
get the stats out of webalizer? Can I manually rebuild the page, or does this
require manually editing all of the webalizer history files in
/var/lib/webalizer?

Thanks in advance,
Rich

I hadn't run webalizer for about a week and I stupidly starting processing the
old logs from newest to oldest until I noticed that after the first log they
were each being processed as if they contained 100% duplicate entries.  Does
webalizer refuse to process logs older than its newest entry?  If so, is there
any way I can process the old logs?

On 2/2/2012 19:54, emailgrant123 wrote:
> I hadn't run webalizer for about a week and I stupidly starting processing the
old logs from newest to oldest until I noticed that after the first log they
were each being processed as if they contained 100% duplicate entries.  Does
webalizer refuse to process logs older than its newest entry?  If so, is there
any way I can process the old logs?

yes... process them oldest to newest ;)

seriously, though...

> > I hadn't run webalizer for about a week and I stupidly starting processing
the old logs from newest to oldest until I noticed that after the first log they
were each being processed as if they contained 100% duplicate entries.  Does
webalizer refuse to process logs older than its newest entry?  If so, is there
any way I can process the old logs?
>
> yes... process them oldest to newest ;)

How can I do that now that I've messed up and processed a newer one?  Can I
delete its records somehow so I can start over going from oldest to newest?

Remove the incremental data file (webalizer.current unless renamed to
something else).  Be careful, as this will remove any data processed
for the current month.  DO NOT remove webalizer.hist, as that contains
totals for _previous_ months.  After that, you can (and will need to)
re-process your logs fron the beginning of the current month.

And yes, when using incremental mode, you MUST process logs in
chronological order.

Cheers,
Brad

--

>> yes... process them oldest to newest ;)
>
> How can I do that now that I've messed up and processed a newer one?
> Can I delete its records somehow so I can start over going from oldest
> to newest?

> Remove the incremental data file (webalizer.current unless renamed to
> something else).  Be careful, as this will remove any data processed
> for the current month.  DO NOT remove webalizer.hist, as that contains
> totals for _previous_ months.  After that, you can (and will need to)
> re-process your logs fron the beginning of the current month.
>
> And yes, when using incremental mode, you MUST process logs in
> chronological order.

That sounds like it would work for February, but I'm missing January 24-30.  The
system has processed January 31 - February 5 in the meantime.  Can I remove
everything after January 30?

I'm curious what other modes there are besides incremental mode?

- Grant

> > Remove the incremental data file (webalizer.current unless renamed to
> > something else).  Be careful, as this will remove any data processed
> > for the current month.  DO NOT remove webalizer.hist, as that contains
> > totals for _previous_ months.  After that, you can (and will need to)
> > re-process your logs fron the beginning of the current month.
> >
> > And yes, when using incremental mode, you MUST process logs in
> > chronological order.
>
> That sounds like it would work for February, but I'm missing January 24-30. 
The system has processed January 31 - February 5 in the meantime.  Can I remove
everything after January 30?

Can anyone help me out here?  Is there anything I can do at this point to fill
in the missing data?  I have the logs.

- Grant

> That sounds like it would work for February, but I'm missing January
> 24-30.  The system has processed January 31 - February 5 in the
> meantime.  Can I remove everything after January 30?
>
> Can anyone help me out here?  Is there anything I can do at this point
> to fill in the missing data?  I have the logs.

Just start processing your logs from January 1st instead of February 1st
after removing the incremental data file.

Cheers,
Brad

> > That sounds like it would work for February, but I'm missing January
> > 24-30.  The system has processed January 31 - February 5 in the
> > meantime.  Can I remove everything after January 30?
> >
> > Can anyone help me out here?  Is there anything I can do at this point
> > to fill in the missing data?  I have the logs.
>
> Just start processing your logs from January 1st instead of February 1st
> after removing the incremental data file.

I'm sorry to say I only have the logs going back to January 5th.  Is there any
way to process the logs I have for January 24-30 without removing all data from
January?

> I'm sorry to say I only have the logs going back to January 5th.  Is
> there any way to process the logs I have for January 24-30 without
> removing all data from January?

Then I guess you are pretty much SOL. You could _possibly_ get it to
work by hand editing the incremental data file, but that is not for
the faint of heart!  That file contains the timestamp of what the
program 'thinks' was the last log entry processed.  You would need
to change that timestamp to something prior to January 24th, process
your January 24-30th logs, then change the timestamp back to what it
was previously.  This assumes that you have already processed January
31st.  Whatever you do, make sure you backup your current stats just
in case (more likely than not) you hose them in the process :)

Cheers,
Brad

> > I'm sorry to say I only have the logs going back to January 5th.  Is
> > there any way to process the logs I have for January 24-30 without
> > removing all data from January?
>
> Then I guess you are pretty much SOL. You could _possibly_ get it to
> work by hand editing the incremental data file, but that is not for
> the faint of heart!  That file contains the timestamp of what the
> program 'thinks' was the last log entry processed.  You would need
> to change that timestamp to something prior to January 24th, process
> your January 24-30th logs, then change the timestamp back to what it
> was previously.  This assumes that you have already processed January
> 31st.  Whatever you do, make sure you backup your current stats just
> in case (more likely than not) you hose them in the process :)
>
> Cheers,
> Brad

I appreciate your help but that doesn't seem to work.  The stats end up
incorrect and a processing gap appears in a different spot.

Is the top portion of the incremental data file before "# -urls-" documented by
any chance?  I think if I could figure it out I could get this to work.

Or maybe a way to remove records matching a log file instead of adding them? 
Yeah right....

- Grant

On 2/21/2012 06:14, Lewatle Phaladi wrote:
> I am delighted to be member of this group, my questions are as follows: is
that
> possible to run webalizer on separate machine and generate stats from other
> machine, I am asking this question because I have difficulty in setting up
> webalizer on Solaris and I have set it up on Ubuntu and webalizer has
generated
> good analysis report for us, please help me on this issue.

sure... you only need a way to copy the log file over for the analysis... the
resulting output will reside on the local machine where webalizer is installed
unless you have some sort of mapped drives or directories over a LAN...

Greetings everyone.

I am sad to say that the main Webalizer site is currently down.

Unfortunately, the hosting company where the Webalizer server was co-located has
decided to close down their operations effective immediately.  They were great
people to deal with, and had been generously donating rack space to us for the
last 8 years.  I am sorry to see them close up shop, and I wish them luck in any
future ventures they may undertake.

However, this leaves the Webalizer main distribution server in a bit of a
situation and in need of a new home.  North Central Florida is where the server
is now, so it would be ideal if I could find a way to keep it here.  If you or
someone you know might be able to help in any way, please send an e-mail to
thewebalizer over at gmail (you know the extension).  I regret this
inconvenience, but it was completely unexpected and unforeseen.  In the mean
time, please use one of the mirror sites if necessary.

Thank You,
Brad

What kind of storage and bandwidth do you need to keep this going?

Peter K Yanke
PY Multimedia Services
support@...
PH: 575.418.1037
FX: 888.808.0099

-----Original Message-----
From: webalizer@yahoogroups.com [mailto:webalizer@yahoogroups.com] On Behalf
Of thewebalizer
Sent: Thursday, March 15, 2012 12:09 AM
To: webalizer@yahoogroups.com
Subject: [webalizer] Webalizer main site temporarly down

Greetings everyone.

I am sad to say that the main Webalizer site is currently down.

Unfortunately, the hosting company where the Webalizer server was co-located
has decided to close down their operations effective immediately.  They were
great people to deal with, and had been generously donating rack space to us
for the last 8 years.  I am sorry to see them close up shop, and I wish them
luck in any future ventures they may undertake.

However, this leaves the Webalizer main distribution server in a bit of a
situation and in need of a new home.  North Central Florida is where the
server is now, so it would be ideal if I could find a way to keep it here.
If you or someone you know might be able to help in any way, please send an
e-mail to thewebalizer over at gmail (you know the extension).  I regret
this inconvenience, but it was completely unexpected and unforeseen.  In the
mean time, please use one of the mirror sites if necessary.

Thank You,
Brad



------------------------------------

Webalizer homepage: http://www.webalizer.org Yahoo! Groups Links

Thanks for the query Peter.

   The requirements are really quite moderate.  The web stuff is all static html,
and I would estimate that less than 200Mb a day of bandwidth is used between the
web, ftp, smtp and dns services combined.  Currently, it is housed in a
mini-tower enclosure, but could easily be swapped to a rack mount pizza box
enclosure to save space.  I would be happy to discuss the specifics in private.

Please contact 'thewebalizer' at gmail dot com for further information.

Thank You,
Brad

--- In webalizer@yahoogroups.com, "Peter Yanke" <pyanke@...> wrote:
>
> What kind of storage and bandwidth do you need to keep this going?