wwwstat's output consists of a set of cross-reference links, the sum totals and averages for the processed data, and a sequence of amount-by-category tables partitioned into sections. The section categories are based on the characteristics evident from the access request, as provided by the common logfile format (see NOTES). These include:

Request Date

e.g., "Feb 2 1996"

Request Hour

e.g., "00" through "23"

Client Domain

The Fully-Qualified Domain Name (FQDN) suffix that corresponds to an organization type or country name.

Reversed Subdomain

The FQDN, usually minus the first (machine name) component, and reversed so that it is easier to read when sorted.

URL/Archive

Grouping based on Request-URI or non-success status code.

Identity

The user identity based on IdentityCheck token or Authorization field.

Each section can be enabled/disabled using the configuration files or command-line options (see Section Display Options).

Inside each section, the statistics are presented as a preformatted table.

%Reqs %Byte Bytes Sent Requests00 category-type

----- ----- ------------ -------- |---------------

NN.NN NN.NN NNNNNNNNNNNN NNNNNNNN | category-value

100.0 100.0 NNNNNNNNNNNN NNNNNNNN | category-value

Requests

Requests received for this category-value.

Bytes Sent

Bytes transmitted for this category-value.

%Reqs

(<Requests>/<Total Requests>)*100.

%Byte

(<Bytes Sent>/<Total Bytes>)*100.

The table can be sorted by category-value (-sort key), number of requests received (-sort req), or number of bytes received (-sort byte). It can also be limited to the -top N entries.

These options define how wwwstat should establish defaults and interpret the command-line.

-F filename

Get system configuration defaults from the given file. If used, this must be the first argument on the command-line, since it needs to be interpreted before the other command options. The file wwwstat.rc is included with the distribution as an example of this file; it contains perl source code which directly sets the control and display options provided by wwwstat. If filename is not a pathname, the include path (see FILES) is searched for filename. An empty string as filename will disable this feature. [-F "wwwstat.rc"]

-f filename

Get user configuration defaults from the given file. If used, this must be the first argument on the command-line after -F (if any). The file is the same format as for the -F option (see wwwstat.rc). If filename is not a pathname, the include path (see FILES) is searched for filename. An empty string as filename will disable this feature. [-f ".wwwstatrc"]

--

Last option (the remaining arguments are treated as input files).

These options provide information about wwwstat usage or about some unusual aspects of the logfile(s) being processed.

-h

Help - display usage information to STDERR and then exit.

-v

Verbose display to STDERR of each log entry processed.

-x

Display to STDERR all requests resulting in HTTP error responses.

-e

Display to STDERR all invalid log entries. Invalid log entries can occur if the server is miswriting or overwriting its own log, if the request is made by a broken client or proxy, or if a malicious attacker is trying to gain privileged access to your system. For the latter reason, the webmaster should run wwwstat with this option on a regular basis.

These options modify the output format.

-H string

Use the given string as the HTML title and heading for output.

-X string

Use the given string as the cross-reference URL to the last summary output. Any occurrence of the characters "%M" or "%Y" are replaced by the month and year, respectively, of the month prior to the first log entry date. The empty string will exclude any cross-reference.

-R

Display the daily stats table sorted in reverse. This option is primarily for use with the gwstat program for producing graphs of the output.

-l

-L

Do (-l) or don't (-L) display the full DNS hostname of clients in your local domain (which is determined by the configured value of $AppendToLocalhost) in the section on subdomain statistics. The default [-L] is to strip the machine name from local addresses.

-o

-O

Do (-o) or don't (-O) display the full DNS hostname of clients outside your local domain in the section on subdomain statistics. The default [-O] is to strip the machine name from outside addresses.

-u

-U

Do (-u) or don't (-U) display the IP address of clients with unresolved domain names in the section on subdomain statistics. The -dns option can be used to resolve some names, but not all IP hosts have a DNS name (SLIP/PPP connections) and sometimes a host's DNS service is inaccessible. The default [-U] is to group all such addresses under the category "Unresolved".

-dns

-nodns

Do (-dns) or don't (-nodns) use the system's hostname lookup facilities to find the DNS hostname associated with any unresolved IP addresses. Looking up a DNS name may be very slow, particularly when the results are negative (no DNS name), which is why a caching capability is included as well. [-nodns]

-cache filename

Use the given DBM database as the read/write persistent DNS cache (the .dir and .pag extensions are appended automatically). Cached entries (including negative results) are removed after the time configured for $DNSexpires [two months]. No caching is performed if filename is the empty string, which may be needed if your system does not support DBM or NDBM functionality. Running -dns without a persistent cache is not recommended. [-cache "dnscache"]

-trunc N

Truncate the URLs listed in the archive section after the Nth hierarchy level. This option is commonly used to reduce the output size and memory requirements of wwwstat by grouping the requests by directory tree instead of listing every URL. The default [-trunc 0] is to display every requested URL.

-files

-nofiles

Do (-files) or don't (-nofiles) include the last component of a URL (usually the filename) in the archive section. This option is commonly used to reduce the output size and memory requirements of wwwstat by grouping the requests by directory instead of listing every URL. The default [-files] is to display the entire requested URL.

-link

-nolink

Do (-link) or don't (-nolink) add a hypertext link around each archive URL. This option is useful for local maintenance, but it is not recommended for publication of the HTML results (it often results in links to temporary or nonexistant resources, and leads people/robots to resources that might not be publically available). [-nolink]

-cgi

-nocgi

Do (-cgi) or don't (-nocgi) prefix the summary output with CGI header fields appropriate for use with the HTTP common gateway interface. Using wwwstat as a CGI script is not recommended - it is usually better to simply run the wwwstat program periodically and serve the static output file. [-nocgi]

These options change the display of entire sections (as opposed to the entries within those sections). They allow the user to enable or disable an entire section, set the sorting method for that section, and limit the number of displayed entries for that section. These options are context-sensitive and processed in the order given.

-all

-noall

Include (-all) or exclude (-noall) all of the display sections. The -noall option is commonly used just prior to one or more of the other section options, such that only the listed sections are displayed.

-daily

-nodaily

Include (-daily) or exclude (-nodaily) the section of statistics by request date and set the scope for later -sort and -top options to this section.

-hourly

-nohourly

Include (-hourly) or exclude (-nohourly) the section of statistics by request hour and set the scope for later -sort and -top options to this section.

-domain

-nodomain

Include (-domain) or exclude (-nodomain) the section of statistics by the client's Internet domain and set the scope for later -sort and -top options to this section.

-subdomain

-nosubdomain

Include (-subdomain) or exclude (-nosubdomain) the section of statistics by the client's Internet subdomain (reversed for display) and set the scope for later -sort and -top options to this section.

-archive

-noarchive

Include (-archive) or exclude (-noarchive) the section of statistics by requested URL/archive and set the scope for later -sort and -top options to this section.

-r

-ident

-noident

Include (-r or -ident) or exclude (-noident) the section of statistics by the identity of the user (if IdentityCheck is ON) or the authentication userid (if supplied) and set the scope for later -sort and -top options to this section. DO NOT PUBLISH this information, as that would reveal security-related identities and be a violation of privacy. This option is provided for administrative purposes only.

-sort (key|byte|req)

Sort this section by its primary key, the number of bytes transmitted, or the number of requests received. [-sort key]

-top N

Display only the top N entries for this section. This option assumes that the -sort option has been set to either bytes or requests.

-both

Display both the top N entries for this section [10, sorted by requests], and then the full section (all entries) sorted by key.

These options are used to limit the analysis to requests matching a pattern. The pattern is supplied in the form of a perl regular expression, except that the characters "+" and "." are escaped automatically unless the -noescape option is given. Enclose the pattern in single-quotes to prevent the command shell from interpreting some special characters.

Multiple occurrences of the same option results in an OR-ing of the regular expressions. Search options are only applied to logfile entries; any summary files input must have been created with the same search options.

-a regexp

-A regexp

Include (-a) or exclude (-A) all requests containing a hostname/IP address matching the given perl regular expression.

-c regexp

-C regexp

Include (-c) or exclude (-C) all requests resulting in an HTTP status code matching the given perl regular expression.

-d regexp

-D regexp

Include (-d) or exclude (-D) all requests occurring on a date (e.g., "Feb 2 1994") matching the given perl regular expression.

-t regexp

-T regexp

Include (-t) or exclude (-T) all requests occurring during the hour (e.g., "23" is 11pm - 12pm) matching the given perl regular expression.

-m regexp

-M regexp

Include (-m) or exclude (-M) all requests using an HTTP method (e.g., "HEAD") matching the given perl regular expression.

-n regexp

-N regexp

Include (-n) or exclude (-N) all requests on a URL (archive name) matching the given perl regular expression.

-noescape

Do not escape the special characters ("+" and ".") in the remaining search options.

The Search Options and many of the configuration file settings allow for full use of perl regular expressions (with the exception that the -a, -A, -n and -N options treat '+' and '.' characters as normal alphabetic characters unless they are preceded by the -noescape option). Most people only need to know the following special characters:

^

at start of pattern, means "starts with pattern".

$

at end of pattern, means "ends with pattern".

(...)

groups pattern elements as a single element.

?

matches preceding element zero or one times.

*

matches preceding element zero or more times.

+

matches preceding element one or more times.

.

matches any single character.

[...]

denotes a class of characters to match. [^...] negates the class. Inside a class, '-' indicates a range of characters.

(A|B|C)

matches if A or B or C matches.

Depending on your command shell, some special characters may need to be escaped on the command line or enclosed in single-quotes to avoid shell interpretation.

HTTP/1.1 Proposed Standard

R. Fielding, J. Gettys, J. C. Mogul, H. Frystyk, and T. Berners-Lee. "Hypertext Transfer Protocol -- HTTP/1.1", U.C. Irvine, DEC, MIT/LCS, August 1996.

http://www.ics.uci.edu/pub/ietf/http/

The Perl Language Home Page

http://www.perl.com/perl/index.html

Johan Vromans' Perl Reference Guide

http://www.xs4all.nl/~jvromans/perlref.html

wwwstat counts HTTP requests received by the server. When a request is successful, it is often referred to as a "hit". Retrieving a single image is one GET request. Retrieving an HTML page is also one GET request, but that does not include the separate requests made for in-line images or related objects. Checking to see if a cached image is still valid (a HEAD or conditional GET) is also one request.

In all sections except the archive section, wwwstat shows the statistics for all requests (successful or not). In the archive section, it normally shows all non-successful requests under a special category for the status code and only successful requests (hits) under the URL or archive tree associated with the request. However, this grouping of non-successful requests is disabled when wwwstat is used with the search options -n, -c, and -C, since those options are normally used for finding error conditions.

wwwstat does not count "visitors" -- individual people or programs making the requests. HTTP does not, by default, provide any information that can be accurately correlated to an individual person, though it is possible (in an unreliable manner) to use HTTP extensions and request profiles as a means of tracking individual client programs. Such tracking requires extensive resources (memory and diskspace) and is often considered a violation of privacy.

With the exception of the ident section, wwwstat does not reveal information about the individual people making requests. Unless the output is limited to a specific URL or a specific hostname, wwwstat's output does not connect the requester to the URL being requested.

The httpd common logfile format (CLF) was defined in early 1994 as the result of discussions among server and access_log analyzer developers (Roy Fielding, John Franks, Kevin Hughes, Ari Luotonen, Rob McCool, and Tony Sanders) on how to make it easier for analysis tools to be used across multiple servers. The format is:

remote_host ident authuser [date-time zone] "Request-Line" Status-Code bytes

where

means

------------

--------------------------------------

remote_host

Client DNS hostname or IP address

ident

Identity check token or "-"

authuser

Authorization user-id or "-"

date-time

dd/Mmm/yyyy:hh:mm:ss

zone

+dddd or -dddd

Request-Line

The first line of the HTTP request, which normally includes the method, URL, and HTTP-version.

Status-Code

Response status from server or "-"

bytes

Size of Entity-Body transmitted or "-"

------------

--------------------------------------

with each field separated by a single space (it turns out that problems occur if the ident token contains a space, which was not anticipated by the original designers).