wflogs - firewall log analyser of the WallFire project.

wflogs [options] [logfile]

wflogs is a firewall log analyser. It can be used to produce a log summary report in plain text, HTML and XML, or even to translate a log file into another firewall log format, for example. Logs can be filtered, summarized, sorted, and obfuscated (in that order), using the following options.

By default, output is not sorted, and may be summarized if the output module has a `summary' option and if this option is set to `yes' (even by default value).

You have to specify a module name that will handle the input (parsing) and another for the output (exportation). See MODULES sections below.

With no logfile, wflogs read /var/log/messages. When logfile is `-', it reads standard input.

-c | --config file

wflogs will use given configuration file. If not specified, wflogs will not use any configuration file and will only use command line options.

-f | --filter expression

Print log entries that match the boolean expression. This expression looks very much like a Perl condition, which must be passed as a single, quoted argument. If no expression is given, all log entries will be dumped. Otherwise, only entries for which expression is `true' will be dumped. See the FILTER EXPRESSION section below.

-i | --input-format format[,format2,...]

Specify the input parsing modules. Wflogs will use the corresponding modules (if available) to parse the logs. If you want to parse a log file with multiple formats mixed (typically a remote syslog file), you can specify several format module names separated by commas, one being probed after another. Use special name `all' to try every available format. If you omit the `-i' option, wflogs will try to guess the local firewalling tool at runtime, and use the corresponding module. Use format `help' to know which modules are available (currently, `netfilter', `ipchains', `ipfilter', `cisco_pix', `cisco_ios', and `snort'), and which is the default (guessed) module. See INPUT MODULES section below.

-I, --interactive

Interactive mode. The program will not terminate, but enter a little interactive shell.

This option can be used in conjonction with real-time mode (-R option). While in non-interactive real-time mode (-R only), signal USR1 enables to fall back into interactive mode.

-o | --output-type type [ output module options ]

Specify the output module type. Wflogs will use the corresponding module (if available) to export the input logs to the corresponding target. Use type `help' to know which modules are available (currently, `text', `html', `xml', `human', `netfilter', `ipchains', and `ipfilter'). Default mode is `text'. See OUTPUT MODULES section below.

Output module configuration can be achieved via the command line. You can specify long options (with a `--' prefix). Three types are supported: boolean (yes or no), integer, and string. A special option `--options' displays the available options of the module, with type, help message, and default value. For example, wflogs -o html --options shows the HTML output module configuration.

-O | --obfuscate [criterias]

This option obfuscates some logging fields according to given criterias, separated by commas. These can be `date', `hostname', `ipaddr', or `macaddr' (or `all' for everything). Default (if no criteria is given) is `all'. If ipaddr is specified, output module options `resolve' and `whois_lookup' (if available) are set to no. If macaddr is specified, output module option `mac_vendor' (if available) is set to no.

Date order is conserved, hostnames are replaced by "hostx" (where x is a growing number), ipaddr belong to 0.0.0.0/8 network, macaddr are of the form 0:0:0:0:0:1, 0:0:0:0:0:2, etc. Note that for all obfuscated fields, each original value is associated with a new unique one (unicity is preserved).

-P | --proceed

If real-time (-R) or interactive (-I) modes are set, first process log entries in the input logfile before entering in these modes, as these entries won't be parsed by default in these modes.

-R | --realtime

``Real-time'' mode: logs are monitored in real-time. Wflogs will wait for new log entries. Entries already present in the input logfile will not be processed as usual, unless you specify -P option.

This option can be used in conjonction with interactive mode (-I option). While in non-interactive real-time mode (-R only), signal USR1 enables to fall back into interactive mode.

-s | --sort[=criteria_list]

Set output lines sort order according to the multilevel sort specified by the sequence of keys key1,key2,... Syntax is --sort=[+|-]key1[,[+|-]key2[,...]]. Choose a key from the SORT KEYS section. `-' reverses direction only on the key it precedes. The `+' is really optional since default direction is increasing numerical or lexicographic order. For example wflogs --sort=dport,-time sorts according to destination port number, then reverse time (for a given port number). If one of the keys is `none', the output is not sorted. Use key `help' to show available keys. If no sort criteria is given, output is sorted by with `-count,time,dipaddr,protocol,dport'.

--strict-parsing type

Set the parsing policy. Available types are: `loose' (even if there are garbage in the input file or incorrect log lines, parse as much as possible and issue no warning at all), `nowarning' (in this case, issue no warnings, ignore non-log lines but do not store incoherent entries), `warning' (issue warnings on stderr, ignore non-log lines but do not store bizarre entries), and `error' (stop parsing if line is not a log entry, or if entry is bizarre). Default type is `warning'.

-v | --verbose [level]

Set verbosity level. If level is omitted, default value is 1.

-V | --version

Display current version.

-h | --help

Show help message on stdout.

wflogs can use extended input modules, each one parsing a specific firewall log format. See option -i.

netfilter

ipchains

ipfilter

cisco_pix

cisco_ios

snort

wflogs can use extended output modules, which enable to export the input logs to a particular format. So it can be used to rewrite the input into another firewall log format or generate a report, for example. See option -o. Summary mode depends on the module, and is configurable through the `summary' module option.

text

html

xml

human

netfilter

ipchains

ipfilter

KEY		DESCRIPTION
count		sort by count (number of original log entries)
time		sort by log entry date (if count != 1, the date of the
			first original log line)
timeend		sort by log entry end date (if count != 1, the date of the
			last original log line)
input_iface	sort by input interface name
output_iface	sort by output interface name
sipaddr		sort by source IP address
dipaddr		sort by destination IP address
smacaddr	sort by source MAC address
dmacaddr	sort by destination MAC address
protocol	sort by protocol number
sport		sort by source port number (if available)
dport		sort by destination port number (if available)
tcpflags	sort by TCP flags
hostname	sort by hostname
chainlabel	sort by chain label
branchname	sort by branch name
datalen		sort by data length
format		sort by firewalling tool format
none		do not sort

This filtering expression looks very much like a Perl condition.

Variables are prefixed with `$'. Pre-defined variables are:

$format (string)

firewalling tool format

$count (integer)

number of original log entries

$start_time ([string] or integer)

log entry date (if count != 1, the date of the first original log line), in date format ([string], see below), or in seconds since the Epoch

$end_time ([string] or integer)

log entry end date (if count != 1, the date of the last original log line), in date format ([string], see below), or in seconds since the Epoch

$hostname (string)

name of the host which logged the packet

$chainlabel (string)

chain label

$branchname (string)

branch name

$input_iface (string)

input interface name

$output_iface (string)

output interface name

$protocol (integer)

protocol number (or name used in /etc/protocols)

$datalen (integer)

data length

$sipaddr (IP network)

source IP address, or source IP network

$sport (integer)

source port number (or name used in /etc/services) if protocol is UDP or TCP, and ICMP type number or name if protocol is ICMP (this may change in the future)

$smacaddr (MAC address)

source MAC address

$dipaddr (IP network)

destination IP address, or destination IP network

$dport (integer)

destination port number (or name used in /etc/services) if protocol is UDP or TCP, and ICMP code number or name if protocol is ICMP (this may change in the future)

$dmacaddr (MAC address)

destination MAC address

$tcpflags (integer)

TCP flags if protocol is TCP (flags can be a combination of SYN|ACK|RST|FIN|PSH|URG|ECE|CWR)

For integer and boolean values, the following operators can be used: ||, &&, ==, !=, <, >, <=, >=, &, |, ^, +, -.

String variables can be compared for strict equality with == and != operators, but also matched with an extended regular expression with =~ operator. Strings are quoted with " (like "foo"), and regexps with / (like /(foo|bar)/). Note that regexp matches only a subset of the string. You have to surround the regexp with ^ and $ if you want to match the whole string (that may change in the future). Like in Perl, you may add an optional i modifier after final /, to do case-insensitive pattern matching.

Date format is one that is accepted by the getdate C function. It must be enclosed in brackets [] and will be converted to an integer value which stands for the number of seconds since the epoch (01 Jan 1970 UTC 00:00). See DATE FORMAT section.

IP network can be an IP address, or an IP network (a.b.c.d/n.o.p.q or a.b.c.d/bitmask, or even things like a.b.*.* for a /16 mask, for example).

MAC addresses are of the form aa:bb:cc:dd:ee:ff. They can only be compared for strict equality (== and != operators).

The string may contain many flavors of items: calendar date items, time of the day items, time zone items, day of the week item, relative items, or pure numbers. As expression can be quite complex, if you have doubt about the dates you specified, activate global verbose mode to show filter expression on stderr using absolute dates.

Calendar date

can be "1974-08-31", "74-8-31", "74-08-31", "8/31/74", "31 August 1974", "31 Aug 1974", "Aug 31, 1974", "31-aug-74", "31aug74". The year can be omitted (current year is then used).

Time of day

can be "02:50:00", "02:50", "2:50am".

Day of week

can be "Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday" or "Saturday", but can be abbreviated to their first three letters. A number may precede a day of the week item to move forward supplementary weeks. It is best used in expression like `third monday'. In this context, `last DAY' or `next DAY' is also acceptable; they move one week before or after the day that DAY by itself would represent.

Relative items

adjust a date (or the current date if none) forward or backward. It can be "1 year", "1 year ago", "3 years", "2 days", for example. You can also use "month", "week", "day", "hour", "minute" ("min"), and "second" ("sec"), or "now" ("today"), "yesterday", and "tomorrow". The string `this' also has the meaning of a zero-valued time displacement, but is preferred in date strings like `this thursday'.

Pure decimal number

precise intepretation depends on the context in the date string. If the decimal number is of the form YYYYMMDD and no other calendar date item appears before it in the date string, then YYYY is read as the year, MM as the month number and DD as the day of the month, for the specified calendar date. If the decimal number is of the form HHMM and no other time of day item appears before it in the date string, then HH is read as the hour of the day and MM as the minute of the hour, for the specified time of the day. MM can also be omitted.

wflogs -i netfilter -o html netfilter.log > logs.html

converts the given netfilter log file into a HTML report.

wflogs --sort=protocol,-time -i netfilter -o text netfilter.log > logs.txt

converts the given netfilter log file into a sorted (by protocol number, then reverse time) text report.

wflogs -f '$start_time >= [this 3 days ago] && $start_time < [this 2 days ago] && $chainlabel =~ /(DROP|REJECT)/ && $sipaddr == 10.0.0.0/8 && $protocol == tcp && ($dport == ssh || $dport == telnet) && ($tcpflags & SYN)' -i netfilter -o text --summary=no

shows log entries (without summary) which match the given expression (refused connection attempts that occured 3 days ago to ssh and telnet ports coming from internal network 10.0.0.0/8).

wflogs -i netfilter --resolve=0 --whois=0 netfilter.log

converts the given netfilter log file into a text report (default mode), disabling IP address reverse lookups and whois lookups.

wflogs -i netfilter -o xml netfilter.log > logs.xml

exports netfilter logs in XML.

wflogs -i ipchains -o netfilter ipchains.log > netfilter.log

converts ipchains logs into netfilter log format. So you may process them with your favorite netfilter log analyser, for example (even if the latter may not be better than wflogs itself. ;-)).

wflogs -i ipfilter -o human --datalen=yes ipfilter.log

produces a report about ipfilter logfile in natural language on stdout, displaying packet length (datalen option) which is not showed by default.

wfconvert(8), regex(7).

Bugs? What's this? ;-) Contributions are welcome, please see http://wallfire.org/.

wflogs has been written by Herve Eychenne. See http://wallfire.org/.

This man page has been initiated by Gregoire Hubert <greg@coolkeums.org>, and written by Herve Eychenne.