fwlogwatch - a firewall log analyzer, report generator and realtime response agent

fwlogwatch [options] [input_files]

fwlogwatch produces Linux ipchains, Linux netfilter/iptables, Solaris/BSD/Irix/HP-UX ipfilter, ipfw, Cisco IOS, Cisco PIX, NetScreen, Windows XP firewall, Elsa Lancom router and Snort IDS log summary reports in plain text and HTML form and has a lot of options to analyze and display relevant patterns. It can produce customizable incident reports and send them to abuse contacts at offending sites or CERTs. Finally, it can also run as daemon (with web interface) doing realtime log monitoring and reporting anomalies or starting attack countermeasures.

These options are independent from the main modes of operation.

-h

Show the available options.

-L

Show time of the first and the last log entry. The input file(s) can be compressed or plain log file(s). Summary mode will show the time of the first and last packet log entry, this log times mode will show the time of the first and last entry overall.

-V

Show version and copyright information and the options used to compile fwlogwatch.

The global options for all modes are:

-b

Show the amount of data in bytes this entry represents, this is the sum of total packet lengths of packets matching this rule (obviously only available for log formats that contain this information).

-c config

Use the alternate configuration file config instead of the default configuration file /etc/fwlogwatch/fwlogwatch.config (which does not need to exist). Only options not specified in the files can be overridden by command line options.

-D

Do not differentiate destination IP addresses. Useful for finding scans in whole subnets.

-d

Differentiate destination ports.

-E format

Specific hosts, ports, chains and branches (targets) can be selected or excluded, selections an exclusions can be added and combined. The format is composed of one of the functions i include or e exclude, then one of the parameters h host, p port, c chain or b branch. In case of a host or port a third parameter for s source or d destination is needed. Finally, the object is directly appended, in case of a host this is an IP address (networks can be specified in CIDR format), port is a number and chain and branch are strings. To show entries with destination port 25 you would use -Eipd25 and to exclude entries which have the class C network 192.168.1.0 as source or belong to the chain INPUT: -Eehs192.168.1.0/24 -EecINPUT

-M number

If you only want to see a fixed maximum amount of entries (e.g. the "top 20") this option will trim the output for you.

-m count

When analyzing large amounts of data you usually aren't interested in entries that have a small count. You can hide entries below a certain threshold with this option.

-N

Enable service lookups. Port numbers will be looked up in /etc/services.

-n

Enable DNS lookups. Host names will be resolved (reverse and forward lookup with a warning if they don't match). This makes summary generation very slow if a lot of different hosts appear in the log file. Resolved host names are cached.

-O order

This is the sort order of the summary and packet cache. Since entries often are equal in certain fields you can sort by several fields one after another (the sort algorithm is stable, so equal entries will remain sorted in the order they were sorted before). The sort string can be composed of up to 11 fields of the form ab where a is the sort criteria: c count, t start time, e end time, z duration, n target name, p protocol, b byte count (sum of total packet lengths), S source host, s source port, D destination host and d destination port. b is the direction: a ascending and d descending. Sorting is done in the order specified, so the last option is the primary criteria. The default in summary mode is tacd (start with the highest count, if two counts match list the one earlier in time first) of which ta is built in, so if you specify an empty sort string or everything else is equal entries will be sorted ascending by time. The realtime response mode default is cd ( ta is not built in).

-P format

Only use certain parsers, where the log format can be one or a combination of: i ipchains, n netfilter, f ipfilter, b ipfw, c Cisco IOS, p Cisco PIX, e NetScreen, w Windows XP, l Elsa Lancom and s Snort. The default is to use all parsers except the ones for NetScreen, Windows XP, Elsa Lancom and Snort logs.

-p

Differentiate protocols. This is activated automatically if you differentiate source and/or destination ports.

-s

Differentiate source ports.

-U title

Set title as title of the report and status page and as subject for reports sent by email.

-v

Be verbose. You can specify it twice for more information. In very verbose mode while parsing the log file you will see "." for relevant packet filter log entries, "r" for 'last message repeated' entries concerning packet filter logs, "o" for packet filter log entries that are too old and "_" for entries that are not packet filter logs.

-y

Differentiate TCP options. All packets with a SYN are listed separately, other TCP flags are shown in full format if they are available (ipchains does not log them, netfilter and ipfilter do, Cisco IOS doesn't even log SYNs).

This are additional options that are only available in log summary mode:

-e

Show timestamp of last packet logged. End times are only available if there is more than one packet log entry with unique characteristics.

-l time

Process recent events only. See TIME FORMAT below for the time options.

-o file

Specify an output file.

-S

Do not differentiate source IP addresses.

-T email

The summary will be sent by email to this address. If HTML output is selected the report will be embedded as attachment so HTML-aware mail clients can show it directly.

-t

Show timestamp of first packet logged.

-W

Look up information about the source addresses in the whois database. This is slow, please don't stress the registry with too many queries.

-w

Produce output in HTML format.

-z

Show time interval between start and end time of packet log entries. This is only available if there is more than one packet log entry with unique characteristics.

The interactive report mode is a summary mode extension with the following additional options:

-i count

Enter interactive report mode. count is the minimum number of log entries you want to start reporting at. A summary of the corresponding entries will be shown and a report generated for each one. The more of the options above you use the more fields of the report will be filled in.

-F email

This is the address the email containing the report will be sent from.

-T email

This is the email address of the abuse contact or CERT the report will be sent to.

-C email

These email recipients will get a carbon copy of the report (e.g. for your archives).

-I file

Template file for report (defaults to /etc/fwlogwatch/fwlogwatch.template ).

-R

Enter realtime response mode. This means: detach and run as daemon until the TERM signal (kill) is received. The HUP signal forces a reload of the configuration file, the USR1 signal forces fwlogwatch to reopen and read the input file from the beginning (useful e.g. for log rotation). All output can be followed in the system log.

-a count

Alert threshold. Notify or start countermeasures if this limit is reached. Defaults to 5.

-l time

Forget events that happened this long ago (defaults to 1 day). See TIME FORMAT below for the time options.

-k IP/net

This option defines a host or network in CIDR notation that will never be blocked or other actions taken against. To specify more than one, use the -k parameter again for each IP address or network you want to add.

-A

The notification script is invoked when the threshold is reached. A few examples of possible notifications are included in fwlw_notify, you can add your own ones as you see fit.

-B

The response script is invoked when the threshold is reached. Using the example script fwlw_respond this will block the attacking host with a new firewall rule. A new chain for fwlogwatch actions is inserted in the input chain and block rules added as needed. The chain and its content is removed if fwlogwatch is terminated normally. The example scripts contain actions for ipchains and netfilter, you can modify them or add others as you like.

-X port

Activate the internal web server to monitor and control the current status of the daemon. It listens on the specified port and by default only allows connections from localhost. The default user name is admin and the default password is fwlogwat (since DES can only encrypt 8 characters). All options related to the status web server can be changed in the configuration file.

You can specify one or more input files (if none is given it defaults to /var/log/messages ). Relevant entries are automatically detected so combined log files (e.g. from a log host) are no problem. Compressed files are supported (except in realtime response mode where they don't make sense anyway). The '-' sign may be used for reading from standard input (stdin). In realtime response mode the file needs to be specified with an absolute path since the daemon uses the file system root (/) as working directory.

Time is specified as nx where n is a natural number and x is one of the following: s for seconds (this is the default), m for minutes, h for hours, d for days, w for weeks, M for months and y for years.

/etc/fwlogwatch/fwlogwatch.config

Default configuration file.

/etc/fwlogwatch/fwlogwatch.template

Default template for incident reports.

/var/log/messages

Default input log file.

/var/run/fwlogwatch.pid

Default PID file generated by the daemon in realtime response mode if configured to do so.

The following features are only available in the configuration file and not on the command line, they are presented and explained in more detail in the sample configuration file.

HTML colors and stylesheet

The colors of the HTML output and status page can be customized, an external cascading stylesheet can be referenced.

Realtime response options

Verification of ipchains rules, PID file handling, the user fwlogwatch should run as, the location of the notification and response scripts, which address the status web server listens on, which host can connect, the refresh interval of the status page and the admin name and password can be configured.

Since fwlogwatch is a security tool special care was taken to make it secure. You can and should run it with user permissions for most functions, you can make it setgid for a group /var/log/messages is in if all you need is to be able to read this file. Only the realtime response mode with activated ipchains rule analysis needs superuser permissions but you might also need them to write the PID file, for actions in the response script and for binding the default status port. However, you can configure fwlogwatch to drop root privileges as soon as possible after allocating these resources (the notification and response scripts will still be executed with user privileges and log rotation might not work).

Boris Wesslowski <bw@inside-security.de>