qmail2dlf - convert sanitized qmail-send logs to dlf

qmail2ldf

qmail2dlf reads qmail-send logs from stdin, writes dlf to stdout, and, in case of errors, complains on stderr.

We expect timestamps which look like e.g. `977359048.466280500' in our log: number of seconds since the epoch, in any precision. The log should feature lines which look something like:

 982584201.511524 info msg 6426: bytes 3537 from 
   <qmail-return-63074-joostvb=mdcc.cx@list.cr.yp.to> qp 21089 uid 70

splogger, as distributed with qmail, writes `a numerical timestamp', ie something like 972802273.627578, indicating seconds and nanoseconds since the beginning of 1970.

multilog, as distributed with daemontools, http://cr.yp.to/daemontools.html, if invoked with action t, writes a `@' and a tai64n format timestamp. These timestamps look like e.g. 4000000039ef8532346bb35c. Note that 0x400000000000000000000000 is 2^62.

One can get the current time in tai64 format by doing:

 echo 40000000`(echo obase = 16; date +%s) | bc`00000000

(assuming GNU date is installed)

To convert tai64n to numerical timestamps, one can use Russ Allbery's tai64nfraq. It's in the public domain, available from http://www.qmail.org/tai64nfrac.

To process a log as produced by splogger:

 $ lr_desyslog qmail < mail.log | qmail2dlf

To process a log as produced by multilog:

 $ tai64nfrac < current | qmail2dlf

qmail2dlf(1) will be rarely used on its own, but is more likely called by lr_log2report:

 $ tai64nfrac < /service/qmail-send/log/main/current | \
     lr_run lr_log2report qmail

We don't deal with bounces:

 email qmail none qmail2dlf info skipping line '986244190.800217 bounce msg 
  6306 qp 28445': first field should be one of 'new', 'info', 'starting', 
  'delivery', 'end' or 'status:', not 'bounce'

qmail2dlf doesn't use any information from the qmail-smtpd(1) logs. (These look like

 2002-06-16 09:34:57.798038500 tcpserver: pid 19385 from 100.61.24.7
 2002-06-16 09:34:58.114198500 tcpserver: ok 19385 foo.example.com:100.163.25.11:25 logreport.iae.nl:212.61.24.7:postfix:1189

.)

$Id: qmail2dlf.in,v 1.22 2004/01/21 07:22:59 wsourdeau Exp $

Copyright (C) 2000, 2001 Stichting LogReport Foundation LogReport@LogReport.org

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program (see COPYING); if not, check with http://www.gnu.org/copyleft/gpl.html or write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA.

Joost van Baal <joostvb@logreport.org>