bind9_query2dlf - convert BIND9 querylogs to dlf

bind9_query2dlf

bind9_query2dlf expects BIND 9 query log files on stdin.

If you have a

        channel query_logging {
                file "/var/log/named_querylog"
                versions 3 size 100M;
                print-time yes;                 // timestamp log entries
        };

in your named.conf, the produced logfiles are supported. Optionally, you could add

                print-category yes;             // print category name
                print-severity yes;             // print severity level

to this channel. Query logs as produced by a patched BIND (see NOTES below) are supported too.

We also support Bind 9.3 log file which uses a new date format (15-Jul-2002) instead of the old-syslog one.

With print-time, print-category and print-severity set:

 Feb 25 11:09:43.651 queries: info: client 10.0.0.3#1035:
  query: 3.example.com.nl IN A
 Feb 25 11:09:48.739 queries: info: client 10.0.0.3#1035:
  query: 3.example.com.nl IN A
 Feb 25 12:50:32.476 queries: info: client 10.0.0.3#1035:
  query: 21.example.com.co.uk IN A
 Feb 25 12:50:34.110 queries: info: client 10.0.0.3#1035:
  query: 22.example.com IN A
 Feb 25 12:50:34.525 lame-servers: info: lame server on
  '22.example.com' (in '23.example.com'?): 10.0.0.4#53
 Feb 25 12:50:34.715 queries: info: client 10.0.0.3#1035:
  query: 24.example.com IN A
 Feb 26 07:30:08.211 queries: info: client 10.0.0.1#1050:
  query: 1.0.0.10.in-addr.arpa IN PTR
 Feb 26 12:26:55.455 queries: info: client 10.0.0.1#1051:
  query: 28.example.com.nl IN MX
 Feb 04 04:02:00.932 general: info: loading configuration
  from '/etc/336.example.com'
 Feb 18 04:02:01.023 security: warning: zone
  '337.example.com.nl' allows updates by IP address, which
  is insecure
 Feb 18 04:02:01.049 config: warning: option 'use-id-pool'
  is obsolete
 Feb 18 04:02:01.049 config: warning: option 'check-names'
  is not implemented
 Feb 18 04:02:01.049 config: warning: option
  'statistics-interval' is not yet implemented
 Feb 18 04:02:01.049 network: info: no IPv6 interfaces found
 Feb 04 16:47:18.289 security: info: client 10.0.0.201#137:
  query denied
 Feb 20 07:26:53.731 general: info: running
 Feb 13 08:01:56.138 general: info: shutting down
 Feb 13 08:01:56.140 network: info: no longer listening on
  10.0.0.3#53
 Feb 14 08:02:13.983 general: info: refresh_callback: zone
  384.example.com/IN: failure for 10.0.0.204#53: timed out

With only print-time set:

 Aug 27 04:07:13.361 client 127.0.0.1#3123: query: foo.com IN ANY
 Aug 27 04:07:13.438 client 127.0.0.1#3123: query: fu.bar.nl IN AAAA
 Aug 27 04:07:13.443 client 127.0.0.1#3123: query: fu.bar.nl IN A

To process a log as produced by bind9:

 $ bind9_query2dlf < dns-query

bind9_query2dlf will be rarely used on its own, but is more likely called by lr_log2report:

 $ lr_run lr_log2report bind9_query < /var/log/dns-query

Bind9 doesn't log wether the query was recursive, therefore the last dlf field (DLF_RESOLVER) is a '-'. However, applying this patch by Wytze van der Raay:

 # patch bin/named/query.c to log recursive/non-recursive query indication
 SRC=bin/named/query.c
 if [ -f ${SRC}.org ]
 then
         echo "Patched ${SRC} already in place"
 else
         echo "Patch ${SRC} for recursive/non-recursive query indication"
         cp -p ${SRC} ${SRC}.org
         patch -p0 ${SRC} <<\!
 --- bin/named/query.c.org       Mon Sep 24 22:57:48 2001
 +++ bin/named/query.c   Tue Sep 25 09:55:21 2001
 @@ -3272,7 +3272,8 @@
         dns_rdatatype_format(rdataset->type, typename, sizeof(typename));

         ns_client_log(client, NS_LOGCATEGORY_QUERIES, NS_LOGMODULE_QUERY,
 -                     level, "query: %s %s %s", namebuf, classname, typename);
 +                     level, "query: %s %s %s%s", namebuf, classname, typename,
 +                     WANTRECURSION(client) ? "+" : "-");
  }

  void
 !
 fi

will yield loglines like

 Nov 11 12:06:42.829 queries: info: client 10.0.0.1#3664:
   query: 6.example.com.nl IN A+

A '+' indicates a recursive query, - indicates a non-recursive query, the lack of + or - indicates a non-patched bind9. See Wytze's message of Fri, 28 Dec 2001 16:56:30 +0100 on bind9-workers@isc.org , archived at http://www.mail-archive.com/bind9-workers@isc.org/msg00501.html .

This type of logfiles is recognised by the script.

If you're running cutting edge bind, the code might be integrated in BIND. BIND 9.3 will very likely offer support for this logging feature.

In a private discussion on Thu, 18 Jul 2002 07:55:22 +0200, Wytze wrote:

 This contains the "ICS-blessed" version of  the patch for getting a
 recursive/non-recursive request logged. ISC decided there should be a space
 between the type and the recursion indicator in the logfile, so be it.

If you want to have your BIND 9.2.1 to log in the ISC-blessed 9.3 style, apply this patch to your BIND sources:

 --- bin/named/query.c.org       Thu Mar 28 06:10:09 2002
 +++ bin/named/query.c   Wed Jul 17 08:14:41 2002
 @@ -3279,7 +3279,8 @@
         dns_rdatatype_format(rdataset->type, typename, sizeof(typename));

         ns_client_log(client, NS_LOGCATEGORY_QUERIES, NS_LOGMODULE_QUERY,
 -                     level, "query: %s %s %s", namebuf, classname, typename);
 +                     level, "query: %s %s %s %s", namebuf, classname, typename,
 +                     WANTRECURSION(client) ? "+" : "-");
  }

  void

. This script understands both Wytze-style and ISC-style recursiveness indication.

Wytze van der Raay, for supplying the BIND 9 query log patch.

bind8_query2dlf(1), The bind9 online documentation, as distributed with BIND (but unfortunately not online at http://isc.org/ , you might like http://doc.mdcc.cx/doc/bind/html/logging.html though)

$Id: bind9_query2dlf.in,v 1.4 2004/01/21 07:23:00 wsourdeau Exp $

Copyright (C) 2001 Joost Bekkers <joost@jodocus.org>, Copyright (C) 2000, 2001, 2002 Stichting LogReport Foundation LogReport@LogReport.org

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program (see COPYING); if not, check with http://www.gnu.org/copyleft/gpl.html or write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA.

Joost Bekkers <joost@jodocus.org>, based on Edwin Groothuis and Joost van Baal's work, now maintained by the LogReport team.