htdig - retrieve HTML documents for ht://Dig search engine

htdig [options]

Htdig retrieves HTML documents using the HTTP protocol and gathers information from these documents which can later be used to search these documents. This program can be referred to as the search robot.

-a

Use alternate work files. Tells htdig to append .work to database files, causing a second copy of the database to be built. This allows the original files to be used by htsearch during the indexing run.

-c configfile

Use the specified configfile instead of the default.

-h maxhops

Restrict the dig to documents that are at most maxhops links away from the starting document. This only works if option -i is also given.

-i

Initial. Do not use any old databases. This is accomplished by first erasing the databases.

-s

Print statistics about the dig after completion.

-t

Create an ASCII version of the document database. This database is easy to parse with other programs so that information can be extracted from it for purposes other than searching. One could gather some interesting statistics from this database.

-u

username:password Tells htdig to send the supplied username and password with each HTTP request. The credentials will be encoded using the between the username and password.

-v

Verbose mode. This increases the verbosity of the program. Using more than 2 is probably only useful for debugging purposes. The default verbose mode (using only one -v) gives a nice progress report while digging.

/etc/htdig/htdig.conf

The default configuration file.

Please refer to the HTML pages (in the htdig-doc package) /usr/share/doc/htdig-doc/html/index.html and the manual pages htdigconfig(8) , htfuzzy(1) , htmerge(1) , htnotify(1) , htsearch(1) and rundig(1) for a detailed description of ht://Dig and its commands.

This manual page was written by Christian Schwarz, modified by Stijn de Bekker, based on the HTML documentation of ht://Dig.