Qmail-Scanner: Email Scanning Harness for Qmail

Description

Qmail-Scanner, (also known as scan4virus) is an addon that enables a Qmail Email server to scan all gatewayed Email for certain characteristics. It is typically used for its anti-virus protection functions, in which case it is used in conjunction with commercial virus scanners. but also enables a site to react to Email (at a server/site level) that contains specific strings in particular headers, or particular attachment filenames or types (e.g. *.VBS attachments). It also can be used as an archiving tool for auditing or backup purposes. Qmail-Scanner is integrated into the mail server at a lower level than some other Unix-based virus scanners, resulting in better performance. It is capable of scanning not only locally sent/received Email, but also Email that crosses the server in a relay capacity.

Features

Uses any commercial Unix command-line virus scanner.
Can call more than one virus scanner for each mail message
Has its own internal scanner that can be used to pick up viruses for which commercial scanner updates are not yet available
The internal scanner can also be used to block attachment types, or Email with certain Email headers... Need to stop *.mp3 files or "Subject: ILOVEYOU" Email getting onto and off your LAN - can do! :-)
Auto-detects Email from "postmaster"-style and mailing-list addresses - and doesn't send virus reports to them (i.e. attempts to act more like a responsible net citizen)
Each message is tagged via a new Received: header with a virus report showing whether it is clean or not and virus scanner version numbers/etc
Messages with viruses are moved into a "maildir" mail folder for later perusal by the appropriate staff
Can optionally add a descriptive header: X-QmailScan to every Email that passes through the system to allow users to see that a scanner has run over their messages
Messages caught by Qmail-Scanner generate an Email message (currently supports English, Italian and German messages) to a configurable combination of the sender, recipients and a "virus-admin" address explaining why their message was rejected
Can archive all processed Email into an archive directory. Useful when debugging Email-based apps, for backup purposes and for audit policy reasons.
Redundant scanning. Not only does it unpack each message before running the scanners over it, it can also scan the original Email message as well as the unpacked messages (if you think a particular scanner can do a better job than Qmail-scanner's internal systems allow.)

Download

The latest release is 0.94 (via http), (via ftp), and is kindly housed by SourceForge

Requirements

Qmail 1.03
reformime from Maildrop 0.73 or 1.1+ (Don't use 1.0, which has a major bug)
Perl 5.005_03+
Perl module Time::HiRes
Perl module DB_File (most distributions come with it pre-installed)
Optional: Mark Simpson's TNEF unpacker. Can decode those annoying MS-TNEF MIME attachments that Microsoft mail servers just love to use. If you don't have this, there are several classes of Email that you basically won't be able to detect viruses in.

Patches

Bruce Guenter's QMAILQUEUE patch is required to enable Qmail to call a different qmail-queue program than the one compiled in by default. Qmail-scanner's qmail-scanner-queue.pl perl script is used instead of Qmail's qmail-queue binary. After qmail-scanner-queue.pl has run, it calls the original qmail-queue binary to resubmit the message back into the system.

Note: Qmail-scanner can be installed without the QMAILQUEUE patch. You can rename qmail-queue to something like qmail-queue.real, and rename the final qmail-scanner-queue.pl to qmail-queue (after ensuring the perl script's $qmailqueue variable points to the new qmail-queue.real of course!). This is not recommended or supported as it would be easy to trash an otherwise working scanning system when upgrading/reinstalling Qmail for example.

Supported Virus Scanners

The following virus scanners are known to work with qmail-scanner. Other Unix-based scanners should be simple to add support for.

CHANGES

There is a separate page listing changes that have been made between releases

TODO

There is a separate TODO page.

FAQ

There is a separate FAQ page.

Performance/Resource Usage

Adding content/virus scanning to an Email server will considerably add to the resource usage of that server. As this "wrapper" is written in perl instead of low-level C, quite a lot of memory and file opens/stats occurs just to get it going. Adding to this the actual scanners memory and CPU usage and it becomes quite complicated (certainly the debugging info shows that the scanner harness spends more time running the commercial scanners than it does doing things itself [that is to be expected as they do quite a lot of thinking...]).

As a "rule of thumb" I'd suggest you look at how many simultaneous SMTP sessions you are willing your box to have going at any one point in time. Each SMTP session can invoke up to 'n' different virus scanners (although they run one after the other - not simultaneously) and I'd estimate that leads to around 5-6Mb of memory usage per SMTP session. Thus if your SMTP host has 256Mb RAM + 256Mb swap - that should mean you can handle - well heaps ;-) The scanners cause the CPU to be thrashed while they're running, so I'm making sure for our site that our Qmail server will only accept up to 30 incoming SMTP sessions at any one time - that way I know the box will handle it. As this leads to an increased memory usage, don't forget Qmail's ulimits will need to be increased to deal with it (set via ulimit or softlimit calls with Qmail system startup scripts).

One thing you should test for is what happens if connectivity between this server and another local SMTP server is down for any length of time (due to failure/power outage). When the link is restored, can your server handle the other trying to dump 1,000's of Email msgs onto it at once? You need to use ulimit and tcpserver's limit options to ensure your box doesn't get killed. Note that this resource issue isn't caused by scan4virus. The same thing will happen with a pure, untouched Qmail (or any other) system - it will just happen sooner...

After that scare-mongering I should say that I have tested scan4virus under ridiculously low resource conditions - and it reacts as it should - so at worst your system should start deferring Email. Thankfully DJB's layering of programs is such that this is easy to accomplish :-)

Installation

Unpack Qmail-Scanner and run ./configure --help. This will show you what options are available to you.
Run ./configure ... [with your options], it will autodetect what software is installed on your system, and will generate a script specific to your system.
Run ./configure again, this time include "--install" along with the options you chose, this will do the same as the previous line, but will also create the directory structure required, and install qmail-scanner-queue.pl
If you want to manually install it, see the Manual Installation page.

At this stage the Qmail startup script(s) (e.g. /etc/rc.d/init.d/qmail ) will need to be updated so that Qmail knows to use qmail-scanner-queue.pl instead of qmail-queue.

echo -n "qmail-smtpd, "
QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl" export QMAILQUEUE

(ulimit -d 5120 -m 2048 && tcpserver -l`hostname -f` -c20 -b30 -P -h -R -t10 \
 -O -Q -v -x/var/qmail/control/tcprules.cdb \
 -gQMAILDUSER -uQMAILDGROUP 0 25 qmail-smtpd 2>&1) | splogger tcpserver &

..or whatever yours is like. Basically set QMAILQUEUE just before you invoke qmail-smtpd - that way only it runs with a "different" qmail-queue - everything else carries on running the standard qmail-queue.

BE VERY CAREFUL ABOUT THAT - YOU DON'T WANT AN INFINITE LOOP FORMING!!!

This will mean from now on all incoming SMTP mail will be scanned for viruses/etc.

If you want finer control, you can define QMAILQUEUE within the tcpserver rules file (man tcpserver for details), so that it only gets defined for certain IP addresses/ranges. That way you could have virus scanning only performed on mail coming from your Exchange server (as an example), and not from your Unix servers.

To scan all mail sent by local shell users, the QMAILQUEUE will also need to be defined within /etc/profile or the like so that when they send mail, it will be affected as well.

Although as they are obviously not Windows users, you may want to save your system the effort and explicitly NOT do that! :-)

If "$DEBUG=1" (the default), then every transaction will be logged to /var/spool/qmailscan/qmail-queue.log - so you'll see how it goes. Regardless of debugging, errors (and attachment info if enabled) should also be recorded in the qmail logs (probably via syslog) - just look for entries containing the string "X-QmailScan".

Any SMTP sessions that are dropped (due to network outages/etc) may lead to files lying around in /var/spool/qmailscan . Running /var/qmail/bin/qmail-scanner-queue.pl -z at least once daily will ensure such files are deleted when they're over 30 hours old - make a cronjob to do that. Also realise that /var/spool/qmailscan/qmail-queue.log will grow without bounds. At some stage turn debugging off $DEBUG=0 off and delete the logfile.

Qmail-Scanner contains an internal scanner which allows you to reject Email based on attachment filenames and/or Email headers. Read the minimal document on it for details.

When Viruses Strike...

When Qmail-Scanner finds a message with a virus, it moves it into a local "quarantine" mail folder (maildir format) - by default /var/spool/qmailscan/viruses/. This means the message can be read in its pure "adulterated" state by maildir clients like mutt - or via IMAP (if maildir format supported - you'll have to work that out for yourself). At worse you can just read it - it's just a MIME file...

If you want a good IMAP server that supports maildir natively - try Courier-IMAP.

I made the decision to write it into maildir format for performance and reliability reasons - and it expressly makes it difficult for any Windows admin to click on it with their vulnerable Windows mailer and read it :-) Qmail actually comes with a program called /var/qmail/bin/maildir2mbox which can do just that... (you could run it from cron to automatically suck all the new mail messages from /var/spool/qmailscan/viruses/new/ into a mbox.)

Also this event is logged in /var/spool/qmailscan/viruses.log in a tab-delimited format (for post-processing). A good script is needed to convert this file into some nice graphs for management :-)

Support

This software is released under the GPL as found in the COPYING file enclosed.

This package is housed on SourceForge.

Any questions, suggestions, etc to jhaar@users.sourceforge.net or to the mailing-list set up to discuss this, subscribe via http://lists.sourceforge.net/mailman/listinfo/qmail-scanner-general , or subscribe to the announcements-only list via http://lists.sourceforge.net/mailman/listinfo/qmail-scanner-announce.

Last Updated: