Anti Virus Anti Spam solution

Project Scope

To setup an email solution for The Creative Store (TCS) that provided virtual hosting capabilities, anti-spam and anti-virus capabilities.


The chance to engineer this from nothing with no oversight appealed to me. Given my previous experience with dovecot and sendmail (and the knowledge that mimedefang would allow me to interact easily with the SMTP transaction) made these an easy starting point. Very recent experience with Qmail ruled it out as a candidate. The existing system was based on courier and postfix.

The Sendmail/Dovecot interface for virtual hosting and Dovecot's ability to use MySQL as a backend for a user database meant that the solution now had a complete skeleton.

Given that everything was being re-engineered and would cause disruption (changing settings) I dispensed with plain-text auth for POP. POP3 over SSL (995) and IMAP over SSL (993) are supported with a self signed certificate. Warnings from outlook were dealt with by importing our CA signature.

Another added benefit was Squirrelmail (via https) with a certificate singed by the same CA. I added a quick form to the squirrelmail page allowing folk to change passwords. There is a simple algorithm to prevent easy exploitation by a dictionary cracker.

One of the big issues (for me anyway) was how to deal with SPAM once identified.

  • It could be flagged...
  • This is just pointless. Folk either just automate delivery into the SPAM folder that is never checked, or they still need to peruse the email (taking time and effort).
  • It could be dropped...
  • This prevents the recipient from having to deal with it, but is really just as bad as the "automated delivery into the SPAM folder".

  • It could be returned to the sender...
  • Possibly the best solution, since the recipient (victim) doesn't have to deal with it AND the sender is informed. BUT with any real spam having a fake return address we would become generators of spam.

The correct solution would be then to return it to the sender, but NOT the fake sender in the From header or even the envelope sender. The transaction should be rejected at the SMTP level, just after the DATA command.

Any mail that is rejected could be returned by the MTA connecting to us, we'd still need a way to be told about false positives, but at least the receipient isn't having to deal with the mails and the REAL sender is informed if their MTA is 'good and proper'.

... so what happens if a message is rejected incorrectly? Since no "pretty message" can be returned to the sender a URL is returned. Hopefully even the most novice of users will follow that link and find a form allowing them to submit the error which is than passed on to me, allowing appropriate action to be taken


As at 2007/07/10 this software is in use:

  • sendmail : sendmail-8.13.7
  • mimedefang : mimedefang-2.58 (/etc/mail/mimedefang heavily customised)
  • dovecot : dovecot-1.0.rc15 (compiled from source)
  • spamassassin : spamassassin-3.1.7-36 (FC4)
  • mysql : MySQL (whatever was already available on the system)
  • ClamScan : The signatures are updated in accordance with Clam's requirements and the software is updated usually within days of a new version being released.
  • Sanesecurity definitions for catching stock spam, phising etc. were added 2007/08/08

The need to handle issues at the SMTP level has meant significant changes to MimeDefang to handle receipient validation (by and implication LDAP alias resolution).

Other sendmail anti-relaying functionality that was lost with use of relaying virtually hosted domains to dovecot was re-written in MimeDefang

In retrospect I've realised that I'm not using much of sendmail's functionality and consequently could quite easily have gone with a lighter SMTP deamon. I didn't, and I have no plans to change now.

How does the system perform?