General Administration and Maintenance Issues - SpamTagger/SpamTagger-Plus GitHub Wiki

MailCleaner has several tools in place which typically allow it to run, without any manual maintenance for quite some time. These features include:

However, there are some occasional problems that will require manual intervention which are discussed here.

Exim Queue Management

MailCleaner uses 3 different Exim queues, one for each of the MTA stages. As shown from Monitoring->Status, these queues can get backed up. This is especially the case for incoming email at Stage 4 when the recipient address does not exist (see Address verification to mitigate this), or if the back-end mail server is malfunctioning. It will also happen for invalid outgoing addresses in the Stage 1 queue, or for bounce messages.

Regardless of the cause, you can search, force and delete items from the Exim queue as discussed here.

Basic commands

MailCleaner ships with a custom compiled version of Exim so that we can enable many of the features not enabled in Debian's package. It installs to a custom path so as not to conflict with the Debian version if it gets installed: /opt/exim4/bin/.

Within that directory, the main commands that you will need are:

  • exiqgrep - to search the queues
  • exim - to operate on the queues

For several of the examples below, we will want to take output from the first command and pipe the discovered IDs to xargs which will then operate on them using the second.

Since there are 3 separate queues, you must always include a -C flag followed by the path to the configuration file for that MTA stage. This means each command will look something like:

/opt/exim4/bin/exim -C /var/mailcleaner/spool/tmp/exim/exim_stage1.conf

Search for messages in a queue (exiqgrep)

The exiqgrep command requires at least one search term and then will also accept an output modifier for the results. Search options include:

  • -f <regexp> - Match sender address sender (field is "< >" wrapped)
  • -r <regexp> - Match recipient address
  • -s <regexp> - Match against the size field from long output
  • -y <seconds> - Message younger than
  • -o <seconds> - Message older than
  • -z - Frozen messages only (exclude non-frozen)
  • -x - Non-frozen messages only (exclude frozen)

Output modifiers include:

  • -c - Display match count
  • -l - Long Format [Default]
  • -i - Message IDs only
  • -b - Brief Format
  • -R - Reverse order
  • -a - All recipients (including delivered)

You will most likely be running a search like:

/opt/exim4/bin/exiqgrep -C /var/mailcleaner/spool/tmp/exim/exim_stage1.conf -r "@domain\.com"

to find the full details of anything destined for a user at "@domain.com". Take care with the search patterns since they are Regular Expressions. You will need to escape special characters like the dot shown above.

Then, if you want to operate on all of these items, you would also append the -i option to get just the IDs and forward them on to exim.

If the exiqgrep search provides a lot of results, you can pipe it to exiqsumm to summarize the results so that they are a little easier to read:

/opt/exim4/bin/exiqgrep -C /var/mailcleaner/spool/tmp/exim/exim_stage1.conf -r "@domain\.com" | /opt/exim4/bin/exiqsumm

Note that the output of exiqsumm will not be useable by exim, so do not try to pipe out from it.

Operating on queued messages (exim)

With -M followed by a message ID, exim will attempt to queue the message with that ID:

/opt/exim4/bin/exiqgrep -C /var/mailcleaner/spool/tmp/exim/exim_stage1.conf -M ABCD-000000000123-1234

With -Mrm it will remove that item from the queue (deleting it):

/opt/exim4/bin/exiqgrep -C /var/mailcleaner/spool/tmp/exim/exim_stage1.conf -M ABCD-000000000123-1234

Note that in MailCleaner, the queue files are located at /var/mailcleaner/spool/exim_stage{1,2,4}/input/. There will be 2 files, both starting with the Exim ID. The one ending -D contains the actual message data. The one ending with -H contains the Exim header information including how long it has been queued, when it was last attempted, etc.

It is likely that you'll want to perform these actions in bulk from the output of exiqgrep, as mentioned above, and as demonstrated in these examples.

Examples

Immediately try to deliver messages to [email protected]:

/opt/exim4/bin/exiqgrep -C /var/mailcleaner/spool/tmp/exim/exim_stage1.conf -f "[email protected]" -i | xargs opt/exim4/bin/exim -C /var/mailcleaner/spool/tmp/exim/exim_stage1.conf -M

This can be useful if the back-end server for that recipient had been offline, but is now up and you would like to deliver all queued messages as soon as possible.

Delete all messages queue in Stage 4 from the sender '[email protected]':

/opt/exim4/bin/exiqgrep -C /var/mailcleaner/spool/tmp/exim/exim_stage1.conf -f "[email protected]" -i | xargs opt/exim4/bin/exim -C /var/mailcleaner/spool/tmp/exim/exim_stage1.conf -Mrm

This can be useful if you did not have recipient verification set up correctly and have some messages for a user/domain that you know will never be delivered.

Unfreezing messages

Messages are frozen when there is no possibility of delivery, for example if there is no destination available (ie. no MX record, or no A record for the host in the MX record). Since it is impossible to deliver the message, there is no use in Exim queueing the message for delivery repeatedly, so it freezes it in order to only be delivered again if you manually un-freeze it.

The search flag -z will print only the frozen messages, then -M will queue it:

/opt/exim4/bin/exiqgrep -C /usr/mailcleaner/etc/exim/exim_stage4.conf -i -z | xargs /opt/exim4/bin/exim -v -C /usr/mailcleaner/etc/exim/exim_stage4.conf -M

Useful links

You can read some documentations on the web, one is easy to find and complete enough: http://bradthemad.org/tech/notes/exim_cheatsheet.php

Repair a database

Occasionally you may see that one of the databases fails to start. They can be checked with:

/usr/mailcleaner/bin/check_db.pl -m --mycheck # On master
/usr/mailcleaner/bin/check_db.pl -s --mycheck # On slave

If there is a problem, you can attempt an automatic repair with:

/usr/mailcleaner/bin/check_db.pl -m --myrepair # On master
/usr/mailcleaner/bin/check_db.pl -s --myrepair # On slave

or perhaps it needs to be updated:

/usr/mailcleaner/bin/check_db.pl -m --update # On master
/usr/mailcleaner/bin/check_db.pl -s --update # On slave

If that database fails to start after all of these actions, you may need to check the logs in /var/mailcleaner/log/mysql_{master,slave}/mysql.log to see what the issue is.

Re-synchronize database

It is possible for the Master and Slave databases to fall out of sync. There is a regular task which should run to try to correct this, and a watchdog which should alert you after several failures of this task to resolve the issue. If for some reason the problem persists, you can try to run /usr/mailcleaner/bin/resync_db.pl.

⚠️ **GitHub.com Fallback** ⚠️