FAQ ‐ Administrators - MailCleaner/MailCleaner8 GitHub Wiki

See General Administration and Maintenance Issues for periodic issues that might arise which require a system administrator to correct problems from the command line.

If you need to find what happened to any messaging transiting MailCleaner, the first place to look is the results of a Management->Tracing search.

You will need to define at least the domain of a local user (recipient for inbound mail or sender for outbound mail) and can optionally provide their full address, an external address (the sender or recipient) and some other detail that might be logged by the Incoming MTA (eg. subject or hostname). This is needed in order to greatly speed up search results. If it is ever too restrictive (eg. you need to find all connections from a specific IP, regardless of the local address), you will need to search the logs (/var/mailcleaner/log) directly using grep. See below.

If searched correctly, your desired email(s) should be listed. If it is not, it is possible that the message you were expecting has not arrived yet, it may have been rejected before the recipient was know (eg. if the IP was blacklisted), or you may have the incorrect search terms.

Once you find your desired item, you can expand the details of the mails by clicking the Magnifying glass icon to the left of the listing. Each section of the filtering process should be labelled, so for example, if the trace is pretty short and only includes the Incoming MTA section, it is likely that it was either blocked at the SMTP stage, or it is still waiting to be filtered. The content of the logs should indicate this and if it was not rejected, you should find the item in the Filtering Queue.

If there are more logs, pay special attention to the contents of the Filtering Engine, which should indicate if it was found to be spam and why, as well as the action that is to be taken. Even if it is spam, it may say 'Want deliver' which indicates that it was whitelisted, or the recipient is in tag mode. In this case, you should see that there is an Outgoing MTA log.

If the message is not still queue, the Tracing table should summarize the results by indicating the status.

Perhaps the most common issue we hear of is that a message is marked as being delivered, but it was never received by the user. This is almost always because of additional filtering by the back-end mail server. The last couple of lines (Outgoing MTA) should show the response when forwarding the message to that server, and normally there will be an 'Accepted' message, including an internal ID that can be used to track it down further. If your mail server went on to reject the message after we determined it not to be spam, it in unrecoverable, but the sender should have received a bounce message from us.

The SMTP specification provides the protocol for how email is transferred between MTAs. One specification within this standard is that lines SHOULD NOT exceed 80 characters (78 plus the carriage-return and line-feed) and MUST NOT exceed 1000 characters (998 plus the carriage-return and line-feed).

However, some source, especially various mail clients from Microsoft, including Outlook, IIS, etc. do not respect this limit at all. This resulted in the message being rejected with an MTA error and logging of lines like this:

T=remote_smtp: message has lines too long for transport

It is still somewhat uncommon for regular plain-text email to exceed this length unless it has several dozen recipients or other long headers, so most mail would go through, but a small percentage would not and the response to the sender was an MTA error code rather than a proper SMTP response.

To resolve this issue, we published an update which allows you to resolve this problem in a couple of ways. Since the issues actually comes from the remote sender, there isn't really a correct option to fix this, so you are able to choose how you prefer to do so:

• Reject these messages. According to the SMTP standard, this is actually the correct solution. The messages should be refused for containing invalid SMTP content. This will enable a proper 5xx rejection code rather than just an MTA error. However, since Microsoft clients represent a rather large proportion of desired mail, it is not practical for most users if/until Microsoft fixes their mail clients. It is unlikely that they will do so and if they do it will be many years for all users to be upgraded to the fixed version.
• Ignore the invalid long lines. This is to return to the pre-4.95 behaviour and simply process all mail and pass it along. This is now the default behaviour. The issue with this solution is that MailCleaner is not responsible for sending invalid SMTP mail as well. If you are relaying to another machine which does enforce the limit, they will correctly reject messages from MailCleaner. This now includes default Exim configurations which are also quite common (eg. unmodified versions of cPanel).
• Fold long lines. This will accept all long lines, but break them across multiple lines of less than 998 characters. This is arguably the best solution because it allows you to accept all mail while also being able to relay valid SMTP which should be accepted by all remote SMTP machines. However, it does introduce possible complications with DKIM signing. Since parts of the message could be modified, this may fail and the messages may be seen as spam. Outgoing messages signed by MailCleaner will get folded before signing, so this is not a problem. We also discourage using DKIM checks on back-end servers since we already add headers to inbound mail, so it could already be broken. If you ignore DKIM/DMARC checks for MailCleaner on back-end machines, this is probably the option to choose.

In all cases, messages with long lines now provide warning messages in the logs.

From Configuration->SMTP->SMTP checks, there is the setting What to do with invalid long lines

This provides the three options discussed above:

• Ignore errors; relay invalid SMTP (default)
• Fold long lines; makes SMTP vaild, but may break SMTP signing
• Reject invalid long lines

The first two options above will set a default line length to 5000000 (5 million) characters. This should be far an above any what any legitimate sender should ever use, however, it is still possible that it is too short for some badly configured MTAs. To set an even higher, or lower, value, you can write a number to the file /var/mailcleaner/spool/mailcleaner/exim_max_line_length

Then restart all 3 MTA (Exim) services.

If the sender has an outdated or incomplete SPF record, and you are enforcing Configuration->SMTP->SMTP checks->Reject wrong SPF (fail result), this will result in domain with a -all designation having their emails blocked. Even without this setting enabled, if they also have DMARC with a 'reject' policy, and if you are using Configuration->SMTP->SMTP checks->Honor DMARC reject policy then it will also be rejected.

The proper solution to this issue is for the sender to update their SPF record. However, if this is not practical, you can ignore bad SPF results for specific sending hosts from Configuration->SMTP->SMTP checks->Don't check these hosts for SPF or DMARC.

You will see this message in the Incoming MTA logs, like:

Incoming MTA stage: 2019-07-09 17:07:46 H=(hostname.domain.tld) [xxx.xxx.xxx.xxx] [email protected] temporarily rejected RCPT [email protected]: Could not complete sender verify"

MailCleaner attempts to validate the sending email address before accepting the email. We do this by verifying if the sending domain has correctly set his MX field. You can disable this behaviour from Configuration->SMTP->SMTP checks->Verify sender domain, but it is recommended that you leave it intact and instead contact the sender to have them set up proper MX records for the sending domain, or change the sender address to one with MX records.

The exact test is described here.

RBL can be used in several places in MailCleaner so if you are having too many problems with a specific IP/hostname/URL or other content being blocked by a specific RBL source, it may be worth considering disabling that source at the SMTP level and instead enable it at the PreRBL, UriRBL or SpamC level as described here.

You can also exempt specific sources from being evaluated by RBLs.

First, check that the message was not also detected as spam. It is possible that we are ignoring the newsletter classification, but blocking it for being a spam.

In the case that the default setting is "Allow Newsletters by default", check that the recipient still has this set for themselves. They may have overriden this default behavior from Configuration->Address settings->For each message detected as newsletter.

You can see this setting by accessing the User Interface as that user by going to Management->Users->[search by user]->Actions, then choose Open user's interface.

• your message was stopped because of UriRBL : this is the most efficient tool versus phishing we have, please ask us to investigate if it is legitimate that the URI that was stopped is listed. If the URL is not harmful anymore, we unlist it.

You may wish to make adjustments to the SpamC scores or create your own as described here.

For False-Positives and False-Negatives for normal spam and phishing emails, see this section in the User's FAQ.

If you are experiencing issues with too many viruses or dangerous content making it through the filter, you may wish to check the various Content Protection filters:

• Configuration->Content Protection->Global settings->disable content controls in archives - Should not be checked. If this is checked, MailCleaner will NOT look for viruses inside archives.
• Configuration->Content Protection->Global settings->Content control maximum archive depth - Should be set to about 4. All archives nested more deeply than this value are blocked. Archives nested more deeply than 4 level are probably trying to avoid being scanned and can be assumed to be dangerous.

Be sure to block attachment filenames that your user's do not have a legitimate need to receive from Configuration->Content Protection->Attachment name. We recommend, at a minimum:

• .bat$
• .cmd$
• .com$
• .exe$
• .hta$
• .mhtml$
• .pif$
• .reg$
• .scr$
• .vb[es]$
• .jse$
• .ws[cfh]$

You can allow identify attachments by their Mime Type using Configuration->Content Protection->Attachment type. These will do a string search within the description of the file type description (the output of file in Linux), and we recommend blocking at least:

• ELF
• executable
• Registry
• script
• self-extract

If you don't currently have the ClamAV Unofficial Signatures enabled, you can enable them as instructed here.

See the info in the User's FAQ for normal false-positive reports. Note that for ClamAV, you can address false-positive reports directly to them. However, ensure that the signature does not contain 'UNOFFICIAL' as these signatures do not come from ClamAV themselves. MailCleaner Enterprise Edition provides a wide variety of additional signatures from SecuriteInfo. Please open a ticket if you find a problematic signature with this in the name.

If you would like to learn about the virus signature, you can search for it like:

mailcleaner:~$ /opt/clamav/bin/sigtool --find-sigs=Sanesecurity.Malware.8890 --datadir=/var/mailcleaner/spool/clamav/ | /opt/clamav/bin/sigtool --decode-sigs
VIRUS NAME: Sanesecurity.Malware.8890
TARGET TYPE: MAIL
OFFSET: *
DECODED SIGNATURE:
<a href="http://{WILDCARD_ANY_STRING(LENGTH<=50)}index{WILDCARD_ANY_STRING(LENGTH<=10)}html">READ FULL STORY</a>

As shown here, leave off a .UNOFFICIAL suffix, if it exists and search by shorter strings, if you don't find anything. This first searches for the signature using --find-sigs, which returns the file that the rule is found in and a machine readable version of the rule. It is then piped to the --decode-sigs option which will show it in a human-readable form.

Note that some newer rules (eg. in the yara format) are fairly human readable and you can just find the definition by locating which file the signature is in with grep -R <signature> /var/mailcleaner/spool/clamav, then browse that file.

You can whitelist a pernicious signature by inserting it's name into a .ign2 file within the signature directory /var/mailcleaner/spool/clamav/. DO NOT use an existing .ign2 file, unless it is one you created. The existing files well be overwritten during signature updates. You need to restart clamd and clamspamd to apply the whitelist.

You can manually scan a message or attachment again with clamd by either uploading it to the MailCleaner server or locating it's path in the content quarantine (as shown here):

/opt/clamav/bin/clamdscan --config-file=/usr/mailcleaner/etc/clamav/clamd.conf /var/mailcleaner/spool/mailscanner/<date>/<exim-id>/<file>

Only an administrator can release items from the Content Quarantine.

When an attachment is stripped, the recipient will find that the email instead contains an attachment like AttentionVirus.txt. This will contain details of the stripped attachment including an ID like:

host1:20240101/1abcde-00000000000-1234

You should request this ID from them if they did not already provide it. Then you will be able to use it to search for the message in Management->Content Quarantine and use your judgement as to whether it is safe to release. Otherwise, you can search based on their recipient address (and the sender, if they provided it), but this increases the risk that you might accidentally release an attachment that should not have.

If you need to inspect the item before releasing it, it will be located with a directory with the name of the ID (less the hostname) within /var/mailcleaner/spool/mailscanner/quarantine.

By default, MailCleaner uses a self-signed certificate when you enable HTTPS for the web interface and when you enable STARTTLS for SMTP connections. It is strongly encouraged that you don't leave this in place for production use. If you do, users will see warnings when they connect to the web interface, and many servers will SMTP connections with your machine(s).

You can configure any valid certificate provided by a recognized certification authority (RapidSSL, Thawte, Comodo, etc.), or generate your own with a service like Let's Encrypt to resolve these issues.

You will generally be provided with:

• A private key
• A 'full chain' certificate or an intermediate chain certificate(s) and a local server certificate

The 'full chain' certificate is just the local certificate and the intermediate certificate(s) in the same file.

Each certificate is formatted

-----BEGIN CERTIFICATE-----
<CERTIFICATE>
-----END CERTIFICATE-----

The format expected by Apache and Exim are different, so you just need to understand that you will either need to split the first certificate from the full chain, or append the intermediates to the local, as described below.

In Configuration->SMTP->SSL/TLS you will need to select Enable SSL/TLS if you have not already. Then there will be two fields:

• Encoded SSL certificate - paste in the full chain (the local server certificate, followed by the intermediate chain certificates)
• Encoded SSL private key - paste in the private key

Upon submitting, you will need to restart Exim Stage 1 and Exim Stage 4 on each node for the new certificate to apply. You may want to test your certificate with an online tool.

Note that it is important that the hostname of the certificate, the public hostname of your machine (HELO name) and the hostname provided by your MX record all match. If you have multiple machines with unique private hostnames behind the same public hostname, you may need to force a different HELO name.

See this article for other changes which can ensure that SMTP connections are more likely to be accepted.

In Configuration->Services->Web Interface you will need to select Enable SSL (HTTPS) if you have not already. Then there will be three fields:

• Encoded SSL certificate - paste in the local server certificate only
• Encoded SSL private key - paste in the private key
• Encoded SSL certificate chain - paste in all of the intermediate chain certificates

Upon submitting, you will need to restart Apache on each node for the new certificate to apply. Your browser should tell you if the certificates appear to be valid, although you may need to do a hard refresh (Ctrl+Shift+R) for the new certificate to be seen. The hostname of the machine will need to match one of the Subjects of the certificate.

There is a tool /usr/mailcleaner/tools/update_cert.sh which can automate the deployment of a new certificate by directly inserting into the correct fields of the database and restarting all necessary services. This script requires that the new full chain certificate be provided in a single file, as well as the private key in another and is called like:

/usr/mailcleaner/tools/update_cert.sh /path/to/fullchain.pem /path/to/private.key

You can add this procedure to any automation tools that you might use, such as with a post installation hook for Let's Encrypt's Certbot.

When you set the feature Configuration->Content Protection->Message format controls->Password protected archives, all password protected archive files (eg. .zip files) will be blocked as if they contain viruses.

You may have the occasional domain which has a legitimate use-case for sending these types of files. In fact, we will actually recommend that you have senders place attachments into a password-protected archiveif they are having a problem with it getting caught by another Content Protection rule, or a virus false-positive. In order to allow this, you can whitelist some domains so that this setting is not enforced.

To do so, please connect to your MailCleaner server (if you use MailCleaner in cluster mode connect to the master). You will need to use the script /usr/mailcleaner/bin/manage_wh_passwd_archives.sh in order to add or remove these exceptions.

To add domain.tld to the whitelist for this feature, use /usr/bin/mailcleaner/manage_wh_passwd_archives.sh add domain.tld.

To remove domain.tld to the whitelist for this feature, use /usr/mailcleaner/bin/manage_wh_passwd_archives.sh del domain.tld

For each server, you then need to restart the Filtering MTA (MailScanner). If you prefer to do it on command line of each machine with /usr/mailcleaner/etc/init.d/mailscanner restart ms.

Similar to the previous section, you can exempt specific domains from being checked for the Configuration->Content Protection->HTML controls checks that all other mail will be held to.

To manage the whitelists for HTML controls use on the master node of your cluster, /usr/mailcleaner/bin/manage_wh_html_controls.sh.

To add an address to the whitelist use /usr/mailcleaner/bin/manage_wh_html_controls.sh add [email protected]

To add a domain to the whitelist use /usr/mailcleaner/bin/manage_wh_html_controls.sh add domain.tld

To delete an entry from the whitelist use /usr/mailcleaner/bin/manage_wh_html_controls.sh del entry

To see all whitelists for this feature use /usr/mailcleaner/bin/manage_wh_html_controls.sh show

For each server, you then need to restart the Filtering MTA (MailScanner). If you prefer to do it on command line of each machine with /usr/mailcleaner/etc/init.d/mailscanner restart ms.

Except where specified (eg. cluster-related values), these settings are configured by /usr/mailcleaner/scripts/installer/installer.pl when you initially set up the machine (the script that runs on first login only). They should not be changed manually unless your really know what you are doing. Here is what they represent:

• SRCDIR (default: /usr/mailcleaner) - This is the directory of the MailCleaner source repository. This should always be the default value. Historically it has been /opt/mailcleaner. That should now be a symlink to /usr/mailcleaner for compatibility. Most parts of the code will reference $SRCDIR to know the source home, or will use a relative path which makes this prefix irrelevant, however, this path has been the expected path for so long that some code and documentation is written to require it to be there. It would be very dangerous to change this.
• VARDIR (default: /var/mailcleaner) - This is the MailCleaner data directory. It is where the logs, database data, and quarantines, as well as other flag files, etc. are located. Similar to the previous point, this should not be changed.
• MCHOSTNAME - This is the hostname of the appliance you are connected to. It is set up by `` and is use in various places.
• HELONAME - Similar to the previous point, this is a customizable value to be used just for the HELO name in the SMTP banner.
• HOSTID - The unique ID for this host within a cluster. This is also set in installer.pl. Enterprise licenses are provided which expect a specific HOST ID and if you want to substitute out a clone of the machine, it will need the same ID.
• DEFAULTDOMAIN - Optional setting, as configured and described here.
• ISMASTER - Boolean (Y or N) indicating if this machine serves as the master node within a cluster.
• MASTERIP - Required for Slave nodes only. This is the IP or hostname of the Master node (configured with /usr/mailcleaner/scripts/configuration/slaves.pl).
• MASTERPWD - Required for Slave nodes only. This is the database password of the Master node (configured with /usr/mailcleaner/scripts/configuration/slaves.pl).
• MYMAILCLEANERPWD - The local database password for this host. On the master node, this will match the MASTERPWD, but this is not necessarily true for the slaves.
• CLIENTID - For Enterprise Edition, this is the Client ID you entered when registering. It will be used to fetch premium updates.
• RESELLERID - Same as previous for the Reseller ID value.
• REGISTERED - Registration status of your machine. 0 = not registered; 1 = registered for Enterprise Edition; 2 = registered for Community Edition.

Each of the Address Verification and Authentication sections of the Domain configuration wizard have mechanisms to test whether the connector works. If it does not seem to be working, the best way to debug this is to try to manually run an ldapsearch with the same details from the commandline. This should have more useful feedback, like:

ldapsearch -h ldap.domain.com -b "dc=BindDN" -D BindUser -w 'Passw0rd'

This may show a problem like:

ldap_bind: Invalid credentials (49)
        additional info: 80090308: LdapErr: DSID-0C0903A9, comment: AcceptSecurityContext error, data 52e, v1db1

which indicates that the password does not match for those bind parameters. Or, you may see that it times out trying to connect.

For 'Address Verification' the -D and -w values should be those of the Bind User and a valid connection should return an LDIF with all of the directory details available to that user. This will be quite long, so you can pipe it to the less command to page through the results. The user you are trying to verify should be found in the LDIF output. Our test in the WebUI expects the 'postmaster' address to exist and for the randomly generated address not to exist.

For 'Authentication' you can substitute the Bind User with the user you are trying to test the authentication for and their password. The actual Bind User details provided just check to see if the address exists (as done in the verification step above) before bothering to make authentication attempts against it. With the actual user address, it should return just their LDIF details.

If you have access to MailCleaner via SSH, but not HTTP(S), you can set up a tunnel to it from whatever machine has SSH access. This can be done like:

ssh [email protected] -L 4443:localhost:443

Then you should be able to access the WebUI via the tunnel if you connect to port 4443 on the machine where the tunnel was set up. If this was your localhost, for example, you can point your browser to:

https://localhost:4443/admin/

Typically, you can manage Web Admin accounts via Configuration->Accesses. However, if you don't have lost your admin credentials, you may need to reset or create the account manually in the database.

For all of these steps, you will first need to connect to the database on the Master node (if in a cluster):

mc_mysql -m mc_config

First, lets just view your admin account since you may just have the wrong login name. Then you can list the members of the administrator table:

SELECT * FROM administrator;

If you would like to create a new admin user, you can do so like:

INSERT INTO `administrator` VALUES ('new-admin', md5('password'),'1','1','1','1','1','*','0','default',NULL);

This will create the user 'new-admin' with the maximum 'admin' permissions. It will be easier to reduce this with a different 'admin' account via the Accesses page, rather than to craft the less privileged user manually.

If you would like to just update an existing account, for example to reset the password, you would do so like:

UPDATE  `administrator` SET password = md5('password') WHERE username = 'admin';

It is best to set a temporary password then login and immediately change it so that the md5('password') in the mysql history does not contain a legitimate password (pertinent mostly for machines with shared admin access).

egrep -o -e "[[:digit:]]+\.[[:digit:]] seconds" /var/mailcleaner/log/mailscanner/spamd.log | egrep -o "[[:digit:]]+\.[[:digit:]]" | awk '{ sum += $1; n++} END { if (n > 0) print sum / n; }'

grep -e "<=" /var/mailcleaner/log/exim_stage1/mainlog | wc -l

grep -e "rejected RCPT" /var/mailcleaner/log/exim_stage1/mainlog | wc -l

grep -e "Profiled SpamCheck" /var/mailcleaner/log/mailscanner/infolog | wc -l

First, you need to ensure that the actual check that the disk size assigned by the hypervisor is correct. Then, check that the OS detects the full size of the disk with:

fdisk -l

If you have recently increased the disk size and the full size does not yet show up, you will need to reboot.

The partition will need to be unmounted in order to resize it. If you want to resize the root partition, this means that you need to mount it from a recovery partition/image. For the /var partition, you can simply stop mailcleaner with:

/usr/mailcleaner/etc/init.d/mailcleaner stop

then unmount that partition. If you aren't sure of the partition name, you can check with lsblk. It should most likely be /dev/sda3 or /dev/vda3.

umount /dev/sda3

Then destroy the desired partition (this will not alter any files within the partition, it will only modify the partition table. The files will be discovered again when you re-create it):

fdisk -u /dev/sda

• p to print the current state
• d to destroy it and indicate its number
• w to write the changes

Recreate it (with fdisk still running, or re-run as above):

• n for new
• p for primary
• indicate its number (same as the one destroyed)
• make sure to use the same starting position as before (should be the default)
• w to write the changes
• q to finish

Force the kernel to take the changes into account immediately:

partx /dev/sda

If this fails, you have to reboot.

Finally, resize the partition table with (this extends the partition size to match the actual volume size that was provided):

resize2fs /dev/sda3