TLS SSL Certificates - SpamTagger/SpamTagger-Plus GitHub Wiki

Note: Most of the information in this guide currently applies to MailCleaner appliances. A note exists at the end of the document regarding changes in SpamTagger Plus. When SpamTagger Plus is released, these will switch places.

TLS/SSL Certificate Usage in MailCleaner

A TLS/SSL certificate and key pair can be used to encrypt both web traffic (HTTPS) and email traffic (STARTTLS via SMTP). MailCleaner automatically deploys the currently installed certificate from the database to the correct location on the filesystem immediately before each of the relevant services starts. You should use the built-in tools to manage these certificates and keys rather than trying to manually install the certificates as they will be overwritten if you do.

Configuring TLS/SSL Certificates via the Web UI

The simplest way to configure TLS/SSL certificates for use with the Web UI itself and for secure SMTP connections is to use the wizard for each in the web interface.

Configuration->Services->Web Interface

Once you enable "Enable SSL (HTTPS)" there will be three fields which must be populated to have functional HTTPS. These are:

  • Encoded SSL certificate - This is your local certificate. If you only have a fullchain certificate, it is the first to appear in that file, inclusive of the first -----BEGIN CERTIFICATE----- through to the first -----END CERTIFICATE-----.
  • Encoded SSL private key - This is the private key file. It will be the entire file beginning with -----BEGIN PRIVATE KEY----- and ending with -----END PRIVATE KEY-----.
  • Encoded SSL certificate chain - This is all of the upstream certificates which provide the chain of trust from the organization which provided you with your local certificate all the way up to a root certificate that is natively trusted by browsers and MTAs. If you only have have a "fullchain" file, this will be all of the remaining contents of the file after the local certificate described above. Otherwise, if you have a chain file, it will be the complete contents of that file, consisting of one or more "intermediate" certificates.

Configuration->SMTP->TLS/SSL

After you have enabled "Enable SSL/TLS" there will be two fields which must be populated to have functional HTTPS. These are very similar to the fields described for the Web Interface, except that "Encoded SSL certificate" should contain the "fullchain". If you only have a local certificate and a "chain" certificate, you should paste them in one after the other.

Updating TLS/SSL Certificates via the CLI

Because TLS/SSL certificates should have a regular expiry between around 30 days and 1 year and an expired key can cause interruptions in service, it is often convenient to automate the renewal of these certificates. For this, there is a tool included with MailCleaner which can be executed like:

/usr/mailcleaner/tools/update_cert.sh fullchain.pem privkey.pem

The actual file names/extensions don't matter as long as the text of the fullchain certificate is at the location of the first argument and the text of the private key is at the location of the second argument.

This script will extract the local certificate from the fullchain then insert the relevant data into the database columns associated with the 5 fields discussed in the previous section. It will then restart each of the relevant services.

If you are doing this for the first time, immediately after installation, you may still need to enable TLS/SSL for each service in the Web UI if you have not done so already, but this can be done after using the update_cert.sh script.

It is only necessary to run this command on the master node of a cluster. However, if you have a cluster, the new certificate will not become active for the MTAs on each of the slave nodes until those services restart. This will happen automatically at midnight, so as long as you don't wait until the day the certificate is set to expire, you should not encounter any downtime.

Sourcing a TLS/SSL Certificate

Warning about self-signed certificates

MailCleaner ships with a self-signed certificate which will allow encrypted traffic but with guaranteed warnings to access the Web UI and a high likelihood for remote mail servers to temporarily abandon SMTP connections. Those servers will continue to retry sending the messages for a day or more until they see a valid certifictae.

Self-sign certificates are not secure, especially the one shipped with MailCleaner which is known to any potential man-in-the-middle attacker. You should endeavor to replace this certificate immediately upon installing MailCleaner. It is possible to generate a new self-signed certificate which would largely make you safe from man-in-the-middle attacks, but the warnings and abandoned SMTP connections will not go away if you do this.

Instead, it is very important that you source a certificate with a full chain of trust up to a root certificate authority trusted by browsers and MTAs.

Using the correct certificate Subject

In order to avoid warnings and abandon SMTP connections similar to those described in the previous section, you need to ensure that the public hostname of your machine(s) are among the list of Subject names of the certificate. If you create the certificate for only example.com, but your MailCleaner machine is mailcleaner.example.com, it will be seen as invalid.

For a single-node cluster, this should be as simple as providing the exact hostname of your appliance when generating the certificate.

Cluster with distinct public hostnames

If you have a multiple node cluster and each node has a distinct public hostname, you will need to ensure that the certificate includes all of the hostnames as a Subject. It is necessary that the A record, the HELO name presented by the node and the Subject of the certificate all match, otherwise it will still result in abandoned SMTP connections.

Load-balanced cluster with single MX record

One common DNS configuration is to have a single MX record for domains using MailCleaner. For example, you may have just one MX record for mailcleaner.example.com and have that A record point to a load-balancing firewall with all of the nodes behind it. If this is your configuration, then you can just create the certificate with the single Subject matching the shared MX record hostname.

However, you need to use a bit of caution with this setup. You will need to:

  • Set Configuration->Services->Web Interface->Base URL to match the same Subject name. HTTPS traffic to that hostname should be routed to the master node.
  • Set Configuration->Base system->DNS settings->Force HELO / EHLO identity with to match the Subject name. This will cause each member of the cluster to identify themselves with the same hostname.

Commercial Certificate Authorities

If you have an existing relationships with a certificate authority you may be able to have them generate a new certificate for your MailCleaner machine(s) at no extra cost. In this case, they should provide you with all of the files described above and you can then copy them to the relevant fields in the Web UI. These certificates usually last a year and require some amount of intervention from you when you receive them anyways, so this process is not easily automated.

Let's Encrypt

Let’s Encrypt is a free, automated, and open Certificate Authority (CA), run for the public’s benefit, which provides SSL certificates at no cost. They provide useful tools to make generating and renewing certificates simple.

The main problem with Let's Encrypt for use with MailCleaner is that it's tools are no longer supported by the version of Debian used by MailCleaner, so you cannot generate the certificate on your MailCleaner machine directly (at least not without use of a Docker container or other difficulty).

Instead, you can generate the certificate on a different machine and then deploy it to MailCleaner by pasting the contents of the files into the fields in the Web UI or by copying the files to the MailCleaner machine and running the update_cert.sh script. The latter would look like:

scp /path/to/fullchain.pem [email protected]:/root/
scp /path/to/privkey.pem [email protected]:/root/
ssh [email protected] -t /usr/mailcleaner/tools/update_cert.sh /root/fullchain.pem /root/privkey.pem

The commands can be used with the certbot --post-hook argument to automatically deploy the certificate when it is generated/renewed, or within a cron task on on the machine which is scheduled to renew the certificate, shortly after that is done.

The machine generating the certificate will need to have a public SSH key installed within /root/.ssh/authorized_keys on the master node (in this example this is mailcleaner.host).

There are a number of ways to generate Let's Encrypt certificates on a third-party hosts, as described below.

Certbot

Any reasonably modern Unix-like machine should be compatible with Let's Encrypt's own certbot program. This can generate a certificate like:

certbot certonly --standalone -d mailcleaner.host

where mailcleaner.host is the "Subject" of the certificate which should match the hostname of the MailCleaner machine. You can list multiple names, if necessary, for each member of your cluster.

Let's Encrypt provides a variety of verification mechanisms to confirm that you actually own the domain that the certificate is being requested for. The main methods for this are DNS-based and will require that the domain that you are requesting the certificate for points to the machine generating the certificate or a similar mechanism. This is made simple if the most generating the certificate shares the exact hostname of your MailCleaner machine by way of serving as a NAT firewall or similar. Please see the Let's Encrypt certificate for more details.

Pfsense

Pfsense provides a graphical interface for generating certificates. You can do so as follows:

  1. Log into pfsense and install the following packages from System -> Package Manager:
  • acme
  • cron

  1. Go to Services -> Acme Certificates, click on the "General Settings" tab, and ensure that "Cron Entry" and "Write Certificates" is ticked.

  2. Click on the "Account Keys" tab, click "Add", and fill in the following fields:

Name = Staging Description = Staging ACME Server = Let's Encrypt Staging (For TESTING purposes) Email Address = use an email address you want to receive important info to

Then click on "Create new account key", and assuming that gets a tick-mark, click "Register ACME account key" (which should also get a tick-mark)

Now add a second account key with the following fields:

Name = Prod Description = Production ACME Server = Let's Encrypt Production (Applies rate limits to certificate requests) Email Address = once again use an email address you want to receive important info to

Then click on "Create new account key", and assuming that gets a tick-mark, click "Register ACME account key" (which should also get a tick-mark)

  1. Click the "Certificates" tab, click "Add", and fill in the following fields (leave others at defaults):

Name = Antispam_LE_Cert ACME Account = Staging Private Key = 2048-bit RSA

in the "Domain SAN list", click the Add button and fill in as follows:

Mode = enabled Domainname = your.spamtagger.fqdn (where this is your actual SpamTagger fully qualified domain name) Method = DNS-Manual

Put in any other alternate names as additional entries into the Domain SAN list (this will allow the certificate to cover the use of these domains alongside your primary domain), just use the up and down arrows to ensure your primary domain is at the top of the list.

DNS-Sleep = 120 Certificate renewal after = 30

  1. Bring up the portal page for your DNS so you can add entries, then click the "Issue" button on the pfsense Certificates tab. I say this because this process can fail and need amending if you take too long between hitting Issue or Renew and having a valid DNS entry in place, so best to be prepared.

When you hit issue, you will get a page of green text as a reply. This will list what TXT record and the value of the TXT record should be for each Domain SAN you configured in step 4. You will need to take this information and enter it into your domain DNS portal, and once done hit the "Renew" button. If you are successful you will get a long bunch of green text back which would include the text version of the certificate with a line before it saying "Certificate Success". If you get any other response then you will need to read the green text, amend, and hit renew again.

  1. When you are successful, go to the Certificates tab and create a second certificate entry with every field the same as the first EXCEPT for changing the ACME Account from Staging to Prod. This is because the production version of Let's Encrypt is rate-limited on giving out certificates so you need to ensure you have all the variables correct prior to requesting a production certificate, otherwise the process could become inordinately long.

  2. Repeat step 5, but for the Prod certificate.

Once you are successful, go to Services -> Cron and you will see all the scheduled tasks that pfsense performs. Please try to avoid deleting any. You should see an entry mentioning acme_command.sh "renewall" which is the 30 day renewal process that has been set to occur. We now need to add a Cron job to copy the certificate and its chain to Spamtagger:

  1. Log into the shell of pfsense either by SSH or locally, and run the command ./ssh-keygen

Press enter to accept the default path (make a note of this path though as it will be used in the next steps - it's in the format id_xxxxx and I'll refer to xxxxx in the subsequent steps) Press enter to leave the passphrase empty (this is important)

  1. Copy the public key to SpamTagger using:

ssh-copy-id -i ~/.ssh/id_xxxxx.pub root@

  1. Run the following command

chmod 600 ~/.ssh/id_xxxxx

  1. Test the file transfer with:

scp -i /root/.ssh/id_xxxxx /conf/acme/Antispam_LE_Cert.crt /conf/acme/Antispam_LE_Cert.key /conf/acme/Antispam_LE_Cert.fullchain root@

This should be almost instantaneous and you can check the contents of the Antispam_LE_Cert.key file on both locations to ensure they are the same.

  1. Once the file transfer has been checked and is working, go back to the Services -> Cron page in pfsense, click Add, and fill out the details as follows:

Minute = 30 Hour = 3 Day of the Month = * Month of the Year = * Day of the Week = 7 User = root Command = scp -i /root/.ssh/id_xxxxx /conf/acme/Antispam_LE_Cert.crt /conf/acme/Antispam_LE_Cert.key /conf/acme/Antispam_LE_Cert.fullchain root@

  1. Finally, we now need to add a Cron entry into SpamTagger to run the update_cert.sh script on a weekly basis. The format for that command should be:

update_cert.sh Antispam_LE_Cert.fullchain Antispam_LE_Cert.key

This should stop the associated SpamTagger services, copy over the certificate, and restart the services.

  1. You can now inspect the certificate by browsing to the SpamTagger website and inspecting its certificate. You should now be in possession of a valid Let's Encrypt certificate.

Changes in SpamTagger Plus

A few things will be different in SpamTagger Plus:

  • A unique self-signed certificate is generated during the installer script which is automatically run upon the first boot of the appliance. This should still be replaced ASAP, but is somewhat more secure that MailCleaner.
  • Commands and services changed slightly, so the exact contents of update_cert.sh are a little different.
  • Some third-party documentation used to recommend overriding the default certificate/key locations by editing the Apache2 and Exim configuration templates directly. These files will be read-only, so they cannot be edited. It will be possible to provide a custom template at a different location, but it is still recommended to just use the built-in database mechanism to store and write the certificates.
  • SpamTagger Plus will be using Debian 13 out of the gate and will aim to provide first-class Let's Encrypt support by integrating certificate generation and renewal into the product. This will probably start as another item in the installer, but will eventually be added to the Web UI.