How Sender Rules Work

What Sender Rules exist

There are a variety of rules that can be created for sender email addresses. These include:

• Whitelist - Prevents messages from that sender from being marked as Spam
• Warnlist - Alerts users when a message from that sender was held in the quarantine
• Blacklist - Forces messages from that sender to be marked as Spam
• Newslist - Prevents messages from that sender from being marked as a Newsletter

Enabling Sender Rules

In order for sender rules to be evaluated and for the user to be able to create their own rules, each type must be enabled both globally (Configuration->Anti-spam->Enable access to...), as well as for the domain (Configuration->Domain->select domain->Filtering->Enable access to...).

Note that Newslists are always enabled.

Evaluation Order

The list types are generally evaluated with their precedence in the order above:

• If a rule that applies to that sender exists in the Whitelist, it will not be marked as Spam. The Blacklist will be ignored, and it will not be possible to activate the Warnlist because nothing will get quarantined. * See Newslist caveat.
• If it is not Whitelisted, but it is Warnlisted, an alert will be sent when and item from that sender is quarantined. Blacklisting will still be enforced to guarantee that it gets quarantined, but it will not prevent the Warnlist alert.
• A Blacklisted without either of the first two rules, will guarantee that the item is quarantined, and there will be no Warnlist alert.

However, the Newslist is evaluated separately. Even if the sender is whitelisted, an email can separately be identified as a newsletter. If the sender is not also in the Newslist, it will still be quarantined as a newsletter. If it is Warnlisted, the user will receive an alert about it being quarantined, despite the Whitelist. Instead, the Newslist prevents being identified as a newsletter, but will not prevent from being identified as spam. So, you need both rules to ensure that the sender is not caught for either.

Note that you have the option to not quarantine newsletters, this is effectively the equivalent to having all senders in the Newslist.

Where Are the Rules Created

These rules can be created at a variety of levels:

• Configuration->Anti-Spam - These rules apply to all recipients no matter what.
• Configuration->Domains->[select domain]->Filtering - These rules will apply to all recipients within that domain (must be enabled)
• Configuration->Management->Users->[search email address] - These rules will apply only to that recipient email address (must be enabled in the Domain's Filtering settings)
• User's Configuration->Address settings - These rules apply only to that recipient email address (must be enabled in the Domain's Filtering settings)
• Releasing a message from the Quarantine Report - Users will get a pop-up message to allow you to create a Newslist and/or Whitelist entry that applies to that recipient (if enabled for the domain)

Sender Rule Formatting

The recognized formats for sender rules have changed somewhat over time, so here are some guidelines on the currently supported formats.

Single sender address:

[email protected]

Note on this format. Support for sub-addressing has recently been added. This means that rules for the above address will also apply to senders with a '+' suffix such as:

[email protected]

Support has also been added to decode variable and obscured sender addresses such as SRS addresses so that the above rule would also apply to:

[email protected]

You can also apply the rule to an entire sender domain with:

@domain.com

Legacy support is provided for the '*' prefix that used to be required (*@domain.com).

These rules are actually matching sub-strings, so without the @, the previous entry would match any domain containing that string. So this rule would also apply to:

otherdomain.com

Sometimes this is useful if you want to match the domain itself, plus all subdomains, but this is slightly risky. You are better off including the domain itself then a separate rule for any subdomain:

.domain.com

Limited regular expressions are also supported. This includes:

• ^ - Matches the start of the address
• $ - Matches the end of the address
• * - Wildcard match for anything in the middle of the address (not equivalent to * in a standard Regular Expression, since it does not match the previous character, ie. .*).

You can use something like:

^sender*@*domain.com$

To match any sender starting with sender and ending with domain.com. We split the search pattern at the last @ and search the "local part" of the address and the domain separately, so the @ is necessary. ^sender*domain.com$ will not work.

Wildcards at the beginning or end are not necessary since they will, again, be matched because we search for sub-strings. You may wish to include the start and end markers with every rule as a best practice to avoid the ambiguity of say: [email protected] matching [email protected] when the end anchor is not used, however the risk with this is limited.

Where Are Sender Rules Stored

There is a table in the 'mc_config' database which is called 'wwlists' (originally short for White/Warn Lists). This contains all of these rules distinguished by the column 'type'. If you ever need to review or amend the rules in bulk, it can be done with SQL commands there.

Bulk insertion

Our advice on Blacklists and Whitelists is to use them as little as possible, so we discourage inserting too many and instead trying to remedy mis-classifications using other adjustments, including sending reports to us so that we can better train the filters.

If you do want to insert these rules in bulk, you can connect to your master MailCleaner node (in case of cluster), then use the tool /usr/mailcleaner/tools/insert_bulk_wwlists.sh. Running it without any options will provide instructions, more clearly described here:

The script requires that you provide it with a file in a very specific format, namely:

sender recipient type

one per line, like:

[email protected] [email protected] white

If you would like to apply the rule to an entire recipient domain, it you need to include the '@' sign at the start, like:

[email protected] @mydomain.com white

If the rule is for all domains please use --- as the recipient:

[email protected] --- white

The type can be one of:

• white
• warn
• black
• wnews

The fields in the file should be separated by a single space.

Once this file is properly formatted, you just run the script with a path to that file:

/usr/mailcleaner/tools/insert_bulk_wwlists.sh /path/to/bulk_entry_file.txt

Testing Sender Rules

There is another script /usr/mailcleaner/bin/check_match_wwlists.pl to check if there is an existing rule that would hit for a specific sender and recipient combination. This uses the same mechanism for querying the rules table that gets used by SpamHandler when the delivery decision is being made, so the results should be identical.

It requires that the arguments be an exact sender address, then an exact recipient, like:

/usr/mailcleaner/bin/check_match_wwlists.pl [email protected] [email protected]

It will return all rules that were found, including more general sender and recipient rules like ones that apply to @mydomain.com as a whole.

Warning: Too many rules

If a user or admin goes overboard with creating too many rule, it can cause the webpage to fail to load. This happens when the max_input_vars value is exceeded in the PHP configuration. This is a limit on the number of configurable page elements which grows every time that a new entry is added. If you reach the point where this is happening, we would strongly encourage you to remove most of these rules and make other adjustments. No recipient should have thousands of senders who they cannot afford to miss mail from, or thousands of senders who couldn't likewise have spams caught by other means.

If, for some reason, this is a supported use-case on your appliance, you can increase the max_input_vars value in /var/home/jpm/coding/MailCleaner/MailCleaner/etc/apache/php.ini. The default is 3000 which should be more than enough for anyone.