Domain Checks Failed Lookup Read As Missing - edmozley/freeitsm GitHub Wiki

A failed DNS lookup was reported as a missing record

Fixed in 3.16.0 · Reported by email by a user setting up the Domains module · Module: Domains

The short version

When the server running FreeITSM could not get an answer from DNS, the Domains checks reported "No SPF record" and "No DMARC record" - and graded the domain down for them - although both records were there. A tool like MXToolbox showed them plainly.

A lookup that fails is now reported as a failed lookup: once, with the error, costing the domain nothing, and with no grade until the checks can run. And on Windows the Automatic resolver no longer gives up when Google's DNS-over-HTTPS cannot answer: it tries Cloudflare's, then the server's own resolver.

What you saw

On the domain's Security and email tab:

❌ No DMARC record - Start with v=DMARC1; p=none; … at _dmarc to collect reports
❌ No SPF record - Without SPF, receiving servers cannot tell your real email from forgeries

and a grade of about C. Checked from outside, the same domain had both:

v=spf1 include:spf.protection.outlook.com -all
v=DMARC1; p=quarantine; adkim=r; aspf=r; rua=mailto:…

What was wrong

A DNS question in the Domains module is asked by domainDns(), which answers in one shape whichever resolver it used:

@return array{ok:bool, nxdomain:bool, records:array, error:?string}

ok: false means "I could not get an answer". records: [] means "nothing here" - and it is also what comes back when ok is false.

The email checks did not ask domainDns() directly. They used a helper that returns only the matching records:

function domainTxtStartingWith(string $name, string $prefix, string $resolver): array
{
    $r = domainDns($name, 'TXT', $resolver);
    return array_values(array_filter($r['records'], fn($t) => stripos(ltrim($t), $prefix) === 0));
}

ok is thrown away on the second line. So:

$spfs = domainTxtStartingWith($name, 'v=spf1', $resolver);   // [] - could not ask, or no record?
...
if (!$spfs) {
    $add('spf_missing', 'email', 'fail', 10);                 // "No SPF record"

An empty list was read as "no record", whatever the reason it was empty.

Why only those two

This is the part worth noticing. When every lookup failed, the checks did not all fail the same way:

Check	On a failed lookup	Why
Name servers	said nothing	it tested `nxdomain`, then `ok && !records` - a failure matched neither
MX	"No MX" - an info, no points	an empty list is a legal answer for a domain with no mail
CAA	said nothing	it only acts when `ok`
SPF	❌ fail, 10 points	empty list = missing
DMARC	❌ fail, 12 points	empty list = missing

So a server with no working DNS produced a page that looked like a domain with a specific email problem, not a server with a general one. That is what made it convincing - and why the report was "the SPF and DMARC lookups aren't working", not "nothing is working".

A run against a real failing lookup reproduces the reporter's screen exactly: dnssec-failed.org is a public test domain that validating resolvers refuse to answer for, and with the old code its checks came out as mx_none, spf_missing, dmarc_missing, grade C.

Why the lookups failed in the first place

FreeITSM cannot know that from here, but the setting explains where it would bite. The Automatic resolver uses Google's DNS-over-HTTPS on Windows servers - PHP's own resolver on Windows gives up on large answers - and the server's resolver elsewhere. On Windows there was no second opinion: if the server could not reach dns.google, every question failed. Networks that filter DNS-over-HTTPS - common in schools and companies, because it gets around their DNS filtering - do exactly that.

It was more than two checks

The same "empty means absent" reading was in several places, all in domainRunChecks():

SPF and DMARC - the visible ones.
Lock-down checks for parked and defensive names - "SPF must be v=spf1 -all" failed on a lookup failure too.
DKIM - "DKIM selector not found" (warn) for a selector that simply could not be asked about.
MTA-STS / TLS reporting - "not set up".
SPF includes - an include whose lookup failed was counted as a broken include.
Extra hosts - "has no address" when the A lookup failed.

The look-alike scanner (includes/domains/watch.php) reads a failure as "not found", which can only ever miss a look-alike, not invent one, so it was left alone. Change detection was already right: an empty value is "unknown", never "removed", so nobody was ever told a record had disappeared.

The fix

1. A failure is reported as a failure

Every question in domainRunChecks() now goes through one helper that remembers what could not be answered, and TXT lookups through another that keeps the difference:

$failed = [];
$ask = function (string $q, string $type) use ($resolver, &$failed): array {
    $r = domainDns($q, $type, $resolver);
    if (!$r['ok']) $failed["$type $q"] = (string)$r['error'];
    return $r;
};
$txt = function (string $q, string $prefix) use ($ask): ?array {
    $r = $ask($q, 'TXT');
    if (!$r['ok']) return null;          // could not ask
    return array_values(array_filter($r['records'], fn($t) => stripos(ltrim($t), $prefix) === 0));   // [] = really none
};

Each check now has three cases, not two:

if ($spfs === null) {
    // could not look it up - said once, below
} elseif (!$spfs) {
    $add('spf_missing', 'email', 'fail', 10);

and at the end, one finding names what failed and why:

⚠️ 3 DNS lookups could not be answered, so the checks that needed them were skipped. This is the server running FreeITSM failing to look records up - it says nothing about the domain itself… First failure: DNS-over-HTTPS query failed: Could not resolve host: dns.google (TXT example.org, …)

It costs the domain no points, and the grade is withheld (shown as –): a score built from the checks that happened to work would look like a clean bill of health it is not.

2. Automatic tries again

if (!$res['ok']) {
    $tries = !$useDoh ? ['google'] : ($auto ? ['cloudflare', 'system'] : []);
    foreach ($tries as $alt) {
        $next = $alt === 'system' ? domainDnsSystem($name, $type) : domainDnsDoh($name, $type, $alt);
        if ($next['ok']) { $res = $next; break; }
    }
}

The system resolver already fell back to Google. Now Automatic on Windows falls back to Cloudflare, then the server's own resolver - slower on huge answers, but an answer. A DNS-over-HTTPS service chosen on purpose under Domains → Settings is respected: no quiet substitute.

📁 Files

	File	Change
🔧	`includes/domains/checks.php`	`$ask` / `$txt`; three-way checks; `dns_lookup_failed`; grade withheld; SPF includes not "broken" on a failure
🔧	`includes/domains/dns.php`	Automatic's second and third opinions
🌐	`lang/en/domains.php`	the new finding; the resolver setting's description
🧪	`tests/domains-dns-failures.php`	13 checks

How it was proved

php tests/domains-dns-failures.php - against the internet, with dnssec-failed.org as a real, repeatable failed lookup (Google and Cloudflare refuse it; a non-validating resolver answers):

no "No SPF", no "No DMARC", no "no MX / no DKIM / MTA-STS off / no name servers / broken include" from a failure;
one dns_lookup_failed, with the error and the names, worth 0 points; no grade;
Automatic on Windows: Google and Cloudflare refuse, the server's own resolver answers; a chosen Cloudflare gets no substitute;
positive control: google.com - SPF found, DMARC found, graded.

Run against the old checks.php, the same test fails six checks - mx_none, spf_missing, dmarc_missing, grade C: the reporter's screen.

Not proven here: what stopped the reporter's server reaching DNS. The fix makes the page say so, with the error - which is the first thing that will tell us.

What this means for you

If your domains showed SPF or DMARC as missing when they are not, Check now after updating. Either the checks now pass, or you see "DNS lookups could not be answered" with the reason.
If you see that warning, the server cannot reach DNS. Make sure it can, or pick a different resolver under Domains → Settings → Monitoring - on a network that filters DNS-over-HTTPS, The server's own resolver is usually the one that works.

Related: Domains · Domains — Developer Guide · Bugs resolved