Domains Internal Monitoring Design - edmozley/freeitsm GitHub Wiki

Internal monitoring in Domains - design (#203)

Status: complete. Slices 0 (4.0), 1 (4.5.0) and 2 (4.8.0) are built, and slice 3 - expected records, the HTTP check and REST (4.9.0), then split DNS (#249, after 4.10.0). Split DNS was built differently from section 6 - as two linked records rather than one record with two perspectives; why is in Split DNS - Developer Guide ยง 3. For users: Domains - Internal domains and Internal websites; for developers: Domains - Developer Guide ยง 18. This page sets out how I am building issue #203. It was written before the code so the approach could be checked first, and @Dr4inder reviewed it on the issue on 26 October 2026. Their answers and clarifications are folded in below and summarised in Where we landed. The decisions are in section 10.

The request, from issue #203 (raised by @Dr4inder):

"The current assessment assumes public domain registration and public web/mail policy ... For a host used only for an internal application, those checks do not describe its intended operation."

In plain words

Domains was built for public names: it checks the registration, the email-security records and the website certificate, and gives an A+ to F grade. Add an internal name - an intranet, an internal application - and it is graded F for lacking public things it was never meant to have. Worse, some of its lookups send the name outside: to Google's DNS service, to the domain registries, and to crt.sh.

This design adds an Internal setting to a domain. An internal domain:

  1. Never leaves the network. No public DNS, no registry lookup, no Certificate Transparency, no look-alike scan - not even as a fallback.
  2. Skips what does not apply. Registration, renewal and email security show as not applicable, not as failures.
  3. Is checked for what matters inside: does each name resolve (slice 1), and - on the endpoints you switch it on for - is the HTTPS certificate on that port valid, trusted and in date (slice 2).
  4. Gets a health status, not a grade. Healthy, a problem, not checked, or stale - never a pass made up from checks that did not run.

It is built in slices. Slice 0 shipped in 4.0, slice 1 in 4.5.0 and slice 2 in 4.8.0. Slice 3 is its own issue.

Where we landed

After review on the issue, the phased approach is agreed, and all four open decisions went the proposed way (section 10). Five clarifications changed the design, and each is reflected in the section it touches:

# Clarification What changed
1 Configured checks only. A name that only has to resolve must not fail because nothing serves HTTPS on it, and there is no implied apex or www HTTPS is switched on per target, never assumed. A name with no checks switched on is Not checked (section 5)
2 The protections come with the first HTTPS probe. The original slice 1 checked certificates on port 443, while administrator-only targets, the network and port limits and resolved-address connections only arrived in slice 2 Slice 1 is DNS-only. Every internal HTTPS check, and every protection that goes with it, is in slice 2, together (section 4)
3 Fresh is not the same as healthy. A completed check that confirms a problem is a fresh observation Each name keeps two dates: the last completed check and the last healthy result. A known problem stays on screen, marked stale or unknown, when later checks cannot complete (section 4)
4 Slice 0's boundary. This server only covers FreeITSM's DNS lookups, not the registry, Certificate Transparency or the look-alike scan Said in section 2 and on the user page. Slice 1's per-domain gate is what covers those for an internal domain
5 The server's resolver can forward. Asking only this server's resolver keeps nothing private if that resolver forwards internal names to the internet Said in section 2 and on the user page. The prerequisite is a resolver whose upstream policy keeps internal namespaces private, and FreeITSM cannot check that for you

Two more points from the review are recorded where they apply:

  • An internal endpoint never inherits permission for public lookups from a Public parent (section 6).
  • The documentation says the scheduled job must run at least as often as the shortest interval chosen (section 7).

1. What Domains does today

Today
DNS domainDns() - the server's resolver, or Google / Cloudflare DNS-over-HTTPS. Automatic is DoH on Windows. With the server's resolver, a lookup it cannot answer falls back to Google, and DS / DNSKEY always go to Google.
Registration RDAP via IANA's bootstrap file, then the registry - includes/domains/lookup.php
Certificates domainTlsCertificate(host, 443) on the domain, www. and the ssl_hosts list. Port 443 only. The connection resolves the name through the operating system - not through domainDns() - so DNS checks and the certificate check can see different answers. Validation uses the server's CA bundle (SSL_CA_BUNDLE).
Watches Certificate Transparency (crt.sh) and a look-alike scan - both send the name, or names built from it, to public services
Grade A+ to F from weighted findings; withheld when DNS lookups failed (3.16.0)
Purpose primary, secondary, email-only, campaign, defensive, parked, redirect - decides which checks weigh, not whether the name is public
Schedule cron/domains.php hourly: registry weekly, checks daily, watches weekly; lock, time budget, job health (#182)

The reporter's observations all hold. An internal HTTPS name gets its certificate read correctly, then an F for missing NS, SPF and DMARC, plus a failed registry lookup.

2. Slice 0 - done in 4.0: "This server only"

Domains โ†’ Settings โ†’ Monitoring โ†’ DNS resolver โ†’ This server only - never public DNS.

  • Every DNS question goes to the FreeITSM server's own resolver, and nothing else - no DoH, no fallback.
  • Record types that can only be asked over DoH (DS, DNSKEY, and CAA where PHP cannot ask for it) are not asked. One information finding - "Not checked: DS" - says so, costs nothing, and does not withhold the grade.
  • tests/domains-dns-failures.php proves a full check run makes no public DNS call, with a control showing the old server's own resolver does.

It is install-wide and covers FreeITSM's DNS lookups only. The registry lookup, the CT watch and the look-alike scan keep their own switches. Slice 1 makes all of this automatic, per domain.

What it cannot promise. This server only means FreeITSM asks nobody but the server's own resolver. Where that resolver sends a question it cannot answer itself is up to the resolver. If it forwards to a public DNS service, an internal name still leaves the network, one step further away (DNS forwarding). The prerequisite is a resolver whose upstream policy keeps internal namespaces private - typically an internal DNS server that answers for them itself. FreeITSM cannot see or check that.

3. The shape: Scope, separate from Purpose

A new per-domain field, Scope: Public (today's behaviour, and the upgrade default) or Internal.

It is deliberately not another Purpose. The reporter is right that a name should not have to pretend to be parked to stop the email checks. Purpose says what a public name is for. Scope says whether it is public at all. An internal domain has no Purpose (it does not apply), so the Purpose field is hidden.

Nothing is guessed from the name: .local, .corp or .internal do not make a domain internal. An internal subdomain of a public domain (intranet.example.com) is just a domain with Scope = Internal. The subdomain support from 3.16.0 already lets it sit beside its public parent.

4. Slice 1 - Internal scope (the core)

What an internal domain never does:

Lookup Internal
DNS This server's resolver only, whatever the install-wide setting - the same code as slice 0
Registry (RDAP / WHOIS) Never asked. Registration, transfer lock, registry lock and renewal show Not applicable
Certificate Transparency watch Never
Look-alike scan Never
Email security (SPF, DKIM, DMARC, MTA-STS, TLS-RPT) Not checked - Not applicable
Name servers, DNSSEC, CAA Not checked by default - an internal host is not assumed to be a delegated zone (slice 3 can add NS / SOA as an expectation)

The rule lives in one function that every outbound path asks - domainMayLeaveNetwork($domain). The registry lookup, the watches, the on-add processing and Check now all call it, so no route out is forgotten. The tests use the same trace as slice 0 on every one of those paths, including the failure and fallback cases.

What it does check: DNS only. Each name it knows - the domain, plus the names listed for it - resolves (A / AAAA, through this server). Slice 1 makes no HTTPS connection at all, not even the certificate check public domains get today. Connecting to internal addresses needs the protections in slice 2, so it arrives with them, as the review asked. An internal domain added before slice 2 simply has no certificate result.

Health instead of a grade

Health instead of a grade. The A+ to F grade is a public security score, so an internal domain does not get one. It gets a health status:

Health Means
โœ… Healthy Every configured check ran, and passed
โš ๏ธ Expiring A certificate is valid but inside its warning window
โŒ Problem A confirmed problem: the name does not resolve (a real "no such name"), the certificate is expired, does not match, or is not trusted, or nothing answers on the port
โ“ Not checked A probe could not run - a resolver error, a timeout - which says nothing about the target; or there is nothing configured to check
๐Ÿ•’ Stale The last completed check is older than twice its interval. Worked out when the page is read, so a stopped scheduler shows up without another run

A domain with no usable checks is Not checked, never Healthy. Unknown, skipped or stale is never shown as a pass.

Two dates, not one. Each name keeps:

  • the last completed check - any check that ran to an answer, including one that found a problem;
  • the last healthy result.

A check that confirms a problem is a fresh observation, so a broken name is shown as a current Problem, not as stale. When later checks cannot complete, the last known state stays on screen with the stale or not-checked mark beside it - "Problem, last confirmed 3 hours ago; checks since have not completed". A known problem never disappears into unknown.

On screen: the register gains a Scope filter and a Health column. An internal domain's Overview shows its health, last check and next due time, and says plainly "Internal - registration and email checks do not apply" instead of a wall of warnings. The Security and email tab becomes Health for an internal domain.

Size: a new column, the gate function, a health calculation, and the screens. Comparable to the subdomains work in 3.16.0.

5. Slice 2 - HTTPS endpoints and internal trust

Endpoints. A Monitoring tab on an internal domain lists the endpoints to check: host, port (8443 is as good as 443), certificate warning window, and trust profile. The endpoint's full identity - host and port - is kept everywhere, in certificates and history. HTTPS is checked only on endpoints listed here - never assumed on the domain itself, its apex or www. A DNS-only name has no endpoints and is judged on DNS alone.

All of it arrives together. This is the first slice that connects to internal addresses. So the administrator-only targets, the network and port limits, resolved-address connections and the trust rules below all ship with it - none of them later than the first probe.

Same perspective for DNS and the connection. The endpoint's address is looked up through this server's resolver (domainDns, system only), and the connection goes to that address, with the host name still used for SNI and for checking the certificate's name. So the DNS check and the certificate check can never quietly look at two different machines. Today they can.

Trust profiles. Most internal certificates are issued by the organisation's own certificate authority. Under Domains โ†’ Settings โ†’ Monitoring, an administrator adds a trust profile: a name and the CA certificate(s), pasted as PEM. An endpoint uses the server's normal CA bundle - which stays available - or a profile. A profile changes which CAs are trusted, never whether the chain and the host name are checked. Validation is never switched off. "Internal" is not permission to accept any certificate; an untrusted chain is a Problem, reported as untrusted, separately from expired and wrong name.

Which addresses may be probed. Slice 2 is the first time FreeITSM connects to addresses an administrator types in, so it must not become a tool for scanning the network:

  • Endpoints, trust profiles and the limits below are administrator-only (a new capability, domains.internal). Read-only users and everyday Domains users can see results, not change or run checks.
  • Permitted networks (CIDR ranges, IPv4 and IPv6) and permitted ports, set under Settings, are checked against the resolved address, not the typed name, at every connection, and a name that resolves somewhere else is refused.
  • No redirects are followed by the certificate check. The slice 3 HTTP check follows none either, unless the target is also permitted.

As built (4.8.0). Three choices were left open after slice 1, and went this way:

  1. No network is permitted out of the box. The private ranges are tick boxes under Settings โ†’ Internal monitoring, none ticked, plus a box for other ranges. Until somebody ticks one, nothing is connected to, and the domain's Health tab says so. An upgrade never starts connecting anywhere on its own.
  2. A new permission, Internal monitoring (domains.internal), its own settings tab and marked sensitive. It gates endpoints, the limits and trust profiles, and whether Check now reconnects to endpoints. Everybody who can see the domain sees the results.
  3. Ports are configurable: 443 out of the box, 8443 a tick box away, and any other port can be added.

Also as built: every address a name resolves to must be permitted (one outside refuses the endpoint, rather than probing at whichever address was allowed); the endpoint's host must be the domain or a name under it, and a name rather than an address; changing an endpoint's host, port or trust clears its old results, which described a different target. REST reports the expiring health state; endpoints and trust profiles in the REST API move to slice 3's issue with the rest. Developer detail: Domains - Developer Guide ยง 18.1.

6. Slice 3 - expected records, split DNS, and an HTTP check

Expected records. For each name, the record types expected - A / AAAA and CNAME to start, NS / SOA where the name is a zone - each with optional expected values. Sets are compared sorted and normalised, so order alone is never a change. A different address is reported only when the domain asks for it (many internal addresses are dynamic).

Split DNS. One domain record can have two perspectives: the public one (today's checks and grade) and the internal one (health). Each keeps its own findings, baselines and timestamps. The internal result never replaces the public grade, or the other way round. Permission is per perspective, never inherited: an internal endpoint under a Public parent domain gets no public lookups because of its parent. This needs perspective added to findings, history and certificates, which is the biggest data-model change in the design, and why it comes last.

Expected records - as built (4.9.0). Table domain_expected_records (host, record_type from the DomainRecordType enum, expected_values one per line or NULL). domainRunInternalChecks() takes them and an optional resolver (tests pass a stub; production is always system_only); domainRecordValues() normalises - trim, lower case, no trailing dot, canonical IPv6, unique, sorted - and drops anything invalid for the type, so a typo can never make a comparison pass. Saving refuses each invalid value by name, a second CNAME or SOA value, a name outside the domain and a public domain. SOA was added to the DNS helper and is compared by its primary name server. Findings int_expected_ok / _missing / _mismatch / _not_checked feed the existing health, so no new state was needed.

HTTP check - as built (4.9.0). Columns http_path and http_expect on domain_endpoints. domainHttpStatus() makes one GET over TLS to the resolved, permitted address with verify_peer on and the trust profile's CA file, reads only the status line, and never follows a redirect (the strictest reading of "unless the target is also permitted"). It runs only after the certificate checks pass, so an untrusted certificate is reported as itself and the page is never asked. Outcomes http_status and http_no_answer are problems; a changed path or codes clears the endpoint's old results, like a changed host.

REST - as built (4.9.0). Endpoints, expected records and trust profiles have 12 routes; changing any of them needs the domains.internal key permission and the Internal monitoring permission on the key's analyst, and trust profiles need a key with no company limit. Developer detail for all of slice 3 so far: Internal Domain Monitoring - Developer Guide.

Split DNS - as built (#249). Not as above. One name may be in the register twice - one Public record, one Internal - linked by being the same name in the same company. Each record already keeps its own findings, history, certificates, grade or health and permission, so the two perspectives are separate with no change to storage, nothing to relabel on upgrade, and nothing inherited between them (domainMayLeaveNetwork() answers per record). The rule is one record per name, per company, per scope. Public -> Internal is one click (Add an internal view); the reverse never is, as it would leak an internal name. The reasoning, the rule and every place it is enforced: Split DNS in Domains - Developer Guide.

HTTP check (optional). For an endpoint, a path and the expected status codes: "the web application answers", beyond "the TLS handshake works". No logins, no page-content scripting. The reporter suggests it can ship separately, and it can.

7. Scheduling, history and alerts

  • One scheduler. cron/domains.php gains an internal-checks step with its own interval (from 15 minutes to a day, default hourly), inside the existing lock and time budget, recorded in Background jobs (#182). The job can only honour an interval it is run often enough for: a 15-minute interval needs the server to run cron/domains.php at least every 15 minutes. The setting says so beside the choice, and the user page says it too. "The probe ran and found the target down" is a check result. "The job could not run the probe" is a job failure. Neither is confused with the other.
  • History records each change of health and each certificate change, labelled by host:port. Changing an endpoint never rewrites earlier observations.
  • Alerts reuse the existing deduplication and owner notifications. Workflow gets the same domain events. The Watchtower Domains card counts internal problems and stale checks.
  • Service Status and incidents: suggest, never act. An internal failure can suggest the linked services, but does not raise an incident, publish an outage, or close one because later data is missing - unless a later, explicit setting says so.

8. REST API

Additive. Scope, health and endpoints appear on the domain resource, and endpoints and trust profiles get their own routes, all through the same DomainsService validation, company scope and audit as the screens. Documented in the OpenAPI contract. Existing fields and behaviour do not change for public domains.

9. The issue's acceptance criteria

# Criterion Where
1 An internal target can be healthy without registration, SPF / DMARC, www or CT Slice 1 - not applicable, health not grade
2 Internal names never leave the configured context, including failure and fallback Slice 0 (DNS) and slice 1 (domainMayLeaveNetwork, traced on every path)
3 Split DNS, ports and trust stay distinct; a changed profile does not rewrite history Slices 2 and 3
4 Negative answers, probe errors and stale results stay distinguishable Slice 1 - the five health states, and the last completed and last healthy dates, across UI, API and notifications
5 Public records and APIs unchanged until opted in; REST through the same service Scope defaults to Public; section 8
6 Administrator-controlled targets, networks, ports, trust; resolved-address and redirect limits Slice 2
7 Runs from the FreeITSM server, no agent Throughout - no new service

10. Decisions

All four went the proposed way on review:

  1. Scope per domain. Internal subdomains are their own domains with Scope = Internal. Split DNS (slice 3) keeps separate public and internal perspectives for the same name, with no permission inherited between them.
  2. Internal domains are not in the grade summary. They contribute to health reporting only, and neither raise nor lower the public grades.
  3. Trust profiles are Domains-only to start. The server's normal CA bundle remains a choice, and a profile never relaxes chain or host-name validation.
  4. Hourly by default, with shorter intervals configurable - documented as needing the scheduled job to run at least that often.

11. Order to build

  1. โœ… Slice 0 - This server only (4.0).
  2. โœ… Slice 1 - Scope, the gate on every outbound path, DNS-only checks, health with its two dates, the screens (4.5.0). The core, and useful on its own. No HTTPS.
  3. โœ… Slice 2 (4.8.0) - every internal HTTPS check, all together from the first probe: endpoints with ports, switched on per target; trust profiles; administrator-only targets; the network and port limits; resolved-address connections.
  4. Slice 3 (#243) - expected records โœ… (4.9.0), the HTTP check โœ… (4.9.0), REST for endpoints, expected records and trust profiles โœ… (4.9.0), then split DNS โœ… (#249, as two linked records - developer guide).

Each slice ships with its own tests, and a write-up for the issue.

12. Deliberately out of scope

As the issue says: AD replication monitoring, DNS zone editing, network discovery, logins or page scripting in the HTTP check, and any agent or collector.

Related pages