How the Domains module is put together: the files, the one service every write goes through, how lookups and checks work, how alerts fire exactly once, and the traps found while building it. For the plain-language version see Domains; for the API, REST API: Domains.

Colour key: 🗄️ schema · ⚙️ engine · ✏️ write · 📖 read · 🔗 cross-module · 🔌 REST · 🖥️ UI · ⏱️ schedule

What you do not touch to add a check: the screens. A finding is a key plus params; its words live in lang/en/domains.php under check.<key>, and every screen, the digest and the API render it from there.

screens (api/domains/*) ─┐
REST API (api/v1)        ├─► DomainsService ─► domains + domain_audit + WorkflowEngine::dispatch()
scheduler (cron, tick)   ─┘

• loadForActor() starts every by-id method: a domain in a company the caller cannot reach reads as not found, never forbidden, so ids cannot be probed.
• Every changed field gets a domain_audit row, whoever changed it; source says who: app, api, lookup, check, monitor, import. Ids are stored as names (auditDisplay()), so the history reads as English a year later.
• The auth code is the exception: its change and every view are recorded, its value never is.
• Deletes remove children by hand - an upgraded install whose foreign keys failed to add has no cascade to rely on. That includes the 3.0.0 link tables, each in its own try so a table not yet created never stops a delete. (Deleting the record on the other side - a CI, a ticket - relies on the cascade; a link left without its other end is never shown, because domainLinks() drops a row whose target fails the access check.)

domainLookup() returns one normalised shape whatever the source, and never writes. DomainsService::applyLookup() decides what to keep:

• the overwrite setting: always (the registry wins) or blanks (fill only what nobody typed);
• silence is not an answer - a null from the registry (a .de expiry) never blanks a typed value;
• registry statuses always follow the registry (people never type them);
• the registrar is matched to a Contracts supplier by name when none is set, so the supplier page lists its domains.

Nominet moves .uk domains by registrar tag, not auth code, so most show only active. Treating that as "unlocked" flagged every .uk domain as unprotected. But some registrars do set client transfer prohibited on .uk (measured: heartinternet.uk has it; freeitsm.co.uk at the same registrar does not). So: a transfer status present → locked; none → not applicable (transfer_lock = NULL, finding reg_transfer_uk, no points). See domainTransferLockNotApplicable().

domainDeriveFlags() once read any reply in EPP vocabulary as "lock off unless a transfer status is present". True for generic TLDs - ICANN's RDAP profile makes their registries publish every EPP status, and RFC 8056 maps ok to active - but many country registries answer active alone and never publish locks. So OFF now needs evidence: domainTldPublishesAllStatuses() (TLD longer than two letters, not xn--) or another (client|server) * prohibited in the reply; otherwise transfer_lock = NULL. Tested by tests/domain-transfer-lock.php. Write-up: Domain-Transfer-Lock-Shown-Off-When-Unknown.

.de WHOIS needs -T dn,ace to say anything, and still publishes no expiry. Its status connect is not EPP vocabulary, so locks stay unknown rather than off.

dns_get_record() on Windows returned false after 10 seconds for a domain with 22 TXT records (it cannot fall back to TCP), and has no DNS_CAA at all. DNS-over-HTTPS answered the same question in 0.3s. The auto resolver (the default) is DoH on Windows and the system resolver elsewhere, with DoH as the second opinion when the system resolver fails - and, since 3.16.0, on Windows Cloudflare then the system resolver as second and third opinions when Google cannot answer (a network that blocks one DoH service must not leave every check failing). An explicitly chosen DoH service gets no substitute.

system_only (4.0, GH #203) is the system resolver with every public route closed: no DoH, no fallback. A type that needs DoH (DS, DNSKEY, CAA where PHP lacks DNS_CAA) comes back ok: false, unsupported: true - and $ask in domainRunChecks() does NOT count that as a failed lookup (which would withhold the grade); one dns_server_only info finding names the types instead. The DS lookup's "system means Google" shortcut does not apply to it. domainHttpGet() records every URL into $GLOBALS['DOMAIN_HTTP_TRACE'] when a test sets it to an array - tests/domains-dns-failures.php uses that to prove system_only calls no public DNS service, with a control showing system does.

domainDns() returns ok: false when it could not get an answer, and records: [] either way - so domainTxtStartingWith(), which returns the matching records, cannot tell "no SPF" from "could not ask". Until 3.16.0 domainRunChecks() used it directly, and a server that could not reach its resolver reported every domain as "No SPF record" / "No DMARC record" and graded it down - while a failed NS or MX lookup passed silently. Now every question in the checks goes through $ask (which remembers failures) and $txt (which returns null for a failed lookup, [] for a real absence); each check says nothing on null, one dns_lookup_failed finding names what failed and why (weight 0), and the grade is withheld. Change detection was already right: domainBaselineDiff() treats an empty value as unknown. Write-up: A failed DNS lookup was reported as a missing record.

A check asks about 25 DoH questions. With a fresh curl handle each time that was 7.5s a domain; one static handle reused (curl_reset() keeps the connection cache) brought it to 1-2s. domainHttpGet() owns that handle.

Verification off to read the certificate whatever its state (an expired one is the one worth seeing), then on, against the same CA bundle as the rest of FreeITSM, to answer "would a browser trust this?".

domainParseSslHosts() keeps only the domain and its own sub-domains. Otherwise "extra hosts to check" would be a way to make the server connect to arbitrary or internal addresses.

domainRunChecks() returns findings ['key','area','level','weight','params']. Warnings and failures subtract their weight from 100; domainGrade() maps the score to A+…F, with expired, redemption or NXDOMAIN forced to F. The purpose changes the questions: domainPurposeIsNonMail() swaps the mail checks for the lock-down checks.

To add a check: add $add('key', 'area', 'level', weight, params) in includes/domains/checks.php, then check.key.title / .advice in the lang file. The key-audit script used during the build (every $add key must have a title) is the quickest way to prove nothing is missing.

After each check, domainBaseline() records the registrar, locks, DNSSEC, live name servers, MX, SPF and DMARC. domainBaselineDiff() compares with the last run:

• known → different known is a change; known → unknown is a gap in our knowledge, never an alert (a registry having a slow afternoon must not page anybody);
• domainBaselineMerge() keeps the last known value through a gap, so its reappearance is not a change either;
• A/AAAA records are deliberately not watched - CDNs change them constantly;
• the record's own name-server list is not watched either: a person editing it is not a hijack, and a real hijack shows in live DNS.

Serious changes (registrar, name servers, MX moving; a lock or DNSSEC going off) are emailed at once via domainAlertOnChange().

domain_alerts_sent holds (domain, kind, fingerprint). The fingerprint carries the date the alert is about, so renewing re-arms every window with nothing to clear. Each audience has its own kind - evt_* (bell + workflows), mail_* (digest), act_* (task/ticket) - so switching email on later still sends the mail for a window whose event already fired.

Claim before sending (INSERT IGNORE), give it back on failure - the LMS reminders rule. A digest item is claimed once and released only if every digest carrying it failed.

workflowEmitOnce() (the contract/warranty expiry path) returns early when no workflow is listening - and the bell is fed from dispatch(), so a bell notification for a time-based event would never arrive on an install with no workflows. Domains keeps its own ledger and calls WorkflowEngine::dispatch() directly.

domainScheduledRun() works to a time budget, not a count: lookups (paced per registry host), then checks, then the weekly watches, then alerts, the calendar (both kinds) and domainStatusRun() - Service Status raising (mode auto) and resolving (§12). Oldest first, never-looked-up first; each domain finished or not started. cron/domains.php holds a MySQL GET_LOCK so two runs never overlap. The page-load fallback (api/domains/tick.php) runs a 25-second batch at most once an hour and never waits the page.

Domains and registrar accounts are scoped data (tenant_id NULL = the Default company). Lists use activeTenantReadFilter() (widens in the All-companies view); by-id reads use analystCanAccessDomain(); the service enforces companyScope; the REST API uses apiKeyTenantFilter() / apiKeyCanAccessTenantRow(). A registrar account from another company cannot be put on a domain - the link itself would leak. The same name may exist once per company.

The Watchtower card is the first card scoped by company and module access - the others leave that to the page. Its "services at risk" count follows the same company filter.

Connections keep to the company (§11): a CI or ticket must be in the domain's company, checked in domainLinkTargetOk() and narrowed in SQL by every picker. Status services are install-wide and knowledge articles have their own audience model, so those two are judged by their own permissions only.

One file decides everything: includes/domains/links.php. Four join tables plus the domain's own contract_id:

🔑 The rule, both directions: a link is shown, made or removed only when the analyst can open Domains and the other module and the record itself. domainLinks() drops rows they cannot see rather than marking them hidden; domainLinkSearch() / domainLinkPickDomains() run the same per-record check as domainLinkAdd(), so a picker never offers something the add would refuse. Removing needs the same right as adding.

One endpoint, both sides: api/domains/links.php - ?domain_id= (the tab, plus the Service Status state), &search=KIND (link from the domain), ?for=KIND&id= (the other side's list), &pick=1 (link from the other side), ?contract_id=, and POST add / remove / set_contract (through DomainsService::updateDomain, so audited and contract-visibility checked) / raise_incident.

The other side is one shared panel, assets/js/domain-links.js + assets/css/domain-links.css: DomainLinks.mount(box, {kind, id, base, cardClass…, editable, hideEmpty, bare, onChange}). Mounted by the CMDB object page (in the page's own o2-card), a contract (bare, under the page's own heading) and Knowledge - list-only and hidden when empty while reading, linkable in the editor, the documents rule (reading is not editing). Tickets use pills in the Links strip instead (loadTicketDomains() in inbox.js), matching CMDB objects there. Every host page links the panel only when the analyst can open Domains and exports the domains translation namespace.

⚠️ The contract picker only offers domains with no contract. A domain has one contract_id; offering one under another contract would silently move it. Moving is a deliberate change, in its Edit.

includes/domains/status_link.php. Settings on their own tab (service-status, Cap::DOMAINS_SERVICE_STATUS, sensitive):

domain_status_mode (off · suggest · auto) · domain_status_on_expired · domain_status_on_cert · domain_status_cert_days · domain_status_impact · domain_status_public (0) · domain_status_auto_resolve (1)

• Trouble = domainStatusProblems(): past expiry_date, or ssl_expiry_date past / within N days - only for a domain whose status wants alerts and which is not Do not renew (the Watchtower card's $live rule).
• Never twice: domain_status_incidents (domain, trigger, fingerprint = the date) - the domain_alerts_sent rule. One open incident per domain; a person resolving it does not let the same dates raise again; a new date re-arms.
• Created and resolved through ServiceStatusService::saveIncident, so the opening update, impact snapshot and workflow events are the same as one raised by hand. The scheduler's actor is actorId 0 ("Domains").
• domainStatusRun() runs at the end of the scheduled run: raise (mode auto), then resolve (auto-resolve) when none of the reasons an incident was raised for still holds.

Tested by tests/domain-status-incidents.php (22): it makes its own domain and service, so it touches nothing real, and deletes them with every incident and update it raised.

domainSyncExpiryCalendar() now draws two kinds, each with its own source, setting and category - the Warranty and lease rules:

Each kind adopts its category before clearing its entries and remembers it by id (<source>_category_id), so a renamed category is never duplicated. A manual Check now that changes the certificate date re-syncs the calendar; the scheduled run syncs once at its end. Tested by tests/domain-calendar-kinds.php (14), including rename-then-sync and adopting a rename from before ids were kept.

The Watchtower card shows when either surface asks for the dashboard, each figure only under its own setting; certificates use domain_ssl_warn_days (was a fixed 21) and skip deliberately-lapsing domains.

In domains-register.js, the Tasks menu's shape: one #domCtx built once, submenus that flip at the edge, keyboard arrows. Every action goes through the endpoint the domain's page uses (bulk_update.php, process.php, delete.php); Edit opens view.php?id=N&edit=1, which opens the dialog after load. Copy uses window.copyToClipboard (works on plain HTTP).

Add, never swap. #162 asked for the owner to be the customer and the technical contact to be "us". Re-pointing owner_analyst_id or tech_contact_id would have changed what every existing record, alert, report and API client means by them, so the new kinds are new columns beside the old ones.

One of each kind — DomainsService::normaliseParties(), run before the field loop in both createDomain() and updateDomain():

• technical contact: tech_contact_id XOR tech_analyst_id. Setting one nulls the other; both non-empty in one request is a ServiceError.
• customer: customer_user_id XOR (customer_supplier_id [+ customer_contact_id]). A contact sets the supplier to its own contacts.supplier_id; a contact plus a different supplier is refused; a new supplier drops a stored contact from another supplier; clearing the supplier clears the contact. Clearing a field otherwise clears only that field.

Because setting one kind clears the other, a caller never needs to know what was stored before — the REST API can PATCH one field.

Readiness. domainPartiesReady() (in includes/domains/customer.php) probes all three columns at once. fieldReady() skips them in both write paths, normaliseParties() returns early, domainListSelect() and apiDomainSelect() select NULL AS … in their place, and domainApiLookups() returns parties_ready so the edit dialog leaves out the Analysts group. Proved by saving through api/domains/save.php on a database without the columns.

Who may see contacts. Supplier contacts are Contracts' records. L.contacts has always been empty for an analyst without Contracts, and the customer search (api/domains/people.php) drops kind: contact rows for them too. Suppliers' names are not hidden — the registrar list already shows them to everyone in Domains.

The edit dialog (assets/js/domains-view.js): the technical contact is one <select> with values a:<id> / c:<id>; the customer is the existing search box, its hidden value now user:<id>, supplier:<id> or contact:<id>. 🔴 Both are sent only when they changed from what the dialog opened with (techAtOpen, custAtOpen). Before 3.1.0 an analyst without Contracts opened the dialog with an empty contact list, so saving posted a blank tech_contact_id and silently cleared it (#2137). The current contact is now also always kept as an option, and an inactive analyst likewise, so the dialog shows the truth.

People pages. The domain page links the technical contact and a supplier customer to people/contact.php / people/supplier.php (only for analysts with People and Contracts — window.DOM_PEOPLE_SUPPLIERS). Those pages list domains through peopleSupplierDomains(), which reports each domain's role (registrar / customer / tech) and keeps to the analyst's accessible companies. See People — Developer Guide.

Tests. php tests/domain-parties.php — 15 checks of the rules above, in a rolled-back transaction.

domains.parent_domain_id (nullable, same company, one level: a parent is never a subdomain). Rules in DomainsService (validateField type parent) and includes/domains/subdomains.php:

• Found by name, both ways. createDomain() sets the parent from domainParentCandidates() - every proper suffix of the name with at least two labels that is a top-level domain in the same company, longest first - unless the caller named one. After the insert, domainOrphanSubdomains() adopts rows already under the new name (when it is not a subdomain itself). No public-suffix list: only names already in the register can be parents, and domainIsUnder() requires the dot (notexample.org is not under example.org).
• Validation refuses a parent in another company, one the name is not under, one that is itself a subdomain, the domain itself, and a domain with subdomains becoming one.
• Rename out from under the parent clears the link; a renamed parent keeps only the subdomains still under its new name. Delete of a parent leaves its subdomains standing alone (the link is cleared, not cascaded).
• No registration. refreshLookup() returns ok:false, subdomain:true without writing last_lookup_error; the scheduled lookup query has AND parent_domain_id IS NULL; domainAlertsDue() skips a subdomain's expiry; the domain_expiry calendar kind excludes it; the look-alike scan skips it. domainDetail() returns parent (the parent's registrar, expiry, days left, renewal, locks) and the page draws that instead of its own registration card.
• Checks (domainRunChecks() gets parent_name and parent_dnssec from runChecks()): registration findings are replaced by one sub_registration info; NS empty is dns_sub_parent_zone (pass), not dns_no_ns; DNSSEC falls back to the parent's flag, then a DS lookup at the parent; CAA climbs to the parent (RFC 8659); DMARC with no record at the subdomain reads the parent's and applies sp= else p= (RFC 7489 §6.6.3) - dmarc_inherited names the parent and policy, and the normal dmarc_* finding grades the policy that applies. A failed parent lookup says nothing (the §4 trap).
• Upgrade. Database Verification runs domainLinkExistingSubdomains() once (marker domain_subdomains_linked), shortest names first so chains end one level deep; each link gets a history row with source import.
• REST: parent_domain_id is returned and accepted (create and PATCH), through the same validation.

domain_manual_certificates (purpose, cert_type_id, supplier_id OR provider_name, expiry_date, url, notes) and domain_certificate_types (an editable list like the statuses; seeded into an empty table; Cap::DOMAINS_CERT_TYPES tab). DomainsService::saveManualCertificate() / deleteManualCertificate() (scope through the domain; purpose kept to one line - it goes into e-mail subjects and calendar titles; http/https addresses only; a supplier clears typed text; an inactive type stays on a certificate that has it but cannot be given to a new one), saveCertType() / deleteCertType() (refused while in use).

One reminder pipeline, not a second one. ssl_expiry_date keeps its meaning (the soonest checked certificate); recorded certificates join each consumer beside it:

Not wired: Service Status trouble (still the checked certificate only) and the REST API (no endpoint for recorded certificates yet).

Tests. php tests/domain-subdomains-certs.php - 49 checks: the rules in a rolled-back transaction, deleting with real rows (deleteDomain() has its own transaction), and the checks against real DNS (www.gov.uk: gov.uk has p=reject; sp=none, so it inherits none; mail.google.com inherits reject; CAA climbs). ⚠️ Count days from UTC today, never strtotime('+5 days') - local time crosses a clock change and is a day out (it did, the night this was written).

domains.scope is public (everything above) or internal - the DomainScope enum in includes/domains/scope.php, with the design in Internal monitoring in Domains.

🔑 One gate, asked by every route out. domainMayLeaveNetwork($row) (and domainIdMayLeaveNetwork($conn, $id) for the watch functions, which only have an id) answers may anything about this domain be sent outside the network? Asked by:

tests/domains-internal-scope.php traces every route: $GLOBALS['DOMAIN_HTTP_TRACE'] now records WHOIS (whois://), TLS (tls://host:port, or tls://host:port@address when connecting to a resolved address - slice 2) and this server's resolver (dns-system://name/type) as well as HTTP. An internal domain with no HTTPS endpoints may only appear as dns-system://; its endpoints are connected to only as § 18.1 describes. Controls prove the trace sees a TLS connection and a registry lookup for a public domain. Part 2 writes domains and settings, so it runs only when FREEITSM_TEST_DATABASE names the database config.php points at.

The check. domainRunInternalChecks(): A and AAAA for the domain and its ssl_hosts, always with 'system_only' whatever the install-wide resolver (system falls back to Google; auto is Google on Windows). No www assumption, no TLS.

TRAP: Windows answers a missing name with an empty A list and a FAILED AAAA query. Requiring both answers made a missing name "not checked" for ever on a Windows server. So an empty A answer is a real answer (no IPv4 address, int_no_ipv4 when AAAA could not be asked), and only a failed A query is "not checked".

Health, stored and worked out. Stored: health_status (the last COMPLETED result), health_completed_datetime, health_healthy_datetime, health_incomplete (the latest attempt did not complete). A completed run moves the completed date - a confirmed problem is a fresh observation; an incomplete one only sets the flag, so the known state survives. domainHealth($row, $interval) works out stale at read time (completed more than twice the interval ago). The interval comes from domainShapeInterval(), set by the readers (domainListRows, domainDetail, the REST apiDomainSelect) because array_map('domainShapeRow') cannot pass a connection.

TRAP: an "internal" the database cannot store is refused, not dropped. A field Database Verification has not added is normally skipped quietly (fieldReady). A dropped internal would leave a public domain - looked up at a registry on the next line of save.php. assertScopeStorable() refuses it on create, update and bulk update.

Counts. Internal domains are left out of the dashboard grades, email posture and registration counts (an Internal tile counts them by health), Watchtower's expiry, unlocked and weak counts, the registration expiry alerts, and domainNeedsAttention() - which gives them health and stale instead. A certificate recorded by hand on an internal domain still counts.

Scheduling. Public checks stay daily; internal ones are due after domain_internal_interval minutes (15, 30, 60, 240 or 1440, default 60), in the same run, lock and time budget.

REST. scope and a health object (state, known, last_completed_at, last_healthy_at, since_incomplete, interval_minutes) on the domain resource, in all three schemas; ?scope= on the list; POST /domains/{id}/lookup on an internal domain is 409 internal_domain, not a 502 failure.

The first time FreeITSM connects to addresses somebody types in, so every protection the review asked for ships with the first probe. Code: includes/domains/endpoints.php (the check and the rules), DomainsService::saveEndpoint() / deleteEndpoint() / saveTrustProfile() / deleteTrustProfile() / runInternalChecks() / storeEndpointResult(), api/domains/endpoints.php (the screen's adapter), domains/settings/index.php (the Internal monitoring tab), assets/js/domains-view.js (endpointsHtml()).

domainCheckEndpoint(), in order - each refusal returns before anything is connected to, and tests/domains-internal-endpoints.php proves it from the trace:

1. The host belongs to the domain (domainHostBelongsTo) and is a name, never an address - re-checked at check time, not only on save.
2. The port is permitted (domain_internal_ports, 443 by default; an emptied list is no ports, not "back to 443").
3. Some network is permitted (domain_internal_networks, empty by default).
4. DNS through this server only (system_only), with slice 1's Windows rule (an empty A answer is an answer).
5. Every resolved address is in a permitted network (domainIpInCidr, IPv4 and IPv6). One outside refuses the endpoint - it is never probed at whichever address happened to be allowed.
6. Connect to the address, IPv4 first: domainTlsCertificate($host, $port, 8, $address, $cafile) connects to ssl://<address>:<port> with peer_name = the host, so SNI and the certificate's name check use the host. The trace records tls://host:port@address.
7. Judge: no answer → problem no_answer; expired → expired; name → wrong_name; chain → untrusted; inside warn_days → expiring; else healthy. A trust profile's PEM is written to a temp file for the second handshake and deleted.

Refusals and a resolver error are not completed: the endpoint's known status stays, incomplete is set, and the domain's latest attempt is incomplete too. A confirmed problem anywhere makes the domain problem (a fresh observation); otherwise any expiring makes it expiring. domainHealth() accepts expiring as a known state.

Who probes. runChecks() probes endpoints for $ctx === null (the scheduled run) or an actor holding Cap::DOMAINS_INTERNAL; anyone else's Check now re-runs DNS and re-uses the stored endpoint results (runInternalChecks(..., $probe = false)).

Feeding what exists. The endpoint certificates go into check_results.certificates (labelled host:port), the earliest valid_to into domains.ssl_expiry_date / ssl_issuer - so domainAlertsRun()'s certificate alerts, the calendar and Watchtower's certificate count work unchanged - and domainNeedsAttention() flags an internal domain that is expiring or whose ssl_days_left is 14 or less. History: endpoint (added / changed / removed) and endpoint_status ("host:port: status"). Changing an endpoint's host, port or trust profile clears its results: they described a different target.

Permission. Cap::DOMAINS_INTERNAL (domains.internal), a settings tab of its own (internal) and sensitive. It gates endpoint writes, trust profiles (list and write) and the two settings. Reading an endpoint's results needs only the domain.

🔴 TRAP: $GLOBALS['DOMAIN_ENDPOINT_RESOLVER'] is a test hook - a pretend DNS answer, because a Windows server's resolver does not answer for localhost. domainCheckEndpoint() honours it from the command line only (PHP_SAPI === 'cli'), so nothing a web request reaches can change where a name resolves. Never read it anywhere else.

TRAP: a deleted trust profile is never quietly swapped for the server's bundle. deleteTrustProfile() refuses while an endpoint uses it, and an endpoint whose profile is somehow gone is profile_missing, not checked against something else.

REST (slice 2): the health state and known enums gain expiring. Endpoints and trust profiles reached the REST API in slice 3 (4.9.0) - see Internal Domain Monitoring - Developer Guide § 4.

Tests: php tests/domains-internal-endpoints.php - part 1 the rules; part 2 real TLS against throwaway local servers signed by a throwaway CA (healthy, untrusted with the server's bundle, expiring, wrong name, nothing listening, and every refusal with an empty trace); part 2b the optional HTTP check (slice 3); part 3 (with FREEITSM_TEST_DATABASE) through DomainsService.

Expected DNS records (domain_expected_records, DomainRecordType, domainRecordValues()), the optional HTTP check on an endpoint (http_path, http_expect, domainHttpStatus()) and the 12 REST routes for endpoints, expected records and trust profiles (api/v1/resources/domain_monitoring.php, permission domains.internal plus Cap::DOMAINS_INTERNAL on the key's analyst) have their own page: Internal Domain Monitoring - Developer Guide. In short: every new DNS question goes through this server's resolver only, the HTTP check runs only after the certificate passes, to the address already permitted, and never follows a redirect, and every REST write goes through the same DomainsService methods as the screens.