Domains - edmozley/freeitsm GitHub Wiki

🌐 Domains

A register of every domain name your organisation holds - or, on an MSP install, every domain each client holds. It keeps itself up to date from the registries, checks how well each domain is protected, and tells you before anything lapses or changes.

Asked for in GitHub #154. For how it is built, see the Domains developer guide; for other systems, REST API: Domains.

What it does, in one screen
πŸ“‹ Register Every domain with its expiry countdown, status, registrar, owner, security grade, locks and certificate expiry. Views for the questions people actually ask - needs attention, expiring in 30 days, transfer unlocked, grade C or below.
πŸ”Ž Lookups Add a name and the registrar, registration and expiry dates, name servers and lock status are filled in from the registry.
πŸ›‘οΈ Security grade A+ to F, from daily checks of DNS, email security and certificates - each finding with advice in plain English and, where it helps, the exact record to publish.
🚨 Change detection Name servers, registrar, mail servers or a lock changing is recorded, and can be emailed at once - it is what a hijacked domain looks like.
⏰ Alerts Renewal and certificate warnings as one digest per person, plus the notification bell and workflows. Can also raise a task or a ticket at the renewal window.
πŸ‘€ Watches Optional: new certificates issued for your names (Certificate Transparency), and look-alike domains registered one keystroke from yours.
πŸ”— Connections Each domain linked to the CMDB items that depend on it, the tickets about it, its runbooks in Knowledge and its contract - shown on both sides. (3.0.0)
🚦 Service Status Link the services that run on a domain, and an expired domain or certificate flags them as at risk - or raises the incident by itself, if you choose. (3.0.0)
πŸ–±οΈ Right-click Open, edit, refresh, check, set status or owner, and more, straight from the register. (3.0.0)

Adding domains

All three are in the action row under the module header (UI style guide), with the count beside the title. The Table view has New too, which opens the register's Add.

  • New - one name. Paste a web address or an email address if that is easier; it is tidied into the domain name.
  • Paste list - as many as you like, one per line. Duplicates are skipped and listed.
  • Import CSV - a spreadsheet saved as CSV. You match its columns on screen; only the domain name is required, and status, owner and registrar are matched by name.

New domains are looked up and checked straight away, in batches with a progress bar. Anything not finished is picked up by the scheduled run.

[!NOTE] About a sixth of domain endings - mostly country codes such as .de, .io and .co - have no modern lookup service (RDAP). FreeITSM uses the older WHOIS service for those. A few registries, .de among them, never publish an expiry date; enter it from the registrar's invoice so the reminders still work.

Purpose matters

Every domain has a Purpose: primary, secondary, redirect only, email only, defensive, parked or campaign. It changes what the checks look for. A domain that should never send email (parked, defensive, redirect) is checked for being locked down - the SPF record v=spf1 -all, a DMARC reject policy, no mail servers - because an unused, unwatched domain is exactly what a phisher wants to send from.

Subdomains

Since 3.16.0. A subdomain - students.example.org, shop.example.org - can be in the register as well, linked to the domain it belongs to.

  • Linked for you. Add students.example.org when example.org is in the register and it is linked straight away. Add example.org after its subdomains and they are linked then. Change or remove the link with Subdomain of in the Edit dialog. Only a domain in the same company can be a parent.
  • One level. A subdomain cannot have subdomains of its own: x.students.example.org is linked to example.org, the name that is actually registered.
  • Its registration is the parent's. A subdomain is never looked up at a registry, its page shows the parent's registrar, expiry and renewal (with a link there), and renewal reminders go to the parent only. Its countdown is the parent's.
  • Its own checks, where they are its own. Mail servers, SPF, DKIM and certificates are checked for the subdomain. What DNS keeps at the parent is read there: a subdomain with no name servers of its own is answered by the parent's (not "No name servers"), its DNSSEC is the parent zone's, and CAA climbs to the parent.
  • DMARC is inherited. With no _dmarc record of its own, the parent's applies - its sp= subdomain policy when it sets one, otherwise its p=. The checks say which: "No DMARC record of its own - example.org's applies (policy: quarantine)". Worth knowing: a parent with p=reject; sp=none protects itself and not its subdomains.
  • No look-alike scan - only a name somebody could register has look-alikes.

The parent's Overview lists its subdomains with their grades. Upgrading an existing register: Database Verification links subdomains already in it, once.

Recorded certificates

Since 3.16.0. The checks read the certificates on the domain, on www and on the extra hosts you list - whatever answers on the internet. Recorded certificates are for the rest: an internal server, a VPN appliance, a SAML or ADFS token-signing certificate, a code-signing or S/MIME certificate.

On a domain's (or subdomain's) Certificates tab, Add:

Field
System or purpose what it is for - this is what the reminders call it
Type from Domains β†’ Settings β†’ Certificate types - DV, OV, EV, Wildcard, Multi-domain (SAN), Code signing, Email (S/MIME), Client / device, Internal CA, Other to start with; add, rename, reorder or switch off your own
Provider one of your Contracts suppliers, or typed in when it is not one (Let's Encrypt, an internal CA)
Expiry date
Address where it is used or renewed
Notes

Each recorded certificate gets the same reminders as the checked ones: the certificate-expiring alert (bell, the e-mail digest - which names the certificate - and the domain.ssl_expiring workflow event, with its purpose in {{certificate}}), an entry in the Certificate renewals calendar category, the Watchtower count, and a place on the register's needs attention list in its last 14 days - all on the same warn N days before setting. Adding, changing and removing one is in the domain's history.

The security grade

Each check is a pass, a note, a warning or a failure. Warnings and failures cost points from 100:

Grade Score
A+ 100
A 90+
B 80+
C 65+
D 50+
F below 50, or the registration has expired / is being deleted

The checks cover registration (expiry, transfer lock, registry lock, how it renews), DNS (name servers, DNSSEC, CAA), email (SPF including its 10-lookup limit, DKIM, DMARC, MTA-STS, TLS reporting) and certificates (the domain, www, and any sub-domains you list).

The domain page's Overview shows the three things most worth fixing; Security and email lists every check.

[!TIP] .uk domains move between registrars by a change of registrar tag rather than an auth code, so most have no transfer lock at all. FreeITSM does not mark them down for it, but says so, and shows the lock as on for the registrars that do offer one.

On, off, or not known (3.9.0, #188). The lock is on when the registry says a transfer is prohibited, and off only when the registry would have said so - a generic domain (.com, .org, .app...), whose registry must publish every status, or a reply that shows other prohibitions but not this one. Many country registries answer just active, which says nothing about the lock; those show not known rather than a false off. See Transfer lock shown off when unknown.

Keeping internal names inside (4.0)

DNS questions are asked by the FreeITSM server or - on Windows, and as a second opinion when the server cannot answer - by Google or Cloudflare over DNS-over-HTTPS. For internal names that should not leave your network, set Domains β†’ Settings β†’ Monitoring β†’ DNS resolver to This server only - never public DNS. Nothing is then asked of a public DNS service, not even as a fallback; DNSSEC, which can only be checked over DNS-over-HTTPS, shows as Not checked (information, costs nothing) and the domain is still graded. The registry lookup, the Certificate Transparency watch and the look-alike scan are separate outbound lookups with their own switches - This server only covers FreeITSM's DNS lookups only. For a name that should never leave at all, make the domain internal (below).

What "this server only" cannot promise. It keeps FreeITSM's questions on this server. Where this server's resolver sends a question it cannot answer itself is up to the resolver: if it forwards to a public DNS service, an internal name still leaves the network, one step further away. Use a resolver whose upstream policy keeps internal namespaces private - typically an internal DNS server that answers for them itself. FreeITSM cannot see or check that.

Internal domains (4.5.0)

(#203, slice 1 of the design.) An intranet or an internal application is not a public domain, and grading it on public things gives it an F it never deserved. Set its Scope to Internal - when adding it (Add, Paste list, Import CSV with a scope column), under Edit, or for several at once with bulk edit.

An internal domain never leaves the network. No public DNS, no registry lookup, no Certificate Transparency watch, no look-alike scan - not on add, not on Check now, not in the scheduled run, not as a fallback when something fails. The only connections it makes are to the HTTPS endpoints you list (below), inside the networks and ports you permit. Registration, renewal, purpose, locks and email security do not apply, so they are shown as a dash, not as failures; Refresh and the Look-alikes tab are not offered.

It gets a health, not a grade. Each name - the domain and its Extra hosts to check - is asked for its address of this server's DNS resolver only, and each HTTPS endpoint's certificate is checked:

Health Means
βœ… Healthy Every name resolved, and every endpoint's certificate is valid
⚠️ Expiring An endpoint's certificate is valid but inside its warning window (21 days unless you change it)
❌ Problem A name has no address, or an endpoint's certificate is expired, for a different name or not trusted, or nothing answers on its port. On Windows a name that does not exist comes back as no IPv4 address; IPv6 could not be checked, and the finding says so
❓ Not checked The check could not run - this server could not ask its resolver, or an endpoint is outside the permitted networks or ports. That says nothing about the name
πŸ•’ Stale No check has completed for twice the interval. Worked out when you look, so a stopped scheduled job shows up

Two dates are kept: the last completed check (a check that found a problem counts - it is a current result) and when it was last healthy. When later checks cannot complete, the last known state stays on screen with a note - Problem, last confirmed ... - the checks since have not completed - so a known problem never turns into "unknown".

Where it shows. The register's Grade / health column shows the health, the Any scope filter picks internal or public, and an internal problem or stale check is under Needs attention. The domain's page shows the health in place of the grade, a Health card (state, last completed, last healthy, next check due) in place of Registration, and the Security and email tab becomes Health. The dashboard has an Internal tile; internal domains are left out of the grade chart, the email posture, registration alerts and Watchtower's counts.

How often. Every hour by default: Domains β†’ Settings β†’ Monitoring β†’ Check internal domains every (15 minutes to a day). The scheduled job has to run at least that often for a short interval to happen - for every 15 minutes, run cron/domains.php every 15 minutes.

Internal websites: HTTPS endpoints (4.8.0)

(Slice 2 of the design.) An internal domain's Health tab lists its HTTPS endpoints: a host and a port whose certificate is checked, such as intranet.example.local on 443 or apps.example.local on 8443. Only what is listed is checked - never the domain itself or www unless you add them, so a name that only has to resolve is never failed for having no website.

How each is checked. The host is looked up through this server's DNS, and FreeITSM connects to that address, sending the host name so the right certificate comes back and checking the certificate against it. DNS and the certificate therefore always describe the same machine. Then the certificate is read and judged: expired, for a different name and not trusted are each a problem, said as itself; a certificate inside its warning window is expiring. A certificate is never accepted just because the site is internal - validation is never switched off. Unless the endpoint also checks a page (below), only a TLS handshake is made: nothing is fetched, and no redirect is followed.

Checking a page (4.9.0). An endpoint can also check that the application answers, not only that the certificate is good: give a path, such as /health, and the status codes it should answer with - 200 when left empty, or several, such as 200, 204. It is asked only once the certificate has passed, over a fully verified connection to the same permitted address. Only the status line is read: nothing is signed in to, nothing else is fetched, and a redirect is never followed - a 301 or 302 is the answer, so list it if that is the right one. A different status, or no HTTP answer, is a problem that says which.

The limits - set first. FreeITSM only connects within limits an administrator sets under Domains β†’ Settings β†’ Internal monitoring:

  • Networks. Every address the host resolves to must be inside a permitted network, or the endpoint is not checked - a name that also resolves somewhere else is refused, not probed at whichever address is allowed. The private ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, fc00::/7) are tick boxes, none ticked until you choose, and other ranges can be added. Until then nothing is connected to, and the Health tab says so.
  • Ports. 443 out of the box; 8443 is a tick box, and other ports can be added.
  • Trust profiles. For certificates from your own certificate authority: a name and the CA's certificate (and any intermediates), pasted as PEM - never a private key. Choose the profile on the endpoint. A profile changes which authorities are trusted, never whether the chain and the name are checked. A profile in use cannot be deleted.

Who can do what. Adding endpoints, the networks, the ports and trust profiles need the Internal monitoring permission (System β†’ Roles; administrators have it, and it is marked sensitive because it decides which internal addresses the server connects to). Everybody who can see the domain sees the results. Check now reconnects to the endpoints only for somebody who holds the permission; for anybody else it re-runs the DNS check and keeps the endpoints' last results. The scheduled run checks them on the internal interval.

What you see. Each endpoint shows its address, the trust it is checked against, its result with the reason, the certificate's expiry, and its last completed and last healthy dates - an endpoint that cannot be checked keeps its last known result beside the reason. The Certificates tab lists the endpoint certificates by host:port. The earliest one's expiry feeds the existing certificate alerts, Calendar and Watchtower count, and changes of health are in the domain's History, labelled by host:port. The dashboard's Internal tile counts domains with a certificate expiring.

Expected records (4.9.0)

(Slice 3 of the design, #243.) Under the endpoints, the Health tab lists expected records: what a name should hold, not only that it resolves. Each is a name (the domain or a name under it), a record type - A, AAAA, CNAME, or NS and SOA for a name that is a zone - and, optionally, the values you expect, one per line.

  • No values: the record only has to exist. Use this for an address that changes.
  • Values: the record must hold exactly those. They are compared tidied and sorted, so order, capitals, a trailing dot or a differently written IPv6 address are never a change. An SOA is compared by its primary name server.

A missing record, or different values, is a problem that says what it found ("app.example.local A is 10.0.0.9, not 10.0.0.5"); a resolver that could not be asked is not checked, never a pass. They are asked through this server's DNS only, like the rest of an internal domain, and need the same Internal monitoring permission as endpoints. Changes are in the domain's History.

How it is built: Domains - Developer Guide Β§ 18, and for expected records, the page check and the REST API, Internal Domain Monitoring - Developer Guide. API users: REST API: Domains - Internal monitoring.

Split DNS: one name, watched from outside and inside

(#249, the last part of slice 3.) Some names answer differently inside your network than outside it. A customer asking for portal.example.com gets your public website; a member of staff in the office gets an internal server, under the same name. That is called split DNS, and the two sides can break independently - the website fine, the internal server's certificate expired, or the other way round.

To watch both, open the public domain and choose Add an internal view, under its name. FreeITSM asks first, then adds a second record of the same name, set to Internal - in the same company, starting with the same owner - checks it straight away, and opens it.

  • Each is its own record. The public one keeps its grade, registration and email checks; the internal one its health, endpoints and expected records. Neither changes the other.
  • Each links to the other, under the name: "Split DNS - also watched from inside the network: the internal view", and back.
  • The internal one never leaves the network, whatever its public twin does.
  • A name can be in the register once as Public and once as Internal - never twice in the same scope. Adding a third, or switching one twin to the other's scope, is refused and says why.
  • Paste list and Import CSV can add the internal one too: a list with the name and Scope = Internal adds it alongside the public record.
  • Subdomains link to the parent of their own scope - an internal intra.example.com to the internal example.com. If the internal view is added after its subdomains, they move to it, and each move is in its history.

There is no Add a public view on an internal domain, on purpose: making a public record of an internal name would send that name to public DNS and the registries. If a name really is public too, add it with New.

How and why it is built this way: Split DNS in Domains - Developer Guide.

Alerts

Email alerts are off until you switch them on in Domains β†’ Settings β†’ Alerts - press Preview first to see who would be emailed today.

  • Warnings go at 60, 30, 14, 7 and 1 days before expiry by default, once per expiry date - renewing re-arms next year's.
  • After a domain lapses, a reminder every 7 days for 45 days, while it can still be rescued.
  • One digest per person, not one email per domain.
  • Domains whose status has alerts off (Letting lapse, Cancelled) and domains set to Do not renew are left out.
  • Optionally raise a task or ticket at the renewal window, assigned to the domain's owner.
  • The owner's notification bell hears about expiring domains, expiring certificates and detected changes whether or not email is on - each switchable in System β†’ Preferences.
  • Workflows can trigger on any domain event.

Accounts and auth codes

Domains β†’ Accounts records the accounts at registrars that hold your domains: which login, who owns it, and who holds the second factor. Never a password - keep that in your password manager.

A domain's auth code (EPP code) lets whoever holds it transfer the domain away. FreeITSM stores it encrypted, never shows it in a list or over the API, and only analysts with the auth codes permission (System β†’ Roles) can see or change one. Every view is written to the domain's history.

The schedule

One script keeps everything fresh: cron/domains.php, hourly. It refreshes registry details (weekly, and daily for anything expiring within 45 days), runs the checks daily, runs the optional watches weekly, then sends alerts, brings the Calendar up to date (domain and certificate renewals) and - as Settings β†’ Service Status decides - raises incidents for linked services whose domain is in trouble and resolves those that have recovered. Each is recorded once done, so running it more often never repeats anything. Domains β†’ Settings β†’ Monitoring shows the command, a web address to call instead, and a Run now button.

Without a cron, opening Domains runs a short batch at most once an hour - enough for a small register, but nothing runs if nobody opens it.

Connections (3.0.0)

A domain's Connections tab links it to the rest of FreeITSM. Every link shows on both sides, and can be made from either:

Linked to On the domain On the other side
CMDB items that depend on it what breaks if it lapses a Domains panel on the CI's page
Service Status services that run on it the services at risk (below) β€”
Tickets about it a list with each ticket's status 🌐 pills in the ticket's Links strip - Link to… β†’ Domain
Knowledge articles - its runbooks how to move its DNS, renew it, who to call listed under the article; linked in the article's editor
Its contract the contract it is renewed or billed under (set in Edit) a Domains section on the contract, with each domain's yearly cost and the total. A domain's cost can be in any currency

You only see, make or remove a link when you can open both ends: Domains and the other module, and the record itself. A CI or a ticket must be in the same company as the domain. Somebody without Domains sees no domain panel anywhere, rather than an empty one.

Service Status (3.0.0)

An expired domain - or an expired certificate - is an outage for the services that run on it. Link those services on the domain's Connections tab, then choose in Domains β†’ Settings β†’ Service Status what happens when the domain is in trouble:

Setting Choices
When a linked domain is in trouble Do nothing Β· Suggest (the default) - the Connections tab lists the services at risk with a Raise incident button Β· Raise by itself - the scheduled run raises it
What counts The domain has expired Β· its certificate has expired, optionally a number of days before it does
Impact on each service Any of your impact levels; blank = the most severe level that counts as downtime
Publish straight away Off by default: the incident opens with an internal update and customers see nothing until somebody writes to them
Resolve when the domain recovers On by default: renew the domain or replace the certificate and the incident is closed with a resolved update

An incident is raised once per problem: the same expiry date never raises a second, and renewing re-arms it. The Watchtower Domains card counts the services at risk.

The tab has its own permission (System β†’ Roles), marked sensitive, because it can publish to a page customers read.

Right-click (3.0.0)

Right-click any domain in the register for: Open, Open in a new tab, Edit, Refresh lookup, Check now, Status β€Ί, Owner β€Ί, Assign to me, Connections, Visit website, Copy name and Delete. Each does exactly what the same action on the domain's own page does, history included.

Cost centre (3.9.0)

Each domain can be charged to a cost centre from System β†’ Cost centres, the same list assets, contracts and people use. Choose it in the Cost centre box when editing the domain. Only the active cost centres of the domain's own company are offered; a domain already charged to one that is later made inactive keeps it.

  • Domains β†’ Table has a Cost centre column (switch it on under Columns) that you can sort and filter by.
  • The bulk editor, the CSV import and the REST API take the cost centre's code (0010, N1414), so a list from your finance system can be applied as it is. A code that does not exist in the domain's company is refused rather than guessed.
  • A cost centre cannot be deleted while a domain is charged to it - make it inactive instead.

Domains that had a cost centre typed as free text before 3.9.0 were linked automatically where the text matched a cost centre code. Any other text is still shown under the box as Previously recorded until someone chooses a real cost centre.

Where Domains appears elsewhere

Where What
Watchtower A Domains card: expiring, certificates, unlocked, weak grades, and services at risk. Renewals and certificates each appear only if their own setting says so
Calendar Two kinds, each with its own setting in Settings β†’ Alerts: domain renewals (Where renewals are shown) and certificate renewals (Where certificate renewals are shown). Each goes in its own category, Domain renewals and Certificate renewals - rename either to your own language or wording and FreeITSM keeps filing into it, never making a second one. Switching one kind off removes only its own entries
CMDB, Tickets, Knowledge, Contracts The Connections above
Service Status Incidents suggested or raised for linked services
⌘K search Domains by name, registrar or tag
Contracts β†’ Suppliers A registrar's page lists the domains it holds
Documents Attach invoices and transfer authorisations to a domain
Workflows Triggers for added / changed / deleted, expiring, certificate expiring, change detected, new certificate, look-alike found
Feature Bingo 20 cards for the Domains features worth setting up

Owner, technical contact and customer

Three people-fields answer three different questions, and none of them replaces another:

Field Who it can be What it is for
Owner One of your analysts Who is responsible. Gets the renewal reminders, the bell notifications and any task or ticket raised at the renewal window.
Technical contact One of your analysts, or a contact at a supplier (the web agency, the host) Who looks after the DNS and hosting. One or the other, never both. (Analysts added in 3.1.0, #162.)
Customer A person from Users in the domain's own company, or a supplier, optionally with one of its contacts Who you look after the domain for. A person or a supplier, never both. (Suppliers added in 3.1.0, #162.)

Both are set in the Edit dialog. The technical contact is one list with Analysts and Supplier contacts groups. The customer is one search box: type two letters and it offers People (in the domain's company) and Suppliers and contacts together. Pick a contact and their supplier comes with them. On the domain's page each name links to its People page β€” a person's, a supplier's or a contact's β€” and that page lists every domain they are named on.

If you only manage your own domains, leave Customer empty and nothing is different. If you manage domains for customers, the customer can be one of your users or an organisation you keep in Contracts β†’ Suppliers, whichever fits how you record them. See also Contracts with customers.

Works on a phone: the register's views and filters live behind a Filters button, and the list, accounts and a domain's page are laid out for a narrow screen - see Mobile: Domains.

On a multi-company install each domain belongs to one company; see the help page's Companies section. Calendar entries have no company, so domain and certificate dates in the Calendar are visible to everyone who can open it. Connections keep to the company too: a CMDB item or ticket can only be linked to a domain in the same company (Service Status services and knowledge articles are not company records).

Connections are not in the REST API yet - they are made and read on the screens.