Developer Tests Integrations - edmozley/freeitsm GitHub Wiki

πŸ§ͺ Developer Tests β€” Integrations & external services

Part of Developer Tests. Issue trackers, AI providers, CalDAV, Microsoft Graph mail folders, and the inventory agent's ingest endpoints.

πŸ”‘ The recurring idea on this page: test the decision, not the network. Most of these suites deliberately stop at the boundary β€” they prove what FreeITSM sends and decides, using a fixture or a local stand-in, and say plainly what they cannot prove. A green run here is not a working integration; it is a correct request. The exception is caldav-provider.php, which really does drive a server.

Test Needs
integrations/run.php Nothing
integrations/templates_check.php Database
azure-openai/run.php Nothing (starts its own server)
caldav-provider.php A running Baikal fixture
mailbox-folder-resolve.php Nothing
agent-empty-report-guard.php Database

tests/integrations/run.php

What it tests

IssueDoc and the provider contract for external issue trackers.

IssueDoc is the piece every later phase writes through β€” descriptions first, comments next, everything after. If it is wrong, the next phase is where you find out and where you would have to rip it up. So it gets a real suite now rather than an eyeball.

Four renderers produce four genuinely different syntaxes from one document, and the risk is that they drift β€” one quietly stops escaping, or handles an empty paragraph differently. So every case runs through all four and each is asserted separately.

⚠️ Every negative assertion is paired with a positive control. "The asterisk did not become italics" proves nothing on its own β€” it is equally true of a renderer that dropped the text entirely. So each escaping test also asserts the surrounding content still arrived.

Run it

php tests/integrations/run.php

353 assertions. No database, no network, pure functions β€” fast and safe to run anywhere. This is the best test to run first on a machine you have just cloned to.

Reading the result

Ends with a passed: / failed: block rather than the one-line summary most other tests use.

If it fails

The label names the renderer and the case. Fix the one renderer β€” do not "normalise" all four to agree, because they are supposed to differ; they target different syntaxes.

tests/integrations/templates_check.php

What it tests

That every workflow starter recipe can actually run.

workflow/includes/templates.php promises in its own header that "every trigger_event here is a real, wired trigger … and every action type is a real handler β€” a recipe that can't actually run would be worse than no recipe at all". Nothing enforced that, and the first tracker recipe shipped with an add_note action that does not exist (it is add_ticket_note).

For every template, not just the tracker ones:

  1. the trigger exists
  2. every action type exists
  3. every arg name exists on that action
  4. every required arg is supplied, or is a $configure marker the user fills in
  5. it resolves against this install without throwing

Run it

php tests/integrations/templates_check.php

⚠️ Separate from run.php on purpose. WorkflowEngine::availableActions() reads webhook formats from the database, and run.php's whole value is that it needs no database and no network. Do not merge this back into it.

If it fails

A recipe offers an action or an argument that does not exist. Someone renamed a handler without updating the templates β€” fix the template, since the handler name is the real one.

tests/azure-openai/run.php

What it tests

Azure OpenAI's deployment-based endpoints. Exactly three things are new compared with the OpenAI path this codebase has shipped for months: the URL shape, the api-key header, and the absence of model β€” and all three are things we emit, so all three are assertable.

It also checks the three ways an administrator might paste an endpoint (with or without a trailing slash, with or without /openai) all normalise to the same URL, and that a deployment name containing a space is escaped β€” an unencoded space breaks the request line.

Since GH #171 it also covers: a pasted Target URI or Foundry project address cut back to the resource; the v1 route (no query string, deployment sent as model); a model that refuses temperature: 0, and one (gpt5-like) that refuses both that and max_tokens and must succeed in exactly three requests - including through tool calling; #171's exact failure (a model version date as the API version answered with a 404, not retried, and explained as the API version); and a behavioural control that an unrelated 400 (a content filter) surfaces after one attempt. That control used to grep the source for the old retry line, which proved nothing about behaviour and broke the moment the retry moved.

How it works

It starts its own php -S on a free port, serving only the tests/azure-openai/ directory, and stops it again at the end. mock.php is a stand-in Azure endpoint that records the request it received; the test then reads that back and asserts on the bytes we sent. The real cURL path is exercised rather than stubbed.

The mock is the one file in tests/ allowed to answer HTTP, and it accepts only the built-in server β€” it refuses under Apache, nginx and php-fpm.

⚠️ On Windows proc_open() must be given bypass_shell, or the server is a grandchild of cmd.exe and proc_terminate() leaves it running.

Run it

php tests/azure-openai/run.php

48 assertions. Needs nothing else running.

Reading the result

πŸ”΄ It ends with a warning, and the warning is the point:

Green here means WE SEND THE RIGHT REQUEST. It does not mean Azure accepts it β€” that needs a real tenant.

Nobody on the project has an Azure subscription to point it at. Content filtering, api-version drift, quota and regional behaviour are all outside what this can prove. Debug tool D014 exists for that gap: it makes one live call, prints the whole exchange and reads back Azure's answer (see AI Providers).

If it fails

"the mock server never came up" is an environment problem, not a code one β€” check nothing is blocking loopback sockets. Anything else means the request we build has changed shape.

tests/caldav-provider.php

What it tests

The CalDAV provider against a real server. The cases that matter:

  • an edit keeps what the analyst added (a reminder) β€” the rule this provider exists to keep
  • a stale change token gives a baseline, never "everything was deleted"
  • a calendar address on another host is refused before any request β€” the per-analyst address must not become a way to reach anywhere
  • the analyst's own appointments are never reported, only ours

Run it

It needs the Baikal fixture, seeded:

docker compose -f docker/carddav-test/docker-compose.yml up -d
bash docker/carddav-test/seed.sh
php tests/caldav-provider.php

Touches no database. Everything it writes is a FreeITSM-named event in the throwaway itsm and tech2 calendars, and it deletes what it made.

Reading the result

If you see:

  FAIL  the address is a calendar server  <- Failed to connect to localhost port 8092
  Is the Baikal fixture running and seeded?

…the fixture is not up. That is an environment failure, not a code failure β€” start the fixture and run it again.

If it fails with the fixture running

The "stale token gives a baseline" case is the one with teeth: getting it wrong means a sync deletes a calendar's contents rather than resynchronising it.

tests/mailbox-folder-resolve.php

What it tests

mailboxResolveFolderId() β€” reading mail from a folder that is not the Inbox.

/mailFolders/<x>/messages does not take folder names. Graph accepts a short list of well-known aliases there and treats everything else as an opaque folder id, so a mailbox told to read freeitsm got:

400  ErrorInvalidIdMalformed β€” "Id is malformed."

INBOX had always worked purely because it is on the alias list. Reading a folder by name had never worked for any name off it.

How it works

The resolver takes its HTTP fetcher as an argument, so the whole thing runs against a fixture with no network and no mailbox. The fixture deliberately contains a custom top-level folder and a same-named one inside the Inbox, so freeitsm and Inbox/freeitsm must not resolve to the same place.

What it cannot prove is what Graph really returns.

Run it

php tests/mailbox-folder-resolve.php

15 assertions.

If it fails

A path resolving to the wrong folder means a mailbox silently collects from somewhere else β€” which looks to the operator like mail going missing.

tests/agent-empty-report-guard.php

What it tests

That an empty agent report does not empty the machine's inventory.

πŸ”΄ Both ingest endpoints wipe an asset's hardware tables and reinsert on every report. The DELETE ran unconditionally while the INSERT was guarded by !empty(...) β€” so a report that collected nothing for a category emptied that category and left it empty. A WMI blip took 226 Device Manager rows with it, with no error anywhere, until the next good run. Any client holding an API key could do it deliberately with {"disks":{}}.

An empty list now means "I did not find out", not "there is nothing there": the wipe is skipped and the section is named in the response, because looking like a successful sync of nothing is the other half of the bug.

Run it

php tests/agent-empty-report-guard.php

ZZEG-prefixed, cleaned up including on failure.

If it fails

This one destroys customer data silently and is only noticed later. Treat any red here as blocking, and check both ingest endpoints β€” the bug was in both.