Freshservice data sync - SS67/project-docs GitHub Wiki

Yeah, got the gist. You want to extend Freshservice as source of truth beyond vSphere into security and patch state per asset. Different sources, same asset records.

Before I draft anything, a few requirements to lock down:

1. Asset scope. Same VMware VCenter VM asset type (55000745586), or does this also touch physical servers, workstations, non VM Windows endpoints? Defender/Intune usually covers endpoints too.

2. Match key strategy. vSphere sync keys on `instance_uuid`. Defender/Intune/SCCM/TVM key on device name, AAD device id, SCCM resource id, or something else. Need to know what unique field each source exposes and which one you trust as join key back to the Fresh asset.

3. Fresh fields. Are the target fields already created in Fresh, or does the Fresh team still need to add them? Rough list I’d expect: `defender_status`, `defender_last_seen`, `intune_compliance`, `intune_last_checkin`, `tvm_risk_score`, `tvm_critical_count`, `sccm_last_patch`, `wsus_pending`, `patch` (yes/no). What’s the actual field id map look like on your side?

4. Data source access.
– Defender/Intune/TVM: Graph API + Defender API, app registration with client credentials? Already have one, or need to request?
– SCCM: WMI on SMS provider, MECM admin service REST API, or direct SQL against CAS?
– WSUS: PowerShell remoting to WSUS server, or SUSDB read?

5. Patch field logic. “Yes/No based on latest Ansible patch play result”. Yes means what exactly. Last patch play succeeded on that host in last N days? Or host was in the target inventory of the last successful play regardless of when? Pilot on Prod first, then Dev, but what defines Prod vs Dev at the AAP layer, separate job templates, separate inventories, tags?

6. Cadence. vSphere sync is hourly. Vulnerability data changes slow (daily). Patch status changes on patch windows (weekly/monthly). Do you want per source schedules, or one unified enrichment run?

7. Write model. Extend the existing `freshservice_vm_sync` role to accept pluggable enrichment sources, or separate roles/playbooks per source that PUT to the same asset? I’d lean separate, cleaner blast radius, but your call.

8. Failure handling. If Defender API is down, do we skip enrichment silently, hold last known value, or stamp the field null? Same question for each source.

Answer what you can, skip what you don’t know yet. Where you don’t know, I’ll flag it as an open question in the design.

1. How to ingest non-vsphere sources ?

Layer 1: collectors. One script per source, same shape as `fs_inventory_snapshot.py`. Each pulls its source and writes a normalized staging file to `/var/lib/awx/fs_staging/latest.json`. Every record carries all identity keys the source knows (hostname, FQDN, serial, BIOS UUID, AAD device id, SCCM resource id) plus payload plus `collectedat`. This mirrors the BMC guidance of giving each data provider its own import dataset and merging into a consolidated dataset instead of writing sources straight into production. Your staging dir is that consolidated dataset.

For Defender/TVM specifically, use the batch export APIs, not per device Graph calls. The JSON export pulls all org data paginated and is fine under 100k devices, and delta export gives changes since a timestamp, with vulnerability data regenerated every 6 hours. Per device Graph loops fall over at your scale. Same lesson someone else learned: querying windowsProtectionState per device works but making thousands of requests is crazy, batch report export is the way.

Layer 2: identity map. One module, `fs_identity_resolve.py`. Builds a persisted map of Fresh asset id to all known aliases. Join precedence: serial or UUID first, FQDN second, bare hostname last. This is exactly what identification rules do in ServiceNow: decide how to uniquely recognize a CI before any reconciliation happens. You already do this implicitly with `instance_uuid` in the reaper. Pull it out into its own layer so all sources share it.

Layer 3: one generic writer. `fs_enrich.py`, which is `fs_apply.py` generalized. Takes staging file plus a field mapping config plus identity map. Computes delta against the Fresh snapshot you already refresh every 15 min. PUTs only changed fields. Dry run default, per run cap, cascade halt on match rate collapse, ledger append. One engine, one YAML config block per source. You never write five bespoke apply scripts.

The one rule that makes it stable: single writer per field. Each Fresh field is owned by exactly one source. Defender owns `defender_*`, SCCM owns `sccm_*`, your patch play owns `patch`. No two sources ever write the same attribute. This is the core of every enterprise reconciliation engine. ServiceNow makes precedence explicit because without data source precedence rules, data sources overwrite each other and CI data flips back and forth between integration syncs. You avoid building a precedence engine entirely by never sharing a field. Cheaper and cannot flip flop.

Also give every field group its own timestamp: `defender_last_seen`, `sccm_last_seen`, `patch_last_run`. Provenance per record is standard practice, capture discovery provenance like last_seen_by on every record so reconciliation decisions stay explainable. It also gives you per source staleness reporting for free, same trick as your refresh rule.

Patch field specifics. Do not make it a raw event flag. Make it computed. Patch play on success runs `set_stats` per host or appends to `ansible_patch_latest.json` in staging. Then `fs_enrich` computes `patch = Yes` if last successful run is within N days (30 matching your patch cycle, tune it), else `No`. Two wins: the value decays on its own when a host misses a window, and a failed play run cannot silently leave a stale Yes. Pair it with `patch_last_run` timestamp so Yes/No is auditable. Pilot: scope the enrichment config to the Prod inventory group only, Dev is a config change later, zero code.

Failure handling. Collector fails, staging file does not update, enrichment skips that source because `collected_at` is stale. Hold last known value in Fresh, never null fields on API failure. The per source last_seen tells consumers the data is old. Add a small coverage report per source, reaper style.

Cadence. Defender/TVM daily (data only regenerates every 6h anyway). Intune daily. SCCM daily plus a run after each patch window. Patch field: event driven from the play plus a daily recompute for decay. All staggered like your existing UTC slots.

Where I push back. Two things. First, confirm assumption 5 before building anything for WSUS. If SCCM drives updates, WSUS is not a dataset, it is plumbing. Second, the asset type gap is the real design risk, not the pipeline. Defender and Intune will return laptops and physical boxes with no Fresh record and no VMware asset type home. Decide with the Fresh team where non VM endpoints land before the first collector runs, otherwise your identity map fills with unmatchable records on day one.

Rollout order. Build `fs_enrich` engine plus identity map first. Then one collector end to end as the pattern proof, I would pick SCCM patch status since it feeds your pilot goal directly. Then Ansible patch field. Then Defender/TVM. Then Intune. WSUS only if assumption 5 is wrong.

1. Will audit ledger ensure insights and with that the accuracy of Asset Inventory?
2. Can I use that as live feed to Grafana dashboard for each sync action, spot inaccuracy and correct?

Honest answer in two parts.

Does the ledger ensure accuracy? No.

The ledger records what our pipeline changed. It says nothing about whether Fresh matches reality. It is a write log, not a truth check. Two gaps prove it:

1. It only sees our writes. Your bulk import incident (1835 external assets) never touched the ledger. Fresh drifted and the ledger was silent.
2. A wrong write gets logged just as happily as a right one. Ledger says what happened, not whether it should have.

Accuracy comes from the reaper. That is your reconciliation engine. Match rate, category counts, staleness distribution, halt events. Those are the accuracy signals. The ledger gives you traceability, forensics, and volume trends on top.

So the dashboard needs both feeds. Ledger for actions, reaper output for accuracy. Ledger alone gives you a very confident view of half the picture.

Can you feed it to Grafana? Yes, and it fits well.

`audit_ledger.jsonl` is append only JSONL. That is the ideal shape for log shipping.

Clean pattern:

- Grafana Alloy (or Promtail) on cp01 tailing `/var/lib/awx/fs_reports/audit_ledger.jsonl` into Loki. Live tail of every sync action, filter by action, reason, asset. Near real time within seconds of a write.
- Small exporter or a textfile collector job that parses `reaper_latest.json` into Prometheus metrics after each reaper run. Cat 1 through 5 counts, healthy match count, needs_review count, eligible count. Hourly resolution, matching your reaper cadence.
- Alerts on: cascade halt (exit 2), Cat 5 spike, match rate drop, ledger silence during a scheduled window, retire count above cap.

Two caveats:

1. “Live” is bounded by your schedules. Ledger lines appear only when a job writes. Between runs the feed is quiet by design. That is fine, just do not expect a stream.
2. Everything lives on cp01 local disk because of the `cp01_only` pinning. So the shipper must run on cp01, or you accept that this is one more thing tied to that node. Loki ingestion actually softens that problem since the history leaves the box.

On “spot and correct”: spotting in Grafana works. Correcting should still go through fs_apply and fs_retire with their guards. Do not build a correction path from the dashboard. Dashboard finds it, existing tools fix it, ledger records it. Loop closed.

One schema note before you wire this. Make sure every ledger line has consistent keys: timestamp, action, source, asset id, field, old value, new value, reason, run id, result. Consistent keys make LogQL parsing trivial and dashboards cheap.

Question before I go further: do you already have a Loki and Prometheus stack running, or would this be net new infra?

1. Describe staging dir. Is it just a comprehensive JSON format report to be consumed in fresh?
   Not a report, and not consumed by Fresh directly. Think of it as a decoupling buffer between collectors and the writer.

`/var/lib/awx/fs_staging/` holds one normalized snapshot file per source:

/var/lib/awx/fs_staging/defender_latest.json
/var/lib/awx/fs_staging/intune_latest.json
/var/lib/awx/fs_staging/sccm_latest.json
/var/lib/awx/fs_staging/ansible_patch_latest.json

Each file is the full answer to one question: what does this source say about every host it knows, right now.

What it is:

1. A cache of source state. Same role your `fs_vmware_assets.json` snapshot plays for Fresh. Collector talks to the API once, everything downstream reads the file.
2. A normalization boundary. Defender speaks Graph, SCCM speaks admin service, the patch play speaks set_stats. Collectors translate all of that into one common record shape. `fs_enrich` never learns any vendor API.
3. A failure buffer. Defender API down means the file just stays old. `fs_enrich` sees stale `collected_at` and skips that source. No partial writes, no nulls pushed to Fresh.
4. Evidence. When Grafana shows a weird value in Fresh, you diff the staging file against the ledger and know whether the source lied or the writer did.

What it is not: nothing in Fresh ever reads it. Fresh only ever receives targeted PUTs of changed fields from `fs_enrich`. The staging file might hold 40 attributes per host while only 5 are mapped to Fresh fields. That is deliberate. Collect wide, write narrow. You can map new fields later without touching collectors.

Record shape, roughly:

{
  "source": "sccm",
  "collected_at": "2026-07-08T10:15:00Z",
  "record_count": 2014,
  "records": [
    {
      "identity": {
        "hostname": "winapp01",
        "fqdn": "winapp01.example.com",
        "serial": "VMware-42 1a ...",
        "sccm_resource_id": 16778341
      },
      "payload": {
        "last_patch_time": "2026-07-01T03:12:00Z",
        "pending_updates": 0,
        "client_health": "Active"
      }
    }
  ]
}

Identity block feeds the identity map. Payload block feeds the field mapping. `collected_at` feeds the freshness guard. Same three ingredients for every source, which is why one generic writer works.

So the flow per source is: collector writes staging file, `fs_enrich` reads staging plus identity map plus Fresh snapshot, computes deltas, PUTs changes, appends ledger. Staging is the middle stop, never the destination.

Same cp01 local disk treatment as your other dirs. Placeholder dirs on the other nodes, add to `AWX_ISOLATION_SHOW_PATHS`, pin jobs to `cp01_only`.

1. Makes a lot of sense.
2. I was finishing up list of new asset types – Should I create new ones for each like defender/Rapid7/SCCM/WSUS/Intune/Jamf
   or group them based on their relevance? like Rapid7/Defender – Security
   SCCM/WSUS/Linux Patch/Ansible Patch Status – Patch Compliance
   Intune/Jamf – <NOT SURE, YOU HELP>

Hold on. Pushback needed before you build that list.

Asset types describe what a thing is. Not where data about it comes from. Defender, SCCM, Intune are data sources, not kinds of assets. If you create an asset type per tool, the same host ends up as three or four separate asset records, one per source. That kills the single record model, kills the identity map we just designed, and recreates your bulk import mess on purpose. Freshservice would happily let you do it and you would regret it within a month.

The rule: one host, one asset record. Sources land as field groups on the existing types.

Your grouping instinct is right, just apply it to field groups, not asset types:

The missing label you asked for: Endpoint Management. Intune and Jamf are both MDM. Same domain, different OS. Device Management works too if the Fresh team prefers.

Two details that matter:

1. Keep the vendor prefix in field names even inside a domain group. Defender and Rapid7 both report vulnerability posture, but if they share one field they will fight over it. Vendor prefixed fields preserve the single writer rule. The domain grouping is for display sections and human sanity, not for merging data.

2. Place fields at the highest common parent, not per leaf type. Patch and security apply to VMs, physical servers, and laptops alike. Define them once at Hardware (55000367248) or Computer (55000367253) level so every child inherits, same way your `asset_state` already works. Do not duplicate them on the VMware VM type.

Where new asset types are actually legit: device classes that have no home today. Defender, Intune, and Jamf will surface laptops, desktops, maybe mobiles. Those need types like Laptop or Desktop under the Computer parent because they genuinely are different kinds of assets. Type by device class, fields by domain group, never type by tool.

Also noting two scope additions that just appeared: Rapid7 and Jamf. Rapid7 slots in cleanly as one more collector, same staging pattern. Jamf means macOS endpoints are in scope now. Confirm that is intentional, because it widens the identity map and the Fresh field conversation.

1. Now that we have finalized source intake(collectors) with single writer per filed.
2. I was wondering how to ensure an intelligent intake/reconcile process for similar category sources like – Defender and Rapid 7?
   Both act as Vulnerability detection source. Defender is better for vSphere objects, whereas Rapid 7 is better for Laptop/Desktops and other physical devices – printers, network devices, Load balancers, Firewalls etc.
   We need to come up with a very intelligent, efficient and highly optimized workflow with maximum accuracy.

Good instinct to stop here. This is the one place where two sources legitimately overlap on the same domain, and it is exactly the problem ServiceNow IRE and BMC solve with precedence rules. We can steal the pattern without building their machinery.

Core idea: three field layers, single writer preserved.

1. Raw layer. `defender_*` and `rapid7_*` fields stay vendor prefixed, each owned by its own collector. They never merge, never conflict. Unchanged from what we designed.
2. Rollup layer. New computed fields: `vuln_risk`, `vuln_critical_count`, `vuln_last_scan`, `vuln_source`, `vuln_coverage`. Owned by exactly one writer, a small reconciler step inside `fs_enrich`. Single writer rule holds because the reconciler is the only thing that touches `vuln_*`.
3. The reconciler picks which raw layer feeds the rollup, per asset, using a precedence map.

Precedence by device class, not global. This is the ServiceNow model: precedence is defined per class and per attribute, so different sources win for different CI classes. Yours looks like:

Class comes from the identity map plus Fresh asset type. One YAML block, easy to change, fully auditable. Making source precedence explicit and auditable per class is the documented enterprise practice, along with recording provenance on every record so decisions stay explainable. Your `vuln_source` field is that provenance.

Staleness failover, the intelligent part. Static precedence alone is dumb. If the Defender sensor on a VM dies, Defender keeps winning with month old data. So the reconciler applies one rule per asset:The stale primary override is not my invention. ServiceNow added dynamic reconciliation rules for exactly this: specify when a CI is stale for a specific discovery source and override the static rule so a lower priority source can update it. We get the same behavior with one timestamp comparison.

Score normalization, do not skip this. Defender exposure score and Rapid7 risk score live on different scales. Blending them into one number is garbage math. Reconcile on what both agree on: CVE derived severity counts (critical, high, medium). Those are comparable because both vendors map to CVSS. Vendor proprietary scores stay in the raw vendor fields for anyone who wants them. `vuln_risk` becomes a simple bucket computed from counts, one formula, source agnostic.

The accuracy multiplier: cross source coverage reporting. The overlap you were worried about is actually your best audit tool. Reconciler emits a coverage report per run:

- Expected primary has no record at all: Defender sensor never onboarded on a VM, or Rapid7 not scanning a subnet. This inventory delta comparison is the standard way to find silent onboarding gaps, since devices existing in inventory sources but failed to onboard come from broken policy application, network segmentation, or broken deployment workflows.
- Primary stale, running on fallback: sensor or scan engine dying.
- Both sources fresh but severity counts wildly disagree: usually one side scanning a stale snapshot or wrong host resolution. Small bucket, high value, manual review.

That report is reaper shaped. Same JSON plus CSV output, same Grafana metrics. Coverage percentage per device class becomes a dashboard number.

One guardrail on intake. Rapid7 will discover printers and network gear that have no Fresh record. Do not let a scanner auto create assets. Scanners see DHCP ghosts and duplicate interfaces. Same concept exists in IRE: a data source can be blocked from inserting new CIs for a class while still being trusted to update existing CIs. So Rapid7 is update only, and unmatched records land in a pending seed report, human reviewed, exactly like your Cat 4. Seeding automation can come later once you trust the match quality.

Net build cost: one YAML precedence map, one reconciler function inside `fs_enrich`, one coverage report. No new services, no new pattern. Everything else you already have.

One requirement to confirm: do you want per CVE detail in Fresh, or posture summary only? My strong recommendation is summary plus a deep link to the Defender or Rapid7 console per asset. Fresh is an asset register, not a vulnerability management platform. Stuffing CVE lists into asset fields bloats the API payloads and duplicates what the consoles already do better.

1. end to end workflow in one flow diagram

Here is the full picture, top to bottom. Read it as data flowing downward: sources, staging, engine, Freshservice, audit.Walkthrough to pair with it when you talk to the Fresh developer:

Stage 1, collectors. Six independent scripts, one per source, each on its own schedule. Each talks to exactly one external API and knows nothing about Freshservice. Output is a normalized JSON snapshot: identity keys (serial, UUID, FQDN, hostname, source native id), payload attributes, and a `collected_at` timestamp. A collector failing just means its file goes stale. Nothing downstream breaks.

Stage 2, staging. One file per source on disk. This is a decoupling buffer, not a report. Fresh never reads these files. We collect wide (everything the source offers) and write narrow (only mapped fields). New Fresh fields later are a mapping change, not a collector change.

Stage 3, the engine. Three steps inside one run. Identity map resolves every staged record to exactly one Fresh asset, matching serial or UUID first, FQDN second, bare hostname last. Records that match nothing go to a pending report, they are never guessed. Vuln reconciler applies only to the overlapping security sources: Defender wins for VMs, Rapid7 wins for laptops, desktops, and network gear, and if the winner’s data is older than 7 days the other source takes over and the record is flagged. Delta compute then compares mapped fields against the 15 min Fresh snapshot and keeps only actual changes. Guards wrap all of it: dry run by default, per run write cap, and a cascade halt that refuses the whole run if the match rate collapses, because that means something upstream broke, not that a thousand assets really changed.

Stage 4, the write. Only changed fields, only via PUT, spaced to respect the roughly 0.5s rate limit we have measured. The enrichment path never creates assets and never retires them. Create and retire stay exclusively with the existing vSphere sync and its guarded tools. So from Fresh’s side there are exactly two writers with clearly split powers: the vSphere sync owns lifecycle, the enrichment engine owns attribute updates.

Stage 5, audit. Every write, skip, and halt appends one line to the ledger with old value, new value, source, reason, and run id. Reports capture coverage per source and reconciliation results per run. Grafana tails both. Any value in Fresh can be traced to the exact run and source that wrote it.

What the Fresh developer specifically owns in this picture:

1. Create the field groups, vendor prefixed, at the highest sensible parent type (Hardware or Computer level) so all children inherit. Roughly: `defender_*`, `rapid7_*`, `sccm_*`, `intune_*`, `jamf_*`, `patch` plus `patch_last_run`, and the computed `vuln_*` rollup group including `vuln_source` and `vuln_coverage`. I can hand over the exact field list with types when you are ready.
2. No new asset types per tool. New types only for real device classes that lack a home today: laptop, desktop, printer, network device.
3. One API key scoped for asset read and asset update. No delete permission needed for this path.
4. Confirm the workspace rate limits so we set the write pacing and per run cap correctly.
5. Optionally two URL fields per asset for deep links into the Defender and Rapid7 consoles, since CVE detail stays in those tools, not in Fresh.

1. Jobs
   1. FS Inventory Snapshot Refresh – Pulls all VMware VCenter VM assets from Freshservice and writes them to a local JSON snapshot on cp01. Necessary because per-VM lookups against the Fresh API were too slow at 8500 VMs, so we cache the whole asset table locally and refresh every 15 min.

2. FS Reaper Report – Read-only comparison between the Fresh snapshot and current vSphere inventory, categorizing drift into 5 buckets (orphans, UUID mismatches, duplicates, pending seeds, missing-from-vSphere). Necessary to surface data quality issues before any apply or retire tool acts on them, without touching either system.

3. FS Apply – Fixes safe UUID discrepancies flagged by reaper (Cat 1 single name match, Cat 2 unique name). Necessary because manual UUID cleanup at scale was untenable, and these categories have unambiguous fixes that can be automated with cascade halt and audit ledger safeguards.

4. FS Retire – Flips Freshservice asset state to Retired for Cat 5 entries whose sync_last_seen has aged past 12h. Necessary to keep Fresh accurate as VMs are decommissioned in vSphere, without relying on manual cleanup or risking mass retires during a vCenter outage (protected by cascade spike halt and per-run cap).

1. Flow Diagram
   Yes, I read it. It is your generalized intake tree: detect asset, check the asset DB, branch by type, and if no collector exists, build one. Solid instinct, but it has three problems worth fixing before we generalize it:

1. “Exists → Ignore” is wrong. In your slide, a known asset gets ignored twice. But known assets are where most of the work lives: update changed fields, refresh staleness. Ignore on match means data never updates after day one.
2. It asks the same question twice. “Is asset in Asset DB” appears at two levels. One identity check is enough if it happens in the right place.
3. It mixes two timescales. “Create collector” is an engineering decision made once per source. “Is this asset known” is a runtime decision made thousands of times per run. One diagram mixing both confuses every audience.

The fix that makes it universal: every record from every source, existing or future, answers the same three questions. Do we know this device? Does this source own these fields? Did anything change? That is the whole pipeline, and it is the version anybody can follow.Here is the generalized version. One flow, any source, three questions.How it maps to your slide, so you can explain the changes:

1. Your “Is asset in Asset DB? Yes → Ignore” became “Known device? yes → Owned fields changed?”. Known assets get maintained, not ignored. That single change is what makes the flow work for enrichment sources, not just discovery.
2. Your “Is there a Collector? No → Create collector” moved to its own strip at the top, because it happens once per source, not per asset. Audiences stop confusing build work with runtime behavior.
3. Your “Asset Type? → VM” branch became “Source owns lifecycle?”. That one question generalizes the type check: vSphere may seed VMs, Intune may seed laptops, Rapid7 may seed nothing. Same diagram, every source, just different answers.
4. Added what your slide was missing entirely: the pending review outcome for unmatched records, and the retire lane. Without those two, the flow only ever grows the inventory.

The pitch line for any audience: every record from every source, current or future, answers three questions. Do we know this device? Does this source own this data? Did it change? Everything else is guardrails.

Adding a new source like Rapid7 or Defender never changes this diagram. You only fill in a row of answers: what it may seed (nothing), which fields it owns (rapid7 group), when it counts as stale. That is the slide after this one, a small table with one row per source, and it proves the design scales without redrawing anything.

If you want this in your deck or the project docs, I can output the same flow as mermaid source so it lives next to your existing diagrams and stays editable. Say the word.

flowchart TD
    subgraph ONBOARD[Once per new source]
        SRC[New data source] --> COLL{Collector or plugin exists?}
        COLL -->|yes| SCHED[Joins the schedule<br>writes staging file]
        COLL -->|no| BUILD[Build collector<br>dry run soak] --> SCHED
    end

    SCHED --> STAGE

    subgraph RUNTIME[Every record, every run]
        STAGE[Record in staging<br>identity, data, timestamp] --> KNOWN{Known device?<br>serial, UUID, FQDN match}
        KNOWN -->|no| OWNER{Source owns lifecycle?<br>vSphere owns VMs}
        KNOWN -->|yes| DELTA{Owned fields changed?<br>single writer rule}
        OWNER -->|yes| SEED[Seed asset<br>stamp last seen]
        OWNER -->|no| PEND[Pending review<br>human decision]
        DELTA -->|yes| UPD[Update fields<br>PUT plus ledger]
        DELTA -->|no| SKIP[Refresh or skip<br>bump last seen if stale]
    end

    subgraph REAPER[Separately each run, the reaper]
        UNSEEN[Unseen past threshold<br>no source reports it] --> RETIRE[Guarded retire<br>capped, halted, logged]
    end

Two notes. I expanded the collector decision into explicit yes and no paths since mermaid handles that cleanly. And the reaper subgraph is intentionally disconnected, it runs on its own schedule, mermaid renders it as a separate block which is exactly the message.

Drops straight into the project docs next to your existing diagrams.

1. Phase2:
   please review the following “Purpose” description for Intune collector.

Build an Intune collector that retrieves managed device data from Microsoft Intune, writes a normalized JSON snapshot, and feeds the Global Asset Reconciliation engine for identity matching, owner attribution, enrichment, reporting, and future automation.

The collector should follow the established non-vSphere pattern: one collector per source, one external API, no direct Freshservice writes, normalized snapshot output, and downstream reconciliation handled by the identity/enrichment engine.

Once the above has been confirmed / rewritten, we can get into design principles, target arch, data flow, and conclude mvp scope.

Workflow *
Microsoft Intune
↓
Intune Collector
↓
Normalized Snapshot
↓
Identity & Reconciliation Engine
↓
Enrichment Writer
↓
Freshservice
↓
Reporting / AI / Automation