Polling Methods - rogers-group-inc/polaris GitHub Wiki

Polling methods

Every stream on every asset resolves to exactly one polling method, computed per stream from a four-tier hierarchy.


The hierarchy

per-asset  *Polling column
  → MonitorClassOverride
    → Integration.config.monitorSettings.polling
      (or the Manual tier, for assets with no integration)
        → the source default

Where each tier is edited:

Tier Where
Manual Assets → Settings → Manual Monitoring. Accepts every method except fortimanager — it covers any source
Integration the integration's edit modal → Monitoring → Cadence & Retention. Filtered to the integration's source kind
Class override Assets → Settings → Class Overrides. Re-renders when the source picker changes
Per asset the asset's edit modal → Monitoring. Filtered to the asset's source kind

GET /assets/:id/effective-monitor-settings returns the resolved values plus per-field provenance (asset / class / integration / manual), which is what draws the tier badges in the asset modal.

Every Inherit option is labelled with what you would actually inherit, so you can see the fallback without clicking through. Where a tier sets the method it names the tier and the method — "Inherit (FMG-CORE: REST API)", "Inherit (class override: SNMP)" — and otherwise the source default: "Inherit (Source Manual: ICMP)", or "… : not delivered" for a stream the source has no default for. Manual Monitoring, the bottom tier, has no Inherit option at all.

A method that does not apply to the asset's source kind is silently ignored at resolution time; the route layer rejects it at write time with a clear 400.


The methods

Method Notes
icmp the cheapest universal liveness probe. Batched
snmp v2c / v3 authenticated GETs
ssh Linux / Unix hosts — see agentless host streams
winrm Windows hosts — same
rest_api FortiOS REST. Outside a Fortinet integration it drives the response-time probe only
agent the Polaris Agent installed on the host — every stream but LLDP. Never appliances
vcenter reads the vCenter server, not the guest
fortimanager reads FortiManager's device database, not the device
unraid reads the Unraid host's API, not the VM or container
truenas reads the TrueNAS SCALE host's API, not the VM or App
proxmox reads the Proxmox VE cluster's API, not the node's OS or the guest. Shown as Proxmox
genericapi reads the monitor queries you wrote on a Generic API integration, not the device. Shown as Generic API
disabled universally allowed — do not poll this stream

An HTTP check is configured as a manufacturer custom widget (rule 33); see below.

Which source takes which method

The dropdowns offer only what the asset's source kind accepts (plus disabled everywhere):

Source Methods
FortiManager icmp, snmp, ssh, rest_api, fortimanager
FortiGate icmp, snmp, ssh, rest_api
AD / Entra / Windows Server / Azure Arc icmp, winrm, ssh, agent, vcenter, unraid, truenas, proxmox, genericapi
vCenter icmp, snmp, winrm, ssh, agent, vcenter
Unraid icmp, snmp, winrm, ssh, agent, unraid
TrueNAS SCALE icmp, snmp, winrm, ssh, agent, truenas
Proxmox VE icmp, snmp, winrm, ssh, agent, proxmox
Generic API icmp, snmp, winrm, ssh, agent, genericapi
Manual everything except fortimanager

vcenter, unraid, truenas, proxmox and genericapi appear on the directory sources because a device that Active Directory or Entra found first can later be merged with its vCenter, NAS, Proxmox or Generic API record. A workload method is never offered on another workload platform's assets. Within a source, each stream narrows the list further: ICMP is response-time only, and a method with no collector for that stream is not offered (see compatibility vs capability).


Source defaults

Source responseTime cpuMemory temperature interfaces lldp storage
FortiManager / FortiGate icmp rest_api rest_api rest_api disabled disabled
AD / Entra / Windows Server / Azure Arc / Manual icmp — — — — —
vCenter vcenter vcenter — vcenter — vcenter
Unraid icmp; unraid for an asset with no IP unraid unraid (host) unraid (host) — unraid (host)
TrueNAS SCALE icmp; truenas for an asset with no IP truenas truenas (host) truenas (host) — truenas (host)
Proxmox VE icmp; proxmox for an asset with no IP proxmox — proxmox — proxmox (node)
Generic API icmp — — — — —

— means not delivered until you pick a method. processes and eventLog default to disabled on every source — they are opt-in — except that an installed Polaris Agent turns its own process inventory on (below).

Response time defaults to ICMP wherever there is an address to ping, because ICMP is the cheapest universal liveness probe. The exceptions read a manager instead: vCenter assets (vcenter — power and connection state, no guest IP needed) and Unraid / TrueNAS / Proxmox VE workloads with no address of their own. Operators wanting a heavier transport — FortiOS /sys/status, SNMP sysUpTime — opt in per asset, per class, or at the integration tier.

FortiOS storage defaults to disabled because FortiOS appliances expose no meaningful mountable storage, so the SNMP walk would burn cycles every scrape. Flip it to snmp per asset where you have a device that does.

The FortiManager proxy gotcha

Those rest_api defaults assume Polaris can make a FortiOS call to the device. On a FortiManager on the proxy transport with no FortiGate API token it cannot: every FortiOS collector dials the asset's own IP and needs that token.

So on that configuration the defaults become disabled for cpuMemory, temperature and interfaces, and a stored rest_api is skipped the same way — non-destructively, since the stored value returns the moment a token is supplied. The dropdowns grey out with a note naming the token as the thing that unlocks them.

A per-asset REST API credential is the other way out: a restapi-typed credential selected on an asset's stream authenticates that one gate, and a stream carrying one is exempt from the skip (per-asset FortiOS REST credentials).

Response time is deliberately unchanged (icmp): moving it would silently change how up/down is decided for every gate on every affected install.

Note the condition is not "proxy mode" — proxy with a token is a legitimate setup (discovery and writes ride FMG, monitoring reaches the gates directly) and keeps the normal REST defaults.

The one exception is a managed FortiSwitch / FortiAP's response time, which stays rest_api: that read goes to the parent gate's controller table, the one monitoring path that honours the proxy setting, and works on FMG's own credential.


Agentless host streams

ssh and winrm deliver cpuMemory, interfaces, storage, processes and eventLog.

Linux (SSH) Windows (WinRM)
CPU /proc/stat read twice, a second apart — CPU is a rate Win32_OperatingSystem
Memory /proc/meminfo (MemAvailable) same
Interfaces /sys/class/net/* Get-NetAdapter + Get-NetAdapterStatistics
Storage df -PkT Get-Volume
Processes inventory + a 60 s pinned/mapped sub-pass same
Event log journalctl -o json -p warning Get-WinEvent levels 1–3

Interfaces and storage ride one connection per host per tick when both are on the same transport.

Two design notes on the event-log stream:

  • The window is derived from the stream's own interval (1.5×) rather than a row count. Asking for "the last N entries" either misses a burst or re-ingests a quiet hour; asking for "everything since a bit longer ago than my last run" does neither. The deliberate overlap is cheap because the sink dedupes.
  • The cadence anchor is stamped even on failure. An agentless stream that re-fires every tick against a bad credential is an AD-lockout risk, and this is the one stream where a retry storm would also flood the table it writes into.

It stays opt-in everywhere (PII and volume), gated by the same global master switch the agent's event log uses — an operator who turned the feature off must not have an agentless poller keep filling the audit log.

processes is the exception: it is agent-default-ON, so an agent host collects its process inventory automatically.


The vcenter method

Covers four streams — responseTime, cpuMemory, interfaces and storage — and is the source default for all four, on both VMs and ESXi hosts. That is what lets a vCenter fleet be monitored with no credential in the guest, no SNMP on ESXi, and no reachable guest IP.

Two warm caches back it (30 s TTL, one SOAP round trip per integration per tick): VM quick-stats, and a host snapshot paired with the datastore inventory in one session.

Absent is not unreachable. vCenter answering without this asset in it is a real finding and fails the probe; Polaris being unable to ask skips it. So a vCenter outage can never declare an entire virtual fleet down at once.

A host's interfaces arrive nested: each vSwitch is written as an aggregate row with its uplink pNICs and VMkernel ports stamped as children — the FortiSwitch-trunk shape, so the System tab nests it unchanged. A vSwitch's operational status is derived (up while any uplink is up, and null with no uplinks at all — an internal-only vSwitch is not an outage).


The unraid and truenas methods

Each reads the host's own API for every asset on that host — the host, its VMs and its containers (Apps on TrueNAS). No credential in any guest, no SNMP, and no reachable workload IP is needed.

Stream Host VM Container / App
responseTime the host's API answered the platform's running state the platform's running state
cpuMemory yes no — use the agent or SNMP / SSH / WinRM in the guest yes
interfaces yes — —
storage the pools: the Unraid array and cache pools, TrueNAS ZFS pools — —
temperature disk temperatures — —

One cached read per integration per 30 s answers every asset on the host. Response time defaults to ICMP for any of these assets that has an address of its own (the host, a container on its own network, a VM whose IP another source supplied). The method answers response time only for a workload that cannot be pinged — a bridged container, an App, a VM with no known IP — and there the platform's own state decides: running is up, stopped is down, a state in transition (TrueNAS's DEPLOYING, for example) is skipped with no verdict, and the reading is charted at 0 ms, because a state read has no latency. If the host's API cannot be reached, a host on the method is reported down (its own API is the finding) and its VMs and containers on the method are skipped — no down storm across the workloads. A host on its ICMP default is judged by its ping instead.

Unraid samples per-container CPU and memory over a short WebSocket stats window (4 s by default); TrueNAS reads host usage from its reporting.realtime event and per-App usage from app.stats. See Unraid and TrueNAS SCALE.


The proxmox method

Reads the Proxmox VE cluster's own API for every node, VM and LXC container the integration discovered. No credential in any guest, no SNMP, and no reachable guest IP is needed. Any configured node answers for the whole cluster.

Stream Node VM Container
responseTime the cluster reports the node online the guest's running state the guest's running state
cpuMemory one node-wide CPU figure; memory as Used / ZFS ARC / free a share of the VM's own vCPUs, while running same, while running
interfaces one all interfaces row (Proxmox has no per-NIC counters) one row, every NIC summed one row, every NIC summed
storage the node's ZFS pools and other active storage, with Ceph health on Ceph storage — —

One cached read per integration per 30 s answers every asset in the cluster; storage is re-read at most every five minutes. Response time defaults to ICMP for any asset with an address — every node, and every guest whose IP Polaris knows. The method answers response time for a guest with no known address: running is up, stopped is down, charted at 0 ms. A node its peers report offline probes down, and its guests are then dependency-suppressed. If no configured address answers, nodes on the method fail and guests on it are skipped. See Proxmox VE.


The genericapi method

Reads the values a Generic API integration's monitor queries return for the assets it discovered — up / down, CPU and memory, interfaces, storage, temperatures and LLDP neighbors. You write the queries on the integration's Monitoring tab; see Generic API.

It is not a source default. A monitor query left on Use for this feed's assets makes genericapi the integration's default for its stream, the same as choosing it on another integration's Monitoring tab. You can still pick another method per asset.

What happens Response-time probe
the API lists the asset with an up value up, charted at the latency the API reports (or 0 ms)
the API lists the asset with any other value down
a complete read does not list the asset, or a per-asset request answers 404 down
the API cannot be reached, refuses the credentials, or a read stops part way skipped — one API outage never marks a fleet down
the integration has no query for the stream down, with the reason

Interfaces, storage and LLDP ride the system-info pass. Temperatures ride the CPU / memory schedule, as every sensor reading does, so they need a CPU / memory method on the asset.


The fortimanager method

Response-time only, on the FortiManager source only. It asks FortiManager's own device database instead of touching the device: one native /dvmdb call covers the whole fleet, so ~187 gates cost one round trip — and it reaches a gate Polaris has no route to, since only FortiManager must be reachable.

Keyed by serial, never the FMG device name. FortiManager's device name diverges from the gate's configured hostname on real fleets, and a name mismatch would read as "gone from FortiManager".

FortiManager unreachable ⇒ skipped, never a miss. One manager outage must not down a fleet.

What it trades away, and why it is one stream rather than a transport: the reading is FMG's, on FMG's check-in cadence, so it is lagged and second-hand — it says the chassis is talking to its manager, not that it is serving traffic. It carries no uptime, so it drives no reboot detection. And FMG's database holds no CPU, memory, temperature, interface counters or session count.

Not a source default. Response time stays ICMP, because repointing a default would silently change how up/down is decided for every gate on every FMG install.


The HTTP check

A manufacturer custom widget, keyed by manufacturer plus an optional model pattern (rule 33).

The reason is ownership of the definition. As a polling method the check would ride an http-typed credential, so the row answering "how do I log in to this vendor" would also carry "which path, expecting what" — and a second path would mean a second copy of the same password. A check varies by vendor and model; a login varies by vendor.

How it behaves:

Body match load-bearing — a 200 whose body does not match fails
Order status is judged before content
Redirects never followed
Body capped at 64 KB
Credential authentication only, with no "none" mode
Auth mode stated, not guessed. Digest is a handshake — fire bare, read the challenge, re-fire exactly once
Writes AssetStateSample 0/1 pass-or-fail plus a success-only RTT gauge
monitorStatus never moved

FortiManager authentication

FortiManager 7.4.7+ / 7.6.2+ do not accept an access_token query string. Polaris uses the Bearer Authorization header exclusively, and the standalone FortiGate integration uses the same pattern.

Never call /sys/logout. Polaris authenticates with a predefined REST API Admin api-key, which per Fortinet's own best-practices guide is permanent and shares one session per user. A logout tears that shared session out from under the split-role monitor and discovery processes and causes RPC -11 "no valid session" churn.

The transport retries transient faults only (5xx / network, ≤ 2 retries, exponential backoff, serialised) and fails fast on permanent ones (401, 403, 404, 405, -11).