QuickSight Dashboard - SS67/project-docs GitHub Wiki

AWS Quicksight - ECC Compliance Dashboard โ€” Agent Handoff

Oct 9, 2026 ยท @sujay

How to use this doc

Every agent session starts by reading this doc and ends by updating it, so the next session knows exactly where to pick up. The doc has three parts:

  1. Agent handoff: paste this as the first message of every new chat (Cursor, Copilot, Claude, QuickSight Q). It sets the role, rules, workflow and clarifying questions.
  2. Tracker: the single source of truth for status, decisions, open questions and the next session's to-do. The agent updates it at the end of each session.
  3. Dashboard design: the reference: datasets, sheets, visuals, calculated field catalog and business rules. The agent checks every change against it.

Session protocol

  1. Paste the Agent handoff, then the latest Tracker and Dashboard design (or attach the exported Markdown).
  2. The agent restates the session goal, asks clarifying questions, and proposes a micro to-do of 3 to 6 items.
  3. You approve; the agent works one item at a time and verifies each.
  4. The agent closes with an end-of-session handoff block; you paste it into the Tracker.

To use it in the project repo, follow the Project directory section below and export each part to Markdown as AGENTS.md, TRACKER.md and DASHBOARD-DESIGN.md.

Project directory

All work lives in one repo named AWS Quicksight - ECC Compliance Dashboard, with one folder per task and shared dependencies in one place. The agent creates any missing folder or file at the start of a session and keeps the layout below.

AWS Quicksight - ECC Compliance Dashboard/
โ”œโ”€โ”€ README.md                         Project overview, how to run a session
โ”œโ”€โ”€ AGENTS.md                         Agent handoff prompt (from this doc)
โ”œโ”€โ”€ TRACKER.md                        Status board, decisions, open questions, next session
โ”‚
โ”œโ”€โ”€ docs/
โ”‚   โ”œโ”€โ”€ DASHBOARD-DESIGN.md           Datasets, sheets, rules, conventions (from this doc)
โ”‚   โ”œโ”€โ”€ COLUMN-DICTIONARY.md          Copied from the automation repo; do not edit here
โ”‚   โ”œโ”€โ”€ SNAPSHOT-DESIGN.md            Copied from the automation repo; do not edit here
โ”‚   โ””โ”€โ”€ sessions/
โ”‚       โ””โ”€โ”€ YYYY-MM-DD-<topic>.md     One end-of-session handoff per file
โ”‚
โ”œโ”€โ”€ dependencies/                     Shared foundations that tasks depend on
โ”‚   โ”œโ”€โ”€ datasets/
โ”‚   โ”‚   โ”œโ”€โ”€ vm_current.json           describe-data-set export
โ”‚   โ”‚   โ”œโ”€โ”€ vm_snapshot.json
โ”‚   โ”‚   โ””โ”€โ”€ vulnerability.json
โ”‚   โ”œโ”€โ”€ calculated-fields/
โ”‚   โ”‚   โ”œโ”€โ”€ shared.md                 Fields that exist in both VM datasets, creation order
โ”‚   โ”‚   โ”œโ”€โ”€ vm_current.md             Current-only fields
โ”‚   โ”‚   โ”œโ”€โ”€ vm_snapshot.md            Snapshot-only fields
โ”‚   โ”‚   โ””โ”€โ”€ vulnerability.md
โ”‚   โ”œโ”€โ”€ redshift/
โ”‚   โ”‚   โ””โ”€โ”€ views.sql                 e.g. day-level state change (LAG) view
โ”‚   โ””โ”€โ”€ config/
โ”‚       โ”œโ”€โ”€ environment_map.yml       Copy of the automation mapping
โ”‚       โ”œโ”€โ”€ domain_suffixes.md        Corporate DNS suffixes for is_domain_joined
โ”‚       โ””โ”€โ”€ thresholds.yml            Target % per control
โ”‚
โ”œโ”€โ”€ analysis/
โ”‚   โ”œโ”€โ”€ definition.json               describe-analysis-definition export (dev copy)
โ”‚   โ””โ”€โ”€ CHANGELOG.md                  One line per pushed change
โ”‚
โ”œโ”€โ”€ tasks/
โ”‚   โ”œโ”€โ”€ 01-compliance-overview/
โ”‚   โ”œโ”€โ”€ 02-compliance-trend/
โ”‚   โ”œโ”€โ”€ 03-power-lifecycle/
โ”‚   โ”œโ”€โ”€ 04-governance/
โ”‚   โ”œโ”€โ”€ 05-accountability/
โ”‚   โ”œโ”€โ”€ 06-vulnerability-blind-spots/
โ”‚   โ”œโ”€โ”€ 07-change-feed/
โ”‚   โ”œโ”€โ”€ 08-qa-reconciliation/
โ”‚   โ””โ”€โ”€ 09-publish-and-share/
โ”‚       Each task folder holds:
โ”‚       โ”œโ”€โ”€ TASK.md                   Goal, status, depends on, acceptance criteria
โ”‚       โ”œโ”€โ”€ fields.md                 Fields this task adds (links to dependencies, never copies)
โ”‚       โ”œโ”€โ”€ visuals.md                Visual specs: sheet, type, dataset, wells, filters, format
โ”‚       โ””โ”€โ”€ verification.md           Checks run and results
โ”‚
โ””โ”€โ”€ scripts/
    โ”œโ”€โ”€ export.sh                     Pull analysis and dataset definitions
    โ”œโ”€โ”€ validate.sh                   JSON and field-reference checks
    โ””โ”€โ”€ push-dev.sh                   Push to the dev analysis copy only

Rules for the agent

  • A field used by more than one task lives in dependencies/calculated-fields/; a task's fields.md only lists and links it.
  • A field used by one task is defined in that task's fields.md and promoted to dependencies/ the moment a second task needs it.
  • Every TASK.md has a Depends on list naming dependency files and earlier tasks; work never starts on a task whose dependencies are not Done in the Tracker.
  • New metrics get a new numbered task folder; never mix two sheets in one task.
  • TRACKER.md links each status-board row to its task folder; each session handoff is saved in docs/sessions/.
  • Files copied from the automation repo are read-only here; changes go back to that repo.

Task dependencies

Task Sheet Datasets Depends on
01-compliance-overview Compliance Overview vm_current shared.md, vm_current.md, domain_suffixes.md, thresholds.yml
02-compliance-trend Compliance Trend vm_snapshot shared.md, vm_snapshot.md, 01
03-power-lifecycle Power and Lifecycle vm_snapshot shared.md, vm_snapshot.md
04-governance Governance vm_snapshot shared.md, vm_snapshot.md
05-accountability Compliance Overview, Governance vm_current, vm_snapshot 01, 02
06-vulnerability-blind-spots Vulnerabilities vm_current, vulnerability vulnerability.md, 01
07-change-feed Power and Lifecycle vm_snapshot, Redshift view views.sql, 03
08-qa-reconciliation All All 01 to 07
09-publish-and-share All All 08

TASK.md template

# <NN>-<task-name>
- Status: Not started | In progress | Done | Blocked
- Goal: <one line: question this answers and for whom>
- Sheet: <sheet name>   Datasets: <list>
- Depends on: <dependency files and earlier tasks>
- Fields: see fields.md   Visuals: see visuals.md
- Acceptance criteria:
  - [ ] <numbers reconcile with ...>
  - [ ] <filters apply across datasets>
  - [ ] <title and subtitle state the population>
- Open questions: <numbered>

Agent handoff

Paste the block below as the first message of every new session, followed by the Tracker and Dashboard design sections.

# Role
You are a senior BI engineer and cloud infrastructure analyst working on an Amazon QuickSight
dashboard for VM health and compliance (vSphere estate, ~8k VMs). You help with three kinds of work:
1. Formulas: write, fix and verify QuickSight calculated fields.
2. Views: review sheets and visuals for correctness, clarity and leadership value.
3. Metrics: plan and add new metrics, trends and visuals end to end.

# Context (read before anything else)
- The Dashboard design section below is the source of truth for datasets, fields, rules and visuals.
- The Tracker section below holds status, decisions, open questions and the NEXT SESSION pointer.
- Data is produced by a Red Hat AAP automation (Python dynamic inventory over vSphere) and
  ingested into Redshift by the BI team. I own the QuickSight analysis; BI owns ingestion only.
- If I attach analysis/definition.json or datasets/*.json (from describe-analysis-definition /
  describe-data-set), treat them as the live state and reconcile them with the design doc.

# Project directory
- Project: "AWS Quicksight - ECC Compliance Dashboard". Follow the layout in the Project directory
  section: README.md, AGENTS.md, TRACKER.md, docs/, dependencies/, analysis/, tasks/, scripts/.
- If the repo or a folder is missing, propose creating it, then create it after approval.
- Group work by task: one numbered folder per sheet or workstream under tasks/, each with
  TASK.md, fields.md, visuals.md and verification.md.
- Shared foundations (dataset exports, calculated fields used by 2+ tasks, Redshift views,
  config such as environment map, DNS suffixes, thresholds) live under dependencies/.
  Tasks link to them, never copy them.
- Before starting a task, check its Depends on list; if a dependency is not Done, say so and
  propose doing the dependency first.
- Save the end-of-session handoff to docs/sessions/YYYY-MM-DD-<topic>.md and update TRACKER.md.

# Start of every session
1. Read TRACKER.md (NEXT SESSION block) and the active task's TASK.md; restate the goal in one or two sentences.
2. Ask clarifying questions (see question bank) ONLY where the answer changes what you build.
   Ask at most 5, numbered, each with your recommended default.
3. Propose a micro to-do of 3-6 items, each small enough to finish and verify in this session.
4. Wait for my approval before writing formulas, JSON or visual specs.

# Working rules
- Every calculated field belongs to ONE dataset. Say which dataset for every field you write.
- Use identical field names across datasets so cross-dataset filters work.
- Only reference columns that exist in that dataset (check the column table in the design doc).
- Create dependencies first and list fields in creation order.
- VM counts use vm_uuid with has_uuid = 1. Current dataset: sum of 0/1 flags is exact
  (one row per VM). Snapshot dataset: always group or filter by snapshot_date.
- Trend visuals on vm_snapshot must keep is_first_snapshot = 1.
- Compliance = compliant / eligible, per control, using the <control>_elig and <control>_ok flags.
  Appliances and powered-off VMs are excluded except where the design doc says otherwise.
- Keep Unknown separate from No. Never fold Unknown into non-compliant silently.
- Defender compliant = "Active". rapid7 must go through rapid7_norm. risk = Exception only in snapshot.
- Do not change an existing field's meaning; create a new field and mark the old one for removal.
- For visuals give: sheet, visual type, dataset, X/Value/Color/Group wells, filters, sort,
  format (%, decimals, axis range) and a one-line title and subtitle.
- Verify every change: give a sanity check (an expected number, a reconciliation, or a table to build).
- If editing definition JSON: targeted edits only, keep IDs stable, validate before pushing,
  push only to the dev analysis copy, never to the published dashboard.
- Be concise. Code blocks for formulas. Tables for visual specs.

# Question bank (pick only what applies)
- Audience: leadership, ops team, or VM owners? What decision should this view drive?
- Scope: which environments, OS, clusters, owners? Include appliances or powered-off VMs?
- Definition: what exactly counts as compliant for this control? Any OS-specific rule?
- Grain: current state (vm_current) or trend (vm_snapshot)? Daily or weekly?
- Threshold: target %, alert level, or SLA to show (e.g. 95% Defender)?
- Format: KPI, gauge, line, bar, table, heat map? Where on which sheet?
- Data: is the needed column in the dataset, or does the automation need to emit it?
- Validation: what known number should this match (a report count, a gauge, a manual check)?

# End of every session: output this block exactly
## Session handoff - <YYYY-MM-DD>
- Goal: <one line>
- Done: <bullets, with dataset and field/visual names>
- Changed fields: <name | dataset | new/changed/removed | formula summary>
- Verified: <what was checked and the result>
- Decisions: <decision | reason>
- Open questions: <numbered>
- Blockers: <none or detail>
- NEXT SESSION: <goal> | <3-6 micro to-do items> | <inputs I must bring>

Tracker

The dashboard is about 90% built: data pipeline, datasets, calculated fields, gauges and the first trends are done; remaining work is trends, governance views, QA and publishing.

Next session

  • Goal: set up the project directory (create the repo layout, export datasets and analysis into dependencies/ and analysis/, record existing fields in dependencies/calculated-fields/), then finish the Compliance Trend tab (task 02).
  • Micro to-do:
    1. Add per-environment coverage trend (one control, Color = environment).
    2. Add non-compliant VM count by control (stacked bar).
    3. Add Unknown / data-gap trend (patch, Rapid7, Defender).
    4. Add week-over-week KPI row (compliance, Defender, patch, zombie).
    5. Reconcile latest trend point against Compliance Overview gauges.
  • Bring: screenshot or definition.json of the Compliance Trend sheet; any gauge numbers to match.

Status board

Item Area Status Notes
Report contract (workbook + snapshot CSV) Data Done 23-column workbook, 39-column snapshot, daily first-of-day snapshot
Redshift tables V3 and V3 SNAPSHOT Data Done BI ingests; snapshot appended, no date de-dup
QuickSight datasets (vm_current, vm_snapshot, vulnerability) Data Done Separate datasets, no join
Calculated fields in both VM datasets Formulas Done Per-control _elig / _ok flags in place
Compliance Overview: gauges per control Views Done compliant vs eligible, on vm_current
Compliance Trend: VM count trend Views Done Combo chart; low daily variance, consider net-change bars
Compliance Trend: coverage % per control Views Done Line chart, six controls
Compliance Trend: remaining visuals Views In progress See Next session
Power and lifecycle trends (zombies, reclaimable disk) Views Not started vm_snapshot
Governance trends (owner tag, risk exceptions, hold power) Views Not started vm_snapshot
Accountability (top owners, resource pool heat map) Views Not started Both datasets
Vulnerability blind spots (vulns on VMs missing agents) Metrics Not started Aggregate vulns per VM before joining
Day-level state change feed Metrics Not started Redshift LAG view over snapshot
Dashboard as code (export definition.json to Git) Tooling Not started Dev analysis copy first
QA pass: reconcile counts across datasets QA Not started Latest snapshot vs current within a small gap
Q Topics and executive summary Views Not started After QA
Publish and share with stakeholders Release Not started After QA

Decisions log

Date Decision Reason
2026-10-05 Per-control eligibility (_elig / _ok flags) instead of one eligibility rule Each control applies to a different VM population
2026-10-05 Gauges built on vm_current Snapshot would sum across days
2026-09-30 Appliances and powered-off VMs excluded from agent compliance Appliances need no agents; powered-off VMs cannot report
2026-09-30 SCCM scope = domain-joined Windows; WSUS scope = non-domain Windows WSUS serves only non-domain or custom-domain VMs
2026-09-30 Keep vm_current and vm_snapshot as separate datasets A join multiplies rows by days and inflates counts
2026-09-28 Snapshot once per IST day, first run of the day; is_first_snapshot uses minOver Trends only need daily grain; matches documented recovery rule
2026-09-28 Unknown kept separate from No in every metric Unknown is a telemetry gap, not a failed control
2026-09-25 Lower snake_case column names in both reports Clean SQL and QuickSight formulas
2026-09-25 BI owns ingestion; dashboard logic lives in QuickSight Clear ownership

Open questions

  • Is Defender required on Linux, or Windows only? (changes defender_elig)
  • Is Ansible required on Windows, or Linux only? (changes ansible_elig)
  • Full list of corporate DNS suffixes for is_domain_joined?
  • Final cluster to environment mapping for the calculated environment field on vm_current?
  • Target % per control for gauge thresholds (e.g. 95%)?
  • Does the vulnerability dataset carry vm_uuid, or only asset names? (decides the join key)
  • Who are the dashboard readers and what sharing groups are needed?

Session log

Paste each end-of-session handoff block here, newest first.

Dashboard design

The dashboard answers four questions for leadership and ops: how compliant the fleet is now, whether it is improving, who owns the gaps, and what can be reclaimed.

Data flow

  1. Red Hat AAP job runs the Python dynamic inventory over vSphere (every few hours).
  2. It writes vSphere-HnS-Report-Current.xlsx every run and vm_inventory_snapshot_<timestamp>.csv on the first successful run of each IST day, to the SMB share (56-day file retention).
  3. BI ingests them into Redshift: Table-Name V3 (replaced) and Table-Name V3 SNAPSHOT (appended, no de-dup by date).
  4. QuickSight datasets read the tables; SPICE refresh runs after ingestion (snapshot: incremental on snapshot_ts).

Datasets

Dataset Source Grain Use for
vm_current Table-Name V3 One row per VM (vm_uuid), latest run KPIs, gauges, owner tables, current compliance
vm_snapshot Table-Name V3 SNAPSHOT One row per VM per snapshot_date Trends, week over week, zombie aging, change feed
vulnerability Rapid7 vulnerability table One row per vulnerability per asset Severity, packages, owners, vuln age

The datasets are not joined. Shared filters use cross-dataset filters on identically named fields.

Columns by dataset

Columns vm_current vm_snapshot
resource_pool, vm_name, vm_owner, os, ip, dns_name, cluster, vm_state, vo_tag, vm_tools, patchability, defender, rapid7, ansible, appliance, hold_power, tenant_admins, sccm, wsus, vm_uuid, snapshot_ts Yes Yes
risk Yes / No only Yes / No / Exception
power_state_since Raw text (UTC) Parsed ISO 8601 UTC
snapshot_date, environment, vm_owner_email, vcpu, memory_gb, disk_gb, vcenter_host, vm_moid, duplicate_uuid No Yes
vm_created_date, last_patch_date, defender_last_seen, rapid7_last_seen, shutdown_reason, hold_power_reason, hold_expiry_date No Always empty (placeholders)

Value domains

Column Values Compliant value
vm_state poweredOn, poweredOff, suspended n/a (power state)
defender Active, Inactive, Unknown Active
rapid7 Yes, No, Unknown (not normalised; use rapid7_norm) Yes
ansible, patchability, sccm, wsus Yes, No, Unknown Yes
vm_tools Yes, Unknown (never No) Yes
vo_tag, appliance, hold_power Yes, No vo_tag = Yes; others descriptive
environment (snapshot) Prod, Dev, Lab, Backup, Unknown n/a

Sheets and visuals

Sheet Visual Type Dataset Status
Compliance Overview Coverage per control (Defender, Rapid7, Ansible, Patch, SCCM, WSUS, Tools, VO Tag) Gauge: value x_compliant_vms, target x_eligible_vms vm_current Built
Compliance Overview Compliance %, compliant, non-compliant, eligible, appliances excluded KPI row vm_current Planned
Compliance Overview Non-compliant VMs by owner (top 10) Horizontal bar vm_current Planned
Compliance Trend Total VMs by environment with powered-on line Combo chart vm_snapshot Built
Compliance Trend Coverage % per control Line chart vm_snapshot Built
Compliance Trend Coverage % by environment (one control) Line chart vm_snapshot Next
Compliance Trend Non-compliant count by control Stacked bar vm_snapshot Next
Compliance Trend Unknown / data-gap trend Line chart vm_snapshot Next
Compliance Trend Week-over-week KPIs KPI row vm_snapshot Next
Power and Lifecycle Zombie VMs and reclaimable disk Combo chart vm_snapshot Planned
Power and Lifecycle Daily net change by environment Bar vm_snapshot Planned
Governance Owner tag, risk exceptions, hold power trends Line charts vm_snapshot Planned
Vulnerabilities Critical/High KPIs, top packages, top owners, vuln age KPI, bar, table vulnerability Built earlier

Sheet controls: environment, cluster, resource_pool, vm_owner, os_family, power_status (cross-dataset where the field exists in both).

Business rules

  • Eligibility base: appliance = No, vm_state = poweredOn, has_uuid = 1.
  • Defender, Rapid7, Ansible, Patch: base eligibility and Windows or Linux.
  • SCCM: base eligibility, Windows, domain joined (dns_name ends with a corporate suffix).
  • WSUS: base eligibility, Windows, not domain joined.
  • VM Tools: powered on (appliances included). VO Tag: all VMs with a UUID.
  • Overall compliance (is_compliant): Windows needs Defender Active, Rapid7, Ansible and the right patch channel; Linux needs Defender Active, Rapid7 and Ansible.
  • Zombie VM: poweredOff for more than 30 days.
  • Daily trends: one snapshot per day via is_first_snapshot = 1.
  • Unknown: always shown separately, never counted as No.

Calculated field catalog

Create in the order listed. Fields marked (both) must exist in vm_current and vm_snapshot with identical names.

# ---- Dimensions (both) ----
power_status     ifelse({vm_state}="poweredOn","Powered On",{vm_state}="poweredOff","Powered Off",{vm_state}="suspended","Suspended","Unknown")
os_family        ifelse({is_windows}=1,"Windows",{is_linux}=1,"Linux",{os}="Unknown","Unknown","Other")
rapid7_norm      ifelse(toLower(trim({rapid7}))="yes","Yes",toLower(trim({rapid7}))="no","No","Unknown")
defender_status  ifelse({defender}="Active","Active",{defender}="Unknown","Unknown","Inactive")
environment      (vm_current only) ifelse(in(toLower(trim({cluster})),[...]),"Prod", ... ,"Unknown")  # mirror config/environment_map.yml

# ---- Base flags (both) ----
has_uuid         ifelse(isNull({vm_uuid}) OR trim({vm_uuid})="",0,1)
is_powered_on    ifelse({vm_state}="poweredOn",1,0)
is_appliance     ifelse({appliance}="Yes",1,0)
is_windows       ifelse(locate(toLower({os}),"windows")>0,1,0)
is_linux         ifelse(locate(toLower({os}),"linux")>0 OR locate(toLower({os}),"red hat")>0 OR locate(toLower({os}),"ubuntu")>0 OR locate(toLower({os}),"centos")>0 OR locate(toLower({os}),"debian")>0 OR locate(toLower({os}),"suse")>0,1,0)
is_supported_os  ifelse({is_windows}=1 OR {is_linux}=1,1,0)
is_domain_joined ifelse(endsWith(toLower(trim({dns_name})),".dev.example.com") OR endsWith(toLower(trim({dns_name})),".prod.example.com"),1,0)
base_eligible    ifelse(toLower(trim({appliance}))="no" AND {vm_state}="poweredOn" AND {has_uuid}=1,1,0)

# ---- Per-control flags (both) ----
defender_elig    ifelse({base_eligible}=1 AND {is_supported_os}=1,1,0)
defender_ok      ifelse({defender_elig}=1 AND {defender}="Active",1,0)
rapid7_elig      ifelse({base_eligible}=1 AND {is_supported_os}=1,1,0)
rapid7_ok        ifelse({rapid7_elig}=1 AND {rapid7_norm}="Yes",1,0)
ansible_elig     ifelse({base_eligible}=1 AND {is_supported_os}=1,1,0)
ansible_ok       ifelse({ansible_elig}=1 AND {ansible}="Yes",1,0)
patch_elig       ifelse({base_eligible}=1 AND {is_supported_os}=1,1,0)
patch_ok         ifelse({patch_elig}=1 AND {patchability}="Yes",1,0)
sccm_elig        ifelse({base_eligible}=1 AND {is_windows}=1 AND {is_domain_joined}=1,1,0)
sccm_ok          ifelse({sccm_elig}=1 AND {sccm}="Yes",1,0)
wsus_elig        ifelse({base_eligible}=1 AND {is_windows}=1 AND {is_domain_joined}=0,1,0)
wsus_ok          ifelse({wsus_elig}=1 AND {wsus}="Yes",1,0)
tools_elig       ifelse({vm_state}="poweredOn" AND {has_uuid}=1,1,0)
tools_ok         ifelse({tools_elig}=1 AND {vm_tools}="Yes",1,0)
owner_elig       ifelse({has_uuid}=1,1,0)
owner_ok         ifelse({owner_elig}=1 AND {vo_tag}="Yes",1,0)
patch_unknown    ifelse({patch_elig}=1 AND {patchability}="Unknown",1,0)
rapid7_unknown   ifelse({rapid7_elig}=1 AND {rapid7_norm}="Unknown",1,0)
defender_unknown ifelse({defender_elig}=1 AND {defender}="Unknown",1,0)

# ---- Overall compliance (both) ----
compliance_eligible ifelse({base_eligible}=1,1,0)
is_compliant     ifelse({compliance_eligible}=0,0,
                   {is_windows}=1 AND {defender}="Active" AND {rapid7_norm}="Yes" AND {ansible}="Yes"
                     AND (({is_domain_joined}=1 AND {sccm}="Yes") OR ({is_domain_joined}=0 AND {wsus}="Yes")),1,
                   {is_linux}=1 AND {defender}="Active" AND {rapid7_norm}="Yes" AND {ansible}="Yes",1,0)

# ---- Power age (both) ----
days_in_state    dateDiff({power_state_since},now(),"DD")   # vm_current: parse raw text first if Date type fails
is_zombie        ifelse({vm_state}="poweredOff" AND {days_in_state}>30,1,0)

# ---- Measures (both) ----
total_vms        distinct_countIf({vm_uuid},{has_uuid}=1)
powered_on_vms   distinct_countIf({vm_uuid},{vm_state}="poweredOn")
zombie_vms       distinct_countIf({vm_uuid},{is_zombie}=1)
<x>_eligible_vms sum({<x>_elig})
<x>_compliant_vms sum({<x>_ok})
eligible_vms     sum({compliance_eligible})
compliant_vms    sum({is_compliant})
compliance_pct   {compliant_vms}/{eligible_vms}
windows_vms      distinct_countIf({vm_uuid},{is_windows}=1 AND {is_powered_on}=1 AND {has_uuid}=1)
linux_vms        distinct_countIf({vm_uuid},{is_linux}=1 AND {is_powered_on}=1 AND {has_uuid}=1)

# ---- vm_current only ----
last_updated     max({snapshot_ts})

# ---- vm_snapshot only ----
is_first_snapshot ifelse({snapshot_ts}=minOver({snapshot_ts},[{snapshot_date}],PRE_FILTER),1,0)
is_risk_exception ifelse({risk}="Exception",1,0)
reclaimable_disk_gb sumIf({disk_gb},{is_zombie}=1)
<x>_pct_trend    sum({<x>_ok})/sum({<x>_elig})
vms_daily_change periodOverPeriodDifference({total_vms},{snapshot_date},DAY,1)
compliance_wow   periodOverPeriodDifference({compliance_pct},{snapshot_date},WEEK,1)

Conventions

  • Field names: lower snake_case; flags are 0/1 and named is_*, *_elig, *_ok, *_unknown; measures end in _vms, _pct, _gb, _wow.
  • Percent format with 1 decimal; line charts use Auto (exclude zero) on the Y axis; bars always start at zero.
  • Every visual has a title and a subtitle that states its population (e.g. powered-on, non-appliance VMs).
  • Prefer dataset-level calculated fields over analysis-level so they carry across analyses.

Known caveats

  • vm_tools means a guest state is present, not that Tools is running; measure it on powered-on VMs only.
  • Windows patchability is almost never No (needs SCCM and WSUS both No); watch patch_unknown instead.
  • dns_name falls back to vm_name when the guest hostname is missing, so some domain-joined VMs look non-domain.
  • Workbook and snapshot snapshot_ts differ (workbook refreshes every run); never join history to the current workbook.
  • Blank vm_uuid rows exist; filter has_uuid = 1 everywhere.
  • Snapshot trends lag up to a day (first run of the IST day).

Metrics backlog

Metric Dataset Value to stakeholders
Vulnerability blind spots: Critical/High vulns on VMs with Defender or Rapid7 missing vm_current + vulnerability Highest-risk gaps
Owner scorecard: compliance %, critical vulns, zombie VMs per owner vm_current + vulnerability Accountability
Resource pool heat map of compliance % over time vm_snapshot Spots regressing tenants
Day-level state change feed vm_snapshot (Redshift LAG view) Explains spikes from shutdowns or new builds
Reclaimable vCPU and memory from zombie VMs vm_snapshot Cost savings
Threshold alerts (e.g. Defender below 95%) vm_current Proactive follow-up
Mean time to remediate vulnerabilities vulnerability snapshots Needs vulnerability snapshot history
โš ๏ธ **GitHub.com Fallback** โš ๏ธ