12.0 Explainability System Overview - ravkorsurv/kor-ai-core GitHub Wiki
This document defines the explainability architecture, design principles, and implementation strategy for the Kor.ai Surveillance Platform. It supports transparency, analyst efficiency, audit readiness, and regulatory traceability for all Bayesian-generated risk alerts.
The explainability module exists to:
-
Justify risk scores generated from Bayesian inference.
-
Provide human-readable rationales for alert decisions.
-
Support STOR preparation and internal compliance reviews.
-
Enable trust, filtering, and UI clarity.
Field | Description |
---|---|
node | BN node name (e.g. AccessToMNPI) |
value | Observed or inferred state (e.g. Yes, Extreme) |
score_impact | Normalized contribution to posterior (e.g. 0.25) |
confidence | System’s certainty in that input or state (e.g. High, Medium) |
input_description | Human-readable rationale (e.g. "Trader accessed sales report") |
tag (optional) | Typology marker (e.g. SpoofingPattern, MNPIAccessEvidence) |
-
Stored in columnar format (e.g. Parquet)
-
Retained in low-cost cold storage (e.g. AWS S3 Glacier)
-
Fully replayable by model version for drift analysis and audit
Purpose: Guarantees long-term explainability, regulatory defensibility, and model evolution tracking.
Most surveillance systems ignore non-alerts. Kor.ai captures them because:
-
They help identify false negatives.
-
They form the denominator for metrics like Surveillance Coverage Rate (SCR).
-
They allow replaying with new model logic to find newly relevant patterns.
-
They surface data quality or fallback usage patterns invisible in alert-only logs.
Without this, it’s impossible to prove that you “didn’t miss something” — a common regulatory ask.
Analysts and audit stakeholders can:
-
Filter observations by
final_risk_score
ordq_flags
-
Compare alert vs. non-alert cases across time/model versions
-
View posterior score distributions across desk/region/product
-
Perform targeted reviews of borderline cases
Result: Explainability isn’t just about why an alert was fired — but why others weren’t.
-
Each observation ties to a specific
model_version
-
Past data can be reprocessed through updated models to:
-
Measure alert rate drift
-
Evaluate tuning effectiveness
-
Validate new CPT logic or structure changes
-
-
Sensitivity metrics: Show alert sensitivity to node value flips.
-
Evidence sufficiency index: Estimate if alert was generated from strong or sparse evidence.
-
Analyst feedback learning: Capture reviewer input to reinforce useful drivers or suppress noise.
-
Scenario traces: DAG animations showing causal propagation leading to alert.
This updated explainability design closes a major industry blind spot by:
-
Capturing non-alert scoring metadata
-
Supporting threshold tuning and backtesting
-
Enabling future-proof compliance under evolving regulatory scrutiny
It fundamentally shifts explainability from being post-hoc and alert-centric to being observation-first, model-aware, and analyst-ready.