12.2 Evidence Sufficiency Index (ESI) - ravkorsurv/kor-ai-core GitHub Wiki
The Evidence Sufficiency Index (ESI) is a metric developed by Kor.ai to complement the Bayesian risk score. While the Bayesian model outputs a probabilistic risk value, ESI provides a measure of how well-supported that score is, based on input diversity, quality, and distribution. This enables analysts to differentiate between alerts that are high-risk and well-evidenced versus those driven by sparse or noisy data.
ESI enables:
-
Trust calibration for analysts and reviewers.
-
Filtering and triage based on evidence quality.
-
Better prioritisation in high-volume alert environments.
-
Enhanced explainability and defensibility in audit and STOR reviews.
Component | Description |
---|---|
node_activation_ratio | Proportion of active (populated) nodes in the BN. |
mean_confidence_score | Average confidence level of inputs (mapped to numeric). |
fallback_ratio | Proportion of nodes relying on priors or latent defaults. |
contribution_entropy | Entropy of node contributions โ measures distribution evenness. |
cross_cluster_diversity | Evidence spread across distinct node groups (e.g. trade, comms, PnL). |
ESI = W1 * node_activation_ratio +
W2 * mean_confidence_score +
W3 * (1 - fallback_ratio) +
W4 * contribution_entropy +
W5 * cross_cluster_diversity
{
"evidence_sufficiency_index": 0.84,
"node_count": 6,
"mean_confidence": "High",
"fallback_ratio": 0.0,
"contribution_spread": "Balanced",
"clusters": ["PnL", "MNPI", "TradePattern"]
}
-
Primary: Bayesian Risk Score
-
Secondary: ESI Score + Badge (
Strong
,Moderate
,Sparse
)
-
Badges on Alert Cards (e.g.
ESI: Strong Evidence
) -
Sort/Filter Controls:
-
Filter alerts with
ESI > 0.7
-
Sort by
ESI
descending
-
-
Explain Panel Tooltips:
-
โThis alert is supported by evidence from 5 distinct factors with high confidence and no fallback reliance.โ
-
-
ESI can be used as a multiplier in simulation:
Adjusted Risk Score = Risk Score * ESI
-
This can help:
-
Evaluate impact of noisy nodes
-
Filter low-ESI alerts in QA workflows
-
Simulate precision/recall tradeoffs
-
-
Analyst Feedback Learning: Use confirmed STORs or dismissals to adjust ESI weightings.
-
Sensitivity Maps: Show how much ESI would drop if a key input was removed.
-
Percentile Labels: Show where an alertโs ESI sits within a weekly/monthly percentile range.
The Evidence Sufficiency Index is a key enabler for moving beyond opaque alert scores. It allows Kor.ai to explain not just why an alert was scored as risky, but how trustworthy and complete the supporting evidence is. This separation of risk and sufficiency enables a new standard of transparency in AI-powered surveillance.