09. Kpis monitor - m-marini/wheellyj GitHub Wiki

The Wheely and batch training applications display KPIs during the learning phase.

KPI Normalization Rules

All probability-based KPIs are normalised to percentage scale [0–100].

Value-based KPIs are reported in internal micro-units.

Policy Entropy Definition

All entropy KPIs are reported as:

H% = 100 · H

$$ H = -\frac{1}{\ln |A|} \sum_{a \in A} \pi(a \mid s(t)) \ln \pi(a \mid s(t)) $$

Where $|A|$ is the number of available actions.

Higher entropy indicates more exploratory (stochastic) policy behavior, while lower entropy indicates more deterministic (greedy) behavior.

---

Avg Rew

Average reward (micro-units).

• Positive: desirable behavior
• Negative: undesirable behavior

Δ RMS

RMS of the critic TD error.

Lower values indicate more accurate reward prediction.

Critic

Estimated differential state-value function (micro-units).

Head P (%)

Maximum probability assigned to a head rotation action. These represent the maximum action probability (argmax over the policy distribution), not entropy or overall distribution uncertainty.

Higher values indicate stronger directional preference.

Move P (%)

Maximum probability assigned to a movement action. These represent the maximum action probability (argmax over the policy distribution), not entropy or overall distribution uncertainty.

Higher values indicate stronger action preference.

Move H (%)

Normalised movement policy entropy (range: 0–100%).

• 0: deterministic policy (Exploitation)
• 100: full random policy (Exploratory)

Head H (%)

Normalised head rotation policy entropy (range: 0–100%).

• 0: deterministic policy (Exploitation)
• 100: full random policy (Exploratory)

---

Revision 0.27.0