Architecture - mohamedfadil12/Arachne GitHub Wiki
flowchart LR
A[Audio + transcript] --> B[Validate & decode<br/>16 kHz mono]
R[(Reference<br/>same text)] --> C
B --> C[Word alignment]
C --> D[Features<br/>F0 · energy · pauses]
D --> E[Matched-text comparison<br/>robust z-scores]
E --> F[Rubric rules]
F --> G[Regions + scores + explanations]
G --> H{{JSON API}}
| Module | Responsibility |
|---|---|
audio_io.py |
Decode and validate; typed errors (audio_too_short, silent_audio, unsupported_audio, ...) |
align.py |
Word alignment backends |
features.py |
Frame energy, speech mask, Praat F0, per-word features |
compare.py |
Four detectors, region grouping, explanation rendering |
rubric.py |
Rule metadata and feedback templates |
scoring.py |
Dimension and overall scores |
pipeline.py |
Orchestration, quality checks, output assembly |
api.py |
FastAPI app (no UI) |
synth.py, tier1.py, dataset.py
|
Dev dataset: synthetic baselines and flaw injection |
evaluate.py, stress.py
|
Evaluation and robustness harnesses |
Deterministic, rule-based scoring. No LLM in the scoring path. Same input plus same config gives byte-identical output (tested).
Matched-text comparison. Both recordings contain the same words, so word i in the reference corresponds to word i in the performance. This turns hard open-ended judging into aligned differences.
Speaker-relative features. Pitch is converted to semitones relative to each speaker's own median F0 and scaled by the speaker's robust spread. Energy is median-centred per recording. Raw cross-speaker pitch or loudness is never compared.
Robust z-scores. Deviations are standardised with the median absolute deviation, with a documented floor per rule, and every rule also requires a minimum absolute change. A z-score here is a standardisation within one recording pair, not a universal calibration.
Alignment (default proportional).
- Split audio at silences of at least 0.15 s into speech chunks.
- Assign transcript words to chunks with dynamic programming: chunk duration share is matched to word-weight share, and boundaries are pulled toward the reference's pause positions.
- Place word boundaries inside each chunk with a Viterbi search over energy valleys, weakly regularised toward reference word durations.
It needs no model download and is deterministic, but boundaries are approximate. It relies on energy valleys between words, which are cleaner in synthetic speech than in real speech, so expect a real forced aligner (WhisperX) to be needed for human recordings.
Fail honestly. When SNR is too low, voiced coverage is too small, or alignment fails, the pipeline returns insufficient_evidence or a clear error rather than a score.
Each result carries schema_version, engine_version, rubric_version, config_hash, the rule metadata used, and standing limitations, so any displayed number can be traced to a rule and a configuration.