Researched: 2026-04-27 Domain: Scoring system refactoring -- eliminate hardcoded dimensions, make ScoringRubric the SSOT Confidence: HIGH

This phase is a refactoring of an existing, working scoring system. The core problem is that 5 scoring dimensions (key_message, objection_handling, communication, product_knowledge, scientific_info) are hardcoded in 7 locations across the codebase: the Scenario ORM model (5 weight columns), the scoring engine prompt template (dimension-specific instructions), the mock score generator (5 hardcoded dimension blocks), the frontend ScoringWeights component (typed to exactly 5 keys), the frontend Scenario TypeScript types, the analytics recommendation service (dimension-to-column mapping), and the i18n locale files (hardcoded dimension translations).

A ScoringRubric model already exists with a JSON dimensions field that supports arbitrary dimension names, weights, and criteria. The rubric editor UI already supports dynamic dimensions. The refactoring goal is to make this rubric the single source of truth so all scoring flows read dimensions from the rubric rather than from hardcoded scenario columns. This is a structural cleanup, not a feature addition -- the user-facing behavior (multi-dimensional scoring with configurable weights) remains the same, but becomes truly configurable.

Primary recommendation: Add a rubric_id FK to the Scenario model, remove the 5 weight_* columns via Alembic migration with data migration (converting existing weight values to rubric records), then refactor all downstream consumers (scoring engine, mock generator, frontend components) to read dimensions from the rubric. The two separate scoring systems (session scoring and Skill quality scoring) must remain independent -- they have different dimensions and different purposes.

This refactoring uses exclusively existing libraries. No new packages need to be installed. [VERIFIED: codebase audit]

Scenario model
  ├── weight_key_message: int = 30
  ├── weight_objection_handling: int = 25
  ├── weight_communication: int = 20
  ├── weight_product_knowledge: int = 15
  ├── weight_scientific_info: int = 10
  └── get_scoring_weights() -> dict  # Returns hardcoded 5-key dict

ScoringRubric model (exists but underused)
  └── dimensions: JSON  # [{name, weight, criteria[], max_score}]

scoring_service.py
  ├── Reads scenario.get_scoring_weights() as fallback
  └── Reads rubric dimensions only if a default rubric exists

scoring_engine.py
  ├── SCORING_PROMPT_TEMPLATE: hardcoded dimension instructions
  └── dim_names dict: maps 5 keys to display names

Scenario model
  ├── rubric_id: FK -> scoring_rubrics.id (NOT NULL per D-05)
  ├── pass_threshold: int = 70
  └── (weight_* columns REMOVED)

ScoringRubric model (SSOT)
  └── dimensions: JSON  # [{name, weight, criteria[], max_score}]

scoring_service.py
  ├── Resolves rubric: always via scenario.rubric_id (NOT NULL, no fallback needed)
  └── Passes rubric dimensions to scoring engine

scoring_engine.py
  ├── Builds prompt dynamically from rubric dimensions
  └── No hardcoded dimension names or instructions

Frontend
  ├── ScenarioEditor: rubric selector (required field) instead of ScoringWeights
  └── All scoring components: read dimensions from score.details (already dynamic)

What: Direct rubric lookup via scenario.rubric_id (NOT NULL per D-05) When to use: Every time a session needs to be scored Example:

# Source: [codebase pattern from rubric_service.py, simplified per D-05]
async def resolve_rubric_dimensions(db: AsyncSession, scenario) -> list[dict]:
    """Resolve rubric dimensions for scoring.
    
    Per D-05: rubric_id is NOT NULL, so direct lookup always succeeds.
    get_default_rubric() fallback is no longer needed for scoring.
    """
    import json as _json
    from app.services.rubric_service import get_rubric
    
    rubric = await get_rubric(db, scenario.rubric_id)
    dims = rubric.dimensions
    return _json.loads(dims) if isinstance(dims, str) else dims

What: Build scoring prompt from rubric dimensions, not hardcoded names When to use: LLM scoring engine Example:

# Replace hardcoded dim_names dict with rubric-driven config
def build_dimensions_config(rubric_dimensions: list[dict]) -> str:
    lines = []
    for dim in rubric_dimensions:
        name = dim["name"]
        weight = dim["weight"]
        criteria = dim.get("criteria", [])
        criteria_text = "; ".join(criteria) if criteria else "General assessment"
        lines.append(f"- {name}: weight={weight}%, criteria: {criteria_text}")
    return "\n".join(lines)

What: Generate mock scores for arbitrary dimension sets When to use: Mock scoring fallback when LLM unavailable Example:

def _generate_mock_scores(
    rubric_dimensions: list[dict],
    scenario: Scenario,
    messages: list,
    key_messages_status: list[dict],
) -> dict:
    """Generate mock scores for N arbitrary dimensions."""
    dimensions = []
    for dim_config in rubric_dimensions:
        score = min(95, max(60, base_score + random.randint(-8, 10)))
        dimensions.append({
            "dimension": dim_config["name"],
            "score": score,
            "weight": dim_config["weight"],
            "strengths": [...],
            "weaknesses": [...],
            "suggestions": [...],
        })
    # Calculate weighted overall
    overall = sum(d["score"] * d["weight"] / 100 for d in dimensions)
    ...

• Merging session scoring with Skill quality scoring: These are two separate systems with different dimensions (5 MR-facing vs 6 content-quality). They must remain independent. [VERIFIED: codebase -- Skill scoring uses sop_completeness, knowledge_accuracy, etc.]
• Merging with DryRun scoring: DryRun uses executability_score and coverage_percent, which are completely different metrics. Do not touch DryRun scoring. [VERIFIED: codebase]
• Breaking backward compatibility on stored data: Existing ScoreDetail rows reference dimension names like key_message. These must remain readable even after the refactoring. New sessions will use rubric-defined names.
• Removing ScoringWeights component entirely: Deprecate but keep the file until all references are migrated. The rubric editor already handles dynamic dimensions.

Key insight: The rubric system already has ~80% of what's needed. The refactoring is mainly about removing the parallel hardcoded system and wiring the existing rubric system as the only path.

What goes wrong: SQLite does not support ALTER TABLE DROP COLUMN natively. Attempting to drop the 5 weight columns will fail. Why it happens: Alembic generates standard ALTER TABLE SQL that SQLite cannot execute. How to avoid: Use render_as_batch=True in Alembic's env.py (already configured per CLAUDE.md Gotcha #1). The migration must use with op.batch_alter_table('scenarios') as batch_op: to recreate the table. Warning signs: Migration fails with "near DROP: syntax error" on SQLite.

What goes wrong: Existing scored sessions have ScoreDetail rows with dimension values like key_message, objection_handling, etc. If the frontend tries to display these using new rubric-based labels, they may show raw snake_case keys. Why it happens: Dimension names in ScoreDetail are stored as strings, not FK references. They persist the name used at scoring time. How to avoid: The frontend already displays detail.dimension as a raw string. Add a dimension display name mapping utility that checks rubric first, then falls back to i18n translation, then to the raw key. Historical data remains readable. Warning signs: Old session reports show key_message instead of "Key Message Delivery".

What goes wrong: get_recommended_scenarios() in analytics_service.py maps dimension names to Scenario.weight_* columns to find scenarios targeting the user's weakest dimension. After column removal, this query breaks. Why it happens: The weight_map dict directly references ORM column attributes that no longer exist. How to avoid: Rewrite the recommendation algorithm to: (1) find user's weakest dimension from ScoreDetail records, (2) for each active scenario, load its rubric, (3) rank scenarios by the weight of the weakest dimension in their rubric. This is slightly more complex but correct. Warning signs: 500 errors on user dashboard after migration.

What goes wrong: After adding rubric_id FK and removing weight columns, existing scenarios have rubric_id=NULL and no weight data. Why it happens: The data migration must create rubric records from existing weight values before dropping the columns. How to avoid: Three-step migration within a single Alembic file: (1) Add rubric_id column as nullable, (2) create rubric records from existing weight values using op.execute raw SQL with uuid4() and update scenarios to point to them, (3) alter rubric_id to NOT NULL, then drop weight columns. Warning signs: All existing scenarios lose their scoring configuration.

What goes wrong: There are 42+ backend test files and 44+ frontend test files referencing the 5 hardcoded dimensions. Updating all at once creates massive, error-prone diffs. Why it happens: Tests hardcode scenario weights and dimension names in fixtures. How to avoid: Create a shared test fixture/factory that generates rubric-based scenarios. Update tests to use the factory. Tests that only assert on "some dimensions exist" (not specific names) may need minimal changes. Warning signs: Hundreds of test failures after model change.

What goes wrong: The LLM scoring prompt template has dimension-specific instructions ("For key_message, consider which key messages were delivered..."). After making it dynamic, the LLM may produce lower-quality scores because it lacks domain-specific guidance. Why it happens: Generic instructions produce generic scores. The current per-dimension instructions encode domain expertise. How to avoid: Move the dimension-specific instructions INTO the rubric's criteria field. The prompt builder reads criteria from the rubric and includes them in the prompt. The default rubric should contain the existing detailed instructions as criteria entries. Warning signs: Score quality drops after refactoring.

# In Alembic migration, first batch_alter_table call
with op.batch_alter_table("scenarios") as batch_op:
    batch_op.add_column(
        sa.Column("rubric_id", sa.String(36),
                  sa.ForeignKey("scoring_rubrics.id"), nullable=True)
    )

# In the same Alembic migration, use op.execute / connection.execute
# For each unique weight combination in the scenarios table:
# 1. Create a ScoringRubric record with those weights as dimensions JSON
# 2. For scenarios with that weight combination, SET rubric_id to the new rubric's id
# The default 30/25/20/15/10 combination gets is_default=True

# After all scenarios have rubric_id populated:
with op.batch_alter_table("scenarios") as batch_op:
    batch_op.alter_column("rubric_id", nullable=False)
    batch_op.drop_column("weight_key_message")
    batch_op.drop_column("weight_objection_handling")
    batch_op.drop_column("weight_communication")
    batch_op.drop_column("weight_product_knowledge")
    batch_op.drop_column("weight_scientific_info")

Most scenarios likely use the default weights (30/25/20/15/10). The migration should:

1. Create ONE default rubric with these weights and is_default=True
2. Point all default-weight scenarios to this rubric
3. Create separate rubrics only for scenarios with custom weights

These three systems are architecturally separate and must remain so. The Skill Quality dimensions evaluate content quality (is the training material good?), while Session dimensions evaluate MR performance (did the MR perform well?). DryRun dimensions evaluate SOP executability. They serve fundamentally different purposes.

# Source: [CLAUDE.md Gotcha #1 pattern, adapted for this use case]
import uuid
import json

def upgrade() -> None:
    # Step 1: Add rubric_id column as nullable
    with op.batch_alter_table("scenarios") as batch_op:
        batch_op.add_column(
            sa.Column("rubric_id", sa.String(36), 
                      sa.ForeignKey("scoring_rubrics.id"), nullable=True)
        )
    
    # Step 2: Data migration -- create rubrics for each unique weight combo
    conn = op.get_bind()
    
    # Find unique weight combinations
    scenarios = conn.execute(sa.text(
        "SELECT id, weight_key_message, weight_objection_handling, "
        "weight_communication, weight_product_knowledge, weight_scientific_info "
        "FROM scenarios"
    )).fetchall()
    
    # Group by weight combo, create one rubric per unique combo
    weight_combos = {}
    for row in scenarios:
        combo_key = (row[1], row[2], row[3], row[4], row[5])
        if combo_key not in weight_combos:
            weight_combos[combo_key] = []
        weight_combos[combo_key].append(row[0])
    
    for (wkm, woh, wc, wpk, wsi), scenario_ids in weight_combos.items():
        rubric_id = str(uuid.uuid4())
        is_default = (wkm == 30 and woh == 25 and wc == 20 and wpk == 15 and wsi == 10)
        dims = json.dumps([
            {"name": "key_message", "weight": wkm, "criteria": [...], "max_score": 100.0},
            {"name": "objection_handling", "weight": woh, "criteria": [...], "max_score": 100.0},
            {"name": "communication", "weight": wc, "criteria": [...], "max_score": 100.0},
            {"name": "product_knowledge", "weight": wpk, "criteria": [...], "max_score": 100.0},
            {"name": "scientific_info", "weight": wsi, "criteria": [...], "max_score": 100.0},
        ])
        conn.execute(sa.text(
            "INSERT INTO scoring_rubrics (id, name, description, scenario_type, dimensions, is_default, created_by) "
            "VALUES (:id, :name, :desc, :stype, :dims, :is_default, :created_by)"
        ), {"id": rubric_id, "name": f"Migrated {'Default' if is_default else 'Custom'} Rubric",
            "desc": "Auto-created from scenario weight columns during migration",
            "stype": "f2f", "dims": dims, "is_default": is_default, "created_by": "system"})
        
        for sid in scenario_ids:
            conn.execute(sa.text(
                "UPDATE scenarios SET rubric_id = :rid WHERE id = :sid"
            ), {"rid": rubric_id, "sid": sid})
    
    # Step 3: Enforce NOT NULL and drop weight columns
    with op.batch_alter_table("scenarios") as batch_op:
        batch_op.alter_column("rubric_id", nullable=False)
        batch_op.drop_column("weight_key_message")
        batch_op.drop_column("weight_objection_handling")
        batch_op.drop_column("weight_communication")
        batch_op.drop_column("weight_product_knowledge")
        batch_op.drop_column("weight_scientific_info")

# Source: [adapted from existing scoring_engine.py build_scoring_prompt]
def build_dimensions_instructions(rubric_dimensions: list[dict]) -> str:
    """Build dimension-specific scoring instructions from rubric criteria."""
    lines = []
    for dim in rubric_dimensions:
        name = dim["name"]
        weight = dim["weight"]
        criteria = dim.get("criteria", [])
        lines.append(f"- {name} (weight={weight}%)")
        if criteria:
            for criterion in criteria:
                lines.append(f"  * {criterion}")
    return "\n".join(lines)

// Source: [adapted from existing scenario-editor.tsx pattern]
// In ScenarioEditor form, replace ScoringWeights with:
<div className="grid gap-2">
  <Label>{t("scenarios.scoringRubric")}</Label>
  <Controller
    control={form.control}
    name="rubric_id"
    render={({ field }) => (
      <Select value={field.value ?? ""} onValueChange={field.onChange}>
        <SelectTrigger>
          <SelectValue placeholder="Select scoring rubric" />
        </SelectTrigger>
        <SelectContent>
          {rubrics.map((r) => (
            <SelectItem key={r.id} value={r.id}>
              {r.name} ({r.dimensions.length} dimensions)
            </SelectItem>
          ))}
        </SelectContent>
      </Select>
    )}
  />
</div>

Deprecated after this phase:

• ScoringWeights component -- replaced by rubric selector in ScenarioEditor
• Scenario.get_scoring_weights() method -- replaced by rubric resolution
• ScoringWeightsProps TypeScript interface -- no longer used
• WEIGHT_KEYS and I18N_KEYS constants in scoring-weights.tsx -- no longer used

1. Should rubric_id be required or nullable on Scenario? (RESOLVED)

   • Decision: NOT NULL per D-05. rubric_id is NOT NULL on Scenario. The data migration creates rubric records for all existing scenarios before enforcing the constraint. No fallback chain needed for scoring -- every scenario always has a rubric. get_default_rubric() is retained only as a UI convenience for pre-selecting a rubric when creating new scenarios, not as a scoring fallback.

2. Should scenario editor show rubric dimensions inline or just a selector? (RESOLVED)

   • Decision: Selector with read-only preview per UI-SPEC IC-01. The scenario editor shows a rubric selector dropdown. When a rubric is selected, a read-only dimension preview (name + weight bar + criteria summary) appears below. Full dimension editing goes to the rubric management page via a "Manage Rubrics" link.

3. Should the data migration run in Alembic or as a seed script? (RESOLVED)

   • Decision: Single Alembic migration with inline data migration per D-04. The migration uses raw SQL via op.get_bind() to: (1) read existing weight combinations, (2) create ScoringRubric records, (3) update scenario.rubric_id, (4) enforce NOT NULL, (5) drop weight columns. This keeps the schema change and data migration atomic. The seed scripts (seed_phase2.py, startup_seed.py) are updated separately to create scenarios with explicit rubric_id references.

• NEVER modify schema without Alembic migration -- all column changes require proper migrations
• render_as_batch for SQLite -- column drops require batch mode (Gotcha #1)
• async with for all DB sessions -- all new service code must use async patterns
• Service layer = business logic, routers = HTTP only -- rubric resolution belongs in service
• Create returns 201, Delete returns 204 -- maintain API conventions
• No raw SQL -- use SQLAlchemy ORM or Alembic for all queries (exception: Alembic data migration uses op.execute for raw SQL within the migration itself, which is the standard Alembic pattern)
• db.flush() per project convention -- not db.commit() (session middleware handles commit)
• Pydantic v2 schemas with from_attributes=True -- all schema updates must use ConfigDict
• TypeScript strict: true -- no any types in frontend changes
• TanStack Query hooks per domain -- any new hooks follow existing pattern
• Conventional commits -- e.g., refactor(scoring): remove hardcoded dimensions from scenario model
• server_default in migrations -- for SQLite compatibility with existing rows

• [Codebase audit] -- All 13 hardcoded locations identified by grep + file read
• [backend/app/models/scenario.py] -- Current 5 weight columns
• [backend/app/models/scoring_rubric.py] -- Existing rubric model with JSON dimensions
• [backend/app/services/scoring_engine.py] -- Hardcoded dim_names and prompt template
• [backend/app/services/scoring_service.py] -- Mock generator and rubric fallback logic
• [backend/app/services/analytics_service.py] -- weight_map recommendation query
• [frontend/src/components/admin/scoring-weights.tsx] -- 5-key typed component
• [frontend/src/components/admin/rubric-editor.tsx] -- Already dynamic with useFieldArray
• [frontend/src/components/scoring/radar-chart.tsx] -- Already data-driven
• [frontend/src/components/scoring/dimension-bars.tsx] -- Already iterates ScoreDetail[]
• [CLAUDE.md] -- Project conventions and gotchas

• [backend/app/services/meta_skill_templates/] -- Skill quality scoring dimensions are separate
• [backend/scripts/seed_phase2.py] -- Seed data patterns

Confidence breakdown:

• Hardcoded locations: HIGH -- complete grep audit of entire codebase
• Migration strategy: HIGH -- follows established Alembic patterns in project
• Frontend impact: HIGH -- verified each component's data flow
• Backward compatibility: HIGH -- ScoreDetail stores dimension as string, historical data safe
• Prompt quality after refactor: MEDIUM -- depends on criteria field quality (A2)

Research date: 2026-04-27 Valid until: 2026-05-27 (stable internal refactoring, no external dependency risk)

Visual and interaction contract for the Scoring Criteria Refactor frontend. Generated by gsd-ui-researcher, verified by gsd-ui-checker. This phase is a refactoring -- the primary UI change is replacing the hardcoded ScoringWeights component with a Rubric selector in the Scenario Editor. Scoring display components (RadarChart, DimensionBars, FeedbackCard) are already dynamic and require no changes.

Source: Existing project design system (Phase 01 + Phase 10 polish), confirmed via frontend/src/styles/index.css.

Declared values (must be multiples of 4):

Exceptions: Dimension preview list items use 28px row height (7 x 4px) for dense dimension display without excessive vertical space.

Source: Established project tokens from frontend/src/styles/index.css @layer base rules. No new typography scales introduced in this phase.

Accent reserved for:

• Rubric selector focus ring and selected state border
• "Manage Rubrics" navigation link text
• Dimension weight progress bar fill in rubric preview
• Primary CTA button ("Save Scenario")

Scoring semantic colors (unchanged, already in design system):

• var(--strength) #22C55E -- strengths in scoring feedback
• var(--weakness) #F97316 -- weaknesses in scoring feedback
• var(--improvement) #A855F7 -- improvement suggestions in scoring feedback
• var(--chart-1..5) -- RadarChart dimension colors (already dynamic, no changes)

Trigger: Admin opens Scenario Editor dialog (create or edit).

Layout: The rubric selector replaces the ScoringWeights card. Position in form: after Key Messages, before Pass Threshold.

Behavior:

1. Selector loads available rubrics via useRubrics() TanStack Query hook (already exists).
2. Each <SelectItem> shows: rubric name + dimension count badge (e.g., "Default F2F Rubric (5 dimensions)").
3. Default rubrics (where is_default === true) are listed first with a "(Default)" suffix.
4. When editing an existing scenario, the selector pre-fills with scenario.rubric_id.
5. When creating a new scenario, the selector defaults to the default rubric for the selected mode (f2f or conference).
6. Changing the selected rubric immediately updates the dimension preview below.

Dimension Preview (read-only):

• Rendered inside a <Card> with bg-muted/50 background, padding 16px.
• Each dimension row: dimension name (left, text-sm font-medium), weight percentage (right, text-sm text-muted-foreground), and a thin progress bar (h-1.5 bg-primary rounded-full) showing weight relative to 100.
• Criteria text shown as text-xs text-muted-foreground truncated to 1 line with ellipsis per dimension.
• If no rubric selected, show placeholder text: "Select a scoring rubric to see dimensions".

"Manage Rubrics" link:

• Below the dimension preview card.
• Text: "Manage Rubrics" (en-US) / "管理评分标准" (zh-CN).
• Style: text-sm text-primary hover:underline cursor-pointer.
• Behavior: opens /admin/scoring-rubrics in the same tab (uses navigate()).

Trigger: Admin selects a scenario mode (f2f or conference).

Behavior: When mode changes, if rubric_id is currently the default rubric for the previous mode, auto-switch to the default rubric for the new mode. If rubric_id was manually selected (non-default), keep it unchanged.

Trigger: User views scoring feedback for any session (old or new).

Behavior:

• New sessions: dimension names come from the rubric associated with the scenario at scoring time. Names display as stored in ScoreDetail.dimension.
• Old sessions (pre-refactor): dimension names are stored as snake_case keys like key_message. Display using a fallback chain:
  1. Check i18n scoring:dimensions.{key} -- if translated, use translation.
  2. Otherwise, convert snake_case to Title Case (e.g., key_message becomes "Key Message").
• This fallback utility function is shared across RadarChart, DimensionBars, and FeedbackCard.

Trigger: Admin submits Scenario Editor form.

Validation:

• rubric_id is required (zod: z.string().min(1, "Scoring rubric is required")).
• The 5 weight_* fields are removed from the zod schema entirely.
• pass_threshold remains unchanged (0-100 number).

// REMOVE: ScoringWeights interface
// REMOVE from Scenario: weight_key_message, weight_objection_handling, 
//   weight_communication, weight_product_knowledge, weight_scientific_info
// REMOVE from ScenarioCreate: all weight_* optional fields
// REMOVE from ScenarioUpdate: inherited weight_* fields

// ADD to Scenario:
rubric_id: string;
rubric?: Rubric;  // optional populated relation

// ADD to ScenarioCreate:
rubric_id: string;  // required

// ADD to ScenarioUpdate:
rubric_id?: string;

The Rubric, RubricCreate, RubricUpdate, DimensionConfig types are already correct.

No new UI components are installed from any registry for this phase. All UI elements are composed from existing project components.

A shared utility function is required for consistent dimension name display across all scoring components.

File: frontend/src/lib/dimension-display.ts

Contract:

/**
 * Resolve a display-friendly name for a scoring dimension.
 * 
 * Priority chain:
 * 1. i18n translation key `scoring:dimensions.{camelCase(key)}`
 * 2. Title Case conversion of the raw key (snake_case -> Title Case)
 * 
 * This ensures backward compatibility: old sessions with dimension
 * names like "key_message" display as "Key Message Delivery" via i18n,
 * while new sessions with rubric-defined names like "Clinical Data Accuracy"
 * display as-is (no i18n key exists, Title Case of the original is the name).
 */
export function getDimensionDisplayName(dimension: string, t: TFunction): string;

• Dimension 1 Copywriting: PASS
• Dimension 2 Visuals: PASS
• Dimension 3 Color: PASS
• Dimension 4 Typography: PASS
• Dimension 5 Spacing: PASS
• Dimension 6 Registry Safety: PASS

Approval: pending