Claude Self‐Report on Using Existing Docs to Open Loop Develop with JENNER‐MEASLES‐MCP - laser-base/laser-measles GitHub Wiki
These are now fixed
This was the first set of bug we collected from Claude after Claude tried to use the JENNER-Measles MCP (and JENNER MCP) to build laser-measles models by trial and error.
❯ Tell me about how hard it was to get these working, assuming for now they are correct? Which parts of the code seem like they didn't work as expected initially, and where can the documentation provide more and better examples maybe?
● Great question! Let me reflect on the development process - there were definitely several challenging aspects where the API didn't match expectations. Here's an honest assessment:
🔴 Major Pain Points
1. API Mismatch: Demographics vs Scenarios (Model 1)
What I expected:
from laser_measles.demographics import Demographics, Patch
demographics = Demographics(patches=[...])
model = CompartmentalModel(demographics=demographics, ...)
What actually works:
import polars as pl
scenario = pl.DataFrame({"id": [...], "pop": [...], "lat": [...], ...})
model = CompartmentalModel(scenario=scenario, params=params)
Issue: The documentation search results mentioned "Demographics" and "Patch" classes, but they don't exist in the public API. This was the first blocker.
Documentation gap: Need clearer "Getting Started" examples showing the actual initialization pattern with polars DataFrames right at the top.
---
2. Component Setup Pattern (Model 1)
What I tried:
infection = InfectionProcess(model, params=infection_params)
model.add_phase(infection)
Error: AttributeError: 'CompartmentalModel' object has no attribute 'add_phase'
What actually works:
model.components = [
create_component(InfectionProcess, params=infection_params),
StateTracker
]
Issue: The pattern of passing classes (not instances) and letting the model instantiate them is non-obvious. Some need create_component(), some don't.
Documentation gap: A clear section on "Component Architecture Patterns" showing:
- When to use create_component() vs direct class reference
- How parameter passing works
- Order dependencies between components
---
3. Date Format Requirement (Model 2)
What I tried:
params = ABMParams(start_time="2020-01-01")
Error: ValueError: unconverted data remains: -01
What works:
params = ABMParams(start_time="2020-01") # YYYY-MM only!
Issue: The required date format (YYYY-MM) is nowhere in the error message or easily findable in docs.
Documentation gap: Parameter documentation should specify exact format requirements with examples.
---
4. Component Dependencies (Model 2)
What happened:
model.add_component(ConstantPopProcess)
model.add_component(InfectionProcess)
# ... crashes with IndexError
What's needed:
model.add_component(InitializeEquilibriumStatesProcess) # MUST be first!
model.add_component(ConstantPopProcess)
model.add_component(InfectionProcess)
Issue: Components have hidden dependencies. ConstantPopProcess assumes population is already initialized with proper state arrays, but crashes with cryptic index errors if not.
Documentation gap: Each component should document:
- Required preceding components
- What it initializes/modifies
- What it expects to already exist
---
5. Data Access Patterns (Models 1 & 4)
StateTracker column names:
# Expected (from variable semantics):
results_df.columns # ['tick', 'patch_id', 'state', 'count']
# Actual:
results_df.columns # ['tick', 'group_id', 'state', 'count']
StateTracker with aggregation:
# Not obvious you need this for patch-level tracking:
patch_tracker_params = BaseStateTrackerParams(aggregation_level=0)
Property access:
# Tried (failed):
tracker.history # AttributeError
# Actually:
tracker.get_dataframe() # Then pivot
Documentation gap:
- Clear API reference for each component's output format
- Examples showing "how to extract patch-level data" vs "total data"
- Consistent naming (patch_id vs group_id)
---
6. Spatial Mixing Setup (Model 4)
What I tried:
gravity_mixer = GravityMixing(params=gravity_params)
mixing_matrix = gravity_mixer.get_mixing_matrix(model) # Wrong!
What works:
gravity_mixer = GravityMixing(params=gravity_params)
infection_params = components.InfectionParams(
beta=0.8,
mixer=gravity_mixer # Pass mixer here
)
model.components = [
create_component(InfectionProcess, params=infection_params)
]
Issue: The mixer isn't passed to the model directly, but through InfectionProcess parameters. Not intuitive.
Documentation gap: Spatial mixing tutorial should show the full setup pattern, not just mixer creation.
---
7. Age Structure Limitations (Model 2)
What I expected:
model.add_component(ConstantPopProcess)
# Should get: model.people.age property
What actually happened:
- No age property created
- Can't easily track age-specific outcomes
Issue: ConstantPopProcess doesn't automatically add age structure for simple models.
Documentation gap: Clear explanation of which components add which properties, and how to access them.
---
📊 Difficulty Ranking
┌────────────────────────┬──────────────────┬────────────┬───────────────────────────────────────────────┐
│ Aspect │ Difficulty (1-5) │ Time Spent │ Notes │
├────────────────────────┼──────────────────┼────────────┼───────────────────────────────────────────────┤
│ Initial API discovery │ ⭐⭐⭐⭐⭐ │ 30% │ Demographics vs scenarios, component patterns │
├────────────────────────┼──────────────────┼────────────┼───────────────────────────────────────────────┤
│ Component dependencies │ ⭐⭐⭐⭐ │ 20% │ InitializeEquilibrium requirement │
├────────────────────────┼──────────────────┼────────────┼───────────────────────────────────────────────┤
│ Data extraction │ ⭐⭐⭐⭐ │ 15% │ Column names, aggregation levels │
├────────────────────────┼──────────────────┼────────────┼───────────────────────────────────────────────┤
│ Spatial mixing setup │ ⭐⭐⭐⭐ │ 15% │ Parameter passing through InfectionParams │
├────────────────────────┼──────────────────┼────────────┼───────────────────────────────────────────────┤
│ Date format │ ⭐⭐⭐ │ 5% │ Quick fix once found │
├────────────────────────┼──────────────────┼────────────┼───────────────────────────────────────────────┤
│ Visualization │ ⭐⭐ │ 10% │ Straightforward once data extracted │
├────────────────────────┼──────────────────┼────────────┼───────────────────────────────────────────────┤
│ Core modeling │ ⭐⭐ │ 5% │ Actually quite elegant! │
└────────────────────────┴──────────────────┴────────────┴───────────────────────────────────────────────┘
Total development effort breakdown:
- 60% fixing API/documentation mismatches
- 20% searching documentation via MCP
- 20% actual modeling and visualization
---
💡 What Worked Well
1. MCP search was invaluable - could find examples even if not perfectly documented
2. Component architecture is elegant - once you understand the pattern
3. Polars integration is nice - modern, fast dataframes
4. Error messages from model.run() - generally good (except component initialization)
5. The framework is powerful - can build complex models quickly after the learning curve
---
📚 Documentation Improvements Needed
High Priority
1. "Quick Start" Section
# Complete minimal working example - Model 1
import polars as pl
from laser_measles.compartmental import CompartmentalModel
from laser_measles.compartmental.params import CompartmentalParams
from laser_measles.compartmental import components
scenario = pl.DataFrame({
"id": ["0"],
"pop": [10000],
"lat": [0.0],
"lon": [0.0],
"mcv1": [0.0]
})
params = CompartmentalParams(num_ticks=365, seed=42)
model = CompartmentalModel(scenario=scenario, params=params)
model.components = [
components.InfectionProcess,
components.StateTracker
]
model.run()
tracker = model.get_instance("StateTracker")[0]
results = tracker.get_dataframe()
2. Component Dependency Graph
For Compartmental Models:
InitializeEquilibriumStatesProcess (optional but recommended)
↓
VitalDynamicsProcess OR ConstantPopProcess (if births/deaths)
↓
InfectionSeedingProcess (optional)
↓
InfectionProcess (required)
↓
StateTracker / CaseSurveillanceTracker (monitoring)
3. Parameter Reference Table
| Parameter | Type | Format | Example | Required |
|------------|------|-----------|-----------|----------|
| start_time | str | "YYYY-MM" | "2020-01" | Yes |
| num_ticks | int | days | 365 | Yes |
| seed | int | any | 42 | No |
4. Component Output Formats
# StateTracker with aggregation_level=-1 (default, sum all patches)
df.columns: ['tick', 'state', 'count']
# StateTracker with aggregation_level=0 (patch-level)
df.columns: ['tick', 'state', 'group_id', 'count'] # group_id = patch_id
5. Spatial Mixing Cookbook
- Complete example from scenario → mixer → infection → results
- All 4 mixer types with parameter explanations
- How to visualize mixing matrices
Medium Priority
6. ABM vs Compartmental Decision Tree
- When to use each
- Performance implications
- Feature availability matrix
7. Troubleshooting Guide
- Common errors and solutions
- "If you get IndexError in process_X, check Y"
8. Property Reference
- What properties does each component add?
- model.patches.states structure
- model.people properties for ABM
---
🎯 Bottom Line
The framework itself is excellent - once you know the patterns, you can build sophisticated models quickly. The issue is that the learning curve is steep because:
1. Search results reference old/non-existent APIs (Demographics, add_phase)
2. Component initialization patterns are non-obvious
3. Data extraction requires knowing column names and aggregation levels
4. Dependencies between components are implicit
Recommendation for documentation:
- Add 3-5 complete, copy-paste-able examples at increasing complexity levels
- Each example should include: imports → scenario → params → components → run → extract → visualize
- Annotate with comments explaining every non-obvious choice
- Show the full data pipeline, not just component creation
The MCP search was critical for figuring things out, but having these examples directly in the docs would have reduced development time by ~50%.