JENNER BVT Testing Notes - laser-base/laser-core GitHub Wiki
We love our LASER AI, JENNER, but as software, we need a way of testing it. And that means not manual testing but automated. In short, assuming we can define a set of prompts that we expect to "work", we want to have a way of automatically sending those prompts to the AI, getting code back, and making sure that code "is good".
For our set of test prompts, we want to:
- Send them to an AI webservice (enterprise hosted MCP server), and get python code back.
- Run the code in an environment with laser(*) fully installed (docker image).
- Capture the output from running the code.
- Compare the output of the code to an expected result.
- Report pass/fail.
- Note that we do not here propose testing the code by inspection, just by running and inspecting the output.
We want these prompts/tests to go from something very simple to become progressively more demanding.
Here’s a curated set of 15 progressively complex prompt ideas you could use to test and reuse laser-core features. Each prompt is designed to return runnable Python code that imports and invokes at least one function or class from laser-core. This suite gradually introduces core concepts, starting with minimal LaserFrame usage and building toward demographic modeling, components, squashing, and migration.
-
Prompt 1:
"Create a minimal example that initializes a
LaserFramewith a capacity of 10 and adds a scalar property namedage, then populates it with 5 agents and prints the age array."
Sample Result 1
python3 jenner_bvt1.py
Age array:
[25 30 18 45 60]
-
Prompt 2:
"Add a vector property called
positionwith length 2 to aLaserFrameof capacity 20. Populate 3 agents and set each agent's position manually, then print the property."
Sample Result 2
python3 jenner_bvt2.py
Position array (2D):
[[1. 3. 5.]
[2. 4. 6.]]
Position array (agent-wise rows):
[[1. 2.]
[3. 4.]
[5. 6.]]
-
Prompt 3:
"Write a function that adds 100 agents to a
LaserFramewith a scalarstateproperty, sets the state of all agents to 1, and verifies that all entries are equal to 1."
Sample Result 3
python3 jenner_bvt3.py
After step 1: age = [1 1 1 1 1]
After step 2: age = [2 2 2 2 2]
After step 3: age = [3 3 3 3 3]
After step 4: age = [4 4 4 4 4]
After step 5: age = [5 5 5 5 5]
- Prompt 4: Sample Initial Ages from Population Pyramid
Use AliasedDistribution and load_pyramid_csv() to assign realistic ages (in days) to a new population of 1,000 agents using LASER’s bundled US 2023 pyramid.
Key Features:
- Tests load_pyramid_csv()
- Exercises AliasedDistribution.sample()
- Converts age bins to age-in-days
- Stores in scalar age property
Sample Result 4
python3 jenner_bvt4_realisticages.py
First 10 sampled agent ages (in days):
[ 3457 4743 1204 18044 27217 889 26394 26076 3436 1024]
Converted to years:
[ 9 12 3 49 74 2 72 71 9 2]
- Prompt 5: Set date_of_birth Based on Sampled Age
Convert sampled agent ages into date_of_birth (i.e. -age_in_days) and add date_of_birth as a scalar property in a LaserFrame. Show the first 10 values.
Key Features:
- Reinforces LASER's design: “born in the past”
- Introduces date_of_birth as a required field for demographic realism
- Ensures compatibility with later use in mortality models
- Keeps age dynamic (instead of static scalar)
Sample Result 5
python3 jenner_bvt5_dob.py
First 10 agents:
Agent 0: age=1436 days, date_of_birth=-1436
Agent 1: age=25241 days, date_of_birth=-25241
Agent 2: age=5825 days, date_of_birth=-5825
Agent 3: age=9828 days, date_of_birth=-9828
Agent 4: age=12434 days, date_of_birth=-12434
Agent 5: age=21007 days, date_of_birth=-21007
Agent 6: age=15258 days, date_of_birth=-15258
Agent 7: age=12134 days, date_of_birth=-12134
Agent 8: age=24851 days, date_of_birth=-24851
Agent 9: age=25432 days, date_of_birth=-25432
- Prompt 6: Predict date_of_death with KaplanMeierEstimator
Using KaplanMeierEstimator, predict date_of_death for all agents based on their age in days, and store it as a property in the LaserFrame.
Key Features:
- Uses create_cumulative_deaths() helper or synthetic data
- Introduces date_of_death
- Sets up agents for mortality-aware step functions in Level 3
Sample Result 6
python3 jenner_bvt5_dob.py
First 10 agents:
Agent 0: age=1436 days, date_of_birth=-1436
Agent 1: age=25241 days, date_of_birth=-25241
Agent 2: age=5825 days, date_of_birth=-5825
Agent 3: age=9828 days, date_of_birth=-9828
Agent 4: age=12434 days, date_of_birth=-12434
Agent 5: age=21007 days, date_of_birth=-21007
Agent 6: age=15258 days, date_of_birth=-15258
Agent 7: age=12134 days, date_of_birth=-12134
Agent 8: age=24851 days, date_of_birth=-24851
Agent 9: age=25432 days, date_of_birth=-25432
-
Prompt 7:
"Define a
Modelclass with aLaserFrame, addinfection_stateand a component that randomly infects 1% of agents at each step. Run 10 steps and count infections per step."
Sample Result 7
python3 jenner_bvt7.py
Step 1: infected = 10
Step 2: infected = 20
Step 3: infected = 30
Step 4: infected = 40
Step 5: infected = 50
Step 6: infected = 60
Step 7: infected = 70
Step 8: infected = 80
Step 9: infected = 90
Step 10: infected = 100
-
Prompt 8:
"Create a LASER model that includes a component for aging (increments
ageeach day), and a component that kills agents when theirageexceeds a lifespan threshold."
Sample Result 8
Step 1: alive = 100
Step 2: alive = 100
<snip!>
Step 16: alive = 100
Step 17: alive = 100
Step 18: alive = 100
Step 19: alive = 100
Step 20: alive = 100
Step 21: alive = 99
Step 22: alive = 98
Step 23: alive = 96
Step 24: alive = 95
<snip!>
Step 74: alive = 16
Step 75: alive = 15
Step 76: alive = 10
Step 77: alive = 8
Step 78: alive = 4
Step 79: alive = 2
Step 80: alive = 0
- Prompt 9: "Create a LASER model that which has 2 components that interact with each other. Maybe a timer which counts down and changes state when the counter reaches 0."
Sample Result 9:
Step 1: infected = 20, recovered = 0
Step 2: infected = 20, recovered = 0
Step 3: infected = 20, recovered = 0
Step 4: infected = 16, recovered = 4
Step 5: infected = 14, recovered = 6
Step 6: infected = 9, recovered = 11
Step 7: infected = 6, recovered = 14
Step 8: infected = 0, recovered = 20
- Prompt 10:
"Create a LASER model that includes a LaserFrame for storing time-series results (per timestep). Add a reporting component that counts the number of agents in each infection state — Susceptible (0), Infected (1), and Recovered (2) — and records their fractions in a results frame. Run the model for 20 timesteps and print the values stored in the result frame at the end."
Sample Result 10:
python3 jenner_bvt10.py
Timestep | S I R
1 | 0.990 0.010 0.000
2 | 0.980 0.020 0.000
3 | 0.970 0.030 0.000
4 | 0.960 0.040 0.000
5 | 0.950 0.050 0.000
6 | 0.940 0.060 0.000
7 | 0.930 0.070 0.000
8 | 0.920 0.080 0.000
9 | 0.910 0.090 0.000
10 | 0.900 0.100 0.000
11 | 0.890 0.110 0.000
12 | 0.880 0.120 0.000
13 | 0.870 0.130 0.000
14 | 0.860 0.140 0.000
15 | 0.850 0.150 0.000
16 | 0.840 0.160 0.000
17 | 0.830 0.170 0.000
18 | 0.820 0.180 0.000
19 | 0.810 0.190 0.000
20 | 0.800 0.200 0.000
-
Prompt 11:
"Write a component that sets agents with
status = 2as recovereds and squashes them from theLaserFrame. Print the reduced agent count."
Sample Results 11:
python3 jenner_bvt_squash.py
Before squashing: count = 100
After squashing: count = 70
-
Prompt 12:
"Demonstrate saving a population snapshot to HDF5 using
save_snapshot()and reloading it withload_snapshot(). Show that data matches before and after."
TBD
-
Prompt 13:
"Create a migration matrix using
laser_core.migration.gravity()with 3 nodes and use it to simulate movement of agents between nodes. Print before/after node distributions."
Sample Result 13: (Might not be correct)
python3 jenner_bvt_migration.py
Before migration:
Node 0: 100 agents
Node 1: 100 agents
Node 2: 100 agents
After migration:
Node 0: 153 agents
Node 1: 147 agents
Node 2: 0 agents
-
Prompt 14: Spatial
"Build a spatial SIR disease model on a 1-D ring of patches, e.g., 10 patches where the ends are connected (patch 0 neighbors patch 1 and patch 9). Population: Distribute populations heterogeneously across patches (e.g., patches 0–2 and 7–9 are high-density, others are sparse). Seeding: Infect a small fraction of agents in one high-density patch (e.g., patch 0). Migration: Implement agent migration between adjacent patches, with ring wraparound. Use a fixed per-tick migration probability. Transmission: Model SIR infection dynamics within each patch. Transmission depends on local susceptible and infectious fractions. Objective: Show that the infection spreads to distant patches via agent migration, not external seeding. The contagion should eventually appear in patches not directly connected to the seed."
Sample Result 14:
Step 1
Patch 0: S=1226 I=11 R=0
Patch 1: S=1248 I=2 R=0
Patch 2: S=1243 I=0 R=0
Patch 3: S=644 I=0 R=0
Patch 4: S=625 I=0 R=0
Patch 5: S=632 I=0 R=0
Patch 6: S=624 I=0 R=0
Patch 7: S=1220 I=0 R=0
Patch 8: S=1261 I=0 R=0
Patch 9: S=1263 I=1 R=0
----------------------------------------
<snip!>
Step 50
Patch 0: S=508 I=213 R=486
Patch 1: S=507 I=194 R=389
Patch 2: S=534 I=243 R=252
Patch 3: S=681 I=136 R=113
Patch 4: S=649 I=65 R=27
Patch 5: S=715 I=11 R=17
Patch 6: S=786 I=59 R=36
Patch 7: S=713 I=173 R=125
Patch 8: S=630 I=242 R=329
Patch 9: S=528 I=198 R=441
(This prompt produced 150 lines of perfectly working LASER spatial SIR code, with above output.)
-
Prompt 15: All The Things
"Build a minimal SIR model using
LaserFrame,PropertySet, and 3 components:SeedInfection,Transmission, andRecovery. Run it over 30 timesteps and print S, I, R counts per day."
Sample Result 15:
TBD
We see above that we can enumerate prompts which return code which can be run and produces outputs which can be measured against a reference output. The next step is to host the JENNER-MCP service at some known endpoint and run these tests automatically. It's possible that some outputs may need to be compared to the reference within some statistical bound.