How does your team know a rate change is right? For many, the honest answer is a handful of quotes run by hand before release. That catches the big mistakes. It misses the quiet ones: two factors applied in the wrong order, a band edge that's out by one, a refer rule that stopped firing three releases ago.
Software teams settled this long ago with regression tests. Pricing can borrow the same habit, and with AI speeding up how quickly models change, it can't really afford not to.
A test is a quote with a known answer
A pricing test fixes a full set of inputs to the result you expect. Here are three from our pet insurance model:
- A neutered Persian cat, aged 3, in Greater London, with a 4,000 vet-fee limit and a 100 excess, prices at 255.34 and is valid.
- A cat aged 15 is declined.
- A French Bulldog on 7,000 cover still prices, but is referred to an underwriter.
Run all of them after every change. If any result moves, you know at once, and you know which case.
Building the suite
Good suites draw on a few sources:
- Worked examples from the rating guide. These are the reference. If the model disagrees with the actuary, the model is wrong.
- Band edges. Ages, limits and sums insured either side of each boundary, because that is where off-by-one errors live.
- Every rule, both ways. For each decline and refer, one case that triggers it and one just inside that doesn't.
- Real quotes. A sample of saved production quotes guards against side effects when a model is restructured.
- Old bugs. Every pricing bug you fix becomes a test, so it can't come back.
Treat a failing test as a broken model, and expect results to match to the penny.
In Swallow, tests live on each version of the model. You can add them one by one, upload them as a CSV, or generate a starting set and trim it down. The dashboard shows each test's expected result next to the actual one, with the difference, plus an overall view of how the suite moved. For very large sets you can run a bulk CSV and download the results. See Swallow Test for more.
Testing a change you meant to make
When you change rates on purpose, some tests should move. The question is whether only those tests moved, and by the right amount. The routine:
- Record the suite's results on the current version.
- Make the change in a new draft.
- Run the suite again and look at every difference.
- Update expected results only where the change was meant to land, with a reason for each.
- Approve and publish only when nothing moves that you can't explain.
A worked example: raise the Greater London region factor in the pet model from 1.30 to 1.35. The Persian cat goes from 255.34 to 265.16. The risk premium is 142 × 1.0 (age) × 1.3 (breed) × 1.35 (region) × 1.0 (excess) × 0.95 (neutered), rounded to 236.75, and 12% Insurance Premium Tax of 28.41 takes it to 265.16. The Labrador in the North, the two declines and the French Bulldog refer don't move at all. That's the whole review: one expected change, nothing else.
AI makes tests more important, not less
An AI agent can change a model in seconds, so the check has to be just as quick. Any agent editing a model should run the suite after each edit and report what passed and what failed. The person reviewing reads that report first, then the edit. The tests become the contract between the agent doing the work and the person signing it off.
Swallow's MCP server gives agents the tools for this: test_swallow_project prices every test through the engine, generate_swallow_tests suggests cases by varying each input in turn, and validate_swallow_tests reports the price spread and which refers, declines and endorsements the suite covers, so gaps stand out. More on that in using Claude and ChatGPT to build pricing models.
One rule matters above the rest: an agent should never change a test's expected result to make it pass. Expected values change when a person decides they should, for a stated reason.
Tests as evidence
A versioned model with a suite saved on every version shows what the model was meant to do, that it did it, and who approved each change. That record is useful to compliance teams and to anyone who has to explain a price later. Pair it with a quote debug trace and you can show exactly how any single premium was built.
Build the suite when you build the model
The cheapest moment to write tests is at the start, from the same worked examples used to design the model. Every model on our model building page ships with its suite, and you can watch it pass before anything goes live.
