# AI in regulated industries: why explainability has to come first > Insurers can get real gains from AI and still meet what the FCA, the EU and US regulators expect, provided every decision can be explained, reproduced and signed off. Published 20 April 2026. Category: Regulation. Source: https://www.swallow.app/post/ai-in-regulated-industries-explainability Regulated industries get a reputation for being slow with AI. Look closer and most of them aren't slow, they're careful. In insurance, lending or healthcare, a decision has to make sense to the customer, stand up in front of a regulator and, sometimes years later, be reproduced exactly as it was made. That doesn't rule AI out. It changes how you build with it. The firms getting results are the ones that design for those demands from day one, not the ones that bolt them on at the end. ## Where AI earns its keep In a regulated business, the gains tend to show up in four places: - **Time to market.** Work that used to take months, such as building a new product or reworking a rate structure, can be drafted and tested in days. - **Consistency.** A well-tested process applies the same rules every time, where manual work varies from person to person. - **Where experts spend their time.** Less keying in and reading through documents, more judgement. - **The paper trail.** Processes built around AI usually log, version and test more than the ones they replace. That last one surprises people. Done properly, bringing in AI can leave a firm with a clearer record of how its decisions are made than it had before. ## What regulators are looking for The rules differ by market, but the themes are much the same. - **UK.** The FCA's Consumer Duty expects firms to deliver good outcomes and fair value for retail customers, and its general insurance pricing rules, in force since January 2022, stop firms charging renewing home and motor customers more than equivalent new ones. Firms need to be able to show how a price was set. We cover this in more detail in [fair value under Consumer Duty](https://www.swallow.app/post/consumer-duty-pricing-fair-value). - **EU.** The AI Act treats AI used for risk assessment and pricing of individuals in life and health insurance as high-risk, which brings duties around risk management, data governance, documentation, transparency and human oversight. - **US.** The NAIC has adopted a model bulletin on insurers' use of AI, which a growing number of states have taken up. It expects a written AI governance programme and evidence that AI-supported decisions comply with insurance law, including the rules on unfair discrimination. The common message: use AI by all means, but be able to show how each decision was reached. ## Build it in, don't bolt it on One approach is to take a model that was never meant to be explained and try to generate explanations for it afterwards. That tends to be fragile. It is far more dependable to make explainability part of how the system works: 1. **Logic people can read.** Hold rates and rules as readable configuration, not program code, so the model documents itself. 2. **A trace for every decision.** Record the inputs used, the rules checked and the value produced at each step, so any decision can be replayed. 3. **A version for every change**, with each decision tied to the version that made it. 4. **Tests as proof.** Run a set of known cases with expected results on every change, and keep the results. 5. **A person signs off.** AI can draft and propose. A named person approves before anything goes live. ## Let AI draft, and people decide The pattern that works for the business and the regulator is the same. AI does the drafting, extraction and checking at speed. Accountable people review and approve. The AI makes the work faster, and the way the system is designed makes it defensible. That is how Swallow approaches pricing. Models are plain JSON with labelled inputs, rate tables as rows and formulas written out, so an underwriter can review a change without reading code. Tests live on each version. Any quote can return a [full trace](https://www.swallow.app/developer/quoting) of every input, lookup, formula and rule, and is saved with the version that priced it. AI clients connected through the [Swallow MCP server](https://www.swallow.app/developer/mcp) can draft a model, validate it and run its tests, but no MCP tool creates, edits or publishes a project. The agent hands the model to a person, who uploads it to a draft in the dashboard. From there it is approved and published by people your team's roles allow, and each step is logged with who did it and when. Our post on [using Claude and ChatGPT to build pricing models](https://www.swallow.app/post/using-claude-and-chatgpt-to-build-pricing-models) shows the workflow end to end. ## In short Regulated firms don't have to pick between using AI and meeting their obligations. Explainability, testing and human sign-off are what let an AI project go live at all, and the firms that plan for them early usually end up moving faster.