With pricing critical to growth and profitability, businesses must understand the impact of any price changes, as well as ensure what goes live to customers is identical to that modelled. With pricing driven by complex financial models, this is particularly true - a percentage point error in the code deployed could easily lead to a significant financial impact.
Testing is vital to any pricing change process, whether it be calculating the expected impact on quotability and sales, or monitoring each step, such as a lookup, calculation or an API call.
Through our team’s experience leading financial services businesses and our conversations with more than 100 pricing and technology leaders, we’ve identified the biggest challenges facing pricing teams, and by extension their engineering teams, when it comes to testing:
Comparison Challenges: Many companies discuss difficulty in comparing versions of pricing models to see how they change over time and impact target customers and business KPIs. With minimal version tracking and difficulty in running tests against historic models, teams are limited in their ability to iterate effectively.
Modelled to Live Variance: Without access to the realtime APIs or latest data sets it is hard to effectively test pricing in the modelled stage. As a result, the live pricing can be different to that modelled, with companies often having to deploy prices live to a small number of customers in order to properly test.
Manual Testing: As it is time consuming, many companies only run a handful of tests to check the pricing is returning as expected. With limited scope to these tests, it is likely they won’t accurately reflect the breadth of customer inputs, making it hard to ensure pricing is accurate for the entire target market, as well as sufficiently capture edge cases.
Oversight Questions: Without comprehensive testing, it can be difficult to ensure pricing is in compliance with local regulations. In addition, as companies look to more sophisticated pricing, such as gradient boosted machines in insurance, being able to test and explain pricing outputs to stakeholders and regulators becomes challenging.
A/B Test Complexity: Technical complexity prevents pricing teams from doing A/B tests to better understand demand elasticity and find the most optimal price for their products.
From our conversations with pricing leaders, 51% of them talk about testing as being a pain point for them and their team
We believe there are many opportunities to improve such governance and approvals processes. We’ve focused on 3 areas to solve with the first version of our platform:
Visualise: Quickly see the impacts of a pricing change on a standardised test set. Customise charts to show, for example quotability or average price. With an easily digestable view, identifying pricing errors becomes easy.
Tools: With testing available for every version of your model, it is easy to see the differences based on a standardised test set. In addition, with an API endpoint for every version, it is easy to run your own tests against your models.
Monitor: With ongoing monitoring, if errors haven’t been caught in testing, or internal or external APIs have issues, the pricing team will be notified by email.
Swallow Testing is just one of 4 initial products we’re releasing on our platform, please see our other posts for more on Process, Publishing and Data.
Powerful apart but awesome together. Transform how you price.
We’d love to hear more about your testing pain points, so if you’re keen to share, or would like to try out the Swallow platform - please do get in touch at contact@swallow.app
