How To Do
Pricing Experiments
Pricing is one of the most powerful levers in any business, yet it remains one of the least tested. Whilst companies routinely experiment with their product features, marketing messages, and user interfaces, pricing decisions are often made once and left untouched for years, or worse, copied directly from competitors without any empirical validation.
At 173tech, we have worked with dozens of businesses on their data infrastructure, and we have seen firsthand how the companies that treat pricing as an ongoing experiment rather than a one-time decision consistently outperform their peers. Yet we have also witnessed countless pricing experiments that fail to produce actionable insights, waste resources, and sometimes even damage customer relationships.
The difference between a successful pricing experiment and a failed one rarely comes down to the sophistication of the statistical models or the complexity of the pricing strategy. Instead, it comes down to understanding the fundamentals: what to test, how to test it, and how to interpret the results in the context of your broader business goals.
This article will walk you through the practical reality of running pricing experiments that actually work, from avoiding the common pitfalls that doom most tests before they begin, to understanding the anatomy of a proper experiment, to knowing when to pull the plug on a test that is not delivering value.
Why Most Pricing Experiments Fail
Before we discuss how to run a successful pricing experiment, it is worth understanding why so many fail. The graveyard of abandoned pricing tests is littered with experiments that seemed promising in theory but collapsed in practice.
Testing Too Many Variables Simultaneously: The most common mistake we see is trying to test too much at once. A company might simultaneously change the price point, the billing frequency, the packaging structure, and the feature set, then wonder why they cannot determine which change drove their results. When you alter multiple variables simultaneously, you create a combinatorial explosion of possible explanations for your outcomes. Did revenue increase because the price was higher, or because the annual billing option reduced churn? Did conversion drop because the price was too high, or because the new packaging was confusing? Without isolated variables, you are left guessing.
Failing to Define Success Metrics in Advance: Perhaps the most insidious failure mode is not deciding what constitutes success before launching the experiment. Without pre-defined success criteria, teams fall prey to motivated reasoning, interpreting ambiguous results in whatever way supports their pre-existing beliefs about what the price should be. We have seen experiments where revenue increased but conversion decreased, leading to endless debates about whether this constituted success or failure. Had the team agreed in advance on which metric took precedence, or how to weight multiple metrics, these debates would have been unnecessary.
Insufficient Sample Sizes: Pricing experiments require statistical power to detect meaningful differences, yet many businesses launch tests without calculating whether they have enough traffic or customers to reach significance within a reasonable timeframe. A software company with 100 new signups per month might launch a pricing test expecting clear results within a fortnight. But with that volume, they would need an enormous effect size (perhaps a 30 per cent difference in conversion rates) to detect statistical significance. Subtle but meaningful differences of 5 to 10 per cent could take six months or longer to validate, by which time market conditions may have shifted entirely.
Not Accounting for Customer Segmentation: Pricing sensitivity varies dramatically across customer segments, yet many experiments treat all customers as a homogeneous group. A 20 per cent price increase might be completely imperceptible to enterprise clients but devastating to startups. A monthly billing option might be essential for small businesses but irrelevant to larger organisations with annual budget cycles. When you aggregate results across disparate segments, you often end up with muddy, inconclusive findings that mask clear signals within specific customer groups. The experiment appears to show “no significant difference” overall, when in reality it revealed strong positive results for one segment and strong negative results for another.
Testing for Too Short a Duration: Pricing changes often have lagging effects that take weeks or months to fully materialise. A price increase might initially boost revenue whilst simultaneously increasing churn rates, but that increased churn may not become apparent for 60 or 90 days. Similarly, the customers who convert at a higher price point might have different lifetime value characteristics than those who converted at the lower price. But you will not know this until you have observed their behaviour over a complete customer lifecycle. Many experiments are concluded prematurely based on early signals that later prove misleading. The business sees an immediate revenue boost, declares victory, and rolls out the change, only to discover three months later that customer quality has deteriorated significantly.
Ignoring Cohort Effects and Seasonality: Businesses have rhythms. Software purchases spike in Q4 and Q1 when budgets refresh. Consumer spending patterns shift around holidays. B2B purchasing grinds to a halt in August and December when decision-makers are on holiday. If you launch a pricing experiment in November and see strong results, is that because your new pricing is better, or because November is simply a strong month for your business? Without accounting for these temporal patterns, you risk attributing normal seasonal variation to your pricing changes.
Not Considering the Qualitative Feedback Loop: Numbers tell you what happened, but not why it happened. An experiment might show that a higher price point reduces conversion by 15 per cent, but without understanding why customers are balking, you cannot determine whether this represents genuine price sensitivity or simply poor value communication. Many failed experiments ignore the qualitative signals entirely. Customer support tickets spike with questions about the new pricing structure, but this feedback never makes it back to the team running the experiment. Sales representatives report that prospects are confused by the new options, but this intelligence remains siloed.
The Anatomy Of A Proper Pricing Test
A well-structured pricing experiment is built on five foundational elements: clear hypotheses, isolated variables, adequate sample sizes, appropriate duration, and pre-defined success metrics. Let us examine each in detail.
Forming a Clear Hypothesis
Every pricing experiment should begin with a specific, falsifiable hypothesis. Not “we think our price might be too low” but rather “we believe that increasing our standard plan price from £99 to £129 per month will increase revenue per customer by at least 20 per cent whilst decreasing conversion by no more than 10 per cent, resulting in a net revenue increase of 8 per cent.”
A proper hypothesis includes three components: the specific change you are testing, the expected impact on key metrics, and the threshold at which you will consider the experiment successful. This forces you to think through the trade-offs before launching the test and prevents post-hoc rationalisation of ambiguous results.
Your hypothesis should be grounded in evidence; customer research, competitive analysis, value proposition assessment, rather than pure intuition. Perhaps your customer success team reports that enterprise clients regularly describe your product as “cheap” relative to the value it delivers. Perhaps competitive analysis reveals that you are priced 40 per cent below comparable solutions. Perhaps your high-touch sales process is attracting too many low-value customers who could self-serve at a higher price point.
Isolating Variables
Once you have a clear hypothesis, the next step is to isolate exactly one variable to test. If you want to test pricing, test only pricing, not pricing plus a new feature plus a different billing option plus revised packaging. This seems obvious in principle but becomes surprisingly difficult in practice. Product teams want to bundle their feature launches with pricing changes. Marketing teams want to adjust the messaging alongside the price point. Finance teams want to restructure the plans whilst increasing prices. The discipline required to test one thing at a time is considerable, but it is non-negotiable if you want interpretable results. Every additional variable you introduce multiplies the complexity of your analysis and reduces your ability to draw clear conclusions. There is one important exception to this rule: sometimes you need to test multiple price points simultaneously against your control. Testing £99, £129, and £149 against your current £79 price point is not the same as testing multiple variables, you are still testing one variable (price) but exploring multiple values of that variable. This is not only acceptable but often advisable, as it helps you understand the shape of the demand curve rather than just a single point on it.
Calculating Required Sample Size
Before launching any experiment, you need to calculate whether you have sufficient volume to detect a meaningful effect within a reasonable timeframe. This requires some basic statistical knowledge, but the fundamental inputs are straightforward: First, determine your baseline conversion rate or whatever metric you are attempting to move. Second, decide on the minimum detectable effect, the smallest change that would be meaningful to your business. Third, select your desired confidence level (typically 95 per cent) and statistical power (typically 80 per cent).
With these inputs, you can calculate how many observations you need in each test group to reliably detect your target effect size. There are numerous online calculators that will perform this computation for you; at 173tech, we typically use Evan Miller’s sample size calculator as our starting point. If the calculation reveals that you need six months to reach adequate sample size, you have three options: increase your traffic, accept a longer test duration, or decide that you can only reliably detect larger effect sizes. What you cannot do is proceed with inadequate sample size whilst expecting statistically significant results.
Determining Appropriate Duration
The duration of your pricing experiment should be driven by three factors: the time required to reach your target sample size, the length of your typical customer decision cycle, and the need to observe at least one complete seasonal cycle for your business. For a high-volume consumer application with daily purchase decisions, a test might reach statistical significance within a week or two. For a B2B software company with month-long sales cycles, you might need to run the experiment for 90 days just to capture complete purchase journeys.
Beyond statistical considerations, you also need to consider customer lifecycle effects. If your typical customer takes 60 days to fully onboard and begin extracting value, you need to run your experiment long enough to observe whether the customers acquired at the new price point exhibit different retention or expansion behaviour. As a general guideline, we recommend running pricing experiments for a minimum of four to six weeks, even if you reach statistical significance earlier. This provides some protection against weekly fluctuations and allows enough time for word-of-mouth effects, support load changes, and other lagging indicators to emerge.
Defining Success Metrics and Decision Criteria
Before launching your experiment, document exactly how you will determine success. This should include both your primary metric (the key indicator you are trying to move) and secondary metrics (other indicators that provide important context). For a typical pricing experiment, your primary metric might be revenue per visitor, which elegantly combines both conversion rate and average order value. Secondary metrics might include conversion rate, average order value, customer acquisition cost, and initial customer cohort retention at 30 and 60 days.
Crucially, decide in advance how you will weigh these metrics if they move in different directions. If revenue per visitor increases by 15 per cent but conversion rate drops by 20 per cent, does the experiment succeed or fail? There is no universal right answer, but there must be a predetermined answer agreed upon by all stakeholders. Also establish your decision thresholds: at what point will you conclude the experiment? This should include both a success threshold (if metric X improves by Y per cent with Z confidence, we will roll out the change) and a failure threshold (if metric X declines by Y per cent, we will end the test immediately). These pre-commitments protect you from the temptation to extend experiments indefinitely in hopes of reaching significance, or to end them prematurely when early results support your preferred outcome.
Expert help is only a call away. We are always happy to give advice, offer an impartial opinion and put you on the right track. Book a call with a member of our friendly team today.
Experiment Types
Not all pricing experiments are created equal. The optimal approach depends on your business model, customer base, and what you are attempting to learn. Here are the most common experiment types we see working effectively in practice.
A/B Tests for New Customers
The most straightforward pricing experiment is a simple A/B test where new customers are randomly assigned to either your current pricing (control group) or a new pricing structure (test group). This is the gold standard when you have sufficient new customer volume and want to test a comprehensive pricing change. The key advantage of this approach is its simplicity and interpretability. You are measuring the impact of pricing on real customer behaviour in real purchase situations. There is no need for complex modelling or assumptions, you simply observe which group generates better outcomes according to your pre-defined metrics. However, new customer A/B tests have limitations. They tell you nothing about how existing customers would respond to the pricing change. They require substantial time to reach significance if your new customer volume is modest. And they typically only measure initial conversion and early retention, not long-term customer value.
Price Discrimination Tests
Rather than showing different prices to random segments, some businesses test pricing variations across naturally occurring segments; geography, company size, industry, traffic source, or other characteristics that might correlate with willingness to pay. For example, you might test whether customers arriving from paid search can bear a 20 per cent higher price point than those arriving organically, or whether enterprise customers in financial services will accept pricing that would be untenable for startups in e-commerce. This approach requires careful thought about fairness and transparency. Customers who discover they are paying more than others for an identical product can feel justifiably aggrieved. You need clear, defensible rationales for any pricing variations, ideally based on genuine cost-to-serve differences or meaningfully different value propositions.
Grandfathering Experiments
For businesses with established customer bases, one option is to introduce new pricing for new customers whilst grandfathering existing customers at their current rates. Whilst not a “pure” experiment in the statistical sense, this approach allows you to test new pricing with minimal risk to your existing revenue base. The challenge with grandfathering is that it creates a permanently two-tier pricing structure, which can complicate your billing systems, customer support, and future pricing changes. It also means you cannot directly measure how your existing customers would respond to the new pricing.
Bundling and Unbundling Tests
Sometimes the most impactful pricing experiments involve restructuring how value is packaged rather than changing absolute price points. You might test bundling previously separate products into a single offering, or conversely, unbundling a comprehensive package into component parts that customers can purchase individually. These experiments can reveal surprising insights about how customers perceive value and make purchase decisions. We have seen businesses discover that customers happily pay more for fewer features when the packaging is clearer, or that offering more granular options reduces rather than increases revenue because it introduces decision paralysis.
Value Metric Tests
Your value metric (the unit by which you charge customers) has profound implications for both revenue and customer behaviour. Testing alternative value metrics can be enlightening, though it is also among the most complex experiment types. For instance, a marketing automation platform might test pricing based on number of contacts, number of emails sent, or number of active users. A data analytics tool might test pricing based on data volume processed, number of queries executed, or number of dashboard viewers. The challenge with value metric experiments is that they often require significant product and billing system changes, making them expensive to test and difficult to reverse. They also fundamentally change your incentive alignment with customers, which can have far-reaching effects on product usage, expansion, and retention.
Discount and Promotion Tests
Testing temporary discounts or promotional pricing can help you understand price sensitivity without committing to permanent changes. However, discount tests require particular caution because they can train customers to wait for promotions rather than purchasing at full price. We generally recommend time-boxing any discount experiments very clearly and being thoughtful about which customer segments see promotional pricing. A discount offered to new customers who have never seen your regular pricing is very different from a discount offered to existing customers or prospects who have previously declined to purchase.
Measuring Impact Beyond Revenue
Revenue is the most obvious metric for evaluating pricing experiments, but it is far from the only important one. A pricing change ripples through your entire business, affecting customer behaviour, operational costs, competitive positioning, and strategic optionality. Here is how to measure these broader impacts effectively.
Customer Quality and Lifetime Value
A higher price point typically attracts a different customer profile than a lower one. These customers may be more established, have larger budgets, face more significant pain points, or simply have different expectations about the product and support experience. To understand customer quality shifts, track cohort-level metrics for customers acquired at each price point: retention rates at 30, 60, and 90 days; expansion revenue over the first year; support ticket volume per customer; feature adoption and engagement levels; and net revenue retention.
These metrics take time to mature; you will not have meaningful 90-day retention data until 90 days after launching your experiment. But they are essential for understanding whether a price increase that boosts immediate revenue also improves or degrades long-term customer value. We have seen pricing experiments that appeared highly successful based on initial conversion and revenue metrics but proved disastrous once long-term retention data came in. Conversely, we have seen experiments that initially looked marginal but proved transformative once higher lifetime values became apparent.
Customer Acquisition Cost Efficiency
Pricing changes affect your customer acquisition cost efficiency in subtle ways. A higher price point might reduce conversion rates, but if it attracts customers with lower support costs and higher retention, your actual cost to acquire a valuable customer may decrease. Track your customer acquisition cost to lifetime value ratio separately for each pricing cohort. This gives you a more complete picture of acquisition efficiency than either metric in isolation. A 20 per cent increase in customer acquisition cost might be perfectly acceptable if customer lifetime value increases by 40 per cent. Also monitor your payback period, how long it takes for a new customer to generate enough gross margin to cover their acquisition cost. Shorter payback periods give you more capital efficiency and strategic flexibility, even if absolute lifetime value remains similar.
Sales Cycle and Conversion Funnel Impact
Pricing changes can accelerate or decelerate your sales cycle in ways that have significant operational implications. A higher price point might require more stakeholder approvals, extending your sales cycle by several weeks. Alternatively, it might quickly disqualify poor-fit prospects, shortening the cycle by removing tyre-kickers who were never going to close.
Track stage-by-stage conversion rates through your funnel for each pricing cohort: visitor to trial or demo, trial to sales conversation, conversation to proposal, proposal to close. Where do the funnels diverge? Are prospects dropping out earlier, or making it further but failing to close? Are sales cycles becoming longer or shorter? This granular funnel analysis often reveals opportunities to adjust messaging, qualification criteria, or sales process to better support your new pricing. You might discover that prospects need different value proof at higher price points, or that your sales team needs new objection handling techniques.
Customer Support and Success Costs
Different price points attract customers with different support needs and expectations. Generally speaking, customers paying more expect more; faster response times, more proactive outreach, more sophisticated troubleshooting. Monitor support ticket volume, time-to-resolution, and customer satisfaction scores across pricing cohorts. Also track customer success team time allocation, are higher-priced customers requiring disproportionate hand-holding, or are they more self-sufficient? These operational cost differences can significantly impact unit economics. If a 30 per cent price increase comes with a 50 per cent increase in support costs, your margin improvement may be far less than the headline price increase suggests.
Market Positioning and Competitive Dynamics
Pricing is not just a revenue lever, it is a powerful positioning signal. Moving your prices substantially higher or lower repositions you in the competitive landscape and changes which alternatives customers consider. This is difficult to measure directly through an experiment, but you can track several proxy metrics: competitive mentions in sales conversations and lost deal reasons; changes in the competitive alternatives prospects evaluate alongside your solution; win rates against specific competitors; and shifts in the buyer personas and company profiles you attract. Consider also running surveys or interviews with recent customers and prospects to understand how they perceive your pricing relative to alternatives. Have you moved from the “budget option” to the “premium choice”? Does your pricing now suggest enterprise-grade capabilities, or does it signal a simple, lightweight tool?
Team Morale and Sales Confidence
Finally, do not underestimate the psychological impact of pricing changes on your team. Sales representatives who have sold successfully at one price point can become anxious about higher pricing, even when it is well-justified. Support teams may worry about increased customer demands. Product teams might fear that higher prices will invite more feature requests and complaints. Conversely, appropriate price increases can boost team morale by signalling that you are building a sustainable, premium business rather than competing in a race to the bottom. They can increase sales confidence by providing better margins for enterprise deals and complex negotiations.
Whilst these factors are difficult to quantify, they are important enough to monitor through regular team surveys, one-on-one conversations, and observation of team dynamics during and after your pricing experiment.
When To Stop A Pricing Experiment
Knowing when to conclude an experiment is just as important as knowing how to design one. End too early and you might miss crucial signals. Run too long and you waste time, confuse your go-to-market teams, and potentially damage customer trust. Here is how to make sound decisions about experiment duration and conclusion.
When You Reach Statistical Significance
The most straightforward reason to stop an experiment is that you have reached your pre-defined sample size and achieved statistical significance on your primary metric. If your hypothesis was that the new pricing would increase revenue per visitor by at least 10 per cent, and your experiment shows a 12 per cent increase with 95 per cent confidence, you have your answer. However, statistical significance alone should not be your only consideration. You also want to examine your secondary metrics to ensure you are not generating unexpected negative side effects. Perhaps revenue per visitor increased, but 60-day retention declined precipitously. Perhaps average order value rose, but customer satisfaction scores plummeted. In these situations, you need to return to your pre-defined decision criteria. Did you anticipate these trade-offs? Are they acceptable given the revenue gains? If your decision framework did not contemplate this specific scenario, you may need to extend the experiment or gather additional data before proceeding.
When You Observe Unexpected Negative Impacts
Some experiments should be stopped immediately when they begin causing obvious harm to the business. If your new pricing causes conversion rates to collapse, support tickets to explode, or negative social media sentiment to surge, you do not need to wait for statistical significance to recognise a problem. Define circuit breakers before launching your experiment, specific thresholds that would trigger an immediate halt. These might include: conversion rate declining by more than 30 per cent, support ticket volume increasing by more than 50 per cent, customer satisfaction scores dropping below a specific threshold, or negative revenue impact exceeding a specific amount. When you hit a circuit breaker, stop the experiment immediately and conduct a thorough post-mortem. What went wrong? Was the hypothesis fundamentally flawed, or was the execution problematic? Were there warning signs you missed? What would you do differently next time?
When External Conditions Change Materially
Sometimes the market shifts beneath you whilst your experiment is running. A major competitor announces a dramatic price cut. Economic conditions deteriorate sharply. A regulatory change affects buying behaviour. A global pandemic disrupts everyone’s budgets and priorities. When external conditions change materially, your experiment results become difficult to interpret. Is the decline in conversion rates due to your new pricing, or because everyone’s budgets just got cut? Is the revenue increase due to your pricing change, or because your competitor’s outage sent a flood of new customers your way?In these situations, you generally have two options: stop the experiment and start fresh once conditions stabilise, or extend it substantially to see whether the external effects prove temporary. There is no universal right answer, it depends on the severity of the external change and your confidence that you can still extract valid insights.
When You Have Answered Your Question
Sometimes an experiment does not reach traditional statistical significance but nevertheless provides a clear answer to your question. Perhaps you were testing whether a 50 per cent price increase would completely destroy conversion rates, and whilst you do not yet have statistical significance on the exact conversion rate, it is abundantly clear that conversion has only declined by about 15 per cent, far less catastrophic than feared. In these cases, perfect certainty may not be necessary. If the directional signal is clear and consistent, and if waiting longer would not materially change your decision, you can conclude the experiment and proceed with confidence.This is particularly relevant when running tests with limited sample size where reaching traditional statistical significance would take many months. You may decide that a strong directional signal with 80 per cent confidence is sufficient to inform your decision, particularly if the decision is reversible.
When Opportunity Costs Become Too High
Every day you run a pricing experiment is a day you are potentially leaving revenue on the table if your new pricing is better, or damaging your business if it is worse. At some point, the cost of continued experimentation exceeds the value of additional certainty. This is especially true for experiments that require maintaining complex testing infrastructure, training sales teams on multiple pricing structures, or creating customer confusion through inconsistent pricing messages. If you have been running an experiment for six months and still do not have conclusive results, you need to make a hard decision: either commit more resources to get to significance faster (perhaps by expanding the test to more customer segments), or make a judgment call based on the imperfect data you have. Perfectionism is the enemy of good decision-making. Sometimes an 80 per cent confident decision made today is better than a 95 per cent confident decision made six months from now.
When You Are Ready to Test Something Else
Pricing experiments are just one type of experiment you might run, and your organisation’s capacity for experimentation is finite. If you have a potentially transformative product change, messaging test, or positioning shift waiting in the queue, you may need to conclude your pricing experiment, even if results are still somewhat ambiguous, to free up resources for the next test. This is where having clear decision criteria established upfront becomes invaluable. If your experiment has run for the pre-defined duration without reaching your success threshold, that is itself a form of result. “No significant difference” is an answer, even if it is not the answer you hoped for.
Avoiding the “Just One More Week” Trap
Perhaps the most common mistake in ending experiments is repeatedly extending them “just one more week” in hopes that significance will materialise. This is motivated reasoning dressed up as scientific rigour. If your experiment was properly powered from the start and has run for its intended duration, adding another week or two is unlikely to change your conclusions. If it was not properly powered, if your sample size calculation suggested you needed six months but you hoped to see results in six weeks, then you need to either commit to the full duration or accept that you will be making decisions with less certainty. The discipline to end experiments according to your pre-established criteria, rather than based on whether the results support your preferred outcome, is what separates rigorous experimentation from theatre. It is uncomfortable to accept ambiguous results or findings that contradict your intuition, but it is essential for learning and improving over time.
Conclusion
Pricing experiments are among the most powerful tools available for optimising your business model, yet they remain remarkably rare in practice. The complexity of designing and executing proper tests, combined with natural risk aversion around pricing changes, leads many businesses to avoid experimentation altogether; defaulting instead to copying competitors, intuition-based pricing, or simply inertia.
This is a missed opportunity. Pricing is not a one-time decision but an ongoing strategic question that deserves the same rigorous, data-driven approach you apply to product development, marketing optimisation, and operational efficiency.
At 173tech, we have seen businesses unlock substantial value by treating pricing as an experimental practice rather than a static decision. But we have also seen experiments fail; sometimes because of inadequate technical implementation, but more often because of fundamental flaws in experimental design, insufficient patience to gather meaningful data, or lack of organisational discipline to follow pre-established decision criteria.
The framework outlined in this article provides a foundation for effective pricing experimentation. But it is only a starting point. Your specific context, business model, and customer base will require adaptation and refinement of these principles.
The most successful pricing experimenters share several characteristics. They approach pricing with intellectual humility, recognising that their intuitions may be wrong. They invest time upfront in proper experimental design rather than rushing to launch. They resist the temptation to conclude experiments prematurely or to add “just one more variable” to the test. They measure holistically, looking beyond immediate revenue impacts to customer quality, operational costs, and strategic positioning. And they have the organisational discipline to act on experimental findings, even when those findings challenge their preconceptions.
If you take nothing else from this article, remember this: a mediocre pricing experiment properly executed and rigorously analysed will teach you more about your business than perfect pricing chosen by intuition or copied from competitors. The goal is not to find the theoretically optimal price, it is to systematically learn about your customers’ willingness to pay, your positioning in the market, and the trade-offs inherent in different pricing strategies.
That knowledge, accumulated over time through repeated experimentation, is what ultimately allows you to price with confidence. Not because you have discovered some universal truth about what customers will pay, but because you have developed a deep, empirical understanding of how your specific customers respond to different value propositions and price points and because you have built the organisational capability to continue experimenting and adapting as markets, customers, and competitive dynamics evolve.
Get In Touch
Our friendly team are always on hand to answer questions, troubleshoot problems and point you in the right direction.
