--- title: "How to A/B Test Your Shopify Cart Abandonment Popups" description: "NavonaAI now supports A/B testing. Here's what to test on your cart abandonment popups, in what order, and how to read the results without fooling yourself." canonical: https://navona.ai/blog/how-to-ab-test-shopify-cart-abandonment-popups generator: scripts/generate-agent-md.mjs published: 2026-08-19 category: product tags: ["a/b testing","cart abandonment","conversion optimization","shopify"] --- # How to A/B Test Your Shopify Cart Abandonment Popups ![NavonaAI graphic titled "How to A/B Test Your Shopify Cart Abandonment Popups" showing two prevention popup variants, Group A at 15% off and Group B at 25% off, each applying its discount directly with no email required](https://navona.ai/images/blog/how-to-ab-test-shopify-cart-abandonment-popups/cover.png) To A/B test a cart abandonment popup, run two versions of your prevention settings at the same time and split your traffic between them. Start with discount amount, then test one setting at a time: discount type, minimum purchase threshold, and stacking. Run each test for at least 7 to 14 days. ## A/B testing is now live in NavonaAI You can now run two versions of your prevention setup side by side and let real shopper data tell you which one wins, instead of guessing. One group of carts sees version A, the other sees version B, and NavonaAI tracks how each one performs. When the test is done, you pick the winner and it becomes your new default. It's available on every plan, no upgrade needed. You choose your two versions, pick what you're measuring, and decide when the test should stop, either manually or once you've seen enough cart activity. Only one test runs at a time per store, and every shopper gets a consistent experience across their visit, so nobody sees version A one minute and version B the next. ![NavonaAI A/B testing setup screen showing Group A (Control) at 10% off and Group B (Treatment) at 15% off, with matching minimum purchase and stacking settings](https://navona.ai/images/blog/how-to-ab-test-shopify-cart-abandonment-popups/01-ab-test-setup.png) Each experiment splits your traffic into two groups. Group A is your control, usually your current setup. Group B is the change you want to test. Configure both panels the same way you'd configure your regular prevention settings: turn AI Prevention on or off, pick a discount type and amount, set a minimum purchase, decide whether it stacks with other discounts, and exclude any products you don't want it applying to. The only rule: change one setting between groups, not several, or you won't know which change caused the difference. ![NavonaAI experiment termination rules screen, set to volume-based with the experiment stopping after 1,000 total carts](https://navona.ai/images/blog/how-to-ab-test-shopify-cart-abandonment-popups/02-termination-rules.png) Before you start, pick your goal and how the test should end. Manual runs until you stop it. Volume-based runs until a set number of carts have gone through the test, which is the better choice if your traffic is lower, since it guarantees a real sample instead of stopping on a fixed date. Every experiment moves through the same four stages, start to finish: ![Diagram showing the four-stage experiment lifecycle: Draft, Running, Completed, and Winner Promoted](https://navona.ai/images/blog/how-to-ab-test-shopify-cart-abandonment-popups/03-experiment-lifecycle.svg) That's the full setup, and it's included on every plan already. If you're not running NavonaAI yet, you'll have this available from the moment you install it. [Start Free Trial](https://navona.ai/pricing) ## Why guessing costs you money Most stores pick a discount and stick with it. Ten percent sounds reasonable, so they go with ten percent. But there is no way to know if ten percent is right for your store. Maybe five percent would convert just as well and save you margin on every order. Maybe fifteen percent would convert enough extra shoppers to be worth the cost. The only way to find out is to test it on your own shoppers. ## Test 1: discount amount This is the most common test and usually the most profitable. For example: - **Control:** 10% off - **Treatment:** 15% off The question is whether the extra five points buys you enough additional orders to cover what you give away on every order. Sometimes it does. Often it doesn't, and you find out you can go the other direction. Try 10% vs. 5% too, or 15% vs. 25%. If the smaller discount converts about the same, you keep the difference on every single order from then on. ## Test 2: percentage vs. fixed amount The same discount can feel different depending on how you write it. - **Control:** 10% off - **Treatment:** €10 off On a €100 cart these are identical. But shoppers don't do the math. A fixed amount feels concrete and immediate. A percentage feels bigger on expensive carts. Which one wins depends on your price points. Stores with low average order values often do better with percentages, because 10% off a €25 order sounds better than €2.50 off. Higher-ticket stores often do better with fixed amounts. If your store sells across a wide range of cart values, this test is worth running early. ## Test 3: minimum purchase threshold A minimum cart value protects your margin by making sure the discount only applies to orders worth discounting. For example: - **Control:** $20 minimum - **Treatment:** $50 minimum cart value The risk is that shoppers below the threshold see an offer they can't use and leave anyway. The upside is that some shoppers add an item to qualify, which lifts your average order value. A reasonable starting point is around half your average order value. If your AOV is $60, test a $30 minimum. Set it too high and the offer stops applying often enough to matter. ## Test 4: discount stacking Stacking decides whether your prevention discount combines with other discounts you're running, like order-level discounts, product discounts, or free shipping offers. For example: - **Control:** Stacking off - **Treatment:** Stacking on Turning stacking on makes the offer stronger, which can lift conversions. It also means a shopper who already has 20% off a sale item gets another discount on top. If you run frequent promotions, this test matters more than it looks. The result tells you how much of your prevention lift depends on stacking, and whether the combined discount is costing more than it earns. Most stores should start with stacking off and test turning it on, not the other way around. ## Test 5: product and collection exclusions You can restrict the discount so it never applies to certain products or collections. For example: - **Control:** Discount applies to everything - **Treatment:** Discount excludes your lowest-margin collection Worth testing if you sell across very different margins. A discount that makes sense on accessories might not make sense on your flagship product. Run this after you've tested amount and threshold. It's a refinement, not a starting point. ## How to run these tests well A few rules make the difference between a test that tells you something real and one that wastes two weeks. **Change one thing at a time.** If your treatment has a different discount amount and a different minimum purchase, and it wins, you have no idea which change did it. Isolate the variable. **Pick one goal before you start.** Decide upfront whether you're judging on conversion rate or abandonment rate, and stick with it. Changing what you're measuring partway through makes the result meaningless. **Run it long enough.** Give any test at least 7 to 14 days. Shopping behaviour shifts across the week, and a test that ran Tuesday to Thursday is really just measuring which days landed in your window. Two weeks covers two full cycles. **Get enough carts.** Aim for at least a few hundred carts per version. If your traffic is lower, use a cart-volume target instead of a fixed number of days, so the test runs until you actually have the data. **Don't call it early.** It's tempting to stop the moment one version looks like it's winning. A handful of big orders landing in one group on day three can make it look like a clear winner when it isn't. Let it run its full course. Each test takes about two weeks. Work through discount amount, discount type, minimum purchase threshold, stacking, and exclusions in that order, and by the end you'll have settings based on your own shoppers rather than on a default someone else picked for you. Keep a record of what you tested and what happened. Over time that record becomes the clearest picture you have of what your specific shoppers respond to, and it saves you from re-testing the same thing six months from now because nobody wrote down the answer. ## Start your first test Discount amount takes about two minutes to set up. Head to A/B Testing in your NavonaAI dashboard, set your control at your current discount and your treatment a few points higher or lower, and start the test. Don't have NavonaAI yet? Start with a free trial and you'll be able to run this exact test from day one. [Start Free Trial](https://navona.ai/pricing) ## FAQ ### How long should I run an A/B test on my abandonment popup? At least 7 to 14 days. Shopper behaviour varies by day of the week, so a shorter window can skew results based on which days happened to fall inside it. Two weeks covers at least two full weekly cycles. ### What's the first test I should run? Discount amount. Test your current discount against one a few points higher or lower, with everything else identical. It's the test most stores get the clearest signal from, and the result either protects margin or lifts conversions immediately. ### How many carts do I need for a reliable result? Aim for at least a few hundred carts per group. If your store has lower traffic, use a volume-based termination rule instead of a fixed time window, so the test runs until you actually have enough data. ### Can I test more than one setting at a time? You can, but you won't know which change caused the result. Change one setting between your control and treatment groups and keep everything else identical. ### What happens after the test ends? Your store keeps running on whatever settings were active before the test. Nothing changes automatically. You review the results and promote the winning variant, which applies its full configuration as your new default.