Shopify Retention Engine That Measures Itself: The Unlapse Case Study
The retention engine behind unlapse.io: five models retrained nightly, one decision per customer, sends paced across the day, and a permanent 5 percent group that is never emailed and shows what the emails caused.
- 1.5M+
- orders read every night
- 5
- models retrained every night
- 5%
- held back permanently
- 0.79
- reorder model roc-auc
Retention tools send a lot of email and claim a lot of credit. Unlapse is the engine we built to do the opposite: decide, for every customer, which one message would help right now, hand that decision to the brand's own email platform, and measure what it caused against customers it deliberately never emails. It has run every night for a subscription-driven DTC brand since July 2026 and is now the product at unlapse.io.
The problem
A replenishment brand's next repeat customer is already in its order history. But a "days since last order" column cannot tell a buyer who is in rhythm from one who is overdue, or a paused subscriber from one who has left. And no attribution report can say what the emails actually caused, because loyal customers would have bought anyway. On the first brand, 31.3 percent of first-time buyers placed a second order within 90 days on their own. Any email programme has to beat that, and most quietly take credit for it.
What we built
Every night:
- Five models, retrained from scratch. Four predict who will buy: who will reorder within 60 days, who will start a subscription within 90 days, who will restart one they cancelled, and who will buy a product new to them within 90 days. The fifth chooses which products to offer. Each is graded on customers it has never seen, against the rule it replaced, and the report records it when a model loses.
- One decision per customer. Across more than 1.5 million orders and about 0.6 million customers, each customer gets exactly one campaign with a calibrated probability, one to three products, and a plain sentence saying why. The highest score anywhere in the system is 0.598. Nothing reads 99 percent.
- The decision, delivered. Each one lands in the brand's Klaviyo account as an event: the customer, up to three products, one link that adds all of them to the cart. The brand's own flows render the email.
- Restraint, computed. Anyone who just bought, anyone emailed in the last 21 days and anyone the email platform cannot deliver to is left alone. On the first brand that is about three quarters of the people it could contact, every morning.
- Paced, not blasted. An even slice goes out every 20 minutes across the day. The hourly counts stay flat within about five percent, untuned.
- Five of every hundred, never emailed. Chosen by a hash of the customer id and held back permanently. What the emails added is a subtraction, not a story.
What the models do, in their own numbers
- Predictive reorder. On a recent night the reorder model scored a ROC-AUC of about 0.79 on more than 300,000 customers it had never seen. 32.5 percent of its top thousand picks bought within 60 days, against a base rate of 0.84 percent.
- Win-back. 5.37 percent of recently lapsed customers came back, against 2.10 percent of the long lapsed and 0.81 percent of the dormant. The three groups are treated differently because they behave differently.
- Cross-sell. 15.5 percent of the cross-sell model's top thousand bought a product new to them within 90 days. The simple co-purchase rule it was built to replace scored 20.9 percent. The model currently loses at the top of the list, and the engine reports that every night.
- Subscriptions. 56 percent of the incumbent retention tool's "upgrade to subscription" list were already active subscribers, because it could not see the subscription platform. Only about 23 percent were genuine candidates.
What it proves
A retention engine earns trust by what it refuses to claim. Unlapse retrains nightly, scores honestly, leaves most people alone, and keeps a group it never emails so every number can be checked. That is the product: not a promise of lift, but an instrument that measures it. The measurement design is written up in measuring marketing lift with a permanent control group.
Unlapse is available as a product.
Frequently asked questions
- How is the effect of the emails measured?
- Five percent of customers, chosen by a stable hash of the customer id, are never emailed by Unlapse. Every report compares the emailed group with them, with a confidence interval, on the brand's own orders. We do not promise a lift. We measure one.
- Why does nothing score 99 percent?
- Every campaign score is a calibrated probability of that action within about 90 days, checked on customers the model has never seen. The highest score anywhere in the system on a recent night was 0.598. Where the data runs out, the score stops rather than extrapolating.
- Does it send the emails?
- No. It sends the decision. Each one lands in the brand's own Klaviyo account as an event with the customer, up to three products, one link that adds them all to the cart, and the reason in a sentence. The brand's own flows render the email.