What happens
After an improvement card is shown, we follow it automatically.
- A card that appears three nights in a row becomes tracked, and the numbers at that time (numerator, denominator, what it’s compared against) are recorded as the values when flagged.
- Four weeks later, we re-measure the same numbers (the 28 days when flagged vs. the most recent 28 days).
- The result appears at the top of “Actions to grow revenue” on the right, as a single card.
The /cart drop-off flagged on August 5: cart → checkout pass-through went from 68% to 82%
Changes over this period include factors other than what you did.
Press “Confirm” and it goes away (the record is kept).
The four verdicts
| Verdict | Meaning |
|---|---|
| Improved | The flagged number got better by a margin that can’t be explained by chance |
| Unchanged | No difference appeared |
| Worse | The flagged number got worse by a margin that can’t be explained by chance. We don’t hide worsening either |
| Inconclusive | Not enough data to say whether there was a difference or not |
The wording always uses numbers, like “18% when flagged → 24% now (+6.0pt)”, and always adds “Changes over this period include factors other than what you did.” We never assert cause and effect — if there was a sale or an ad campaign during the same period, its effect is mixed in too.
The “Mark as done” button
“Mark as done” is at the bottom of a card’s Details ▾.
- Press it and it changes to “Verifying, N days left”, and the starting point of the follow-up resets to “the day you pressed it” (useful when you want to compare before and after your change).
- Press it again to undo.
- Follow-up runs even if you don’t press it. We deliberately didn’t design it so that only people who pressed it get results. There’s also no to-do list or task screen — so as not to add admin work.
Which cards are followed up
Follow-up applies to cards with an estimated recovery (CVR by device, speed, funnel drop-off, low-CVR products, cross-sells, and so on). Cards with only “scale” or “change” aren’t followed, since a follow-up would mean something different for them.
Used in anonymous statistics
Once a verdict is final, it’s copied into our statistics in a form that can’t identify the site or any individual (only the card type and the ratios when flagged and at the verdict). This lets us measure across many sites how much of the CVR gap was actually closed on sites that improved speed, for example, and eventually replace the recovery-rate assumption with a measured value. See Article 11 of the Privacy Policy for details.