← Back to blog
Blog

How to tell whether a fix worked in Shopify Analytics — and why to check orders and sales first

Shopify Analytics shows far more than orders and sales. You can see sessions and where shoppers drop off on the way to checkout. But on September 21, 2026 Shopify changed how it counts sessions, so comparisons across that date are no longer like for like. Here is what Shopify measures reliably, what shifts, and why orders and sales are the right place to check whether a change worked.

6 min read
  • Reading the numbers

In an earlier post (Why your order count and conversion count never match), we explained that ecommerce numbers come from a visit layer and an order layer, and that both are right — they just count different things. This post applies that to Shopify.

Say you changed something on your Shopify store. New product photos, clearer shipping costs, an extra payment method. A few weeks later you open Analytics in the admin to see whether it worked. Which number you look at can change the answer.

The short version: check orders and sales first. Use conversion rate and sessions afterwards, to find where things changed. Here is why, starting with what Shopify Analytics can tell you.

What Shopify Analytics shows — how far can you see?

Shopify Analytics lets you look at both layers in one place. (Everything below is from Shopify’s official help pages as of November 2026.)

The order layer is orders, sales and products. These come straight from your store’s own records, so they are accurate.

The visit layer covers a lot too. The behavior reports include a conversion rate breakdown that shows each step — all sessions, sessions with cart additions, sessions that reached checkout, sessions that completed checkout — and you can compare date ranges. Sessions by device and by landing page are standard reports as well.

If you build your own report, you can go further and get conversion rate by referrer and device together. Many of the cuts in the five numbers to check each week that need a custom exploration in GA4 are available inside Shopify.

That is exactly why the next two points matter.

Comparing across September 21, 2026 — fewer shoppers, or a new way of counting?

From September 21, 2026, Shopify changed how it measures sessions. It classifies bot and system traffic more consistently and focuses the default view on human visitors.

Shopify says that after the change, sessions, conversion rate, add-to-cart rate, reached-checkout rate and similar metrics may look different, and that this doesn’t necessarily mean your traffic or customer behavior changed. It advises treating the date as a new baseline.

Meanwhile, orders, sales and customer counts are not affected.

This matters when you check a fix. Suppose you reworked your product pages in early September, and in October your conversion rate is higher. That gap could be your fix, or it could just be the new counting. The Analytics screen alone can’t tell you which.

Orders and sales use the same yardstick before and after September. So when your comparison crosses that date, start there.

Also, session-based metrics only go back to October 1, 2022 (noted in the acquisition reports help). For long look-backs, orders and sales reach further.

Orders vs sessions that completed checkout — which is the real count?

Shopify Analytics shows both your order count and the number of sessions that completed checkout. They won’t match.

Shopify’s help explains why: a customer can place more than one order in a single session. Orders are the order layer, sessions are the visit layer, and as the earlier post said, they count different things.

The visit layer also has limits on what it covers. According to the acquisition reports help, sessions are cookie-based and end after 30 minutes of inactivity, and they cover the Online Store sales channel only. Orders from a physical shop or entered by hand in the admin count as orders but never appear as sessions.

If you need one count, use orders. Use sessions that completed checkout as the top half of conversion rate, inside the visit layer.

When GA4 disagrees — which one do you trust?

If you also run GA4 on your Shopify store, you’ll notice that sales and order counts don’t match. Shopify lists the reasons in its help page on discrepancies:

  • GA4 only counts visitors with JavaScript and cookies enabled
  • Some shoppers use browser extensions that block GA4 from tracking them
  • Tools differ on whether they count bots as visitors
  • Reports may use different time zones
  • Each service simply tracks visitors in its own way

All of these come down to GA4 not seeing every order, or counting differently. Your source of truth for sales and order counts belongs in the Shopify admin. Use GA4 for comparing shares within the visit layer, like traffic sources or pages. When you put the two side by side, match the date range and time zone first.

Check the fix with orders and sales, then use rates to find why — what’s missing

Put together, Shopify’s numbers fall into two groups.

NumbersSeptember 21, 2026 changeUse it for
Order layerOrders, sales, AOVNot affectedChecking whether a fix worked
Visit layerSessions, conversion rate, step-by-step drop-offRe-baselinedFinding where things changed

When you make a change, write down the date and compare orders and sales over equal periods before and after. If they moved, then open the visit layer and see which step of the funnel changed. When you do, make sure both periods are after September 21.

There is still something missing. Orders and sales are accurate, but they don’t tell you where shoppers stopped buying. The visit layer that does tell you was just re-baselined, so for fixes made before it, you can’t line up before and after on the same yardstick. Without that, when sales move you can’t decide whether the thing you fixed did it, or something else moved at the same time.

For whatever you fix next, you can start a consistent record today. The five numbers to check each week has a simple table to fill in weekly. The cuts Shopify Analytics offers fit straight into it, so try filling it with your Shopify numbers.

Related articles