Why does a strong pattern in your data not prove a cause?
A strong pattern does not prove a cause because the same pattern can appear when a third factor drives both numbers, when the cause runs the other way, or when the match is a coincidence. A correlation measures how closely two things move together; it says nothing about why.
Take a fictional 30-store homeware retailer, Larkfield Home. The trading manager, Priya, is reviewing last quarter. Eight stores ran weekend cookware demonstrations and 22 did not. Her export looks like this: 'Demo stores: average weekly sales 41,200. Non-demo stores: 34,900. Difference: +18%. Note: demos chosen by area managers. Demos mostly Nov to Dec. Demo stores include 5 of our 6 largest sites.' An AI assistant summarising the sheet wrote: 'Cookware demonstrations drive an 18% uplift in weekly sales. Rolling them out to all stores should increase revenue.' The board wants to approve the rollout on Monday.
| Store group | Stores | Average weekly sales | Largest sites included | Months demos ran |
|---|---|---|---|---|
| Demo stores | 8 | 41,200 | 5 of 6 | Mostly November and December |
| Non-demo stores | 22 | 34,900 | 1 of 6 | None |
How do you test whether a correlation is causal?
Test a correlation by ruling out the three alternative explanations one at a time, then, where you can, run a fair comparison in which only the suspected cause differs. Each step below is a question you ask of the data before anyone spends money.
1. Look for a confounder
A confounder is a third factor that affects both things you are comparing. At Larkfield, store size and season are candidates: big stores sell more anyway, and November and December are busy months for cookware. Ask what else is different between the two groups.
2. Check the direction
Reverse causation means the effect is actually causing the supposed cause. If area managers gave demos to stores that were already trading well, rising sales led to demos, not the other way round. Ask how the groups were chosen.
3. Consider chance
With eight stores and one quarter, a gap can appear by luck, especially if someone looked at many promotions and reported the one that stood out. Ask how many comparisons were made and whether the pattern holds in another period.
4. Compare like with like
Compare demo stores with similar non-demo stores in the same weeks: large with large, small with small. If most of the gap disappears, the confounders were doing the work.
5. Run an experiment
The strongest test is a controlled trial: pick comparable stores, assign demos at random to half of them for the same weeks, and compare. Random assignment spreads size, location and trading history evenly, so a remaining difference is far more likely to be the demo.
What does a weak and an improved reading of the Larkfield data look like?
The weak reading takes the AI summary at face value: 'Demos increase sales by 18%, so run them everywhere.' It turns an observed gap between two groups that differ in size, season and selection into a promise about what will happen in 22 other stores.
The improved reading says: 'Demo stores sold 18% more per week, but they were mostly our largest sites and ran demos in our two busiest months, and area managers chose them. Comparing the five large demo stores with large non-demo stores in the same weeks, the gap is about 4%, which is within normal week-to-week variation. We cannot tell yet whether demos help. Proposal: a six-week trial in 12 comparable mid-sized stores, six chosen at random to run demos, measuring cookware sales and staff hours.' That version tells the board what is known, what is not and how to find out, at the cost of one month's delay.
How can AI summaries mislead you with correlations?
AI summaries mislead most often by describing a correlation in causal words such as 'drives', 'leads to', 'boosts' or 'because of', which makes a pattern sound like a proven effect. A language model writes fluent explanations of numbers; it does not know how the stores were chosen unless you tell it, and it rarely volunteers that it cannot tell cause from coincidence.
You can reduce this by giving the assistant the context it lacks (how groups were selected, the dates, the sample size) and asking it to separate what the data shows from what it might explain. Then read the output for causal verbs and replace each one with what was actually observed, for example 'stores that ran demos also had higher sales'.
How do you check a causal claim before you act on it?
Check a causal claim by asking how the groups were chosen, whether the cause came before the effect in both groups, whether the gap survives another period or a larger sample, and whether the summary says 'associated with' where the evidence is only a correlation. Write the answers next to the claim before anyone approves spending.
What are the common mistakes with correlation and causation?
The most common mistake is rolling out a change because a dashboard showed a gap, without asking how the groups were formed. Others: treating a high correlation coefficient as proof, reporting the one promotion out of twenty that looked good, forgetting seasonality, and assuming that no correlation means no effect. The opposite error also costs money: refusing to act until certainty is perfect, when a small randomised trial is usually cheap.
Where does this lesson sit in Tosheo.ai?
This lesson is part of the Data Science and Analytics category and appears early in the data science course, because every later topic, from dashboards to machine learning, depends on not mistaking a pattern for a cause. In Tosheo.ai the hands-on task gives you a fictional dataset and an AI-written summary, and the rubric feedback points to each causal claim the data cannot support. It is a professional foundation, not a statistics degree.
Sources and scope
Australian Bureau of Statistics: correlation and causation (checked 19 October 2026) ↗