Cat Ownership & Customer Feedback Programs: Correlation or Causation

Over the last four posts, we’ve covered dashboards that don’t decide, programs missing their doing wing, flying on a single dial, and feedback data with too few dots to trust. This week: the trap that catches teams even when they’ve done everything else right.

The cat-owner calling plan

Here’s a great story to convey my point.

Early in their career, one of my trusted advisors was working with a UK telecoms provider, They found something odd buried in its data: customers who owned cats made noticeably more phone calls than customers who didn’t. Statistically, the relationship was real. Someone in the room, only half joking, suggested a cat-owner calling plan.

Nobody built the calling plan, thankfully, because the correlation was real and the causation was almost certainly nonsense. Cats don’t make people phone more. But the kind of person who owns a cat may simply also be the kind of person who makes more calls, for reasons that have nothing to do with the cat at all. Two things moving together doesn’t mean one is driving the other. Both might be driven by a third thing neither of them mention.

This should be obvious and is, when the example is cats. It’s a lot less obvious when the correlation reinforces a story you already believe.

Why this trips up feedback programs

Feedback data is a magnet for spurious correlation because it’s so easy to slice. Segment by enough variables and you will find something that correlates with loyalty, with NPS, with anything you like. This may be purely by chance, or because both are downstream of a factor you haven’t considered. Customers who use a particular feature score higher on NPS. Great, roll it out to everyone? Maybe. Or maybe the customers who already loved you were the ones who bothered to explore the feature in the first place, and pushing it on everyone else changes nothing.

The commercial cost of mistaking correlation for causation isn’t abstract. Chase the wrong driver, and you invest budget, time, and change management effort into something that was never going to move the number, while the real cause sits untouched. Worse, you often can’t tell it didn’t work, because something else was moving in the background anyway, and you credit your fix for a change it didn’t cause.

Finding real business drivers, not flattering coincidences

Getting from correlation to something closer to causation isn’t difficult. A few disciplines help:

• Control for other factors. Before concluding a feature drives loyalty, check whether the customers using it also differ in tenure, size, sector, or usage intensity. If they do, the feature might just be a marker of an already-loyal customer, not the cause of their loyalty.
• Test with a change, not just an observation. Where you can, roll a change out to one group and hold another back. If the metric moves for the group that got the change and not for the group that didn’t, you’re standing on much firmer ground than a correlation ever gives you.
• Look for a plausible mechanism. Correlation without a story for why one thing would cause the other is a red flag. “Customers who received a proactive call after a service failure churned less” has an obvious mechanism. “Cat owners call more” does not.
• Use proper driver analysis. Statistical techniques exist precisely to separate the variables that move an outcome from the ones merely associated with it. They’re not infallible, but they’re a considerable step up from eyeballing a dashboard and spotting a pattern that feels right.

A driver of what, exactly?

None of this matters if you’re finding drivers of the wrong outcome. A huge amount of feedback analysis stops at finding drivers of NPS or CSAT, as if moving the score were the goal. It isn’t. The score is a proxy, and proxies can be gamed, misread, or moved for reasons that are unrelated to the business doing better.

The outcome that matters is profitable growth: retention, expansion, reduced cost to serve, advocacy that generates new revenue. Start the driver analysis there, not at the survey score. A driver of NPS that doesn’t touch any of those things may be interesting, but it isn’t a priority. A driver of NPS that also reduces churn and support costs is worth real investment. The second kind is rarer, more valuable, and much easier to find if you’re looking for it directly rather than assuming NPS is the outcome rather than the messenger.

Seek “and,” not “or” answers

This is where correlation-chasing does its quiet damage. Under pressure to move a score, teams often reach for actions that improve the metric without helping the business, or that help the business at the customer’s expense. They may nudge a number but make no impact on the bottom line.

The drivers worth chasing are the ones that satisfy “and,” not “or”: good for the customer and good for the company. Proactively fixing a fault before it causes three more support calls helps the customer and cuts cost to serve. Making onboarding genuinely simpler helps the customer and improves activation and retention. These are rarer to find than a spurious correlation, but they’re the only kind of driver worth building a program around, because they’re the only kind that drive the P&L.

Dave and I still think about that cat-owner chart story occasionally, because the instinct that produced it hasn’t gone away; it’s just got better tools. Modern feedback platforms can slice a dataset a thousand ways in seconds and surface every correlation, real or not, in a tidy dashboard. That’s a feature until it’s treated as a shortcut to real thinking. A machine that finds patterns faster than a human doesn’t make those patterns any more causal. It just makes it faster to build a confident-looking case for the wrong action.

Ask the cat-owner question

Next time a pattern in your feedback data looks like an answer, ask the cat-owner question. Is there a plausible reason one thing would cause the other, or have you just found two things that happen to move together? Would it survive a test, or only a glance at a dashboard? And critically, is it a driver of the score, or a driver of the business?

Correlation will always be easier to find than causation. It’s also nearly worthless on its own. Chase the driver, not the coincidence, and make sure whatever you find is good for the customer and the company, not one at the other’s expense.

PS, if you like spurious correlations, check out Tyler Viglen’s hilarious collection. My favorite is the almost perfect correlation between butter consumption and ticket prices at North American movie theaters. Let me know your favorite!

What’s next in this series

Chasing the right driver still assumes you know what decision you’re trying to inform before you start collecting data. Next week, in the final post of this series, we turn the telescope all the way around – starting with the action, not the survey.

Frequently Asked Questions

1. Why is correlation dangerous in customer feedback analysis?
   Feedback data is easy to slice in countless ways, which makes it easy to find correlations that are purely coincidental or driven by an unmeasured third factor. Acting on these spurious correlations wastes resources on changes that were never going to move the real outcome.
2. How can a company tell if a correlation in feedback data is a real driver?
   Control for other factors that might explain both variables, test the relationship with an actual change (not just an observation) where possible, look for a plausible mechanism connecting the two, and use proper statistical driver analysis rather than eyeballing a dashboard.
3. What should feedback programs optimize for instead of moving an NPS or CSAT score?
   Profitable growth outcomes like retention, expansion, reduced cost to serve, and advocacy-driven revenue. A driver that moves NPS but doesn’t affect any of these is a lower priority than one that moves NPS and measurably improves the business.