Sign In

Don’t Fall for the Correlation Trap

1–2 minutes

Correlation doesn’t always mean causation. Most data analysts (including the author) have, at some point, mistakenly linked the two. Strongly related data compounded by misplaced intuition and excessive extrapolation often leads to this fallacy, which reflects a deeper analytics issue that can result in unnecessary effort and incorrect outcomes.

Typical Problem Setup

A customer seeks help from an analyst to improve a specific aspect of their business. Each is an expert in their own domain, not the other’s. This difference should not be seen as an area of contention but as an opportunity for collaboration.

How it always starts. Healthcare provider is an expert in their field and needs help to increase their sales. Data analyst is an expert in their field and can help increase the healthcare provider's sales.

Deductive Approach Starts with Analyzing Data

The analyst examines the data, finds strong correlations, and assumes causation. The customer then confirms whether the causation is real, and if this step is skipped it will leave the incorrect assumption built into the analysis.

Deductive approach works in two steps. First, data expert identifies causation using their understanding of the data. Then, the subject expert verifies causation using their understanding of the problem. This approach wastes resources and may lead to erroneous results.

Wrong Approach for Identifying Causation

Using deduction may lead to excess work and in some cases can lead to disastrous results.

Data expert identifies causation which is the wrong approach. They find that data from diverse businesses show a clear positive link (correlation) between discounts and sales thus deducing that discounts cause more sales and recommend using discounts as an option to increase sales.
Subject expert then verifies causation suggested by data expert and clarifies that their customers are not price driven and that discounts may actually decrease trust and lead to lower sales. Thus, they will reject using discounts as an option to increase sales.

Leverage Diverse Expertise

Both the customer and analyst bring expertise in their own fields, making them essential to solving the problem when effectively engaged.

Difference in expertise must be acknowledged and properly utilized. Data expertise can measure correlation but cannot identify causation. Subject expertise can identify causation but cannot measure correlation.

Inductive Approach Starts with Subject Expertise

Customers, as subject matter experts, are best positioned to identify actual causation. Analysts, with their data expertise, are best suited to measuring correlation. By working together, they can quickly arrive at an accurate and effective solution.

Inductive approach starts with subject expert identifying causation based on their understanding of the problem. Data expert then measures correlation based on their understanding of the data. This approach is correct and makes proper use of resources and generates correct results.

Correct Approach for Identifying Causation

Using induction will lead to reduced work and the best possible results.

Subject expert identifies causation. Healthcare provider knows that capability matters most to their customers and that certifications strengthen trust and can lead to (cause) increased sales. Thus, they recommend adding certifications as an option to increase sales.
Data expert measures correlation. They will use data to identify precise correlation between certifications and sales in similar businesses. Analyze and calculate then assign the proper rate between certifications and sales.

The Correct Approach is Vital

The underlying issue is that data alone rarely reveals true causation. This is why an inductive approach is essential, where human intelligence identifies causation based on actual subject expertise and then data intelligence measures correlation based on analytics expertise. Advanata strikes the right balance between human insight and data analysis, speeding up the analysis process and delivering the best possible results.

Sign up to get the latest articles and product updates.

Ready to see Advanata in action? Explore more resources, read related articles, and access the full application at advanata.com.

Want a personalized walkthrough? Contact info@advanata.com to get application access and schedule a demo.

Leave a comment