Sign In

How to Tackle Analytics Slop and Score Better Results

2 minutes

Quite a bit is being said about AI slop, but not enough about analytics slop. The problem is similar: outputs are abundant, impressive looking, yet with questionable value. This article takes a closer look at why this happens and what can be done about it.

It Starts at the Beginning

Many analytics projects are set up for failure from the start: vague problem definitions inevitably produce vague solutions. Precision is essential if analytics is to solve real business decision problems.

In a typical analytics problem setup the decision owner wants to use analytics to improve their coffee shop business while avoiding to define the precise problem to maximize project results. The analytics expert says they can provide all the support they need while avoiding to generate precise results to minimize project risk. The result are poor analytics that benefit no one and this is an outcome we want to avoid.

The Quantity Conundrum

Without a precise problem, analytics can become endless: forecasts, charts, and insights pile up without producing a solution or even a path to one. It is simply more content waiting for more analysis.

First issue is with data generation. The analytics expert generates interactive dashboards, regular insight reports, and automatically updated KPIs and states that the answer is within this output. In reality, they provide excessive data when information is required. The decision owner needs analytics solutions not the creation of analytics work. Data dumps, regardless of what they're called should not be requested, generated, or accepted.

The most efficient way to arrive at a precise solution is to start with the required outcome and work backward. This minimizes the work required while ensuring the solution is precise and actionable.

First solution is to request information. Use the inductive approach and provide the analytics expert with the exact actions that require analytics support. The problem is reversed and a precisely framed problem is created. Decision owner states their goals and options and requests forecasting their behavior. The analytics expert the provides supporting forecasts. Precise information is requested and generated.

Distant Forecasting Horizons

Forecasting far into the future to avoid verification may interest futurists but rarely helps businesses facing constantly changing conditions and short decision horizons.

Second issue is with excessive horizon. The analytics expert can declare that growth will fluctuate but is expected to reach 20% in the next decade. This horizon is excessive and is impossible to predict, verify, or act upon. Decision owner needs a forecast for the next quarter not the next decade. Excessive horizons regardless of the time period are a wasteful exercise.

Time frames must fit within operational constraints to produce the best possible, rather than perfect, actions to solve problems quickly and maximize business impact.

Second solution is useful timeframes. Using the data effectiveness methodology forecasts are for operational analytics that can be verified and refined. We create practical forecasting time frames. Decision owner needs the best possible actions for the upcoming quarter, and the analytics expert provides the supporting forecasts. The precise time period is requested and generated.

What Really Drives Results

Combining problem ignorance with obsessive data following can push analytics off course, turning business decision problems into open-ended research projects.

Third issue is misguided correlation. Analytics expert can say that data shows that using free trade coffee beans will increase revenue and that this action should be considered. This is just initial bias, data cherry picking, and analytic manipulation. Decision owner understands their business and knows that such an action is irrelevant to sales. Correlation doesn't mean causation and be weary of analytics drift.

Problem and subject experts are best equipped to identify causation, while careful problem specification keeps analytics focused on the problem at hand.

Third solution is to enforce causation. Using the business problem framework, causal actions are based on actual problem expertise to prevent causal detection through observation. Decision owner knows what actions affect their goals and asks for their behavior to be forecast. Analytics expert understands this assignment and provides the supporting forecasts. Don't allow analysts to drift into patter recognition.

Expertise Replaces Methodology

A CV padded with projects and the ability to produce mountains of content means little if it doesn’t add business value. Be wary of happy coincidences where analytics takes credit for improvements it did little to cause.

Forth issue is irrelevance. Analytics expert says that their output speaks for itself since their last project delivered higher sales. All claims of success need to be investigated thoroughly. Decision owner need to see the results and how they were used and in fact the analyst may have prevented a bigger increase in sales. Never take claims at face value and don't allow value research results

Analytics problems must be grounded in sound methodology, so results are relevant, measurable, and support continuous refinement.

Solution four is the scientific method which is used by Advanata throughout the analytics process to construct precise scientific problems. Decision owner provides the precise business problem and exactly where an analytics expert's expertise is needed. It is easy to take credit for improvements that would have happened anyway.

Making Analytics Matter

Analytics expertise is too often wasted producing needless output instead of solving actual problems. This will catch up with the profession if analytics cannot demonstrate tangible business value. Generative AI can already produce far more content than any analyst, and if the goal is to stockpile results, it is also a much cheaper option. Advanata puts analytics experts where they are needed most, embedding their expertise directly into the solution process to solve business decision problems and deliver measurable added value.

One quick poll question (3 seconds):
I’d really value your input.

Sign up for early access to articles, delivered every two weeks. Please check your spam folder for the confirmation email.

Ready to see Advanata in action? Explore more resources, read related articles, and access the full application at advanata.com.

Looking for a personalized walkthrough or decision intelligence consulting? Contact info@advanata.com to get application access, schedule a demo, or discuss your decision optimization challenges.

Leave a comment