The past few years have centered around AI hype, highlighting the promises of transformative innovation and significant returns. However, the narrative is shifting with increased investment into AI initiatives, because while speed and convenience are benefits of AI, they don’t replace quality outcomes.
Study after study points to the same three failure modes: 1. low-quality data, 2. models without the right context, and 3. end-users who simply don’t trust what they’re seeing. These aren’t technology problems. They’re fundamentals that can cause investments to lapse on the path to production.
Working with clients across industries, including highly regulated and consumer-facing sectors, we’ve identified three foundational layers central to the AI investments that will actually drive value. Each layer depends on the one beneath it. Here’s how the AI value stack works, and why each layer matters.
Data
There’s a recurring irony in AI development: despite decades of “garbage in, garbage out,” data quality remains the most neglected investment on the path to production. Two dynamics make this worse in the current era.
First, poor data is harder to detect with large language models because the output reads fluently regardless. A model that confidently produces wrong answers is more dangerous than one that visibly fails. Second, the FOMO-driven rush to deploy generative AI has led many organizations to assume that model intelligence compensates for data noise. It doesn’t.
Data quality isn’t glamorous. It wasn’t when Dr. Fei-Fei Li spent years building ImageNet’s image corpora, and it isn’t now. But consider the analogy: no one asks how attractive the bedrock beneath a building is. Similarly, data is the bedrock of every AI system. Without it, even the most expensive models produce outputs that are statistically probable but factually useless.
The cost of ignoring this is well-documented. Gartner reports that poor data quality costs organizations between $12.9M and $15M annually. A 2025 BARC survey of 421 global AI deployments found that data quality issues jumped from 19% to 44% year-over-year, making it the top obstacle to AI success. Forrester found that over 25% of data professionals report losing more than $5 million annually as a direct result. IBM estimates the total cost to U.S. businesses at $3.1 trillion per year.
The underlying problem is structural. Organizations have spent decades building governance frameworks around structured data. But most enterprise data is unstructured, including contracts, support tickets, emails, call recordings, clinical notes, and documents.
When that unstructured data is incomplete, inconsistently formatted, or stale, AI systems produce less accurate results and hallucinations. They generate plausible, confident wrong answers.
Ask any AI practitioner whether data quality matters, and they’ll say yes without hesitation. Look at what most organizations actually fund, and you’ll find a glaring contradiction.
Context
Once you have high-quality data, the next consideration is architectural. How does the AI system know what matters based on the use case, data, task, decision, risk and end-user?
This is where we enter the domain of context, and it’s more complex than it sounds. Traditional machine learning addressed this with feature engineering. Data scientists dove deep into the data, then identified which variables might matter, baking those factors into the model.
Modern LLM applications work differently. Errors stem from incomplete, irrelevant or poorly structured context windows. Context poisoning can occur when misleading or contradictory information contaminates the model’s reasoning, causing it to draw confident conclusions from flawed premises. Even though the output looks coherent, the conclusion can be wrong.
In response, context engineering has matured into its own discipline. Gartner predicts that by late 2026, 40% of enterprise applications will feature task-specific AI agents. These will require robust context pipelines to function reliably and deliver the correct outputs in the moment.
Effective contextual layering involves capabilities like semantic search, metadata enrichment, and user intent modeling. This allows us to design systems that understand what a user really means, to access the most relevant information, and to adapt accordingly to different situations.
Trust
Trust is both the outcome of a well-built AI system and a prerequisite for adoption. You can have high-quality data and precisely engineered context, but still fail if the end user doesn’t trust what they’re seeing.
Regulatory requirements, consumer expectations, and industry best practices are converging on a single principle: Trust must be designed in, not bolted on. Trust is becoming a technical requirement that must be specified, measured, and maintained. This means moving away from black-box AI to explainable systems.
A trustworthy AI system is understandable, controllable, accountable, grounded, and improvable. In practice, this translates into concrete mechanisms:
- Confidence scores that communicate model certainty to end users;
- SHAP and LIME for feature attribution and local explainability;
- Source attribution that grounds outputs in traceable, auditable evidence.
Organizations doing this well are building feedback loops so that systems improve when users push back, adding audit trails so accountability is traceable, and building interfaces that make model reasoning legible to non-technical stakeholders.
The Path to AI Value is Sequential
To extract long-term value from AI, businesses can’t afford to take shortcuts. Trust can’t be built without context, and context cannot be engineered without clean data. Organizations that skip or underfund earlier layers will keep running into the same ceiling. They will have impressive-seeming demos that don’t produce desired outcomes.
The next competitive advantages in AI won’t belong to the organizations with the largest models or the biggest compute budgets. It will belong to the organizations with the cleanest data, the most precisely engineered context, and the most transparent trust protocols that make AI systems both usable and worth using.
Special Thanks to ODSC
This article was originally published in collaboration with the ODSC Community. We’re grateful to the Open Data Science Conference (OSDC) team for supporting and amplifying conversations across the data science and AI community.
The ODSC Community is made up of passionate data science professionals and contributors from across the industry, helping share insights, perspectives, and practical expertise with the broader AI and analytics ecosystem.
You can explore more from the ODSC Community here: ODSC Community
.png)
.png)