Enterprises are spending heavily on AI infrastructure, from GPU clusters to inference platforms and foundation model licenses. But for many organizations, AI performance is stalling well before compute becomes the limiting factor. The bottleneck is found earlier in the stack, and it has nothing to do with model capability. It lives in the data pipeline.
Unstructured data (such as emails, documents, file shares, collaboration logs, transcripts) makes up the vast majority of enterprise information. This data also carries the richest organizational context for AI to draw from. It is also the data type least suited to the pipeline architectures that many organizations have in place. The result is an infrastructure mismatch that quietly caps what AI can do, regardless of how much compute sits behind it.
Why the Pipeline Breaks Down
AI inference at the enterprise scale requires data pipelines that can deliver on three dimensions simultaneously: low latency, high throughput, and consistency. Unstructured data pipelines, built around passive storage rather than active delivery, often struggle on all three.
Ed Beauvais, Director of Product Management for AI and Cloud Data Infrastructure at Hewlett Packard Enterprise, put it plainly in a recent industry discussion: “To be able to really provide and process inference at scale, you really need to rethink how we’re doing data pipelines end-to-end. That’s not just the processing at the end, but how data is ingested and how it works with the broader ecosystem of the data pipeline.”
When pipelines introduce latency at ingest or between tiers, expensive compute resources sit idle. When throughput is inconsistent, inference workloads become unpredictable. Beauvais noted that one of the most consistent requests HPE hears from enterprise customers is simply: keep the GPUs busy.
The architecture problem compounds further when organizations try to source data from enterprise systems like Microsoft 365. Bulk exports through standard APIs are subject to throttling, rate limits that slow down how fast data can be pulled. For teams trying to feed AI pipelines continuously or at large scale, this overhead is a structural drag on performance.
Fragmentation Undermines Data Quality
Pipeline performance is only part of the problem. The other part is fragmentation.
The majority of enterprise data environments are not unified. Email lives in one system. Case files live in another. Documents are spread across file shares, cloud repositories, and collaboration platforms; each governed differently, each with its own access patterns. When AI systems draw from this environment, they draw from whatever data is most accessible, not necessarily what is most complete or most relevant.
Beauvais spoke on the risk directly: “When you think about the intelligence of data, if it’s in a silo and it’s a critical piece of business information that you’re not aware of, that’s risk to the business. You might be making decisions without all that data.”
The instinct to solve fragmentation through centralization — migrating everything into a data lake or central repository — runs into practical limits quickly. Large-scale migration is expensive, operationally disruptive, and in many cases prohibited by compliance, security, or sovereignty requirements. Data volumes continue to grow faster than migration projects can keep pace. Silos persist, and AI tools get layered on top of them.
What the Pipeline Actually Needs
Solving the unstructured data pipeline problem requires addressing both dimensions: pipeline performance and data completeness. That means moving past the idea that better storage or faster transfer speeds are sufficient answers.
What AI pipelines need from unstructured data infrastructure includes:
- Low-latency access to content without bulk exports
- Consistent, governed data rather than raw exports that include redundant, obsolete, or unclassified content
- Full content indexing alongside metadata: metadata alone provides structural context, but the actual content of documents is what enables precise, high-quality AI inputs
- A unified view across repositories, so AI systems can draw from the full scope of enterprise information
In-Place Management as the Pipeline Architecture
In-place data management addresses these requirements at the data architecture level. Rather than moving or copying files to a central location, in-place management extracts and indexes the “essence” of every document, including its metadata and full content, while the original files remain in their source systems. Email stays in the email platform. Documents stay in file shares or cloud repositories. What changes is the organization’s ability to access, govern, and act on all of it as a unified information environment.
From a pipeline perspective, this architecture solves several problems at once:
- It eliminates the need for bulk exports from source systems. Because content is already indexed, AI pipelines query the index rather than requesting files in bulk from M365, SharePoint, or other platforms, bypassing API throttling entirely.
- It reduces storage cost and token cost. Delivering the “essence” of a document to an AI pipeline requires a fraction of the storage and token overhead of passing full files.
- It enables continuous, low-latency data delivery. The essence is always available; there is no waiting for exports or transfer queues.
- It creates a governed data layer before content reaches the AI pipeline. Classification, sensitivity identification, and ROT remediation happen in-place, so AI systems receive clean, relevant content rather than unfiltered data sprawl.
Hybrid Environments and Unified Governance
The data challenge is further complicated by hybrid environments. Many enterprises now operate across cloud and on-premises systems simultaneously. Beauvais noted that customers want the flexibility to use cloud infrastructure and bring data back on-premises, and that governance, sovereignty, and compliance are consistently cited as the reasons.
In-place management is well suited to this environment. Because governance travels with the index rather than requiring data to be in a specific location, organizations can apply consistent classification, retention policies, and access controls across on-premises and cloud repositories from a single platform.
This unified governance layer addresses Beauvais’ assertion that “the key aspect of getting a massive ROI is having a platform that’s not just limited to a single use case.” Infrastructure that serves only one purpose forces organizations to maintain multiple overlapping platforms. An in-place architecture that governs data continuously and serves multiple use cases simultaneously compounds the return on investment.
The Foundation Comes First
Enterprise AI ambitions are growing, and the compute infrastructure to support them is increasingly available. What is less available, in most organizations, is a data pipeline that can sustain inference at scale: one that delivers complete, governed, low-latency access to the unstructured content AI systems depend on.
The organizations that move fastest on AI will be the ones that resolve this constraint first. Governing data where it lives, indexing its full content, and delivering it to AI pipelines without bulk migration or export throttling is the architecture that makes that possible.
