AI is reshaping the way federal agencies operate. From improving service delivery to enabling real-time decision-making, it is now central to mission success. However, a recent MIT study showed that 95% of AI projects fail to deliver any return on investment.
That’s because scaling AI across the agency is not just about deploying more algorithms. Success requires scalable infrastructure, strong governance frameworks, and the ability to adapt to an evolving landscape. As Sanjeev Pulapaka, principal solutions architect at AWS, put it: “Agencies are moving from experimentation to execution because the mission demands it.”
And at the center of that mission execution is data.
Why Federal AI Needs a New Data Approach
Agencies are rapidly shifting from small pilot projects to agency-wide AI deployments, and mission delivery is the main motivator behind the change. Whether it’s faster services, better decision-making, or more efficient operations, the demand for AI-driven outcomes is growing stronger every day.
Yet even as algorithms improve, data remains the biggest obstacle to widespread production. Federal IT environments are complex, with information spread across hybrid and federated systems—some in the cloud, some on-premises, and some at the “edge” (as close to the source as possible). Traditional archiving approaches that require copying and moving all data in a single repository are no longer practical. They are costly, slow, and often introduce unnecessary risk.
For federal leaders, the question is not just how to scale AI, but how to scale it in environments where data cannot easily be moved.
Governance in a Federated World
Data governance is the cornerstone of this challenge. Agencies must balance availability with control, ensuring that data can fuel AI systems without violating compliance, privacy, or access control requirements.
Pulapaka summarized the problem well: “You don’t have to move the data. You just need to know where it is, how to access it, and who controls it.”
This philosophy aligns with the reality agencies face: data must remain in place but still be discoverable and governed. By adopting a federated approach, agencies can share across silos while maintaining ownership and oversight. Managing data without breaking down silos is an essential capability in environments where collaboration is necessary, but control cannot be compromised.
However, governance cannot stop at structured systems. The real frontier for federal AI lies in the unstructured data that agencies create every day.
Unstructured Data: The New Frontier
While structured data has traditionally powered digital initiatives, the real opportunity for government AI lies in the massive volumes of unstructured data—emails, messages, documents, case files, transcripts, and more—that remain largely untapped.
Unstructured data, created by humans for humans, is both high-risk and high-reward. It carries the sentiment, intent, and context that AI models need to automate work, understand nuance, and deliver mission value—but only if it is governed effectively.
In-Place Data Management Explained
Wrangling intelligence and meeting governance requirements across petabytes of scattered files and messages is no small task. This is where in-place data management becomes a practical, scalable solution. Rather than copying or migrating vast amounts of information into new repositories, in-place management allows agencies to govern and act on unstructured data where it resides.
In-place works by extracting and indexing the “essence” of every document—metadata and content—without copying or moving the original file. High-value documents, such as contracts, can still be selectively archived, but the majority of data remains in its source location. From a single, unified platform, agencies can execute governance functions like:
- Records management: classification, retention, and defensible deletion.
- Policy enforcement: manual or automated tagging to align with regulatory requirements.
- Risk management: remediation of personally identifiable information (PII) and elimination of redundant, obsolete, or trivial (ROT) files.
- Flexibility: rapid reclassification and policy updates without having to re-index or restructure systems.
The benefits include reduced storage costs, lower legal and compliance risks, and unified management across data silos. In-place management provides agencies with the agility to respond to evolving mission demands without being weighed down by data migration.
Powering AI with Curated Data
By consolidating the unstructured data ecosystem into a “virtual” data lake, in-place management enables agencies to curate datasets for internal AI training. Instead of feeding unvetted or incomplete information into models, agencies can search, cull, and deliver only the most relevant and governed data.
This approach has several critical advantages:
- Improved accuracy: High-quality data reduces hallucinations and strengthens AI outputs.
- Risk reduction: Sensitive or regulated data can be excluded from AI training sets, protecting agencies from compliance or legal exposure.
- Tailored insights: Department-specific applications can be trained on curated datasets aligned to mission needs.
Such tailored applications allow agencies to surface institutional knowledge that may be scattered across documents, emails, and reports—knowledge that often lives only in the heads of a few subject-matter experts, or in some cases isn’t concentrated anywhere at all. By governing and curating this information for AI, agencies can make it broadly accessible, ensuring that trusted expertise is available on demand rather than dependent on a handful of individuals.
In-place ensures that federal AI systems are not just powerful, but trustworthy.
Building the AI Advantage
In-place data management offers a path forward by aligning with the federated environments agencies already operate in. It allows leaders to unlock the value of unstructured data without the cost and risk of typical centralization. It provides confidence that AI initiatives are fueled by relevant, accurate, and secure information.
As agencies move from experimentation to execution, they need solutions that make scaling practical. In-place governance delivers exactly that.
Ready to move your AI projects from pilots to agency-wide mission impact? See how in-place data management can help your agency scale AI responsibly.
