Despite Big Data scandals, the government often struggles to manage its own corpus of business records and data.
Any time the words “data” and “government” appear in the same sentence, NSA scandals and Edward Snowden mentions usually follow. That isn’t the purview of this particular post, however. Rather, what I hope to explore are some of the unique challenges facing our governmental agencies with respect to Big Data.
The government, of course, generates a lot of data from the course of daily business. Just like any large enterprise, a federal agency can regularly expect to be inundated with emails, files, and other forms of “unstructured data”, or human-to-human communication. Unlike a private organization, however, a federal agency can be called upon, at any moment, to make readily available a particular piece of data via the Freedom of Information Act (FOIA); FOIA gives Americans the right to access information held by the federal government.
As data has gone digital, federal agencies have lagged somewhat in keeping pace. This isn’t really their fault, though; the mandate of the government is to serve the people, rather than to maximize profit, and as such, some of these long-range internal arrangements can sometimes fall by the wayside. Perhaps for that reason, the government issued its "Managing Government Records Directive" on August 24, 2012. The primary requirements of this directive were that federal agencies should manage permanent electronic records, as well as both permanent and temporary email, in an electronic format. While this may sound rather rudimentary, it turns out that many agencies were simply printing this electronic information and retaining it in paper format. This is not a caricatured example of government inefficiency; governments are necessarily and intentionally designed to change slowly and deliberately, and digital information has thus far taken a decidedly more rapid trajectory.
That being said, there are surely unique challenges to transforming such an entrenched paradigm. Like the most heavily-regulated private industries, the federal government must be very careful about deleting anything, since doing so might compromise its obligations with respect to FOIA. In order to assist in its compliance with the Records Directive, and with the aforementioned FOIA obligations, the federal agency responsible for overseeing the government’s recordkeeping recently deployed ZL Technologies’ Unified Archive within its organization.
In our commitment to engaging in the evolving discussion over information management generally, and in the context of government agencies specifically, our CEO, Kon Leong, will be speaking at a couple of conferences in the coming week. The Digital Government Institute (DGI) E-Discovery, Records & Information Management Conference & Expo, took place on April 3rd in the Ronald Reagan Building in Washington, D.C. And this week, the Security Innovation Network IT Security Entrepreneurs Forum (ITSEF), where Mr. Leong will join luminaries such as the Deputy Secretary of the US Department of Homeland Security in discussing the data challenges facing the government, business, and the investment community.
As ZL Technologies grows its presence within the private and public sectors, we are constantly looking to engage with our audience to both share our expertise and learn as much as we can about the unique challenges facing today’s organizations with respect to information management.