What does ‘Big Data’ mean to you? For consumers, it could mean the data retailers collect that tracks individual purchasing patterns. For Millennials, it may be the pictures they post to Instagram, statuses they post to Facebook, and hashtags they post to Twitter. For enterprises, big data could mean the trillions of emails, files, and IMs generated on a daily basis in order to conduct business. The term ‘Big Data’ was originally coined to describe the volume, variety, and velocity of data creation, but as a side-effect, we’re increasingly facing the reality of its subsequent accumulation and storage. These consumer statistics, Facebook and Twitter posts, corporate emails and messages, and countless other types of electronically stored information (ESI) are piling up somewhere, and at an astonishingly faster rate than you might expect. In 2013, 50% more data was produced than in 2012, which is 400% more than in 2010. To make it relatable, the current amount of data in the world could fill more than 57.5 billion 32GB iPads. In 2015, that amount of data will grow to fill over 246 billion 32GB iPads… and as of October 2013, Apple had only sold 170 million iPads worldwide (IDC inforgraphic here). For the most part, consumers and Millennials don’t worry much about securing, managing, searching, and discovering this data… even if (in theory) they should. Enterprises, however, have immediate business motives that make these tasks a priority.
I recently spoke with a company that is starting fresh – they are comprised of few thousand employees, have been in business for over thirty years, and are just now deciding to get their act together from a data management standpoint. They are hiring a CTO/CDO (the D = Data) to spearhead the change and are allowing her free range to bring the company into the 21st century. We collaborated to outline a plan of action for the company, which they will be executing over the next several months.
The CTO/CDO’s first step will be to determine a retention schedule for both physical and electronic documents. After that, she plans on dedicating a few weeks to combing through the current ESI to determine what documents fall under which retention schedule and what documents can be immediately purged. Lastly, she plans on implementing a solution that can capture all newly created ESI (including emails, IMs, files, records, corporate social media, etc.) and apply retention periods, conduct end-user searches, address electronic discovery requests, and adhere to compliance regulations. It is a large project and admittedly will be time-consuming and tedious, but she is excited and eager to see it through.
The biggest excuses I hear from enterprises are those of denial – “we have too much data to even begin to manage it,” or bad-timing – “we don’t think right now is the best time for us to address our data,” or even worse, unaccountability – “this is someone else’s problem, go talk to them.” What the enterprise must realize is that there are a handful of reasons they can conjure up to avoid facing the ‘Big Data’ management issue. The longer you wait to address this topic, the more difficult it will be. Just ask the CTO/CDO that has to deal with 30 years of unmanaged data. Large organizations must face reality and initiate projects focused on managing data. It needs to happen eventually. If not now, then when?