This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.
Dark data has for some time been the buzz word on the data block, dropped into conversations and delivered with a tone that puts it on a par with being related to the likes of Darth Vader or Bane. Yet many businesses are unsure of what dark data means and whether or not it’s cause for concern.
So what exactly is dark data? Put simply, it constitutes all of the unknown data within an organization. It is a subset of big data that makes up the largest volume of information collected by organizations each year. It’s untapped, unstructured and is normally found within data repositories having not been analysed, processed or cleaned.
According to a recent study carried out by Splunk, 55% of all data globally is unquantified, rising to 75% when it comes to businesses. Many either struggle to find the data in the first place, do not have the inclination to try to make sense of it or simply lack the resource and commitment to locate and quantify it.
A new Veritas study only furthers this issue. It found that half of company data is unclassified and businesses have limited or no visibility over vast volumes of potentially business-critical information, pointing to an increased risk of security breaches. Businesses need to safeguard against potential security breaches, so using and making sense of as much dark data as possible is the best course of action to achieve this.
However, there are several obstacles when it comes to overcoming dark data within businesses that are, fundamentally, team issues. The first and possibly the most problematic is volume of data. Although businesses have some understanding of the value that data can have, they are overwhelmed with the volume of it and do not have the resource to tackle it. As much as a simple solution would be to train more employees in how to analyse data, many businesses don’t have the budget or don’t want to expend their time on data analytics training.
With such a fast evolving, technologically-driven business world, this is overwhelming. It demonstrates how data strategy – and making sense of data – has to be regarded as an essential part of a business for it to have an effect at all. Businesses need to be educated on the value of all their data and the hidden insights that it can provide because, after all, the benefit of having the right data at the right time can be extremely advantageous.
In today’s data-rich landscape, any business that can extract the true value from their internal and external data to make informed decisions is at a huge advantage. For businesses that have an absence of employees trained in data – and to make the solution of tackling cyber security issues simpler – they shouldn’t be afraid to use machine learning and AI.
Data intelligence platforms can be used to help quickly clean large volumes of data from different sources and make sense of it, which is necessary to avoid security breaches. However, by employing these advanced techniques on dark data, businesses will also uncover hidden patterns and insights.
In fact, CBR suggests that between 61% and 67% of organizations see value in AI technologies which would enable them to look more closely at insights in operational efficiency, innovation, strategic decision-making, recruitment, and customer experience.
In the sectors we currently work with, including oil and gas, mining and utilities, our own data intelligence platform, is used to simplify and overcome a wide variety of issues, from improving supply chain management to identifying gas flaring and locating off-grid mines. The wide variety of data that AI can make sense of only goes to show how beneficial data intelligence platforms can be for businesses when it comes to creating competitive advantage for themselves.
As the volume of dark data within organizations increases, the potential to miss opportunities through untapped insights is growing. It is therefore imperative for organizations to be assured that, although dark data can often be associated with missed information and cyber-security threats, through overcoming obstacles and training employees to handle the data, it can produce valuable information which can be translated into viable business opportunities.
Furthermore, as businesses adopt a more open approach to disruptive data analytics and machine learning, they can open up the possibility of harnessing competitive advantage through information they didn’t even know they had, helping them to make faster, more efficient business decisions before the competition has even got out of bed. It’s time to turn the light on dark data so we can start seeing it as a friend rather than a foe.
First published 29th July on DataIQ.
