Photo by Johan Mouchet on Unsplash

AI: Shining a Light on Dark Data

Is Dark Data something I need to be scared of?

Demonstration of Extracting Dark Data from Video and Audio

What is dark data?

This dark data is highly unstructured. This means that most data is captured by structured means. The user interface demands that the data is in a certain structure; humans structure the data in the right format as demanded by the User Interface (UI). Whereas structured data can be categorized, sorted, indexed etc. and hence be made useful; unstructured data on the other hand is deemed as dark and hard to impossible to derive value from.

Traditional data capturing

In the 80’s many organizations had scores of people capturing data from documents into their mainframe. Subsequently organizations started to digitize their environments, and entering or capturing data happened as a matter of course and part of the workflow. The armies of data capture teams were deemed redundant.

Enter Unstructured Data

Video should soon represent up to 90% of all consumer internet traffic; customers are having conversations with companies via email, voice, chatbots and the like. Just think of all the sensors, social media activity etc. generating data which is highly unstructured.

Examples of Interpreting Dark Data

Here are some examples of how AI and ML can be used to search video and audio.

Video One: Video’s being analyzed
Demo: IBM Watson Visual Recognition — Tornado Path Detection
Video 3: Tesla Detector
IBM Watson Visual Recognition: Identify Cities from Space
Analysis of Sections of a larger Image



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Cobus Greyling

Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; NLP/NLU/LLM, Chat/Voicebots, CCAI.