This post was originally published here on In Context Magazine.
The term dark data reminds me of an old Star Trek episode, in which the Starship Enterprise travels to the far reaches of the galaxy to investigate what the crew believes to be a lightning storm in space.
They search the darkness, unsure whether a natural phenomenon or an evil mastermind lies ahead. What they find was not what they expected—sort of like the dark data within the enterprise.
Gartner defines dark data as “the information assets organizations collect, process, and store during regular business activities, but generally fail to use for other purposes. Similar to dark matter in physics, dark data often comprises most organizations’ universe of information assets. Thus, organizations often retain dark data for compliance purposes only. Storing and securing data typically incur more expense (and sometimes greater risk) than value.”
Core systems within the enterprise capture a lot of information that may or may not be useful in the future. Rather than delete it, organizations often elect to store it just to be on the safe side. After all, electronic storage is relatively cheap, while data management and governance is expensive. This is structured dark data, found deep within the ERP system.
While a company’s core systems generate the log files that contain structured dark data, business processes generate additional dark data, within documents, videos, emails, etc., that fuel your day to day business. This dark data exists in enterprise content management (ECM) systems, document repositories and even physical filing cabinets.
The ability to access the right dark data can unlock value. For example, with transactional documents such as invoices or utility bills, you key in or capture only the vital information that’s required to process the record and store the image. But what about all the other information to manage process compliance (i.e. is a vendor consistently including PO number and other line item details?), compliance to established discount terms, even to provide ad hoc spend analytics? That is what remains in the dark. There is inherent value in the information but the enterprise is often limited by cost and complexity from finding the information needed to drive new value and insight.
Most Capture solutions are only able to pull content from documents based on templates but Perceptive Intelligent Capture sorts incoming documents based on their content which eliminates the concern over the rise in unstructured, unmanaged content becoming a liability in enterprise systems. Then, it intelligently lifts specific fields and line-item details based on context, then validates and seamlessly passes information to all core business applications. It only needs to learn a small sample set of documents to process high volumes of widely diverse layouts without missing a beat in all different languages and currencies.
Some of the information stored in Capture files could be considered digital clutter today. Bring this dark data into the light, and it could be the key to realizing inefficiencies and opportunities for improvement you never knew existed. Investing in the right Capture technology can ensure the information locked in dark data can be used to bring greater efficiency to your organization.
Resistance is futile.
Learn more about how to use capture to explore your dark data with useful whitepapers, videos and more on the Perceptive Software Intelligent Capture resource page.
