Share this article
In one of a series of essays on the impact of AI on the insights profession, Martin Rückert identifies a key technology to efficiently find patterns and insights from exponential data volumes: discovery miners.
In my last blog on document miners, I described the way they mine relevant structured information within unstructured artefacts to bridge the gap between unstructured and structured data. Once this data is harmonized into a knowledge graph we call the Market Logic, the next logical step on the path to actionable insights is to mine for relevant patterns and insights from all the ingested reports.
Document miners vs. discovery miners
Document miners automatically extract findings from single reports, one after the other, and connect these to the Market Logic to generate a harmonized view. By contrast, discovery miners look at insights from a holistic business standpoint, by uncovering insights from all sources across the Market Logic.
How discovery miners answer a business question
For competitive intelligence, you may want to track a competitor’s price changes over time, or to identify relevant news items based on features (i.e. general similarity for a topic like “discounting”), certain types of insights (i.e. the relationship between specific entities like price and promotion), or to predict anomalies before they occur like a price and volume hike.
In all these business questions, you aren’t interested in the insights extracted from a single report, but rather all the insights you have, extracted from all of the reports and then ordered by one single dimension, which in this case is date/time. If your insights platform is not prepared to introduce a new dimension when it becomes relevant, then you cannot answer this question without changing the entire model and re-importing everything.
At Market Logic, we make sure you can always add additional dimensions at any time, and whenever your analytics questions require it. Our platform ensures that you don’t have to re-import data because the dimensions are flexible and can be easily changed.
Schema-on-read platforms
This ‘business requirement driven analysis paradigm’ is also known as schema-on-read, which allows for a rapid data ingestions process. By “rapid”, I mean you don’t have to plan or depend on the detailed usage of a specific artifact and its insights at ingestion time, but can dynamically adapt the use of this artifact when and how the business requirements demand it.
Technically speaking, schema-on-read platforms have been around for quite some time and their success has revolutionized the way we do business analytics today. From a business perspective, however, schema-on-read solutions are sold to IT services as a generic solution for any business line or function.
As a consequence, they lack the flexibility marketers need to get meaningful insights from the data – fast answers simply aren’t enough when the questions are changing.
Empowering IT to do what they do best
A dynamic analytics platform that is driven by business requirements enables enterprise IT to do what they do best – make the core technology available in a reliable and responsive way, without having to manage the process of business requirement changes. At the same time, business users themselves can do exactly that if they see the need for it.
It’s a dream come true for anyone who’s ever had to plan roles and responsibilities for IT Operations and LoB IT. The business benefits of defining your own questions on the fly is also a huge benefit.
From a cognitive computing standpoint, discovery miners also enable subsequent cognitive computing services with additional business values, such as: services that automate time-consuming tasks like analyzing all insights and then recommending what a proper reaction from business could be, or being able to interface many discovery processes with a conversational interface, where one can ask almost any analytical question and produce reports of higher complexity, like an entire situation analysis.