Why Invest in a Knowledge Graph? Your Digital Transformation and Enterprise AI Initiatives Depend on It

The scope and success for digital transformations and advanced enterprise solutions is driven by a few factors, namely, defined use cases, available data, and people and SMEs. For many organizations, the biggest hurdle is knowing where to start. Starting with defined problems for which the organization has the most valuable data has demonstrated to be the most successful path toward scale and adoption for such solution-based initiatives. 

This has been supported by an interesting trend that I have been seeing in digital transformations and enterprise artificial intelligence (AI) implementations recently — knowledge management is becoming one of the fastest-growing areas of AI spending. Why? Because just 21% of firms are able to adopt AI at scale and the primary challenge is due to the lack of clear strategies for sourcing the knowledge that AI requires from data. 

Our experience at EK also supports this trend. If we look at where our clients are in their AI journey, we are seeing that the organizations that have made the most progress in digitizing their core business processes and institutional knowledge are also on the leading edge of AI adoption. The image below provides a high-level maturity spectrum and where our clients are on the path toward enterprise AI.Percent of clients in each stage of the AI Journey. Pre-AI = 9%; Foundations = 30%; Prototyping = 50%; Enterprise AI = 11%

If you look closely at the findings, what’s interesting is that more organizations have gone past the foundational stage and are conducting pilots or prototypes. These organizations are finding that this is a quick way to show value, create visibility, and reach go-no-go decisions faster. 

Still, we have yet to see many organizations moving to the enterprise AI scale. Out-of-the-box solutions such as IBM Watson Discovery (recently discontinued) have not been enough to get organizations to the unique collective organizational knowledge they need in order to understand and combine the volumes of data that they have to be able to discover, infer, and recommend information. 

A knowledge graph, a machine-readable model for applying business context (semantics) and capturing relationships between your data and content, allows organizations to create a machine-readable version of your collective institutional knowledge. A knowledge graph uses an abstraction layer that aggregates data that is heavily dependent on context, connectivity and relationships – in the way people think about and describe data – even if it is siloed and in diverse formats.

Three Ways a Knowledge Graph Helps Address Digital Transformation and AI Challenges

Applying organizational context and knowledge [in systems]

Deep learning and machine learning practices have been providing significant value in advancing organizational capabilities and allowing organizations to get started with the exploration of big data analysis to figure out what is in their data that spans multiple years and legacy systems. Yet, data teams are starting to recognize challenges with that approach and are getting stalled after some exploration. These challenges include: 

  • Spending a good chunk of their time preparing and modeling information that typically requires expertise in information architecture principles and knowledge management;
  • Requiring additional domain expertise to help them understand or organize how to label data based on business needs; 
  • Not having a clear answer to fundamental questions such as “what problem are we solving?”, “What is considered good training data?”, and “Are we creating the right algorithms?”; and
  • Introducing rework and mistakes in analysis of algorithms, which not only results in bad errors but also creates a lack of trust for their solutions and can ultimately lead to abandonment of AI initiatives.

The best way to make data easier to organize and manage is to enhance it with rich, descriptive metadata. Knowledge graphs leverage taxonomy and ontology to provide the most intuitive description for your organization and enforce a standard vocabulary and user experience to more easily and efficiently structure your content. A knowledge graph helps achieve this by: Normalizing content across disparate systems and processes; Enforcing consistent tags for content; Mapping relationships between disparate content; Optimizing search experience with synonyms; Streamlining discovery with faceting; and Providing the foundational structure for advanced capabilities like natural language processing (NLP) and machine learning (ML) and organizing and integrating structured and unstructured data

Providing the missing link across siloed and diverse data and content 

Over the last decade, a sizable number of organizations have invested in data lakes and warehouses with the goal to integrate and access data at scale. Many organizations are realizing that moving data into one physical location doesn’t really provide answers as to how data is connected or related. Advanced analytics and enterprise AI need to understand the relationship between diverse and siloed data to make any meaningful decisions. This means that enterprise solutions need to go above and beyond simple search and physical integrations of data to allow for natural language answers and specific recommendations. Imagine searching for “who worked on this project with Company XYZ?” and getting a running list of links or irrelevant documents. This is unhelpful and does not address the context behind the question.

To answer these natural language questions with a specific response (without relying on personal networks), organizations need a way to understand and connect their relational data, such as people or customer data usually stored in customer relationship management systems (CRMs), directories, or HR systems, with semi-structured data like project data typically tracked in project management applications, and with unstructured data such as reports in PDF or video form usually stored in digital asset management systems (DAMs). 

How a knowledge graph helps address these AI challenges: Graphs allow for aggregation of information from multiple disparate solutions, linking information that exists in multiple locations and formats using context, relationships, and metadata. This shifts the focus of AI solutions from the data itself to an abstraction layer - the metadata and other meaningful context that people use to find, understand, and interact with their content across systems. While metadata and taxonomy allow us to standardize the descriptions and labels for our data, ontologies and knowledge graphs give us the ability to connect them. As such, a knowledge graph serves as a semantic data fabric layer and works best as a natural integration framework for enabling interoperability of organizational information assets. A knowledge graph: Allows organizations to store data with context (i.e., metadata and data are stored together); Connects structured data, products, and services, with unstructured content like marketing and videos; Is built on semantic web standards that help organizations avoid proprietary solutions and freely move from current system to future systems; and Benefits from open data and caningest large datasets coming from various sources to augment institutional knowledge with industry insights.

Trust in and predictability of the solution

Advanced data and knowledge management solutions are successful when used in areas of the business that create the most value and solve real business problems. However, scalability challenges persist since organizations need to be able to concretely understand and explain how their advanced AI or data solutions arrived at a given recommendation or decision. Over 90% of our clients face challenges with explainability. Errors from advanced solutions are compounded as organizations work with large volumes of data, and so the ability for users to explain or at least predict the path remains a key component. What this means is that in order for any advanced solution or machine to deliver value out of data or information, it first needs to have the context, knowledge, and understanding of how a person would describe, process, or interpret information. 

While the main solutions here are engaging domain experts, using vetted training data, and creating a repeatable way to validate results, organizations need the capabilities to influence machine decisions and transparency through knowledge representations and machine-readable, domain-based explanations.

How a knowledge graph helps address these AI challenges: Knowledge graphs allow organizations to encode context, expose connections and relationships, and support reasoning and inference, thus serving as logic representation layers for advanced AI, data analytics, and machine learning solutions. Knowledge graphs achieve this by: Encoding semantics behind data for text analytics and natural language understanding; Serving as a better model to capture hierarchical relationships and providing logic for deciphering neural networks; and Encoding knowledge as context to data and structuring ML models in an interpretable way.

Closing 

Organizations are embracing digital transformation and advancing technological capabilities to stay relevant and competitive. Companies with good scaling practices spend half their digital transformation and enterprise AI budgets planning for integrations, and adoptions. A successful advanced data or knowledge management solution is one the organization understands and can support. This means that it is not enough for advanced digital solutions to be cool and exciting for the data or technical teams only; rather, they need to be compelling to the business. Organizations need to design systems that employees can appropriately understand, manage, and trust. A knowledge graph or a semantic approach is one proven way that allows us to get to a working solution. Check out our diverse set of case studies to learn more about how our clients are adopting graph solutions to solve real world problems. Contact us if you are looking to get started and are looking to advance your digital capabilities.

Lulit Tesfaye Lulit Tesfaye is a Partner and Division Director for Data and Information Management strategy, design, and implementation. Lulit is driven by innovation in efficiency, user-centered design, and empowering organizations to smartly leverage their data and information assets along with the best today’s technologies have to offer. Lulit focuses on employing advanced capabilities such as Artificial Intelligence (AI) to ultimately deliver value. More from Lulit Tesfaye »