KMWorld webinar: Building a KM Foundation for AI

Welcome to today’s KMWorld webinar, brought to you by M Files and Market Logic Software. I’m Marydee Ojala, editor in chief of KM World magazine. I will be the moderator for today’s broadcast. Our presentation today is titled building a KM foundation for AI. Now before we get started, I want to explain how you can be a part of this broadcast. There will be a question and answer session, so if you have a question during the presentation, just type it into the question box provided and click on the submit button. We will try to get to as many questions as possible, But if your question has not been selected during the show, you will receive an email response to it within a few days. And now to introduce our speakers for today, Frank Taliano, senior director, product management, M Files. Joseph Rini, director, product management, Market Logic Software. And now let me pass the event over to Frank Taliano, senior director, product management, M Files. Go ahead, Frank. Alright. Thank you, Marydee. And, hello, everyone. Thanks for joining. Thanks for listening today. As Marydee said, my name is Frank Tagliano. I am the senior director of product management at M Files. At M Files, we specialize in in knowledge work automation solutions and and products that leverage AI to help people, find, use, and and collaborate on information. So I’m really happy to be here today to help share some insights into how, how you can maximize the value that you’re getting from your AI solutions. If we if I wanna take a look and and sum up some of the key points that I’m going to talk about today, first thing I’ll mention is that there’s that generative AI holds a lot of tangible benefits, that that can be realized. But to realize those benefits, you need a solid information management foundation. In a lot of conversations I’ve had with CIOs, early adopters of generative AI, a lot of them have had to temporarily scale back their deployments because they realize, that they first need to get their information management structure right before they can move on to to their AI strategy. But the good news with that, is that with the right information management system in place, a solution that aligns to your business strategy, there is a clear path to unlocking the potential of generative AI for knowledge management use cases. So that’s what I’m gonna focus on today. We’ve all heard the promises of AI. I I don’t think I need to build on that hype. There are countless opportunities for for improving on, and adding automations, assistive user experiences, and really creating the the smart knowledge management solutions we’ve been thinking about for for a long time. It’s already started to change the way we work and interact with technology, and and it’s gonna continue to revolutionize how we work with and and create information. But while there is a lot of promise to AI, there are also some challenges to it as well. And as I mentioned, a lot of organizations are are struggling to get a return on their AI investments because they’re grappling with information chaos. And information chaos can manifest itself in a lot of ways, data silos, duplicate data, version control issues, lack of data classification. And this sort of disorganized information landscape, poses a real challenge to the to the effectiveness of how well generative AI solutions can work. When you when you try to apply AI to this chaotic information management environment, very often you’re going to see inaccurate or unreliable results. So, really, the quality of the output you get is heavily dependent on the quality and organization, of the data that you put in to the solution. So today, what I wanna focus on, is not so much the first two tenants of of the promise that I talked about, the the promise of AI and some of the challenges, But I wanna focus on the building blocks that that are needed to overcome those challenges, the building blocks that that can help you enable effective use of of AI and knowledge management use cases. And there are two main pieces of that foundation that I’m gonna talk about today, curation and context. So if we start with curation, you know, if if we think as a as someone if I’m a user or someone performing a task, I need access to the most current and relevant information to work with, especially if I wanna produce high quality output. The same is true of AI. So, a lot of people think of curation as as just organizing and maintaining documents in a way that that makes them easily accessible, which is true, but it’s more than that. It’s it’s also about ensuring that the most valuable, the most relevant, the most current information is what gets surfaced and is used for any given topic or task. Curation could also mean that that granular access controls are in place, which ensures that users and apps, can only see and can only access and process data that they’re authorized to. So if we if we take a look at a at an example here, an AI example, if you’re if you’re looking at a single document, using generative AI to search for information is is pretty straightforward. Concurrent solutions, they’re they’re really good at at interpreting natural language questions, using that to then look into the document and give an answer. Just like in this example, I’m looking at a document. It’s a contract, and I wanna know what the SLA penalty on a delayed response is. So it’s it could be quite successful at that. But, if I wanted to ask that same question about a set of documents, or from my entire repository or from my entire knowledge base, So maybe I I don’t know what document has the answer that I’m looking for, or maybe the answer that I’m looking for is spread across multiple documents. That suddenly makes the task quite a bit more difficult. So if I ask this question, sometimes I’ll get the right answer. But if your content isn’t well maintained and organized, you may often get answers that that just aren’t accurate or relevant. So so if you have, for example, multiple versions of the same document, that can cause noise and confusion, especially if those multiple versions have conflicting information. So how does the AI know which of the documents has the right answer? Or, if you have outdated documents, documents that that should be archived or purged, that also adds to the confusion. So information in those documents might be out of date. And worst case scenario, you know, if access controls aren’t utilized properly, it could be giving you answers and information from a document that you shouldn’t even have access to. So this is what makes curation so important. So curation is is really all about making sure that we aren’t confusing the AI with unnecessary noise And to provide, the best possible answers, the best possible insights, you need to have a curated collection of information, available for the generative AI application to query. To use the example I used earlier, you know, if we as workers can’t find or are unsure about what the latest version of a particular document is, the latest version of a of a proposal or an agreement, then what chance does the generative AI have to determine what’s the latest version? So as you as you evaluate, how well your information is curated, there are some questions that that you could be asking. So things like, do we make copies of of the document, or, do we know what version is official, or do we archive old data? So all of those questions relate to to data minimization strategies. So making sure that that only recent relevant content is surfaced. Other questions are, you know, are you automating permissions? Do you use workflow automation? Are your processes standardized? All of these are associated with automating governance and processes to ensure that information is secure and compliant. Or are you using templates? Are you automating naming and tagging standards? Are your documents connected to your business data? So what what standard standards are you using? And some of those are related to ensuring that your information is managed in context. So that’s something I’ll I’ll talk a lot about in a few minutes as well. So the good news is that when you’re thinking about AI, and the best practices for challenging for addressing those challenges, that you might uncover as you’re asking those questions, The the solutions, that solve those challenges are very similar to the practices that we’ve we’ve needed to solve the same challenges for people who are looking for information. So you might already be on that path or journey to implementing these solutions. So curation is is really all about establishing a a clear governance framework for organizing your content and information, for ensuring that it’s relevant, it’s accurate, it’s up to date. It could be achieved, by implementing versioning and archiving policies. It’s about using collaboration and management tools that don’t require you to make copies and versions while you’re working. All of that helps reduce the noise. It’s about implementing access controls to ensure that people only see the information that they’re supposed to. And through all of this, the more that you can automate these processes and policies, the more effective and consistent they will be and help create that well curated curated, content. And the benefits of of having that well curated content, are tangible. So going back to the needs for well curated content for effective AI, having that will ensure that the information that’s retrieved and provided by the AI is accurate and up to date, which is going to minimize the chances of of the model generating incorrect or misleading answers. It’ll filter out the inclusion of of sensitive or restricted data, so data that might pose legal or reputational risks if you’re exposed to it. It’ll get more consistent and accurate responses to your users, which will enhance the user experience. It’ll build trust with the users. They’ll start using the AI more. And then lastly, the the removal of of that irrelevant or low quality data will ensure that time and money isn’t wasted processing unnecessary information. So people aren’t wasting time. You’re not wasting time, paying for LLMs, to process and and provide false answers. So if we if we think back to the two of those challenges that I talked about in the beginning, curation is is really the key to overcome the first two of those. So information security, and garbage in, garbage out. The, the second foundational element I wanna talk about is context, and context is the key to understanding. So when I ask a question or I make a request, how can I be sure that the AI solution understands my meaning or intent? Does it have the information that it needs to frame the answer correctly? And context is how we ensure that the AI does. So RAG, or retrieval augmented solutions are are the most probably the most common AI solution for, getting insights or knowledge out of a set of documents and information. And and if we took a look at how a a typical RAG solution works, the user asks a question, they make a request, the solution will look into its sources of information. Those sources could be structured data, or more typically when those sources are documents or emails or other unstructured information, it’ll be looking into semantic index or or vector database. So somewhere it can find chunks of relevant information, among all of your content. So then it’ll it’ll use, those those retrieved chunks of information to formulate a prompt, and then ultimately a response back to the user. And so so RAG solutions are great, for when you have a large volumes of content and you wanna ask questions, you wanna get insights, you wanna get nuggets of knowledge, out of that content. But there can also be, some limitations to their effectiveness as well. So if if the solution, doesn’t understand the nuances or the basis of the question, then it might not be using relevant information to formulate a response, and then that could lead to responses that aren’t really aligned with what the user is asking for. So without context, the system will give you an answer, but it may not be quite the right answer that you’re looking for. So if we go back to that example we used earlier, again, there are a lot of challenges related to curation, but there are also challenges if the solution doesn’t understand the context of the question. And so if I ask that question, what is the, you know, the SLA penalty for delayed response? What if it doesn’t know what customer or project I’m asking about? What if I have multiple projects for that customer with different SLAs? Does it even know what I mean by SLA? Does that acronym mean something different in in different industries? So it’s that context that that it needs to be able to provide good answers. One of the ways that that you could overcome this lack of context is to enrich your documents and content with metadata. And so metadata, of course, helps explain what a document is. The type of document, the customer project that it’s for, the asset that it’s related to. I don’t know what its current status is, who created it. So so metadata is what makes documents more than just files. It gives documents meaning and structure, and it makes them easier to manage and process. And and it and it goes well beyond just simple organization. Metadata really is a powerful tool that can contextualize information to enable better document management processes, and and enable better tools in ways that, other systems that might be based on folders or or file location can’t achieve. So that metadata and the context it brings, it could be used, to do things like automate permissions or automate workflows or classification or retention policies. And in the same way, by leveraging metadata, RAG solutions could also become more contextually aware, and that improves the relevance and coherence of their responses. So it helps ensure that the system is not only working with the right information, but it also understands and communicates those responses effectively. The other thing to understand, I I I should point out when it comes to metadata, When you’re when you’re adding and working with metadata, you aren’t just adding business context to a single document. This layer of information that you’re adding is also building relationships between entities, between documents and people, projects, customers, assets between business objects. So, essentially, what you’re building, with that layer of metadata and those relationships is is a sort of knowledge graph of your business. If you think about a knowledge graph, it’s it’s really just a a structured representation of information that that links, concepts and entities together. And in this case, what we’re building is a structured representation of your business. And this graph becomes becomes really important, because it allows the solution to not just use, the metadata that’s directly referenced in the documents that it’s looking at, but it also allows the solution to dig deeper into the all all of the information that those items are related to. So if I’m, I don’t know, asking a question about a customer, the solution will also know all of the projects that we’re working on for that customer. So it knows to look into documents that are tagged to those projects as well, or it might know who the customer account manager is, and it and it’ll look in information related to that person to help formulate the best possible responses for for that user. So now if we, you know, go back to that RAG implementation we showed earlier, what we’re adding to this diagram is a layer of business context to our unstructured documents and content. And this is going to ensure that the solution has a better understanding of of intent, what what the user means, and context. You know, what is the context of the task that I’m doing? And and it’ll also ensure that it’s pulling from the most relevant sources within that graph of information, which will then significantly improve the quality of the response that we get. Oops. So, again, if if we think about context, that’s really the second vital foundational element for AI systems. It it’s again, it’s what helps the system understand intent. You know, if I’m asking a question, am I asking a question about apple the company or I’m asking about apple the fruit? That’s what the context gives gives the AI. It it provides the information that’s needed to to generate responses that are that are tailored to to the work that I’m doing, to the individual, to the task, and it makes interactions much more meaningful and effective. And in the end, it ensures that that the generated answers or content is grounded in the right information, which provides better answers. The taking kind of back full circle, and the great thing, about AI is that you can also use it to automate the building of that context, the knowledge graph, so to automate the enriching of your documents with metadata. So machine learning models or generative AI, can be used to automatically classify documents to extract relevant information from documents or or to generate metadata based on on predefined prompts. So it doesn’t necessarily have to be information that’s in the document, but information that it derives from the information in the document. So being able to automate that with AI means you could add this needed contextual layer without adding extra work or burden on your users. So implementing, an AI assisted metadata based content management solution like m files, could be a really great tool for maintaining a well organized context driven and up to date information repository. It provides that foundation that we need to ensure that AI solutions, whether those solutions are in M Files or there in other applications, are bringing the most value to your users. So just to to as a kind of final recap on the points we started, again, a lot of tangible benefits that could be realized with AI. But as I mentioned, to realize those benefits, you you do need a solid information management foundation. Two of those core building blocks of that foundation are curation and context. But the good news with the right information management system in place that aligns to your business strategy, there is a clear path to unlocking that potential. But I would suggest because of those potential challenges, it is important to to evaluate your readiness, your alignment as you start or continue on the on the journey to to working with an AI, to getting real value from AI. A lot of good frameworks out there for that evaluation at at m files. One of the things that we developed was the the knowledge work automation capability maturity model. Bit of a mouthful. But it’s a it’s a really nice tool, that we found that that’s very useful when we’re working with customers to ensure that, the entire organization is aligned. It helps them get an understanding of where they are on that journey, and then the next steps that they need to take. So but with that, again, that’s my part today. And, thank you guys for listening, and I will pass it back to Mary d. Well, thank you, Frank. And now let’s pass the event over to Joseph Rene, director of product management, MarketLogic Software. Go ahead, Joe. Thank you, Mary d, and thanks, Frank, very much for that, great talk. Hi, everybody. Joe Rene, director of product management at MarketLogic. And, I’m gonna follow-up today with my own kind of take on building the knowledge management foundation for AI. And what I thought I’d do because actually, I mean, I had the benefit of seeing, Frank’s slides ahead of time was I thought I would consciously really not try to, you know, repeat many of the ideas, around what was in the slides. Upon now seeing the talk as well, there’s a lot of in there that I would agree with, especially around, something that stuck out with what he said around, the idea that the challenges that one finds, kind of in the pre Gen AI world still apply now around creation, around permissions, around context, and so on. So what I’m gonna do is take a little bit more of wouldn’t call it a case study, but certainly learnings and so on that are grounded in what we’re doing in market logic, and I’ll I’ll take you through that as we go. So, key points today. I mean, the first cup the first two, I would say, are maybe not super surprising to anyone. Geni AI capabilities will fundamentally are fundamentally changing, enterprise knowledge management. But maybe the flip side or the specific component of that that I’ll try to drive home today is there’s specific enterprise functions, within the the enterprise, that require differentiated knowledge management and then corresponding, Geni capabilities. I’ll cover the journey from, a knowledge management platform without Geni capabilities to the adoption of Geni capabilities there, and talk about the lessons we’ve learned in market logic and generalize them somewhat, in layering a Gen AI solution right on top of an existing knowledge management platform. And I think, I mean, the talk would be aimed both at, providers knowledge management platforms, but also, of course, you know, consumers, enterprise, users who are using these platforms and other people in the space who, you know, potentially consulting around them and so on. So very quickly, and I mean, it’s not meant to be a salesy thing at all. It’s just just sort of frame who we are, the problem we’re solving, and then linking that all to the this Gen AI talk. MarketLogic, we provide, an insights platform for the insights teams of some of the world’s largest, global customer facing companies. We’re a cloud based SaaS provider, so that’s an important thing to keep in mind as I go through here. And you can see some of the customers we work with. And the point is we work with a specific subset of of the total employees at those companies, the insights teams. And some people in this, space may not be so familiar with what they do. And typically, these are sort of the research powerhouses within, large organizations who are keeping tabs on market trends, on, competitors, on what’s going on, among all of their brands, across different geographies, and so on. And they’re tasked with really understanding a huge amount of knowledge, not very uniform in how it comes to them, mainly, in sort of text based documents, but they’re, you know, sourcing it from all sorts of places. And we provide the platform that lets them, centralize and then do their job of curating that and pushing it out within the business to, other stakeholders. So some kind of specifics of our offering that would then be relevant for the overall session. As I said, we have this insights management platform serving that specific subset, of the of the company’s user base and interfacing with the marketing teams, the research and development teams, business strategy teams, and so on. And sometimes those users will also be in our platform consuming the knowledge that’s largely being managed, if you will, by the insights teams. As I said, we’re a cloud based SaaS solution. And then I’ll get into it a little bit later, but some of the the specifics I’ve kind of already mentioned about what this marketing consumer insights content looks like. It’s certainly not the kind of internal, policy documents, you know, sales documents, or, human resources documents, and so on. It’s rather really, external content that’s either been purchased, procured, cocreated by our users, purchased from big research suppliers and so on. And they’re using that and updating that and so on to make decisions. And then they perform a number of specific tasks in the platform. Many of them are standard to knowledge management, but a lot of them are a lot specific to how they’re serving, the organization. And so very quickly, I’ll just go through these, but this is sort of the, let’s say, the value problem, at least one piece of it that we’re solving, right, this exponential explosion of of content and knowledge that our insights teams are meant to stay on top of and the ensuing, like, ability to just decide, of course, being, ever, ever, harder to to do and the ability to do that dropping. And we don’t don’t need to insights at all, but kind of part of rally prop is the idea that those organizations that I showed you in the the slide that we serve, who really enable these insights teams within their organization to help fuel decisions and so on, we would argue, do better. And I think that’s kind of understood in the space that we’re serving. So I mean, these slides are quite logical in that sense. And, I should say that regardless if we’re talking about our JNI piece or we’re talking about our existing knowledge repository piece that, that I mentioned we’ve had in the space for ten years, I would say, a lot of, what we’re doing still comes down to this kind of, trying to solve this decision that needs to be made by business stakeholders or the insights users themselves around, you know, choices to run ads, develop new products, etcetera. And in order to make those decisions, we let them comb through, their content, in order to do so. Of course, we have many of the aspects, of a traditional knowledge management platform, that Frank spoke about. Things like, of course, taxonomies around all of the content, robust permissions, around the content in the system. We have a search experience letting them access the content and so on. And today, I’m really gonna focus on how we’ve moved that over or added the Gen AI Gen AI piece to that to really help them make that decision, based on all of that rich content which they have. And that content, just to give you an idea of what it looks like, is all sorts of, ad hoc research of reports, recurring trackers. These reports will be laden with infographics, along with text and so on. There’s also visual data coming through, sometimes raw transcripts as opposed to, like, more polished reports and then all sorts of new sources and so on coming through. And our users need to be able to parse that and and make, and use it to serve the business. And just some of the key aspects now of that content that are very specific to our space, but also you can imagine now, we’re a RAG offering upon hearing RAG, Frank explain, RAG already is really, suited and all of the technologies behind the latest sort of Gen AI, offerings are suited to some of this. So first of all, I’ve mentioned a lot of the content is packed with, you know, visually complex charts and graphics and so on. I got a couple examples of what that may look like. And our one key challenge that our user base would face is they don’t have much control over what that content looks like, unlike, you know, maybe an internal uniform policy document or report. They’re getting this from all sorts of different providers out there, and it’s coming in all sorts of shapes and sizes. So the tool needs to be robust to being able to comb through all of that content, quickly, and I’ll show in in our, explanation of the rack piece how we are able to accomplish that. We’re also dealing sometimes with structured and quantitative data, as I’ve mentioned, kind of survey results, transcripts, and then maybe not typical to our not, specific to our space, but certainly something that’s key for us, probably many other spaces in the knowledge management, overall ecosystem insights big ecosystem of third party providers of, tools, reports, dashboards, and so on that need to be linked into the platform. Of course, with permissions around those as well with taxonomies and metadata tagged to those, as well. And then, of course, governance and compliance requirements around all that content. And I’ve thrown this up here just to sort of frame, what I’m now gonna talk about with the RAG diagram and show you how it is able to really solve, a lot of the problems in this space for us. But this is something that we’ll hear from, a lot of, users, end users, of course, also buyers of the platform and so on. And it’s a it’s a logical enough and fair enough question. Why should we be using your kind of proprietary RAG, your Gen AI piece, when there’s, you know, Copilot, ChestCPT for enterprise? Glean. It’s not on here, but, you know, perplexity and and so on and so on as the space continues to change. And these are good questions. And, you know, we have a rationale for why it makes sense to use a tool like ours or, you know, a tool like m files in that particular context. And I think I’ll talk about it towards the end, but that’s one takeaway in in the, you know, the two year journey here is the sort of enablement around rationalizing and explaining what we’re actually doing, to the end users and to the, sort of purchase of the tools is paramount. So let me come to our RAG diagram. I won’t spend too much time. Frank did a good job of, taking us through, the way a RAG would work. We don’t layer a, knowledge graph component on top of our content, because we’re not heavily focusing on like the users who are publishing the content and those type of interrelations between them. We are really focused on vetting, the market research reports that are in our repository. And another piece to what Frank talked about, versioning and so on, we have the benefit, that content that ends up in our system is already the kind of, let’s say, final report, the final draft, the final take of whatever study, it typically is. We do have a workflow tool that also produces the content, but that’s not published to the repository here. So we’ve in a sense, we’re addressing that upstream in coaching our users how to get the content into the system, or integrating into the system in its final form, which means that our rag flow can then really focus on, identifying the most important documents in this huge, subset of content, that’s available. So, of course, you know, as Frank talked about, we are, you know, vectorizing, embedding the content that we’re holding and putting it into a vector database, of course, and then vectorizing user’s question when they come to the query and running a vector search against that. So we’re bringing back the most relevant chunks and so on. Frank covered a lot of this and, and we’re doing something quite similar. What we’re sort of innovating on, what’s proprietary to us now, is a whole bunch of pieces we’ve added around really ensuring that the evidence chunks from these, market research reports and so on to get through, are of the highest quality. So we’ve got a bunch of, re ranking components in here. We also use a large language model to reflect upon each evidence chunk in relation to the user’s question, in relation to some specific context, around, the company and so on that we’ve got, you know, in the prompt in the back end. So again, to what Frank spoke about with context. So we’re able to really identify the most relevant chunks of content before we pass them through, to the model to actually generate the answer. And, what that looks like in the end is a, sort of offering that, again, is specific to the insights space. We call this AI for insights. What we do is we, of course, you know, we generate a synthesized answer, I think, standard enough for the rack space. We also let users tailor the sources to go into the answer because, they themselves are experts insights space. They’ll often know what content, ultimately might be in there and maybe remove and add sources accordingly. And we also do something maybe quite innovative that this group be interested in. We, use a large language model to reflect upon the answer itself. So if you see here, we actually pass the answer. We pass the evidence back to a large language model. We say, hey. Point out issues in this data. Other contradictions between multiple studies being sourced here. You know, is any of the content outdated? Any other considerations there? So because we are not trying to source a single document, for instance, or the single canonical answer to policy an internal policy question, but we’re, you know, intrinsically dealing with these, multiple, positions potentially on a business question that’s out there. We really take an effort, to to flag sort of the the the nuances there and act as a sort of colleague to the insights user or the business user as accessing the tool, leveraging the power of the large language model to, provide them the answer. And then finally, I mentioned it earlier, but we’ve sort of, and I think there’s a strong value prop that maybe could be taken away for from a lot of the people, here is we’ve leveraged models that are able to look at the infographics, the, images, and so on within these documents. As I’ve mentioned, a lot of the data packed into this content is not text form, so it’s important to, one, be able to, in the first pay place, read it and then vectorize it. Right? So it’s in the first place available to the search experience, but then surface it and wrap it into the answer and source it appropriately, and so on and so on. So all those, aspects of a traditional text based rag, but taking them into some of this multimodal, if you will, space. So that’s what I wanted to touch about the Gen AI RAG piece that we’ve, launched on top of our offering. We’re also adding some Gen AI pieces. I think this is, Frank Frank also spoke about it in the extraction of, for instance, metadata. Or plugging stuff into search to start to provide Gen AI provided summaries of search content, more traditional search experience, and so on. And there’s, you know, a lot of, different areas in knowledge management that Gen AI can be can be leveraged there. Where we see the space going, and I think this is really relevant for, for a topic like this in terms of prepping the knowledge management, center, platform for the, the Gen AI transition, is certainly to the adviser, the, AI agent space, and maybe analogous to the initial move from knowledge management to RAG. I think maybe from RAG to advisers, we’re gonna see a a sort of step change in terms of what we’re asking Gen AI to now do. We’re gonna take it from beyond finding evidence and summarizing it to asking it to say, okay. Now take that evidence and do, business tasks with it, or go ahead and, you know, write a report or ping somebody and let them know or monitor that, incoming knowledge as it’s coming and flag it as it comes through. So becoming much more proactive as opposed to reactive. And what I want to drive home with this agent piece is where we see it going is much more, if you will take the analogy towards the, sort of chained or, agents on rails side of the spectrum. So especially in the enterprise space, it’s looking much more likely, and that’s what we’re betting that, the type of agents that will have high impact, that will be reliable, that won’t hallucinate, etcetera, will be those that are really within an agentic framework, where the large language models are sort of stitched together in an actual program, as opposed to just the large language model itself, doing all the reflecting, going off and making decisions and so on. And I think that importantly links back to a lot of, what has happened so far in the Rackspace, as you think about moving into the kind of next step, of the the Jenny I space. So kind of key learnings that we’re seeing in the field here. First of all, I would say knowledge management, consumers, users, customers, they’re still learning what Jenny I can do for them. So, I mean, in the past two years, we’ve we’ve seen, initial absolute kind of like euphoria around these tools, which quickly turned into, caution about really embracing them. The enterprise dynamics is in is in flux. Well, we’ve seen the establishment of a lot of Gen AI committees and other sorts of teams and policies in the enterprise around what Gen AI teams, what what Gen AI tools can be purchased, thinking about things like building rather than buying, is back, I would say, in a way that’s, quite new in the in maybe in the past decade. So a lot of, the customers we’re talking to, would are considering or actually are testing out building their own Geni pieces, even stripping out their whole knowledge management, enterprise knowledge management, not just, you know, our platform and considering building it in house. And that’s where interoperability across data, across platforms, and across use cases is key. We have, you know, API offerings that can make the RAG output available in other platforms internally or externally. And I think when you’re considering, knowledge management platforms with Gen AI pieces, it’s important to look at the degree to which they might be interoperable with other tools in your in your company, and so on. And just generally, like, the the pace of change is, extreme at this point. And we are seeing just a constant demand for innovation, from our customers and for enablement. And I think it’s, talks like this, but also being able to educate the, the end consumers in some way about what’s going on in this space, what the offering’s doing, keeping them, grounded in what’s possible today versus in six months. That’s all really key stuff that that we’re seeing as as super important. Great. And with that, I’ll turn it back over to you, Maritine. Okay. Well, thank you so much, Joe. And now let’s turn to questions from our viewers. And a reminder, if you do have a question now that you’ve heard both of the presentations, do put it into the question box provided. Click on the submit button, and we will try to get to as many as possible. But let me start with this. I think, Frank in particular, some of your comments certainly resonated with our viewers. So we have this question, from someone who sounds a bit frustrated. Our files are unorganized and spread throughout shared folders. What is a good first step to cleaning that up, ensuring our content is curated well enough? Yeah. Not a not an uncommon question and not not something we, and something we we hear a lot. I I think the, probably the the best first step you can you can take there is, is is establishing a a data governance policy or a set of policies, defining things like like data retention. How how long do you do you need to keep these documents? Or ensuring that that you have sort of a way to to organize that content. You know, I couldn’t stress enough that that organizing it using using metadata is so much more effective than, say, folder structures. But, however, you may need to do that and and certainly access controls. So so defining those those governance processes, that’s that’s really the first step. Second thing would be then to figure out how to, you know, how to build the automations to make that happen because it’s you could do it you could do it manually once, but if you if you do it through automations, then it’s something that that stays governed. And and so that’s that’s it. But but, really, it’s really about defining that. Yep. Joe, is there anything you wanna add to that? I mean, yeah, I think I would echo what Frank just mentioned about metadata sort of trumping folder structures and so on. I guess, we have the advantage as as I kinda mentioned there that we’re a SaaS platform. So the content we’re ending up, like, we’re getting from the, companies, we can then tag either with them or with our auto tagging capabilities with GenAI and ensure that it’s, tagged up in a certain way. Mhmm. However, getting it from them in the first place, we, of course, do see challenges in terms of where all the content’s sitting and so on that, I mean, difficult to to address as Frank said. Sure. Sure. We have another question here. Maybe this I’ll throw to you, Joe. How are your customers defining and evaluating success in their decision to go with an AI enabled offering? Yeah. So we have a couple different, I would say, kind of prototypical cases. If if we’re if it’s an existing customer of ours who’s considering, going to the Gen AI piece, then there’s, certain methodologies that we can provide them and and ways of doing comparisons versus, like, the traditional search experience, against, for instance, the, you know, the Gen AI kind of answer summary piece and really try to demonstrate tangible benefits and time savings, in finding content and so on. And we we can guide them with that. They also, you know, are quite good at coming with, frameworks and so on. And then, when it comes to new customers or customers who are piloting and so on, often they’ll come with much more of a structured evaluation framework on their side, maybe from procurement or from someone in the in the larger team who’s, who’s involved in the purchasing decision. And, you know, they’re they’re often evaluating, again, the time savings. And one key point to mention is something kind of intangible, not so measurable, that’s on the the plates of either of the cases I just described is really around just, like, accuracy, perceived hallucination, the answers, and so on. And, I think that’s kind of an outsized or maybe correctly weighted Yeah. Extremely important thing, at least in our space. So that’s a big one that’s, kind of a a red flag for companies if they’re seeing that. Yeah. Yep. It’s sort of an add on question that that that just came in. How do you or your customers quantify the value of investment in knowledge management enabled AI? Which one of you would like to answer that one? I mean I think Joseph actually started kind of answering that one too on the on the Yeah. Sure. So, I mean, I kinda got it the way we do. We we have some, some good key, case studies out there that we have, you know, customers who’ve really run a structured, sort of quantification of time saving time, so let’s say the cost of, employees and so on, or also trying to look at the degree to which, these assets that we’re holding. So remember, we’re holding, you know, very expensive set of market research streams of the tens, hundred millions worth of content in our platform, the extent to which those are being, leveraged, found, used in decisions, and so on. So, yeah, various attempts to try and map that. It’s, of course, difficult to follow decisions made outside of our platform once they actually take place in the business and so on, and that’s always a challenge. Maybe Jenny and I could help there, but I think that’s still enough that needs to be cracked, I would say. Yeah. Very true. Okay. Let’s see. Here’s someone who wants to know what your thoughts are on AI chatbots. How could chatbots fit into the, environment that that we’re describing here, this whole issue of building the Kilometers Foundation for AI? Where do the where do the chatbots fit in? Frank? Yeah. I mean, there’s a lot of ways to approach that. It’s, at its core, a a chatbot is is is provides a a natural language interface for for the users, which I think is extremely valuable. And I think one of the one of the great benefits that that we’re starting to see and will see is is how it affects, you know, the way we interact with technology. So, yeah, that’s that’s something that we do. So we we built a Yeah. A chatbot into m files and and provide that interface so you can communicate with it in natural language and ask questions, ask follow-up questions, and and kind of dig into into those answers. So I I think that from a user interface perspective, I think that I think that plays a huge role. We’re also looking at I mean, there are lots and lots and lots and lots of chatbots out there. Every everybody’s kinda building them, and then some do, you know, various things. And it it also goes back to, you know, being able to understand which, which chatbots are good for particular use cases and potentially, you know, integrating with some of those as well. So that’s that’s something we’re also looking into. Yeah. Maybe just just to add that we we made a kind of initial conscious choice to structure our Geni output in a in a single one shot answer, you know, as you saw me showing there. So not with a chat experience. And I think some of that was, around, taking care to avoid, for instance, hallucination, ensuring that what’s happening in the chat doesn’t get, like, passed back up into the context and sort of spit back out as, sort of truth as though it’s, sourced in the the knowledge base. However, I would say that as Frank said there, like, the just the ease with which users use these chatbots and just the degree to which they’re now permeating throughout the both personal and professional space, like, ChatChippy Tea and so on. Yeah. Yeah. We’re now looking at ways to to introduce chatbots, but, use them in more of a controlled way to, like, launch applications and so on within the platform, but not necessarily generate the knowledge, if you know what I mean. So there there’s definitely considerations in introducing, but definitely a plus for user experience. Yeah. Yeah. Frank, let me throw this one to you. Is the lack of context one of the major shortcomings for AI when providing recommendations or conclusions? Well, to get to get good recommendations, you need a you you definitely need a a a type of context, and so you need to to understand, you know, who the user is, what their role is, what they’re asking, the job they’re doing, the task they do. You you need to have the context about about the the person or or system you’re you’re giving the recommendation to. So absolutely. One of the I think one of the worst things you you see a lot of recommendation systems out there. I question a lot of how how useful some of them are. I think the when when you see when you see some of those, what what they’re really missing is is a lack of understanding of who the person is. They’re they’re they’re probably too generic. So, it definitely context definitely plays a role in in getting good recommendations, for sure. Yeah. I think this sort of ties into the example of the SLA, which might mean something different perhaps in a company where SLA was an acronym for a product. Yep. Exactly. Let’s see. Joe, can you talk about some of the challenges that you’re seeing around positioning specific Gen a offerings within the larger enterprise Kilometers ecosystem, including other Gen AI offerings of which there seem to be many all the time emerging. Yeah. Exactly. So, yes, first of all, the space is, you know, in constant flux, and I I kinda tried to look to it there in those final points around the builder by piece. At least what we’re seeing, I think it’s common enough because we’re being introduced to the Gen AI committee of a big, multinational. Right? And they’re telling us that they’re also talking to many other providers in many other sub, teams within the company is, you know, from our perspective, both a challenge and an opportunity to establish ourself within their larger ecosystem. But there’s certainly an overarching push to, one, try to do things in in house. I think an aspect of it is just components of this Gen AI world with large language models being available via API calls out to, you know, all of the big tech providers means that the the temptation to build in house is stronger than it maybe was ten years ago with other type of offerings. So that that’s certainly a challenge we’re seeing, and that then that comes down to, you know, how do you differentiate out which what your RAG is doing versus twenty other providers? What specific steps are you taking that serve your use case? Right? And getting those across to the user, not just from a sales perspective, really driving home and and letting the user experience what the difference is. Those are all, I would say, challenges, but also opportunities and just illustrative of how much action there is in the space right now. Yeah. I think that that your point about build or buy was was actually pretty interesting because we we haven’t really focused on that all that much until fairly recently. Right. Frank, is there anything you wanna add to that? No. I think you covered it quite clearly. So Okay. Well, we do have a question here that specifically says it’s for Frank. Now wait a minute. Let me find it in this list here. Alright. Well, this question is for Frank. Okay. What is your opinion of the use of SharePoint as the system of record for storing and publishing content to end users? We’re in the process of improving our curation to set us up for AI use, but not sure if we should consider a different system for storage and publication? It’s a good question. People have had, you know, varying degrees of of success with SharePoint. We, we work we do work closely with Microsoft and Microsoft three sixty five as well. And so we made a we made a pretty cautious decision that that we know people are gonna have information in SharePoint and OneDrive and other places. And so what what we do is, we have a a connector framework. So we connect to that information, and and use M Files to add a layer of governance, to that. So rather than kind of force everybody to to migrate all of their information into M Files, we we could still add a add the level of governance that adds the kind of context and and gives you better ability to curate content even if you keep it keep it in place in SharePoint. So there’s there’s ways to do it with with SharePoint, you you know, using m files alongside it or or other ways. So it’s possible, but it’s it’s it’s more challenging than I would say it’s more challenging than using a a solution that was purpose built, with a a metadata infrastructure that that that’s made for that. So Mhmm. Mhmm. Mhmm. And we have this question. Presumably, you are using OpenAI or Google Gemini LLM. I guess lots of customers would get worried in sharing all their data with these LLMs. Once the LLMs have the data, they are probably learning from the customer data, and it’s probably expensive. Can you comment now? This one was not directed to either one of you. So let me start with Joe on this one, and then we’ll switch over to Frank. Yep. So Why are you using OpenAI at Gemini? I guess is where we should start with this one. Oh, first, right off the bat, Azure, OpenAI, and Gemini. So the Kira was, very good in understanding with the big API based LN providers that we would like to be using are. Right. So that’s an excellent point. I was remiss to not mention that in the previous question speaking about, like, integration. So, I mean, I don’t know how many times per week I would be on such a call just on one rep for this call, talking through our rack diagram and explaining where we’re doing the API calls out, which LLM models we’re using, and describing the guarantees we have, you know, in the terms and conditions from Azure, and that they won’t train on on models. Right? So a lot of it comes down to one, doing a deep dive into that rack diagram and really explaining what’s going on there. To Mhmm. The terms and conditions being kind of locked up and and and, you know, all above the board. And but then ultimately, even with all of that in place, he’s right that that is still a challenge that I would say absolutely is the only reason that certain customers, like, haven’t gone to the JNI solution that we would offer them. It’s it’s purely based on that. So that’s certainly, still a challenge out there, and there’s only so much you can do. And it’s also about a matter of time as insights more accepted and so on, I would say. But yeah. Frank? Yeah. Actually, very similar. So we we also mainly use Azure OpenAI services. We we don’t, also assurances. We do we don’t train the the model on on customer data. We typically too we we host, the Azure AI service in the same data centers. We host our SaaS application. So in in many cases, the data never actually leaves, the data center. So we we do take security quite seriously. We we did we did purposely build, our solutions to be somewhat, LLM agnostic. So if if, you know, if we get into certain domains where there are better language models and or fine tune models, we we could switch to to use those. So not not tied into the Azure OpenAI models, but but, yeah, but, yeah, definitely not training the model with customer data. Okay. That’s great point. I’ve got one quick question here for you, Frank. Can M Files automatically tag all of my documents with metadata even if they are in SharePoint or network folders? Yeah. Actually, that goes back to the the the answer I had before a little bit, when we were talking about SharePoint. So that’s that’s one of the things that that we we do is we we we call it IML. It’s our it’s our integration, intelligent metadata layer framework. But it’s it it allows us to connect to other repositories, like network folders to, like, SharePoint. Not necessarily move them physically move them out of there, but just add the governance needed to add a a layer of metadata around them to give them context, to use that metadata to drive things like, permissions and workflow and and retention schedules and things like that. So, it’s it’s it’s governance governance without migration is is kinda what how we describe it. Yep. Okay. Very good. Very good. Well, that is all the time we have for questions today, and we apologize we were unable to get to all of your questions. But as I stated earlier, all questions will be answered via email. And I would like to thank our speakers today, Frank Taliano, senior director product management, Emphiles, Joseph Rene, director product management, MarketLogic software. If you would like to review this event or send it to a colleague, please use the same URL that you used for today’s event. It will be archived for ninety days. Plus, you will receive an email with the URL to view the webinar once the archive is posted. If you would like a PDF of the deck, go to the handout section once the archive is live. Thank Thank you again for joining us.