Watch Market Logic’s 2024 IIEX Health Keynote

Good morning, everyone. Thanks so much for joining us today at IEMX Health. We’re super excited to have you all here and, really excited for this next session by MarketLogic. Very happy to have them as a title sponsor for the event. Before I hand it over, just a few quick reminders so you can put any of your questions in that q and a box, and then we’ll get to as many of those that we can at the end of the session. Christian, I will go ahead and hand it over to you. K. Thanks a lot, Kara. Welcome, everyone. Happy to have you here, and I’ll jump right in. I wanna share a case study with you of one of our health care clients that was putting our generative AI product, DeepSites, to a habit test and, happy to share the results. Just a few words on MarketLogic. We’re a software as a service, supplier, with more than one and a half decades experience in the area of market insights management. We have many global customers and, a lot of integration partners. And over the years, we constantly reinvent and win innovation prizes. So so we’re driving our software market in that section. And we do also have a lot of experience with health care and pharma clients, and we know the market. We know how it’s regulated. We comply to that regulation with our software platform. We deliver end to end solutions. And, of course, in in evolving times and with the evolving AI technology, we were looking into AI very early. And, just to share a little insights to our own journey, we started using, GPT three before even ChetGPT was released, in twenty one and was evaluating it, were playing around with it, building our first products, then we with some, key, cocreation partners, we started developing the first products. And then in April, two thousand and twenty three, we launched our generative AI product, DeepSites, which will I, I’ll share with you in a second. And, at the same time in April and May, of that year, the Philips Healthcare team, the Insights team took deep sides and did a pilot, and they they did a real nice test, a very thorough test, that I’m going to share as well. So deepsides is all about answering business questions and reports, and that is what we see in your day to day life. You have more and more data, more and more questions to answer. You have fewer resources. The insights teams, might be reduced, and, you you, like every company, run under the permission to to be faster, to be more swift, in your turnarounds, and to deliver answers and insights even quicker. And very specifically for the health care market, you have a a very regulated market, but not only regulated, but you have a very sophisticated market. So there’s the regulators, there’s insurance companies, there’s practitioners, there’s health care companies, and there’s, of course, patients. And, the complexity of the market leads to a lot of high complexity and insights and a lot of high complexity into the questions. So, let me share deep sides very quickly with you and then jump to the results of the test. In your everyday life as an insights team, you will run into very similar situations like this. A marketeer, for example, reaches out to you. Hey. We work on this new product. We need this and this questions to be answered. We need these insights. Please help. Please send over quickly. We need to solve it in this meeting. So in these situations, the questions naturally comes up. Can AI help? Can it be supportive? And our answer is yes. Well, we have developed deep sides. It’s a very, simple tool. It doesn’t need any training. You ask it’s your question. It delivers answers directly, and then you actually today, you can already create reports, which means it, extends your initial question and looks for surrounding relevant, other questions and then answers those as well. So you create answers and full, research reports generated by an AI, in within seconds. Now why is, it not just a GPT chat GPT added to a search or added to a platform because it needs a whole architecture? And I will not jump into all the details now just due to the time, but I want to draw your focus on one layer of our architecture, and that is what we call deep evidence analysis. So our AI works with the whole infrastructure and processes behind, the the front end and the back end that it retrieves the findings, it retrieves the knowledge, it understands documents and questions. And then we have it run through another AI, an extra layer that checks the findings, checks the initial questions, and does a real proper matching. And with this deep evidence analysis, we are able to reduce hallucinations dramatically. So our tool is not allowed to make anythings up. It will not behave like, the standard check GPT. It is trained for market insights, and it’s really reducing any any hallucinations. So now let’s look on the more interesting stuff. The case study that the health care team from Philips did was they defined a bespoke set of questions, related to health care, themes, and, they split it into two groups, into experts and users. Those were the testers to have the experts focus on the quality of the answers and also do some manual retrieval of answers and the business users to focus on the easy usability on the, did you understand what the answer was? Did you understand the quality of the answer and so on? And, then they loaded deep sides with just a limited set of documents. You know, there were just hundred and fifty documents, from the existing knowledge base. They loaded to deep sites and went into the test. They also put Bing and Cheggpt to the test. These tools, of course, look into the Internet, not into internal repositories. The time frame was April and May, so two months of testing. And the metrics that we’re using was, was the answer understandable? Did you at least get an answer? Was it able to to, answer your question? Was this answer understandable? How was the quality of the answer in means of trustworthiness, reality in in the content? How many search results did you have to read through to get to the same answer without using an AI platform? How was the source that was mentioned in there? Is it trustworthy? Is it available? Did you find that data in that source? The hit rate, they called it, is within the three first results, was the answer already included in the first three hits? Did you directly get to what you were looking for? And then how much time did it take you to get to that answer or to retrieve the answer out of documents if you not did not use a a, existing knowledge management system and the subject matter around those twenty seven questions, they had about thousand eight hundred documents related to that. They took, search results, most read documents and reports, and loaded those as a small selection of hundred and fifty documents to deep sites. And they did exclude any external sources in the deep sites test as at that time, the contracts with external sources were not yet complete, so they couldn’t be included. Just keep that in mind. The, the database for deep sets was very limited. Now the results that they ended up with, was and we start on the left side, comparing AIs with other AI tools that ChatGPT and Bing, yes, they were able to answer around fifty percent, so every second question, that was asked. At least the answer sounded logical. This the answer sounded good, but, not yet they didn’t yet pay attention to to the quality there. I’ll I’ll get to that in a second. DeepSites was able to answer nearly, two thirds of our questions, so sixty four percent. The, the more important part is the sources and the trustworthiness of the of the mentioned sources in those answers where, naturally, the AI tools that look into the Internet were very Joe in scoring and DeepSites using a much better database of market reports that were specifically made by the Philips, of course, had a much better ranking there. And on the right side, and this is more importantly, I think, more to pay attention to is, comparing deep sides with the business users and with the expert users of the existent knowledge ment, management system they have And the hit rate, so finding the right answer, the the correct answer within the first three results, was at about ninety percent, eighty nine percent with DeepSight. So nine out of ten questions could directly be answered within the first, results. And in the existing management system, that was only at about sixty percent. And let me elaborate on that a little bit Joe you still have the same numbers. Even the experts, when they accessed, the system, they they managed to retrieve the same amount of success rate. So nine out of ten questions could be answered. They were looking two thousand eight hundred documents, not hundred and fifty like deep sites. The search results an expert had to crunch through to make it to that answer was an average hundred and twenty two search results, with it when using deep sites, it was within this first, two or three answers deep sites we, reported. So the hit rate that deep sites created was far higher than the hit rate, the existing management system or the classic search could do. And very importantly, doing a search and reading through takes a lot more time. So the experts took about seventeen minutes in average to come up with the answer. DeepSight only needs a few seconds, and then the insights manager took maybe one minute or less than one minute to read through the answers and have the right information. And the level of quality they were looking for, hundred percent trustworthy, and ready to be shared with any stakeholder we have. So copy paste into an email to the CEO that was the level of quality they were looking for. So to in short, the effectiveness was really good, based on Philips testing. They found that DeepSites is more effective than any other AI tool out there, like CheckatpT being similar. The trustworthiness, it was the only AI tool that could deliver trustworthy answers on a continuous basis. The speed was significantly faster than any human could produce it with its classical system. And therewith, speed, quality, and direct answer to a question. DeepSight enables them to foster a new insights culture within their company, with better insights driven decisions, right on demand, direct access for business users. And since that test, a few months have passed, so we not only have answers today, we have reports as mentioned earlier. We have integration into, chat programs Joe you can interact with, deep sites using Teams or Slack or Google Chat. It’s connected to SharePoint, folders Joe you can automatically pull in data to it. So we evolved the platform in in as fast as we can. And so this was the base about half a year ago, and today, you can imagine it would be even better. I just I will leave these quotes on for you to read. It’ll be a little let’s, in take your time for reading them, and I would offer a few minutes, Cara, now to answer questions. Let me maybe focus you on one aspect next to the trustworthiness, speed, and impact, the effectiveness of a real proper insights focused AI. You see the the quotes down here. It’s they they estimated they saved about a whole workday, seven and a half hours on answering those twenty seven questions when using DeepSites. Now imagine you’re using the reports function of deep sites. You will save hours of time just on one question. That’s the impact it creates. So, Kara. Wonderful. Thank you so much, Christian. That was that was great. We have just a few minutes left for questions. So if anybody has any, please put them in the chat box. I have a few here. So how do users proceed with the DeepSight’s answers and mention reports? Yeah. Okay. So the answers, and that is a very important question is you can share any answer using a classical link function, share it with your colleagues. They can have direct access to that answer. They will see the answer that you you sent them, but they will also see two related answers just to cover little deviations, in their question, maybe from from initial question to when they read the answer. And the reports go even further. Once you have created a report, you can download it as a word document, for example, and make it your own report. Change it, add things, and that would just speed up your own processes. That’s great. Thank you so much. Another question here. How does DeepSets work when faced with multimodal inputs like charts, images, or graphs? Yeah. Yeah. That’s a very good question. A lot of insights reports are very visually heavy documents. They use, big images. They use graphs. They use tables. Now the AI tools, as you might all know from the news, evolve as well, and we are injecting what we call answers v two, very soon that will be multimodal enabled. So, deep sides will not extract copy from from any visual slide or graph. It will look at the slide. It will understand the slide, and then step by step in all detail describe the slide. And our internal tests have, shown that it does that very solidly, very stable. So the amount of insights you get out of a report will dramatically increase very soon. Yeah. That’s great. Question from Lucy here. The case study was based on internal data only. That’s a question. How formatted did the inputs need to be, I. E. If there are raw qualitative transcripts, would that be as effective? Yeah. So the the test they did were with ready reports. So they loaded them as PDFs as they had them in the repository. We are looking into adding more raw data, to deep sites, especially in the area of, voice of customer, that you can load survey, that you can load interview videos that transcribes them and learns from them and finds the right patterns. So because you don’t wanna you don’t want opinions to be part of your answer. You want the insights out of many opinions. And same with data. With data, we most probably work with, some data partners because we we have no intention to become a new BI product. But, of course, raw data is more and more important for us, and, it will be enabled soon. Alright. Thank you. Question from Suzanne here. It’s a bit longer. I’m just gonna read the first part of it. Have you rerun the test using the latest models from ChatGPT? We we internally, we test, all the the competition that we have access to, all the AI tools we have access to constantly. Also for us, it means the the AI engine, we’re using currently, it’s a mixture of, three point five turbo and four. We use it in different layers of that architecture I showed you quickly. Of course, we keep track on that in more but more an aspect of improving our own engine. Philips, as far as I know, hasn’t done this test again. It will not solve the core problem of JGPT looking into the Internet instead of their own documents. And even if they can load their own documents to it, it will still water it down with the general knowledge of the Internet. So it’s very difficult if you try using a tool that uses public knowledge, with no filters or preparation. I hope that answered that question. Help you there. Yeah. Yeah. Yeah. Suzanne, let us know if she’s has follow-up. Just a little bit of time here. One last question. How does DeepSights interact with research management slash desk research tools? Yeah. Okay. So that is maybe someone familiar with our platform. DeepSights is our AI product. We have a full, research platform. One part of it is research management, especially built for pharma and health care industry where you, under all your compliance regulations, can perform research, projects and deal with the confidential data you gather there. And we are currently building a connector that when you build a report and you find gaps or you can actually build hypothesis on on your findings and then send that as a as pre briefing, so to say, to our research management module that then enables you to extend that briefing and perform your research projects right on the platform. Wonderful. Well, we are just a little over time here, so I’m gonna wrap wrap up. But, Christian, thank you so much for your time today, and thank you everybody for for joining and asking questions. And we’ll see you all in the next session. Okay. Thanks for having me. Bye. Bye.