Change Management: Rolling Out AI-Powered Insights

Okay. Welcome everyone to our webinar, change management rolling out AI powered insights. My name is Callie, and I will be your host today. I am here from the marketing team at Market Logic seated in our headquarters in Berlin. I’m joined by speakers Julie Sherman in Chicago and Lorenda Geronimo in the UK. So we have a global team here to, talk us through a very interesting topic. So for those of you who are joining who are unfamiliar with us, we are Market Logic. We are a consumer insights and market research solutions provider, that has been leading the market for over a decade now, and we are proud to work with a lot of innovative global enterprises, some of whom you can see here, across all industries such as health care and pharmaceuticals, CPG, retail, etcetera. Our product portfolio is underpinned by the GenAI enabled DeepSights and DeepSights workspace, which you will be hearing about a little bit later on. And we support a range of insights and research functions across these enterprises. But today, however, we are gonna be focusing on just one topic and one customer. So as reflected in this agenda, today’s topic is change management. This is, of course, a phrase I’m sure we’re all very familiar with, both the strategies involved in change management and the challenges that come with change management initiatives. However, as with so many things, I think the proliferation of AI has changed the way that we think about and communicate about change management. And, additionally, I think AI has, caused extra urgency around this change management conversation so that organizations can effectively both harness and make an impact with the AI solutions that they are procuring. So the agenda will start with, Market Logic sharing our vision for change management, then we will look a little bit more specifically at what the Novartis team is doing around change management specifically as it relates to their knowledge management function, their platform, Sherlock, which is supported by Market Logic, and the rollout of our product, DeepSights. Last but not least, these are our speakers for today. I’m very happy to welcome Julie Sherman, the vice president of global customer success from our Chicago office. She supports many of our clients on their change management initiatives and is an expert on this topic. And her counterpart today will be Lorenza Jeronimo, who is the knowledge management change lead at Novartis and leads their efforts there. So I think we are in very good hands when it comes to this topic. Again, we will be hearing from each of them individually and then bringing everyone together for a panel discussion. I I’m very excited to get into this topic, and I think without further ado, I can hand things over to Julie. So thank you guys all for being here. Awesome. Thank you, Callie. Yeah. Julie Sherman, and I’m really excited to talk to you about a vision for change management at Market Logic. So first, let’s talk about some of the challenges we’re seeing today across organizations worldwide. Some studies have shown that it’s taking people days to access insights, and there’s disconnected decision moments. But most critically, we’re finding that many business decisions are being made without insights. This information comes to us from a study we ran about a year ago from two hundred decision makers, in marketing and product. And we’re seeing these challenges not because insights don’t exist, but because they’re not reaching the decision makers. The challenge isn’t the quality of the research. It’s really connecting the research to those decisive moments. So how do we tackle the challenge of generating impact from your insights? First, we wanna make sure that we are centralizing your insights and establishing a single source of truth for the insights. With that, we can leverage democratizing AI and providing a frictionless access across the enterprise. However, none of this would be brought to life without a change management process. Change management is so important to accelerate, any tools that are brought into the organization and ensure people can access them seamlessly. When we talk about insights democratization, here’s a quick checklist of what we see people want to see when they talk about democratizing their insights across a global organization. There needs to be a low barrier to use. It needs to be available everywhere. We need to be able to trust the results and have actionable responses. It encodes best practices. And almost most importantly, it needs to be built for insights. That’s where MarketLogic’s DeepSights comes in. We have a low barrier to use through a natural question answer chat interaction. It is available, in Microsoft Teams, Slack, Google Chat, so it’s available where you’re working. There are trust trusted results through AI learning and optimization, and we’ve worked with over fifty customers at this point, so we feel confident that your results are trusted. We have actionable responses that compare and contrast and provide watch outs for users. We have best practice through AI guidance and prompt libraries, and then it’s also built specifically for insights with chart extraction, evidence classification, source guidance, and many more features and functionalities. So with that, I wanted to take you through how we roll out our change management framework, at Market Logic. So, we wanna make sure that we’re rolling out this AI powered insights tool to the organization to the best of our ability. So at MarketLogic, our change management process supports all aspects of the platform maturity, and we focus on four key areas where change happens. That’s people, process, culture, and technology. For simplification purposes, we’ve put it into three phases that will take you through of an enablement journey today. Our first phase, the early phase, simply the goal is to drive awareness and engagement. We wanna make sure that people know the tool’s coming, they’re excited about it, and they know how it’s gonna help their team, better drive insights in the organization. Some best practices in this phase are really advertising on billboards, and and TVs in the, office showcasing. Having senior leadership endorsement is super important when rolling out an AI. A lot of people don’t know if they can trust or use an AI with their organization, especially when it comes to using it with a, with insights. So we wanna make sure that that senior leader involvement and endorsement is there. Another thing we like to do is we like to work with our team to, brand it. So kind of influencing and allowing people in the organization to have an impact on how that, tool might look within their organization. So, for example, you’ll hear from Lorenza soon. From Novartis standpoint, they’ve branded the platform Sherlock as that would resonate better with their users. We have a number of tools that we use in the early phases, such as success stories, use cases, and best practices that we like to share early on to really get that excitement going in the early phase. Next, we have our mid phase, which is really, where we’re going to achieve capability. We’re going to roll out the platform to the users. So some of the best practices we have in this is really a short training session that’s tailored to your audience. So by that, I mean, we wanna have we can train DeepSights in fifteen minutes to a user, but we really need to have a story and a question that’s gonna bring impact to that group and that audience that we’re training so they can see themselves working with the tool moving forward. From that, we wanna make sure that people can take away, have enablement videos, trainings, recordings, and they know where to go if they have questions when using the tool moving forward. And finally, our late phase where our goal is to drive adoption and continued usage. How do we do that? Well, we have teams of champions that we create at our customers. We create regular intervals of awareness and engagement as well. We have a number of tools that support this, including drop in sessions, content excellent incentives. But most importantly here, we wanna hear from your end users and your success stories through this process as well. One early success story I’ve heard from many of our customers is using DeepSights keeps our meetings going. So, for example, many times, you might have gone into a meeting where you thought the focus was on one region just to go into the meeting and find out it’s on a completely different region that you’re not prepared to present on. So instead of saying, hey. I’ll get back to you on that. We have end users who literally open up their DeepSights, either app through Teams or on their computer. They ask a question of it and presented with a comprehensive answer where it not only makes the insights person look like a hero, but it keeps the conversation moving and going with very little time to waste as well. So what it’s doing is it’s allowing our end users to spend more time on the impact the data makes in the organization rather than mining and, using just mining data, in the beginning. So next, just a minute, we’re gonna hear from Lorenza at Novartis on how DeepSights has transformed the ways of working within their insights team. So the Novartis Market Logic partnership started with a pilot group of users in twenty twenty, and we just continued to grow and add content over the years. But once DeepSights was turned on in July of twenty twenty four, their platform usage skyrocketed. Today, we’re looking at over ten thousand users on their platform. So I am happy to turn it over to Lorenza who’s gonna tell us more about the change management journey at Novartis. Great. Thank you, Julie. So, how did we do that, really? How did we manage to, drive adoption that way? So if it’s first of all, the cornerstone of our change management strategy has has as been, around three key elements. So first of all, awareness. As Julie said, that’s the the real the key step that, we had to achieve really. Associates needed to know what shared logging insights are, who should use them, and how to access them, and also when the knowledge relevant to them is available. Then the second step, which is, interlinked is engagement. So it’s not enough to be aware, but you really, need to see the value of SHERLOCK and DeepSights and be willing to be part of the journey. And then finally, capability is the third important step because associates need to know how to use SHERLOCK and DeepSights and how to make the most of them. So once these three elements are in place, then we can really drive adoption. And adoption for us meant that associate would use the shellogue and DeepSights, in their day to day job, to find the knowledge that they needed, and, and really use it, every every on a daily basis. So, key levers that, key levers that we, use are definitely sponsorships. So for us, it’s always been important to have the endorsement from the lever leaders in the in the business in order to drive engagement among associates. Then communication is also the key. So, not just to create awareness and understanding, but also to create a two way engagement with our target audience. Enablement is key obviously. So building the capability of your target users. And finally, the community element. So, foster knowledge management culture, to make sure that the reusobier will change in the longer term. But even before, the the real the first key step in all of this is really understanding your audiences. So it all started there. So we we engaged in a two way dialogue with the users across the business, and we work with the personas. So for those of you, who may not be familiar with the concept of personas, these are, archetypical, representatives of your typical user of, of the solution. Right? So we really build these personas, to really put ourselves in their shoes, understand what what is their job to be done, what are their needs, what are their pain points. And, obviously, this has been very important not just to build the solution, but also to build the messaging and the the value proposition. So that they were interested and the message resonated with them. Once we understood our audiences, we moved to the second phase. So as I was saying, we built the right value proposition. So the so called the what’s in it for me. And we did that by, first of all, tying, the the shared lock and insights, with really the business objectives and the priorities so that they understood that it wasn’t just something sitting in a vacuum, but, adoption of the of the solution was really part of the wider, strategy of of Novartis. And then we spent a lot of time gathering and communicating success stories, backed by quantitative business impact data. So it’s not not just, you know, the, the qualitative side, but also showing, for example, how much money have we been able to, save, thanks to, Sherlock and DeepSights. Then the other another important aspect has been identify identifying the behavioral triggers. So what does, make people, change their behavior? What does the so we apply behavioral science, and I’m going to talk a little bit about this, in a few minutes. So that that has been quite, quite important and also the sponsorship from the leaders. So that meant, for example, having the leader support and getting them send communications, so email campaigns or featuring them in our videos, so that the users and, you know, the wider users in the business could see, that, sponsorship. And finally, I would point out recognition and rewards. This is also very important. So recognizing those individuals who are contributing to, successful adoption of the solution. And this could be, you know, a simple thank you email sent, in in, you know, in front of, of their line managers, but also rewards. So in Novartis, for example, we have something which we call spark points. These are points where, that have a corresponding monetary monetary value that you can assign to other associates in the business, and they can buy goods, gifts. So that that was a way, for example, of rewarding users, super users, and, champions. And finally, important element has been proactively managed resistance. Because with the with the gen AI, obviously, there was the big a a AI rush and, but people were afraid. There was there were a lot of concerns about, you know, losing, their jobs. So there was a lot of work around demystification that we did and, making sure that we also build the foundational understanding of GenAI and so that people really understood what it was about. And then there were also some concerns about the compliance. So we engaged with the legal team in Novartis, but also with the ethics risk and compliance teams. And instead of shine away, we we really brought them along, so they, we we we really try to derisk the solution. So for example, as Julie was saying, DeepSights, is very, has been trained not to hallucinate, so to make up facts. So it’s very, rigorous. It it has all the sources, it it provides a consolidated answer, when you enter a business question, but there are all the sources are referenced. They’re also watch out some. So in case, for example, there are any contradictions or in the sources or any contextual consideration that the user should take into account, then DeepSights will, will flag that. We also, included the the results itself validation loop. So DeepSights will only provide an answer if it finds, you know, the the, in the in the in the shared locker approved, in the content which is approved, and on Sherlock. Otherwise, it won’t provide an answer. So it it’s all very rigorous in a way. And so we really and we also included disclaimers, in our comms and training materials. So we really worked hard to make sure that the legal and the ethics risk and compliance seems in Novartis feels comfortable with the with the solution before we rolled it out. And finally, we also piloted with the users in the business. So, and that helped us not just, to surface some, feedback that that we could bring back to market logic, but also, it it, unearthed some, best practices. For example, how to ask good questions. You know, what are the good prompts to ask a question in deep insights? And that was all useful feedback that that we could, embed in our in our training, and in our communications. Finally, once we did all that, we we we could, jump to the, you know, to the last phase. So we could really tailor the message that resonated with our target audiences, and we could reach them through the right channels that, were most effective for them. And we also built, a champions network. So we are a a central team. So we can’t reach all the users, across the world in all the countries where Novartis operates. So we have this, network of super users who are are eyes and ears on the ground. So they not just amplify the communications, but they also bring feedback back to us. So feedback from the users across the world back to us. So it’s a very it’s a very successful partnership. And just to give you an example of, as I said earlier, we that you we we we gather and communicate success stories. And and so I’d like to give an example of this. Obviously, when it when it comes to DeepSights, it’s very important to, approach, each, audience with the, you know, with the with the tailored and targeting messaging. So when it comes, for example, to their marketing teams, for them, it’s really that now times toDeepSights, they can find answers in minutes instead of hours or days. When it comes to the insights teams, the really the the what’s in it for me is that they can generate results in weeks instead of months. And then if we go one step higher when it comes to leaders, now they can make better decisions because they have better knowledge, at their finger fingertips. And and finally, when we talk about the wider organizational impact, thanks to SHERLOCK and DeepSights, we could reduce a necessary span on market research, by millions, annually. Because now the knowledge is all centralized, and instead of being scattered across hundreds of SharePoint sites, it’s all centralized and, and easily accessible. And so, a case study that I think is quite, is quite, it’s quite, significant is that, the business essentially wanted to know, if patients preferred be blister or bottle packaging for one of the drugs that we have in Novartis. And this may seem seem like a simple question, but actually, depending on the answer that, you you you can give to you give to this question, you may decide to build up to build a manufacturing site. So it’s it’s a very important decision. And so one part of the organization decided to to to go with the traditional approach. So they decided to come to conduct commission market research to external vendors. And this cost, between fifty and one hundred k US dollars. It took around three months, and the sample size was quite small, like fifty patients. While another part of the organization decided to, to adopt, SHERLOCK and DeepSights. So they they did desk research and they embedded DeepSights almost they used it almost like a sparring partner, in their in their desk research. And this was for free. It took around three weeks, and the patient sample was over six thousand patients. So it was quite robust. So I think this is one of those stories that have really helped us, quantify the, the value of shared logging insights and and drive adoption. And I’d like to conclude on, our the mentality that we have in the team. So it’s, we call it the test and learn mentality because, really, what we do is that we run our campaigns and, you know, our adoption efforts, and we learn as much as possible. So we we experiment and we learn, and then, you know, we we we track. And then based on, on on on on what we see, what works well, and what doesn’t work well with our audiences, then we we we pivot. So we double down on what works, and we let go of what doesn’t work. And so, as I was saying earlier, we have applied behavioral science, in our adoption efforts. And in a recent campaign, actually, we tested two principles. So one is called the authority principle. So essentially, it says that as human beings, we tend to listen to the authority. So if, the message and the the promotion of shared login websites comes from a a leader in the organization, Associates are more likely to, comply and adopt the solution. And then so we tested these when we, and then we tested the also another principle which is called the availability bias, which essentially says that if you recall the pain of not having, the solution ready so if you recall to people, how bad was their experience when they were trying to find knowledge before Sherlock and Insights, which took them a lot of time. And then they weren’t even sure whether what they found was, you know, accurate and compliant. So if you remember if you remind that and then you present the solution, then they’re more, receptive to the message. So we tested to these two, principle and, and and actually what came out and this is true for, our target audiences in Novartis is that the authority principle was the most effective one when it comes to driving traffic and and usage of, of Sherlock and DeepSights. And, we also noticed that emails are still the most effective channels and expect if especially if they’re sent by leaders, as I was saying. So the authority principle, it for for us, it works quite well. And then just another, highlight was that the demos are are quite effective. So now whenever we do a big campaign, we always tend to include a demo session live with the the users so so that they have a chance to, to to see the the the solution in action and also ask any questions. And, so finally, I would just say that, it’s important to track effectiveness. Right? And because that will, will allow you to see, you know, what what you should, let go and what you should, double down. But it’s also important to track what matters. So we track the impact of our, comms tactics. For example, how many people have opened an email, an email campaign. But then in the near term, we also look at, you know, how did that translate into actual usage of the solution? And then finally, at the never even higher impact, we look at, you know, what what has this meant for the organization? How did we, you know, how how much have, we saved? For example, in terms of, research, spend. So we we really go all the way, and I think it’s very important to understand, you know, to to really understand what what what what you should track and what makes more sense to track. And then, yeah. And and gather your your learnings and apply your learnings in future campaigns. And I think that’s, that’s it from, from me. Thank you. Alright. I will, come back on the screen here, and I will invite Julie back as well because now we will transition into our panel discussion portion. At this point, I can, again, just remind people to get your questions in and, direct them to either Lorenza or, Julie if it is a specific question about either the product or Novartis’ process. But I actually want to begin, with Lorenza, just speaking about, not not your process, but actually taking a step back and and speaking about your team in particular because, I think a lot of people in the audience would, be very envious to have, a team, like yours at their disposal. Not many, organizations have a dedicated change management function, let alone one that is dedicated to their knowledge management, solutions. So I think that sets Novartis apart. And I would just like to ask, how did your team come about, and and why do you think Novartis chose to invest so deliberately in change management in the first place? Sure. Yeah. So I think, the, there was a strategic vision, and, the head of the knowledge management team, Stefano D’Alfanto, and also the knowledge management team lead, Joseph Pizzari, because they understood that the knowledge management team really needed some specific change management capabilities. And that also not anyone can do that. So it wasn’t about, you know, redeploying someone on the team. You know, it need it’s a very specific discipline. And, also, it’s a very strategic one. So, you know, change management is not about just sending a newsletter. You know? It’s a very complicated, and complex discipline with a strategy, and the strategy needs to be integrated with the knowledge management strategy. So I think that’s why they this earlier recognition led them to, hire me. I was the first hire in the chain to management team in two thousand and twenty two. And then now we are a team of four, actually. So I think the the the growth from from one to four, how did we do that? I think by, as I was saying earlier, trying to, unquantify the value that you bring to the business. So we look, as I was saying, at coms, tactics. So, for example, you know, what is the, open rate of this campaign of or but then you also look more longer term. You know, what is the adoption now of the solution? So how how did that convert and actually use it? And then even more long term, we look at what how much money, for example, have we saved? Because, you know, we reduced the amount of unnecessary market research that is commissioned to external vendors. And they have a very specific example actually, on this because, we, there was a specific a part of the of the organization to which we rolled out Sherlock and DeepSights. And last year, actually, they saved the several tens of millions of US dollars in market research. While the other part of organization where we haven’t rolled out, Insights and Sherlock or not yet, they actually saw an increase in the span. So I think it’s stories like these backed by by numbers, which really, you know, can show leadership as well that there is a point in our retained management integrated in the in the team. Yeah. I think that that’s a that’s a great example to share, but, of course, I think that’s a really great point that you brought up that before you can get to the example of, look at this team that saved millions in market research, you first have to, find the quantifiable results in your own work, which is, you know, rate of email opens, how much that leadership principle impacts. So you start, you know, quantifying your own work, and then eventually you get to the really large scale impact of insights after the the change management professionals has really flourished. So, yeah, that’s really, really interesting, and I like how that’s all backed by, behavioral science. I’m gonna kinda quit, flip the question on its head, actually, because, you know, from our side, from the MarketLogic side, every customer that we work with is a little bit different. We don’t always have a Lorenzo that, you know, is is stationed within our customers, knowledge management function. So, for you, Julie, working with our broad base of customers, how much do you think it is, like, still important to have this unified approach, and how much do you find yourself treating each of our customers on a case by case basis? And and what does that really look like in terms of, you know, your focus on each phase of the, change management rollout? Yeah. I think the way we’ve structured with the different phases is great, but allowing ourselves to drive within, like, the people, process, culture, and technology allows us to be a little more flexible with each of our customers. So, within each phase, we look at that. So if we know that we have an issue with people in the beginning, right, we wanna make sure we get senior leadership involved. We wanna make sure we’re establishing that group of champions. But then as we roll out, it might not be the people anymore. We’re allowed to be flexible within that framework and say, okay. Now we actually have a technology problem. You know, there’s a barrier to getting our IT teams together or things like that. So while we have a nice framework to to look at and use and leverage, we really are able to take it case by case, within the framework itself. Yeah. I think, you know, that I think that’s a great answer. For my next question, it actually has been asked a few times in the chat. I think this might be kind of the central question that we’re trying to answer with this webinar, which is, you know, how AI has impacted change management initiatives. Particularly, we have questions. I’m just gonna summarize a few people, so, give me a second here. But, let’s go to Lorenzo first, and then Julie will will bounce back and forth here. But in terms of change management when it comes to AI, what are the main concerns that you face? Because I think this is, something new, especially with generative AI. You have to counteract. You talked about having to demystify some some conceptions about Gen AI, and I think that kind of relates to the kind the the crucial first step of the change management process, which is awareness. So how do you what first of all, what are those common concerns, and then how do you specifically tackle them in the in the beginning stages? Yeah. Absolutely. I think, you know, the the fear, you know, that people were having, especially at the beginning, you know, am I gonna lose my job? I think that was, that was, you know, but also fears about, confusion because there were other AI tools in Novartis that had been were were being deployed. And so I think what we we needed a clear differentiator. Right? And, we needed to explain the difference. And we noticed actually that some were being deployed just because they were AI, while Insights was actually rolled out with a very specific and targeted use case. So, not because it was just, you know, shiny new tech, but because it helps specific groups, in in Novartis doing their job, work smarter, really. So I think that was, a key difference as well, that we highlighted, so really the showing the the impact and the value. And then noncompliance was a big fear. So, you know, the legal team as, as I said briefly earlier, the the legal team and the ethics risk and compliance team were really concerned about that. So we we worked a lot with MarketLogic, to, you know, make sure that, any, you know, the the the as as we saw, there are watch outs, in deep insights now that flag, you know, if there are inconsistencies or contradictions or or if the user needs to, you know, to be to dig deeper in the sources. We also have, disclaimers in our communications and training materials. So instead of shying away from those teams in Novartis, we actually engaged with them and brought them along on the journey. And now they are actually comfortable, you know, with the with the with the solution. We really the risk bit, if you if you will. So yeah. I’ll kinda hand it over to Julie with just, like, a a bit more context because, Novartis, they rolled out DeepSights in, I believe it was July twenty twenty four, so it’s less than a year old there. Obviously, we’ve been working on it, a bit longer. So in your experience, when we’re talking about these these common concerns and and the awareness that we have to, share with people around Gen AI, do you think, Julie, that has changed from when, you know, these AI tools first hit the market? Like, do you think that there is a, an evolution of people’s mindset where maybe they’re a little bit less hesitant and more ready to embrace it, or are those concerns, still part of of, your change management conversations? They’re definitely still part of the change management conversations, but they’ve lessened over time. Right? So, in the beginning, it was we’re never gonna get an AI tool in. Like, go work with our legal team. We’ll never get it approved. And now it turned out to how do we roll this out to enterprise? How do we trust that the answers are correct? So we do see that it takes a little bit longer to roll it out than a simple, you know, platform knowledge of knowledge management just because I think our AI teams are wanting to vet a lot of the questions. They’re wanting to make sure that it’s giving the right answers. And, also, as, Lorenza, as you mentioned, people are are we going to lose our job with this tool? Right? And then when they’re finally realizing, no. This is just an enabler of change for us. This is enabling us to go ahead and dive deeper into the data and the impact of the data rather than just spending all of our time doing that lower level data mining. Then they actually see the benefits of it and and enabling it throughout the organization. So it takes a little bit more time to get that trust built up, and I think insights people, technically, they like to keep it close to the chest on their work and their research. But it is definitely getting out there and showing the value for them creating those stories. And then we see these, numbers like we see at Novartis just kind of skyrocket and take off after that happens. Yep. Yeah. So a little a little bit of of hand holding, but then I think also, Keely, especially at these, big enterprises, differentiating how DeepSights is is different than a generalized AI tool, and and we might touch on that a little bit later in the questions. I’m trying to keep up with all of them that are coming in. If they if I can add to that, because I’ve seen that there was a question around, are you using, like, a sample questions for people to try? And, actually, I think that’s a good point because one of the things that we do in our so we have, you know, demos and, regular regular demos and connects with our audiences. And we have, you know, nailed down what are some of the key questions that people have used, you know, the various personas to do their job that has has enabled them to do their job, you know, in easier in a in a smarter way. And, actually, we often invite some of those, associates on this call so that they can actually demo live these questions. And, and, yes. I think those are powerful examples, and, as well. So I just wanted to call out because I saw there was a point around using the No. That’s really smart not to just, you know, give them a list of prompts, but actually show how someone is using it in their daily work, and that’s very nice of someone to volunteer their time to do that as well. Okay. I did want to slightly circle back, because, Lorenzo, you did speak about how important it was to engage the IT, legal and procurement teams, early on even before DeepSights launched. And I’m wondering if, the type of stakeholders that you’re involving in change management has changed over the past few years, whether that is the, frequency of the touch points with the different stakeholders or just the type of stakeholders that want to be involved in these professionals. Is it making your change management process more complicated or or more streamlined, do you think, based on the the type of stakeholders you’re interacting with? Yeah. So I think we we we engage with the wider business ecosystem. So we often talk about knowledge management as a business transformation. So we also address, you know, when we do, change management work, we really look at, the wider, ecosystem. So there are various stakeholders that we actually engage with. So, obviously, the leaders, these are very important stakeholders. We need their sponsorship. And, you know, and once we you have the official support from them, users are also more likely to, adopt the Sherlock and DeepSights. And then it was quite interesting because we what happened with leaders was also that, when there was the big, you know, the AI rush, they were looking into the organization to see what examples of, you know, what solutions had been, deployed using AI. And so that insights was actually the most advanced. And so that helps us. And now, actually, every every every other month, we report into the board of directors. So we’re quite lucky because we we got the level of visibility that that essentially is is very high. And, and so these leaders, some of those leaders are are champions now. So that that’s quite powerful. And then another important, stakeholders, obviously, we have the target users who are the those that, we, you know, we we we target so that they adopt the the solution, but then we have the champions, as I mentioned, and they are also very important for us because they are they reach where we can’t reach because we are a core team, small and core team. So, they they really help us, drive adoption in the various countries and and regions. And then legal and the the ethics risk and compliance teams are also very important, because, obviously, we need to get them on board and and and procurement. So procurement is, is a very important partner because they influence the relationship with the vendors, with the suppliers, and now and they can change the contracts. And now we have a clause in our contracts, which says that suppliers, once they conduct the prime the market research for us, they need to upload it on Sherlock. So thanks. And so procurement has become a a really important, ally for us, in our adoption efforts. And I I I think the most important thing that we did with these, various stakeholders group is, tailoring the messaging and the what’s in it for for me, essentially. So when we approach leaders, obviously, we highlight how shallow and DeepSights enable them, their teams to reach results faster, you know, and, to enhance productivity and decision making in the organization. At the level of the of the champions, we highlight, you know, the, and and and the users. We highlight how Sherlock and DeepSights enable associates on the ground to work, smarter, to save time, to be more efficient. When we talk to legal, we obviously highlight them the the compliance. Right? The scale and and now, actually, when they go and talk to other teams, in Novartis who are trying to implement, AI based solutions, we this site now is the gold standard. So in a way, they’ve become our ambassadors as well because of of these and and same with procurement. Procurement now has has enabled. So, every time they talk to teams in, who normally commission market research, they they say, hey. Have you checked the chart lock first? Do we really need to commission these, market research? So we now have a very, you know, extensive network of stakeholders that, act as partners and analyze and, and ambassadors as well. And, so yeah. So that’s, that that’s been really, really successful so far. Yeah. That’s amazing. I I wish we could add that procurement clause to all of our clients just for yes. Making sure that all the content actually ends up in the system because it does just exponentially make it so much more usable. And I think also your your fact about, knowledge champions. I mean, for correct me if I’m wrong, but I believe you sit within the Novartis global team. So being able to, you know, tap a knowledge champion that works more regionally, that that’s always gonna be more resonant with those teams. So that’s a really good point that I think anyone, in the audience from a global enterprise can can learn from. Gonna go over to you, Julie, with a more product specific question so you can flex your product knowledge on us. But, just about DeepSights in general as a Jet AI tool, what do you think the advantage of DeepSights is versus, you know, just using a SharePoint or a Google Drive, with an agent, an AI agent that’s more generalized? Can you talk about I I guess this is the a specialized, insights, purpose built insights tool versus a generalized Gen AI platform. What would you, say the difference is? Yeah. I’d like to start by saying, right, DeepSights is the AI, but the first step in kind of our our change in generating insights is really centralizing your knowledge information. And it’s great if you just have primary research that you wanna review, and asking that maybe of your SharePoint. You’ll you’ll probably find it. But everybody has other sources connected. So we partner with a number of sources, secondary content, that will be in the AI as well. We also tie in some trusted RSS sources. So you’re getting a holistic answer, not just from your primary content, but maybe from some sources that you, wouldn’t normally see that reside in your SharePoint drive as well. So it’s really that centralized point of where you know you can get an answer, whether it’s on the SharePoint there or, from your secondary source providers as well. So it really gives you that holistic answer that you’re looking for. And yeah. Yeah. And and, of course, you know, if anyone in the audience wants to see it, for themselves, we’re happy to show you. I think we only had one screenshot of it, so there’s a lot more to be seen about how Insights actually works. Lorentz, there are two questions about, you talk about the reduction in, market research spend. That’s obviously a huge cost saver, for a company like Novartis that, relies so much on on research. But then on the flip side, and you might not have visibility on this within your role. How much are you integrating, like, current data? Whether I mean, Julie just talked about RSS feeds and and, connected secondary sources. Is that, part of the SHERLOCK platform? And, how is that contributing to that that reduction in research spend? So, yeah, SHERLOCK is, centralized with all internal, strategic brand knowledge. So the knowledge about the brands that we have in Novartis. So that’s, let’s say, the the primary source. But then we also have, industry news, are also considered, you know, treated as sources when it comes to DeepSights and syndicated reports. So for example, if, you know, there are databases that that we have, you know, and journals that we have subscriptions to, So that’s also, considered the, yeah, it’s part of the sources in DeepSights. So when a users goes into DeepSights and ask a question, so the the answer will be yeah. It will provide an answer, if it finds some, you know, reliable content either from the internal, that did knowledge that we have in Novartis in Sherlock or from the secondary reports so that both are, both types of sources are considered. Although, obviously, we have more control over the the shared lock internal knowledge because we have a governance in place, where, you know, we we have, a team of knowledge management experts who work in partnership with the the brand teams and the functional teams in the business, and they bet and decide, you know, what knowledge is in scope and should be, uploaded on Sherlock, and they maintain it regularly. So, obviously, we have more control over that. Yeah. But yeah. Yeah. Yeah. So the data data government is is a is a separate part of what, the curation of Sherlock looks like, and I, I think that, is true for for many clients who use such a platform. Alright. I’m gonna try to get to every question, but if we do not, I I wanna be conscious of the time. We will, follow-up offline. But let me just, let me go back to just a straightforward change management question to kind of wrap things up here. So, for Lorenza first, you’ve been working in change management. You’re obviously an expert on these processes. How long do you generally think it takes for these new tech rollouts to go from being new to being adopted as a standard? And do you think the insights rollout, fits that timeline, or has it been slightly accelerated? And, yeah, like, what what can you say in terms of the the general timeline for these things, if you can? In interest now it’s an interesting question. So, I mean, change is a journey. Right? It doesn’t happen overnight. So I think, you know, you start, with the with the as we saw earlier with the awareness, then people not only need to be aware, they need to be willing to to embrace the change. So they need to see the value. They need to know how to make the most of the change. So it does take time. Mhmm. And it’s hard to say. And reinforcement is very important as well because, you know, you it’s not like you run a campaign and then you’re done. I mean, you need to reinforce continuously. You need to you know? And there there are various, you you need to reward to recognize the efforts of the people who are making, adoption, you know? So it it it it takes time. I I can’t you know, there’s no magic one, and there is no timeline, I would say. So it really just takes time. Some people may be more receptive than others depending on various, factors as well, their job. You know? I I think, you know, if, but if you can tie the value of, you know, the solution in a very tangible and concrete way to the job of that people do, and and if you actually, you know, how that is saving them time, making them more productive. I think, you know, that’s all very strong, and you tailor your messaging. So as I was saying, you highlight the what’s in it for me. I think that should, speed up the timeline and and speed up adoption. But, yeah, there’s no magic. I I Yeah. Yeah. I I wish there wasn’t. I’m sure it would make, you know, pitching to to higher ups a lot easier if you had an exact date, minute, hour, how long it would take you. But as we saw in the slide about the the history of our partnership with Novartis, I mean, the numbers speak for themselves, and there there is sort of a exponential turn upward once DeepSights rolled out. So I think the what Julie was saying about the barrier to use, just the idea that this is a a easy to use Gen AI solution that’s on top of, a very comprehensive platform of Sherlock, I think, you know, it it it hopefully makes these job change management conversations easier. And, yeah, maybe, Julie, one more question for you. Well, actually, did you wanna just clarify something? Because we we mentioned watch outs. We mentioned how we had to validate, internally with the Novartis IT and legal team that, DeepSights wouldn’t hallucinate. Obviously, that is a huge issue that people find with some, publicly available tools like chatGPT. So can you explain, what that means, like, when we say there are watch outs and, there’s no hallucinate hallucination within the platform? Yeah. Absolutely. And I do think sometimes showing this during training as well is important to show it does not hallucinate. So, one of the questions I got in in a training was, oh, is it gonna tell me what the trends are gonna be in twenty twenty six for electric cars? And I’m like, unless you have research for that, our tool is not gonna predict the future. So let’s ask it. Right? So we go ahead and ask a question that we know it’s not going to give an answer return for it, and it will return and say, hey. We don’t have enough information to do that and to produce that answer. So we make sure that it’s tailored to not predicting the future insights, and it’s not going to hallucinate, and it’s not gonna give people an answer that’s not supported. People can see the sources within the answer as well, so they’re able to go back and and check the sources and validate that that’s a a regular source that they’ve seen. They could also go in and control the sources by turning off one of the sources. Maybe they accidentally have a voice of a consumer with an opinion piece that’s in the answer that they got. They went out and they validated. They could go out and then turn that off to make sure that they are getting an answer based upon factual research that’s in the platform as well. So we really have tailored it to make sure people are not getting those hallucinations that, that they might see. Yeah. Yeah. I mean, before AI, I think the insights and research teams, always, saw themselves as the single source of truth at these companies, and we want any AI tool that we equip them with that also reflect that. So that is probably first and foremost, what sets Insights apart from some of the other, more generalized tools, that are being used. And, of course, there is a whole conversation about, you know, the next phase of the evolution of AI is how these tools will will start talking to each other. But, needless to say, and especially working with a client like Novartis that is in the health care and pharmaceutical industry that has some of the most stringent, regulations when it comes to confidentiality and and, trustworthiness of, like, the sources and the vendors that it that it onboards. Whatever sort of IT and and legal loopholes you’ve given us, we’ve we’ve managed to pass that test. So, for those those people in the the questions column that are asking about these more product related things, I would say that you can go to our website, and request a demo or a free trial to see it for yourself. But it’s it’s something that is more impressively shown rather than told anyway. Alright. I we will wrap up soon so I can get you guys out of here on time. Real quick, Lorenza, when you are measuring, the the users, Are the could I think you had a slide, actually. Maybe we can, like, bring it back up. Are you more concerned with returning users or engagement with new users? Like, when you are are showing, these platform statistics, what are you really focusing on? Yeah. So, I think, you know, we look at, unique users. Like, we look well, we look at different things, really. We look at total insights, but also unique users and then return users as well. Because, obviously, we want to see whether, you know, the solution is, actually sticks. And, and we also look at some, some knowledge, you know, what are the users actually doing on the platform? What is the type of knowledge that they’re looking for? So it’s very, yeah, so it’s quite comprehensive actually. And then based on what we notice, obviously, that informs our our next actions. And and if we see, for example, that in some countries, we we look at this from a global perspective, but also from a country perspective. So if we see that, for example, in a country specific country, adoption is going really well, then we may go and talk to the champion from this country. Okay. What have you been doing, and what lessons you may share with the champion from this other country? Right? So it’s both at, you know, global and then seeing, you know, at at at a country level what’s going on. Yeah. Yeah. Yeah. Really, really interesting. And and then I think that, you know, both you and Julie talked about the continuous improvement. You know? There is no endpoint for change management. It’s always about, adopting to, new use cases, whether that’s a change in technology or onboarding new user groups or new teams. So, yeah, I think it’s this has been a fantastic discussion. I really hope that I got through all the questions, but, I might have missed, they were coming fast and furious. So, unfortunately, we do have to wrap up. We will reach out if there’s anyone that we didn’t answer specifically. But before we say our goodbyes, I would just like to point you to the two, assets that kind of, you know, further reading some homework if you want to learn more about these topics. First of all, we have a an AI change management road map. This is this just goes over, like, the high level, steps that you can take to make sure that you are effectively harnessing AI solutions. And then if you wanna look more closely at how we’ve worked with Novartis over the years from our beginning twenty twenty, twenty five person rollout to now. I think it’s a really great story, and I I look forward to, like, a continued partnership, and, and more success stories to to show in the future. So, with that, I wanna just say another round of thank yous to Julie and Lorenzo. Your insights were really invaluable, and, I’m sure we can we will have another conversation in the future as the, platform and and its impact, continues to grow. So thank you all for joining us. And with that, I will say goodbye. Thank you. Bye. Thank you.