Learn how Philips deployed generative AI across their enterprise consumer insights function and compared our new DeepSights™ solution against ChatGPT, Bing, and existing insights management platforms.
How Philips tested and deployed a generative AI solution to improve its market insights management
With its rich 130-year-old history, Philips transcends traditional limits, unleashing a powerhouse of innovation and pioneering breakthrough solutions to improve people’s health and wellbeing. At its core, Philips is a health technology firm, dealing in healthcare, lighting, and consumer lifestyle, and continuously redefining boundaries across sectors.
A testament to the company’s relentless ingenuity and humancentric advancement, Philips’ insights function has long been at the forefront of insights solutions, recognizing the benefits of being early adopters of cutting-edge insight adopters like Market Logic Software.
And now, Philips’ Marketing Insights & Analytics division is fearlessly experimenting with generative AI solutions for healthcare customer insights, gaining experience and expertise ahead of its competitors and positioning Philips as an industry leader in insights-driven decision-making.
Why is Philips adopting generative AI solutions for insights now?
Generative AI is like the dawning of the Age of Aquarius … it’s going to differentiate people who adopt this technology from the people who resist it. And the people who adopt it will definitely have an advantage.”
-Tom Mostert, Global MI&A Head of Knowledge Management & Competitive Insights, Philips
Philips isn’t hesitating to explore how generative AI can help shorten its insights discovery pathway, says Tom Mostert, Philips’ Global MI&A Head of Knowledge Management & Competitive Insights, who compares AI to the dawning of the Age of Aquarius (characterized as an age of revelation, expansion of consciousness, enlightenment and harmony).
Tom points out that most enterprise knowledge management solutions today have yet to make the search for commercial insights truly easy. The challenge for most enterprises is that they are stuck using unintuitive search logic to get the best search results from their knowledge assets.
Generative AI is like the dawning of the Age of Aquarius … it’s going to differentiate people who adopt this technology from the people who resist it. And the people who adopt it will definitely have an advantage.”
-Tom Mostert, Global MI&A Head of Knowledge Management & Competitive Insights, Philips
This makes their market insights platforms less useful to those who don’t know how to adapt their queries to the technical confines of the knowledge-management technology. But today, because of generative AI and Large Language Models (LLMs), things are rapidly changing in enterprises’ search for insights:
Generative AI is opening up new possibilities. Not only does it do a really great job at search and discovery based on natural language input queries, it also goes a lot further in the interpretation and summary of large volumes of information. It produces output in natural language that can immediately be absorbed and shaped by human users. It’s a significant step in technology.”
-Tom Mostert, Global MI&A Head of Knowledge Management & Competitive Insights, Philips
It’s undeniable that generative AI holds for instant consumer insights in decision-making scenarios. It has the power to solve a host of insights professionals’ challenges, like cutting the time it takes to gather and synthesize scattered data; empowering market researchers to meet tight deadlines without sacrificing thorough analysis and data quality; and simplifying the path from insights to decision-making across large, complex organizations.
Enterprises that jump in early with generative AI will reap the benefits of a shorter insights-to-decisions pathway over the competitors that wait to adopt this technology — and that’s exactly what Philips is doing.
Before Deepsights™ – the challenges of deploying generative AI for consumer insights
The promise of generative AI is huge, but Philips isn’t missing a beat — including not getting wrapped up in all the generative AI hype.
The generative AI revolution is just beginning, so firstly, insights managers need to know if they can trust generative AI tools available to them. A key challenge to overcome is ensuring the reliability of these sources. It’s likely that the first question on insight and intelligence professionals’ minds, and business stakeholders alike is: Are generative AI-powered consumer insights fit and trustworthy for informing important business decisions?
Because, as Tom points out, in the health technology industry, there’s little room to make mistakes, and there are known risks to indiscriminately relying on AI-generated answers.
ChatGPT AI is powerful in terms of generating comprehensive, well-structured output. But there’s a very big caveat — it does not reference sources, so you cannot check the logic … and it’s not designed to give a non-response. It does not say, “I don’t know” … ChatGPT will make stuff up if it can’t find the answer for you. And that doesn’t necessarily make for a very reliable source of information.”
-Tom Mostert, Global MI&A Head of Knowledge Management & Competitive Insights, Philips
Generative AI can provide wrong answers that sound convincingly logical, which is a term known as “hallucinations”. AI models can be complex and challenging to interpret. Other than verifying the validity of generative AI claims, insights managers may also worry about losing control over the decision-making process and being unable to explain AI-generated insights to stakeholders.
If you’re an insights professional, you have to be able to back up your insights. And as Philips is in a highly regulated industry in healthcare, data quality and reliability is top of mind. So, the organization needs to ensure the high standard of their data, while streamlining the research management process with the help of AI.
With the challenges and opportunities of generative AI in mind, Philips’ insights team partnered with Market Logic to pilot DeepSights™— a generative AI assistant that gives insights pros instant answers to their natural language business questions.
Philips decided to see how DeepSights™ performed in comparison with other widely known generative AI solutions such as ChatGPT and Bing. By putting these AI solutions to the test, Philips was able to effectively assess performance of answer quality and time efficiency (compared to established search methods) and uncover which platform offers the highest-quality of AI-driven market insights.
With the challenges and opportunities of generative AI in mind, Philips’ insights team partnered with Market Logic to pilot DeepSights™— a generative AI assistant that gives insights pros
instant answers to their natural language business questions.
The Test: How does DeepSights™ compare to its alternatives?
Philips conducted a side-by-side comparison of how other existing and publicly available generative AI tools, specifically ChatGPT and Bing, succeeded in answering business questions when compared to Market Logic’s DeepSights™ solution. They also compared DeepSights™ against the classic search function in their existing knowledge management system. While the Market Logic team was available in a support capacity as needed, Philips conducted their study independently of Market Logic.
Methodology
In order to ensure comparability and stable test results, Philips chose to focus their questions and curate their knowledge assets around specific healthcare themes. They asked a group of topic experts to handpick relevant resources in their knowledge assets, resulting in about 150 highquality documents added to their DeepSights™ testing environment.
Next, a team of experts knowledgeable on the topic as well as non-experts posed a series of questions, replicating the kind of queries they commonly encounter in their line of work.
Testing scenarios & metrics
Philips provided test results from two scenarios:
Scenario 1: DeepSights™ vs. publicly
available generative AI platforms
ChatGPT & Bing (ChatGPT and Bing were combined in results)
Philips assessed the following variables:
1. Winning answer (draws possible)
2. Source availability and trustworthiness
3. Time investment (average in minutes)
Scenario 1 results:
DeepSights™ vs. ChatGPT & Bing
DeepSights™ Test: Results Overview
*Customer-led test by experts on representative data set against reference question set
**Fraction of questions for which relevant answer was found in top 3 responses
***Minutes needed to obtain relevant answer per question
Results
In terms of winning answers, ChatGPT and Bing had a moderate success rate of 50%, while DeepSights™ surpassed them, successfully answering questions 64% of the time.
This demonstrated DeepSights™’s refined ability to tap into the resources and provide relevant responses.
You can check sources with Bing… but the problem is it doesn’t discriminate between what is really reliable information and what is unreliable information … Its sources come from the entire internet practically, and there’s a lot of junk information on the internet, there’s a lot of unreliable information, so all the sources you get are junk information designed for either sales or marketing purposes or some other nefarious purpose.”
-Tom Mostert, Global MI&A Head of Knowledge Management & Competitive Insights, Philips
However, where DeepSights™ truly shone was in answer quality. Compared to ChatGPT and Bing, which provided verifiable, trustworthy information only 14% of the time, DeepSights™ offered a much higher level of reliability. The client was able to verify the sources cited in DeepSights™’s answers 71% of the time.
We’re in a highly regulated industry in healthcare. We don’t have room to make mistakes. So we prefer to have a generative AI solution that we have control over what information it draws upon, rather than something that could draw from information that could be potentially damaging or harmful to patients.”
-Tom Mostert, Global MI&A Head of Knowledge Management & Competitive Insights, Philip
The results underline the capability of DeepSights™ to extract information from verified sources and generate more precise answers, demonstrating its value in the highly regulated healthcare industry.
ChatGPT & Bing are not valid alternatives as the availability & trustworthiness of sources are rarely given.”
-Tom Mostert, Global MI&A Head of Knowledge Management & Competitive Insights, Philips
An interesting aspect of DeepSights™, as highlighted by Philips, was its built-in limit to generate up to three answers per query. Each answer appeared to be a summary of the asset containing the most relevant information. While this restricted the range of responses, it also added clarity and precision to the answers.
Scenario 2 results:
DeepSights™ vs. Internal Knowledge Management System
DeepSights Test vs. Existing System
*Customer-led test by experts on representative data set against reference question set N=2 tester; N=27 searches; **Configured to show Max = 3
When compared to Philips’ existing knowledge management system, DeepSights™ demonstrated similar proficiency in answering business questions, both achieving an 89% success rate.
The key differentiator, however, was in the hit rate. While the answers were found within the first three search results only 59% of the time on the internal platform’s search engine, DeepSights™ achieved this 89% of the time. The result underscores DeepSights™’s ability to provide precise answers quickly and efficiently, reducing the time users need to spend searching for relevant information.
If someone’s about to go into an important meeting, or they’re facing a client and a question comes up and they need an immediate answer. DeepSights™ performed really well.”
-Tom Mostert, Global MI&A Head of Knowledge Management & Competitive Insights, Philips
One of the most striking outcomes from the pilot was the dramatic difference in time investment. On Philips’s internal platform, it took the team an average of 16.8 minutes to find an answer. In contrast, with DeepSights™, the answer appeared instantaneously. This level of time efficiency is a testament to DeepSights™’s ability to revolutionize the way businesses search for and find answers to their queries.
As with any experimental setup, the pilot program posed some challenges. Initially, Philips’ team observed some non-responses to queries. However, they soon realized the necessity for a critical mass of information for DeepSights™ to generate comprehensive responses. As they added more assets to the reference library, this issue was largely resolved.
Results overview: Philips finds Deepsights™ produces faster and higher-quality answers than ChatGPT or Bing
The major benefit of DeepSights™ is that within seconds, you can get an answer to a question that would otherwise take you half an hour to a couple of hours to answer if you go through regular search.”
-Tom Mostert, Global MI&A Head of Knowledge Management & Competitive Insights, Philip
Time efficiency and answer quality — the Deepsights™ game changer
The biggest strength of DeepSights™ is the understanding of the relevancy of a report and the huge time saving benefit (~7.5h saved for 27 questions).”
– Philips insights team
Philips’ pilot study with DeepSights™ revealed significant improvements in hit rate, answer quality, and time efficiency. These improvements can streamline the decision-making process and foster an environment where valuable time is not spent sifting through information.
The results also provide invaluable quantitative context to understand DeepSights™’s impact and ROI for consumer insights and market research teams, as well as the larger business.
Key Stats
DeepSights™ results:
- 57% higher level of answer reliability, compared to ChatGPT and Bing
- 30% higher answer hit rate than Philips’ internal platform’s search function
- Estimated 7.5 hours of research saved per person over the course of the test (27 questions).
- 16.8 minutes research time saved per question: DeepSights™ vs. Philip’s search function
After Deepsights™: Final considerations from Philips about a successful partnership
Being able to say for certain that the journey from a question to a solution has been shortened …that really frees up a lot of time where the user can focus on the customer, the customer needs, identifying solutions for customer problems. We all recognize that time wasted searching for information is one of the key obstacles for companies to perform better. I think DeepSights™ will really address that.”
Tom Mostert, Global MI&A Head of Knowledge Management & Competitive Insights, Philip
In today’s business landscape, the ability to access accurate and reliable market and customer/consumer insights at a moment’s notice is an increasingly critical need for organizations across every industry. Markets change in real-time. Global as well as local events can trigger huge swings in demand. Consumers (also B2B customers) change behaviors and preferences almost overnight. To ensure longevity and gain a competitive advantage, businesses need to be able to react faster to what is foreseen by insights experts as well as unforeseen events.
Generative AI holds the key, but insights managers need to be able to trust this technology. The success of DeepSights™ in this pilot holds promise not only for the healthcare industry, but also for other sectors looking for a reliable and efficient AI solution to answer their business queries — and that can effectively bridge the gap between enterprises’ vast library of information and the business intelligence they need to make data-informed strategic decisions, on a daily basis.
Leverage DeepSights™ AI to get faster, reliable answers to your market insight questions – and optimize your insights management at scale. Contact our team for a free consultation.
For more on this topic – read or listen to our interview with Tom here.
Dive in deeper. Watch the webinar
Listen to Sehnaz Arasan, Consumer Insights AI Platform Manager at Philips, and Joseph Rini, Director of Product Management at Market Logic, as they share how Philips and Market Logic co‑built AI Personas to drive deeper consumer understanding across the organization. Hear firsthand how Philips and Market Logic co‑created this capability, the challenges they solved, and the future potential of AI Personas for insight‑led innovation.
