2020 Killing The Focus Group: The Disruption Of Market Research CB Insights helps the world’s leading companies make smarter technology decisions with data, not opinion. Our Technology Insights Platform provides companies with comprehensive data, expert insights and work management tools to drive growth and improve operations with technology. WHAT IS CB INSIGHTS? SIGN UP FOR A FREE TRIAL 3 Killing The Focus Group: The Disruption Of Market Research Table of Contents What is market research? The impact of technology on market research across the product lifecycle • Development: Early-stage market research is becoming more efficient • Launch: Faster and cheaper consumer feedback • Growth: New ways to crunch data will boost performance tracking The future of market research 5 8 20 4 Killing The Focus Group: The Disruption Of Market Research Understanding what people think and how they’ll behave is difficult — just look at the ongoing challenges faced by election pollsters. But as shopping activity increasingly moves online, market research — which refers to the ways a company identifies and analyzes consumer needs — is being reimagined by tapping into new datasets of consumer behavior that are vast, high-quality, and updated in real time. Adding to this, emerging technologies like artificial intelligence (AI), chatbots, and virtual reality are disrupting antiquated data collection processes like in-person focus groups and telephone interviews. In this report, we look at what market research does, how it’s changing amid massive technology breakthroughs, and what the future of the industry holds. Market research is changing as emerging tech enables new ways to gather and analyze consumer data. We dig into the industry’s transformation and look at what comes next. 5 Killing The Focus Group: The Disruption Of Market Research Companies look to market research to understand the preferences and needs of their customers. These efforts can include identifying a target market for new products or services, understanding the efficacy of a company’s current brand campaigns, and keeping track of rivals and the price of competing products. Over the years, researchers have identified various ways to capture quantitative data for market research. The earliest instances of market research can be traced back to the 1920s, when a group of researchers, led by psychologist Daniel Starch, approached people in the street to understand the effectiveness of an ad being displayed in a print publication. They asked people to recall if they had seen certain ads in magazines and newspapers, and then compared the recall value with the publication’s circulation. This came to be known as the Starch Test. In the 1930s, George Gallup created a polling technique — still widely used today — to judge the attitudes of a large population from a small sample. After collecting quantitative data on their customers, market researchers turned their focus onto qualitative research in the 1940s. This helped researchers get an insight into why and how people were making purchases. Researchers did this by collecting data through in-person interviews, as well as focus groups and research panels. What is market research? 6 Killing The Focus Group: The Disruption Of Market Research The next major wave of change in market research methodologies came with the internet. A milestone in marketing research was the development in 1995 of Analog, software that allowed website owners to track their usage patterns. The software gave market researchers access to data on how users were engaging with a website, product, or service. Analog is considered to be a precursor to Google Analytics, a current market leader among analytics software. A sample report created by Analog. Source: Web Archive The internet changed the manual process of conducting market research. In-person and telephone interviews were replaced with online surveys, while online data aggregation took the place of focus groups and research panels. Then came the smartphone — another game-changer for market research. With these communication devices now in the hands of almost half of the world’s population, it has become that much easier for market researchers to get data on customers. 7 Killing The Focus Group: The Disruption Of Market Research Source: B2B International In the past, market research datasets tended to be small, low- quality, and manually created based on information from surveying subsets of customers. Now, researchers are shifting from this often inefficient approach to analyzing consumer behavior passively by tapping into massive, constantly updating datasets generated directly from consumer behavior. Netflix, for instance, is able to capture the viewing habits of all its users and adapt to changes as they happen. These deep wells of customer data — much of it garnered automatically from sources like purchase history, sentiment analysis, and social media mining — can provide a more holistic picture of a customer’s preferences. 8 Killing The Focus Group: The Disruption Of Market Research The goal of market research hasn’t changed much since Starch first went out to discover the effectiveness of print advertisements. Researchers still want to know what consumers think. Market research has been integrated into every step of a product’s lifecycle, which encompasses 3 broad stages: development, launch, and growth. At the development stage, companies are looking to bring a new product or service to market, and are considering how best to sell it. Here, market researchers identify potential customers and distribution channels. Additionally, researchers analyze customer trends and create models to forecast usage of a product or service by the target audience. Once companies have identified their target market, they want to launch a product and measure how intended consumers react to it. Market researchers help with collecting and analyzing feedback from pilot schemes and initial customers. Market research also plays a role in supporting the growth of a product. After the launch of a product or service, researchers may help companies keep track of competing products, tweak offerings to align with shifting consumer trends, and assess how advertising campaigns are performing. Below, we look at how technology is disrupting market research at each stage of the product lifecycle. The impact of technology on market research across the product lifecycle 9 Killing The Focus Group: The Disruption Of Market Research DEVELOPMENT: EARLY-STAGE MARKET RESEARCH IS BECOMING MORE EFFICIENT AI applications like natural language processing (NLP) — software for automatically understanding the context of words — and machine learning are helping to reduce the human effort expended on early-stage market research. Additionally, these advances are making forecasting models more accurate and delivering better shortlists for research candidates. Voice assistants are changing how shortlists are used During the development stage of market research, a researcher is still identifying potential customers and the composition of the target market. One part of identifying potential customers is creating a shortlist. Automation tools are reducing the effort it takes to create a shortlist. Where once market researchers had to manually create shortlists of potential customers, AI algorithms are being developed to parse through databases larger than a human could manage and at a faster rate. Armed with a shortlist, market researchers are also employing NLP-enabled voice assistants — with similar interfaces to Siri or Alexa — to conduct surveys instead of using paper-based or digital forms. NLP helps these digital voice assistants “understand” human responses in those surveys and allows them to follow up with relevant questions. This approach is quickly gaining traction. Research from Qualtrics, a US-based experience management company, suggests that a fourth of all surveys will be conducted through digital assistants within the next 5 years. 10 Killing The Focus Group: The Disruption Of Market Research Automation tech is helping researchers parse unstructured data to create consumer profiles AI applications like machine learning have also allowed researchers to tap into massive amounts of semi-structured and unstructured data coming from sources like emails, social media, and photos. For example, algorithms are being designed to pick up on words and phrases relevant to a marketer’s research while parsing through a customer’s public online presence — like the pictures and comments they leave on websites they visit. MonkeyLearn, an AI-powered text analysis platform, claims that its data analytics tool can parse unstructured data such as emails or customer service tickets 1200x faster than a human. Source: MonkeyLearn Researchers can create predictive models for building customer profiles by combining this unstructured data — which accounts for an estimated 90% of all data generated — with structured data, like purchase history. 11 Killing The Focus Group: The Disruption Of Market Research Appier, a Taiwan-based data and AI company, says that it helped CommonWealth, an economics-focused magazine, increase its subscriptions and purchases by 404% after leveraging unstructured data for predictive modeling. Appier first predicted consumers likely to subscribe to CommonWealth and then scoped for other users based on specific patterns picked from its predictive model. Sentiment analysis is allowing researchers to gather more authentic customer feedback Beyond profiling customers, market research is also concerned with understanding how they behave when shopping or using a service. Sentiment analysis, which seeks to understand someone’s emotional response to something, is one way that market researchers are now looking to analyze customer behavior. For example, eye-gaze tracking tech, like that offered by Karna AI’s SmartGaze, uses cameras to measure the effectiveness of in- store campaigns and packaging design, as well as a consumer’s reception to a brand. The company claims that its software takes 85% less time to produce insights than if humans were tracking these eye movements — while costing 40% less. 12 Killing The Focus Group: The Disruption Of Market Research Source: Karna AI Travel booking website Expedia has deployed similar sentiment analysis technology to track a consumer’s emotions while using its website. At its Usability Lab in Washington, the company brings in customers and then records biometric information through eye trackers and face sensors while they make a booking. Natural language generation is being used to create research reports Producing literature on how competitors or a product are operating in the industry is an essential part of market research. Aiming to increase efficiency, some in the space are beginning to use natural language generation (NLG), an application of AI for representing data as easy-to-understand text, to generate business insights. For example, NLG platform Wordsmith generates business intelligence reports like marketing analysis and narratives from basic input data. Currently, these types of reports are mostly limited to a few paragraphs, but the potential is likely to increase in the near future. 13 Killing The Focus Group: The Disruption Of Market Research Brand image tracking is being automated Another function of market research is tracking how a brand is being represented in the media and on social media. However, it’s a time-consuming effort to manually track brand mentions across the web while also disregarding irrelevant keyword mentions. New services aim to speed this process up. For example, Talkwalker claims it can quickly help brands filter out irrelevant data — like differentiating between mentions of Apple (the company) and apple (the fruit). The AI company also says that its technology can detect visual mentions of a brand and conduct sentiment analysis to understand the context in which a brand is being mentioned. Meanwhile, platforms like Mentionlytics work on “social listening” to help brands manage their reputation. Companies offering this service aim to analyze data like a brand’s social media mentions, customer reviews, and customers’ questions. Source: Mentionlytics 14 Killing The Focus Group: The Disruption Of Market Research LAUNCH: FASTER AND CHEAPER CONSUMER FEEDBACK The next step in market research after gathering data on potential customers and preferences is to test a company’s products or services in the run-up to and during launch. Tech is making this easier. AR and VR reduce piloting costs in some cases Market researchers tend to test a product or service among a smaller group of customers before a company launches it to the entire market. Augmented reality and virtual reality (AR/VR) are making this research faster and easier — though associated costs currently limit use cases. For example, if a company wants customer feedback on a new store layout, it would traditionally survey customers to see if they like the proposed changes. Or it may arrange a section of an existing store according to the new layout and record customers’ feedback to it. This process can be time-consuming and costly. But by using VR, the company can bring in a group of customers and let them experience the new layout by strapping on VR headsets. This is a more reliable and authentic way of recording respondents’ reactions compared with asking them to recall their last visit to a similar store or to imagine how they would feel if they were in such a store. UK-based System 1 Research conducted an experiment for various brands to test out a customer’s shopping choices with respect to variables like shelf displays, pricing, and packaging. Tesco, one of the UK’s largest grocery store chains, employed a similar technique by creating a virtual shopping experience to gauge customer feedback before deciding how best to build a new store.0. 15 Killing The Focus Group: The Disruption Of Market Research Screenshot from System 1 Research’s VR shopping tool. Source: Martech Today AI is enabling new forms of sentiment analysis and observational research Market researchers are applying AI-powered tools to observational research, which looks at how a customer actually uses a product. For example, Karna AI’s Perceptron parses through videos of users interacting with a product and generates insights based on nuanced details that would be challenging for humans to notice. In an experiment to understand which beard trimmer worked best for a user, the data Perceptron collected included total time spent on trimming, the beard density in each part of the face, and the facial expressions of the user while operating the trimmers. Source: Karna AI 16 Killing The Focus Group: The Disruption Of Market Research Tracking real-time usage data is helping researchers collect better customer feedback Analyzing the usage of a product or service can help market researchers gain insights into consumer behavior on a continuous basis. San Francisco-based InterQ says that “capturing a customer’s feedback in the moment yields the most honest data.” For example, if a company wants to see how a consumer interacts with its mobile app, tracking real-time usage data will likely offer more authentic results compared to asking a consumer to recall an experience in a survey. This is another area where unstructured data can help marketers gauge a customer’s reaction to a product or service. Take retail stores as an example. Stores have CCTV footage for security purposes, but this is also being used to capture customers’ reactions to changes in store layout and different products. In addition, retailers are using this approach to help understand customer flow in relation to weather, the time of day, or during cultural or sporting events. Some are taking this approach even further. Microsoft-backed startup Cooler Screens, for example, offers fridges that detect shopper movement, serve up ads, and monitor inventory to measure sales results. 17 Killing The Focus Group: The Disruption Of Market Research GROWTH: NEW WAYS TO CRUNCH DATA WILL BOOST PERFORMANCE TRACKING Once a product or service is launched, market researchers track how it is performing in relation to competitors’ products in the market. This is done through brand tracking and comparing product performance. Businesses are automatically tracking competitors to dynamically adjust prices Companies often adjust their prices to stay competitive with rivals, but this process can be time- and effort-intensive. However, some companies are offering tools that aim to automate this legwork. Turkey-based Prisync, for example, is a subscription-based service that helps e-commerce businesses track pricing. The company’s tool scrapes selected URLs and lets businesses check a competitor’s price, know when a competitor’s product goes out of stock, and adjust product prices. Advertising is being revamped by AI-supported techniques Market researchers keep track of how the advertising for a company’s product is performing among its target audience. For example, AI company Affectiva relied on sentiment analysis to track the effectiveness of an ad campaign by Mars. The company used participants’ webcams to track their facial and emotional responses while they viewed Mars ads — data that was used to inform sales predictions. 18 Killing The Focus Group: The Disruption Of Market Research Companies also use natural language processing to identify celebrities and social media influencers who may be suited to promote products. For its Super Bowl campaign in 2016, carmaker Kia used IBM’s AI software Watson to identify influencers to be part of its campaign. Watson analyzed the language that social media influencers used on their profiles to identify those who demonstrated characteristics that Kia wanted to be associated with its brand — in this case, “openness to change,” “artistic interest,” and “achievement-striving.” Direct-to-consumer models are changing data collection Direct-to-consumer (D2C) brands have become increasingly popular over the last decade. Companies like Casper, Allbirds, and Everlane have pioneered new ways to reach customers, and this approach has allowed brands to build an even closer relationship with customers — and their data. Incumbent companies, especially those in consumer packaged goods (CPG), are now also launching D2C stores. Pepsi, for instance, started selling its branded products on its own D2C platforms in May 2020, while brewery group AB InBev is piloting an online store in Europe. While these moves are partly a response to the surge in online shopping that accompanied the pandemic, they mainly represent a way for these companies to gather better data on their customers and experiment with new products. This D2C approach could prove to be much faster in comparison to waiting to measure the performance of new products across retail partners, helping brands to be nimbler as they adapt to emerging consumer trends. 19 Killing The Focus Group: The Disruption Of Market Research Chatbots are helping to gather more consumer data A new wave of chatbots is helping market researchers engage users and collect information more effectively than alternatives. A survey conducted by CONVRG found that up to 80% of chatbot users answered questions, including open-ended ones — 3x higher than the responses received through email surveys. Questions that a business may ask through chatbots range from the problems a customer is trying to solve to their location. This is all useful data for a company to understand how customers are responding to its products and services. For example, data insights company Kantar uses NLP to conduct market research through conversational chatbots on platforms like WhatsApp and Facebook, as well as through voice assistants. Meanwhile, Sweden-based furniture company IKEA is using a chatbot called ORC to collect feedback from customers. Source: Smart Insights 20 Killing The Focus Group: The Disruption Of Market Research Market research will keep evolving as emerging technology enables new ways to collect consumer data. As people’s attention spans continue getting shorter, micro- surveys that are conducted through messages and chatbots are likely to take precedence over approaches like emailed surveys. The demand for gathering data from video will also become increasingly important, given that over 82% of consumer internet traffic is expected to be in video form by 2022, a 15x increase compared to 2017, according to Cisco. Additionally, market researchers could soon rely heavily on voice-enabled devices and virtual assistants to conduct research. Despite the promise of efficiency and new insights from these tech applications, challenges loom on the horizon. As improving AI tech enables market researchers to crunch more data, they will also need to reckon with identifying biases in their data and taking steps to ensure that automated approaches to analysis hold up against the real world. Additionally, growing moves from regulators around the world to give consumers more control over their data — through privacy protection rules like Europe’s GDPR — may prove to be a hurdle for the market research industry. But what could threaten incumbents even more is that these new tech-driven approaches to market research often require completely new skillsets. This could make some of the industry’s big players more vulnerable to tech-savvy startups, or cause a boom in M&A as incumbents scramble to keep pace with emerging tech and the talent needed to leverage it. The future of market research 21 Killing The Focus Group: The Disruption Of Market Research Market research companies are experts in understanding others. However, now more than ever, they will have to make sure that they understand their own blind spots to avoid being left behind. Clients can view 3500+ adtech startups in this Expert Collection. Expert Intelligence clients can check out our report on targeted marketing companies here. 22 Killing The Focus Group: The Disruption Of Market Research This report was created with data from CB Insights’ emerging technology insights platform, which offers clarity into emerging tech and new business strategies through tools like: • Earnings Transcripts Search Engine & Analytics to get an information edge on competitors’ and incumbents’ strategies • Patent Analytics to see where innovation is happening next • Company Mosaic Scores to evaluate startup health, based on our National Science Foundation-backed algorithm • Business Relationships to quickly see a company’s competitors, partners, and more • Market Sizing Tools to visualize market growth and spot the next big opportunity If you aren’t already a client, sign up for a free trial to learn more about our platform. Additional reading