Priscilla McKinney: I am so excited to have this expert on our show today. I was privileged enough to be invited to a Burke event, and the focus was on AI and technology and just a beautiful, beautiful, complex conversation. So you are absolutely going to love this conversation. But I have with me Mike Deinlein, and he is from Burke. He is the vice president of corporate innovation and he focuses on advancing new methodologies and integrating those emerging technologies and specifically AI into market research.
So you can see I’ve got the right person on the show. But he has a background in econometrics and advanced statistical analysis. I mean, I’ve got a smarty pants today. We also have in common that I’m a native Coloradoan and he’s joining me today from Denver. So, Mike, I love it. I love it. Welcome to the show.
Mike Deinlein: Thank you so much for having me. And I have to say it was wonderful to have a superstar in the audience of our event. So couldn’t thank you enough Priscilla for coming and joining us in Chicago and very happy to be here today.
Priscilla McKinney: Well, awesome. Listen, I have sat through some horrible AI presentations. And this was not one of them. This was so fun. I mean, like I you you you really had to be there for it. So if you get invited to a Burke event, say yes. But you did some really funny and informative stuff. It was so like pairing technology with really great use of some Mark Twain quotes. I was dying laughing in the audience. So kudos to you for making this really interesting.
But you know, I really think it’s the experts who can dial it down to make it a very complex subject, simple enough, but also translate it to what am I supposed to be doing today? You bring that analytical rigor and that innovative thinking, but really what you’re doing at Burke is saying, yeah, but at the end of the day, what should organizations know in order to bring AI to bear? So let’s start there, Mike. I this is going to be such an interesting conversation, but right now people are overwhelmed and business leaders are just bombarded with AI tools.
As we mentioned, they’re bored to death with dumb presentations. But where do you see from your vantage point that organizations are getting stuck before they ever even get to implementation?
Mike Deinlein: Yeah, I think that’s a great question. And I think one of the core challenges that we oftentimes see within this is a lot of our stakeholders are being asked by their stakeholders of do AI. And with the do AI mentality there, is let’s jump immediately to a tool. So how are you all doing AI moderation? Or how are you all doing synthetic data? How are you all creating agentic systems for us to be able to analyze all our data.
I think those are all great applications, but it really jumps past the core issue of what are you trying to solve? What’s your business question? And then how can we best solve those things? I think what oftentimes happens with that is they jump to the out of the box tools. So everyone has created a custom project in Claude or OpenAI, ChatGPT, whatever, Copilot, whatever it might be. And then they’re disillusioned with the results because they haven’t really done some of the hard work to understand what the business questions and then build solutions around that to make sure that you can, I mean, these are wonderful tools, but it is a tool and it needs to be wielded in the correct manner. So I think that’s one of the core things.
Priscilla McKinney: Okay. You know, I think one of the reasons why we have not been disillusioned with some of those tools is to your point, because we were very clear about the business question we needed to do. And at Little Bird Marketing, we have such solid written standard operating procedures. And so we were very clear about the tools that we’re bringing. We want to stay in our workflow. We want to stay in our process. And so we’re bringing those to bear.
But you know that when people take AI tools, you can start going down the rabbit hole of what that tool wants you to do. So I guess from your perspective, because you have had so many conversations like this with business leaders, why do you think that is so hard to convince them to really focus in on the right question or focus in on the business challenge?
Mike Deinlein: I think the shortest answer to that is I think oftentimes they don’t know what quite, I mean, even if we were doing a market research study and you ask someone, what business question are you trying to answer? They might think one thing, but then once you start getting into the nitty gritty of those conversations, you end up somewhere completely different. So don’t think people think of, you know, what’s the customer experience vision that we’re trying to create here? What’s our product vision, the innovation vision, whatever it might be, how do we want to target certain individuals.
Those are timeless questions that we have in the market research space, data space, whatever it is. And I think it is the hardest thing to articulate and people don’t spend as much time. I’m going to butcher this Einstein quote, but he said, if I had an hour to solve a problem, I would spend 55 minutes thinking about it and five minutes trying to solve it. And I think that’s really where people get lost is they jump right in.
And Priscilla, I think at Little Bird Marketing, what you just articulated there is you spent so much time in setting up that foundational bit of how we solve problems. What’s the strategic prism with which we want to put AI through that it’s much easier to be able to deploy those things? That’s what we oftentimes say is the pipes is what you need to get right. So what’s the data that you have? How do you want to connect it, ingest it, make it AI ready? And then it’s the fun stuff. It’s thinking about, now what are the strategic frameworks that we know and love? How do we make AI work through those? Because otherwise you’re getting a genericized large language model, which is powerful and helpful and all of that, but it’s not tailored to you. It’s not tailored to what you want to accomplish.
Priscilla McKinney: Yeah. And what you all do at Burke is so put a fine point on it and tailor it so specifically to someone’s business challenge. So that’s never gonna fly over at Burke. But I asked you specifically on because you are right at the cusp of the emerging technologies that are being considered and being used in market research. So let’s kind of start there. This is the big issue where we’re trying to separate the hype from reality.
And you just said it right there that people just dive into something they’ve heard about. And two things that come to mind to me is digital twins and synthetic data. I am sure you are swimming in that stuff. So talk to me a little bit about what you’re hearing. Let’s kind of set that context and then I’ll ask a few other questions.
Mike Deinlein: Yeah. So this is probably the number one topic that we get asked about within this AI space. And I think it’s so challenging because people use these terms in so many different ways. So I’ll use what I consider a useful framework for synthetic data. And I’m not sure if it’s going to resonate with someone else who’s out there who uses a slightly different framework.
But we oftentimes think about it in two lanes. So one of them is the large language model lane of synthetic data. So being able to basically have a tie into a large language model that can be very rich and very contextualized with a lot of different information, longitudinal panel data, behavioral data, demographics, all of that stuff. And that can get at the digital twins. We can layer in some good long form IDIs to be able to get more of the personality dimensions. And it’s a wonderful technology there.
But all of those large language model applications, I think are separate than what I see as quite powerful in the moment. And I’m not saying Digital Twins is not powerful, but I think that there’s the overall promise of Digital Twins and the application right now, there’s a little bit of a gap. And I think it closes every day because these tools get better and better and better. But I think the generative data models, which is math based, it’s not language based like the large language models is being able to uncover those underlying relationships that we see in whatever data file it is.
So certainly market research data that we’re dealing with every single day, but you know, we’ve been using similar applications for boosting and inferencing and things along those lines. They’re machine learning and statistical properties. But now we just have so much more compute and we have so much more powerful models at our disposal here. I was talking to one of our lead data scientists the other day and we were trying to do one of these generative data models that now underlies some of our synthetic data applications and he said it was gonna take six months to run and now we can run it in 15 minutes. It’s just wild, the advance and that all opens up to the same technology that opened up these large language models.
Now we can apply those to our large market research data sets and, you know, internal data sets, whatever it is, and be able to do some of these things that are synthetic data, but they’re very much math based. They’re not kind of those agentic personas for lack of a better term that digital twins and synthetic avatars or synthetic panels kind of live in, which again, I think have great application, but it’s hard to quantify and validate those where these generative data models, because they’re math based, they’re easier to kind of validate.
Priscilla McKinney: Yeah. In all of that, what do you wish people knew about these technologies that maybe you get tired of repeating yourself about or like say it once and for all on my podcast? Like what is the missing knowledge is there?
Mike Deinlein: Yeah, so I think the core aspect for particularly the large language model ones is that there’s an information density about these things. So there’s applications for low information density things. As I said, all of us have created projects in ChatGPT or Claude and said, you know, here’s a high level persona and maybe here’s a few bits of information here and it’s low information density and you can get some initial reactions to it. You know, what do you think about this new concept? So it’s a great brainstorming partner there.
But I think it’s really critical to have, if you’re really gonna make clear business decisions or well-informed business decisions, you need to have so much information density for these solutions in order for them to be as powerful as you want. You know, it’s kind of a silly example, but if I go and feed a bunch of information about myself, about my buying behaviors and purchase behaviors, and then I ask it, what’s my favorite cereal? And if I hadn’t fed that information, it might give a compelling answer or a seemingly compelling sounding answer, but I haven’t given it that information. So it’s not going to know that I love apple cinnamon Cheerios.
So it’s one of those things where the amount of information density that you provide these, the better they’re going to become, but you’re always going to have these evidence gaps that might arise. And I think that’s the core challenge. And I’ll say something like digital twins. Once again, I think it’s the digitization of data that is one, fast expanding, which makes it a fun time to be in that space because we’re gonna get more information that we can feed into these things, but there’s still information gaps. I mean, you think about the emotional complexity of humans.
And it’s really difficult to be able to digitize that in such a form that these digital twins are going to be as compelling as you might hope. So I think that’s one of the core challenges is if you don’t have the information in a digital format that these digital twins can ingest, you might have some information gaps. And then to go and fill those information gaps just to inform these digital twins, it kind of asks the question, well, why don’t you just go ask real people what their feelings are on that specific topic? It’s kind of this tension of what’s the best way to get at these answers.
But yeah, it’s a fun time to be in this space because I do think those things are showing more and more promise. But I will say that information density, it requires so much information. You even think about the large language models, and I’ll stop after this and let you ask me a question, but the large language models, they’re trained on the entire internet. So do you have the entire internet worth of information about Priscilla or Mike? You likely don’t. It takes a lot of time in order to get enough information to make a compelling Priscilla or a compelling Mike.
Priscilla McKinney: Yeah, and you and I actually talked a lot about this in person. As the VP of corporate innovation at Burke, you get to play. Like that is a part of your job description. And so I could see that in your presentation, that playfulness of like some things you had tried to figure out, even about creating a digital twin for yourself. And you alluded to a little bit this about, you know, cereal, which by the way, lucky charms all the way. I mean, come on.
Okay. I’m also St. Patrick’s Day birthday, so I think I’m obliged to say that. But you know, just for people to really understand this, that the playfulness that happens before you go and apply that rigor, tell us a little bit more about something you mentioned in the presentation about this kind of like awesome playground you did about trying to create a digital twin of yourself.
Mike Deinlein: Yeah, yeah. So I’m actually stealing a little bit of credit from one of my colleagues, Eli Moore. So he actually created this version of it. But it’s a very compelling example here. So one of the things that he fed for his digital twin was his entire Spotify history on who his favorite band was. Well, that was the question he wanted to be able to ask, is who is my favorite band?
And he fed his entire Spotify listening history because that seems like a good way to get it. That is if you’re listening to this band the most, that must be your favorite band. And what happened is that he was listening to Limp Bizkit all the time, not to embarrass him. I don’t know if that’s embarrassing for him or not, but it was because it was on one of his workout playlists and his real favorite artist here is Billy Joel. And what was happening is that it took a shortcut, which large language models and all these things are very good at doing. They’re lazy. They’re inherently lazy of how can I take the shortest answer to be able to answer this question?
So it looked at this is his most listened to artist. It must be his favorite artist, which is a completely different question. And that favorite artist comes with emotion. And I think that’s what’s really difficult to digitize. And maybe that changes. You think about wearables and all of this is, you know, when he’s listening to Billy Joel, his heart starts to flutter a little bit more than when it’s at Limp Bizkit and those sorts of things can start informing.
You know, Noah Yuval Harari, who did Sapiens and Nexus and all of that, he has a great bit on, I think it’s in Homo Deus, his book Homo Deus, where he talks about that. And it’s scary to think about how much information is gonna be digitized about us in the future. But it’s also compelling to say, okay, here’s how we can know what gives us an emotional response. At the moment though, we’re relying on shortcuts and proxies for very emotional decisions that oftentimes we make.
Priscilla McKinney: I love that. And I think it’s such a good story. It’s not just interesting, but now when you really understand what happened there, you can extrapolate it out to understand where the rigor gets lost when then you’re trying to use this emerging technology in market research. And I was talking to you then about this. You know, I’m 450 episodes into my Ponderings from the Perch podcast. And then I’m like, I don’t know, 75, 80, 100 pod episodes into this. And then I wrote a book and I actually did the audible version of it. So it’s me speaking that in.
I have 12 years of blogging prolifically. You know, I have 21,000 followers on LinkedIn. So the amount of LinkedIn posts that I’ve written, I look at my website and everything that’s there and the long format papers, all this stuff has been written by me. But to your point, that’s professional Priscilla. And that yes, it does capture my goodness, that’s tons of content to train the model on and is this useful for me in a work situation? And does that make my Claude sing for sure? But it still will not come up with lucky charms. And I think that’s such a good thing to really understand.
You know, what we’re trying to do moving brands forward is to get them closer and closer to the heart of the consumer. And the heart of the consumer is not always in those things. And we have in market research always relied on that expert moderator, that human moderator, to intuit when someone just placated them with an answer and to dig deeper. And we all know that there’s really bad moderators and there’s amazing moderators, right? And so that’s something to really keep in mind, you know, with AI and as we think about this as an emerging technology.
Mike Deinlein: I love that too. I mean, thinking about the human moderator as another tool that’s coming out and is getting a lot of buzz in the market is the AI moderation. And I was actually just up in the great white North earlier this week talking to a client and we were having this conversation and actually we did a mini debate facilitated with AI to be able to say like, is AI moderation something that is compelling and something that we would want to do?
And one of the points that we came up with is there are certain places where the empathy of a human moderator cannot be touched. And I will say that full stop. I do not think there are places there. But there are also very sensitive topics that people might not be comfortable talking about. You think about healthcare and maybe I’m not taking as good of care of myself as maybe I should. Or finances are, I think, the predominant one where people feel so much shame about being in so much debt, I don’t know what to do. And sometimes it’s very difficult to talk to a human with that. And I think some of these AI tools might actually open up more empathy.
So it’s one of those, once again, where there’s no simple answer to this. And it needs to start with that business question of what are you trying to uncover here? And Burke’s never gonna say, go and use this tool full stop. Hey, there are places for human moderators that you absolutely need their level of empathy and ability to pivot in the moment, but there might be some solutions that AI moderation, if you need that scale, it’s wonderful. It’s so great to be able to have that level. Or if you need a safe space, maybe a robot actually gives a safer space than even the most empathetic moderator.
Priscilla McKinney: Yeah. I think that is so interesting. As a cultural anthropologist, I think about that, you know, that the human connection is one element of it, but we have interesting tacit agreements about how we’re going to interact with each other. And so that is the question is does that hold us back or does that will that push us and help us lean in? And that is the question that we need to ask in every situation.
But with everything that you’ve seen, you’ve really been able to not only play, apply rigor, but then talk to people in the real world about it and make some business decisions. Has there been something in there that you could share that’s really specific or like an example of where synthetic data or a generative data model of some kind actually has changed how a brand has made a decision or informed a brand in a unique way? And I know you probably can’t share specifics. But you know, we want to understand how the technology is really reaching into this.
Mike Deinlein: Yeah, so I think if I go to the, once again, that bifurcation in between the large language models and the generative data models being more math based, I think one of the challenges that we’re seeing in the market and it continues to come up year after year after year is sample quality. And I think that’s one of the things that people are kind of rushing to these large language model solutions is, okay, well, if I can’t trust the human data collection that I’m getting over here, is this synthetic avatar just as compelling.
And I think another way to think about that is if you have a sample budget, go and collect the 300, 500, whatever it is of the highest quality individuals you can imagine there. And maybe you would otherwise have chased the base size of n equals 1000 or whatever it is. But now if you can leverage some of these really robust math-based models to be able to expand that, it can be really compelling, but you don’t have all of the dirt and dust that comes with some of the sample constraints there.
And Burke prides itself on the highest possible sample quality. But taking that to the next level and saying, okay, maybe you’re going to pay 3x the cost to collect these really high quality individuals, but making sure that you know that’s a real person on the other end. And then you’re just leveraging really high quality math in order to build that out. So we’re doing that a lot with our clients these days is to be able to say, let’s actually look at the comparison. And we’re actually doing an R&D study right now on this. Let’s look at that comparison in between what we know is high quality sample versus low quality sample. And then what if we just boosted the high quality sample? And would we get better actionability and decisioning there?
I do think, I mean, that’s kind of a genericized example there, but we have a number of clients who are doing these generative data models with us to be able to fill in. I’ll give an example for a large tech company. And as you said, I have to be high level here. But one of the things that we found is there’s a survey limitation. We can’t ask people to do five brands at a time because they would be on the computer for 20, 25 minutes in order to fill that out. But what we can look at is the response profile for each of those brands and say, who’s missing in this?
Maybe one brand had, just because it was random selection, that doesn’t mean equal selection for all of those brands. So maybe there was one brand that just by the vagaries of randomness got more individuals in their familiar bucket or their user bucket. And that’s going to discount the results of a brand that was only among those that were aware, more heavily skewed to those aware. So you can leverage some of those generative data models to flatten that out and be able to ensure that it’s not just a product of who rated something, but it’s actually a real reflection of that brand.
And that can have very real implications for your key driver models and the decisions that you’re making. And what is our realistic competitive position within this, is there white space in the market that we want to capture without being able to do that generative data, those insights could be lost.
Priscilla McKinney: Yeah. And that goes to the same point you’re making that you can’t just slap an AI tool on if you don’t understand the mathematical rigor that is underneath those decisions, right? Because you wouldn’t know if it’s wrong. If you don’t know what to inspect, then you really can’t expect the answer to be right. And I love that it is an anonymized example, but I think it’s a very good one. And it is a very common problem that people are having. Just side question. I am curious if something has just surprised you in this.
Mike Deinlein: Yeah, so one thing that I will say has surprised me is how adept it is at going through those strategic prisms. So we’ve spent a lot of time in setting the foundation. What is the data foundation? What are some of the high level architectures that we have here? But I would say AI is only getting better and better and better at being able to take on the role of certain strategic agents. We want to look at this through, you know, this jobs to be done lens, or we want to look at this through, we have an entire innovation framework and being able to set agents that are going to go through each and every one of those points.
Same thing with our CX gravity points, which is being able to apply the moments that matter across the journey, those that really shape the experience and have more of a gravitational pull around it. But being able to set all of these specific agents and use it as that expert led thought partner. I think AI is getting better and better and better, but all of that does require the expert led piece. And I think that’s one of the things that has surprised me is it can do a pretty good job when I give it enough context of my knowledge base.
But I think the gap is quite wide when you don’t provide that level of context. So I think that’s actually one of the core things that’s really surprised me is just that gap that oftentimes arises in between no context and great expert-led context. What you get from AI can be quite different.
Priscilla McKinney: Right, right. I could not agree more. There’s some people are like, well, I tried it for content. It doesn’t really do a good job. I’m like, nope, you didn’t try it hard enough, let me tell you. I love it. I love it. Okay. Well, let’s end with this. If you could get every business leader in a room right now, what is the one thing about implementing AI in market research that you wish they would either start doing or stop doing? Like just get it off your chest, Mike.
Mike Deinlein: Yeah, I would say ask your question first. Think about your question. Do Einstein. Think about it for 95% of the time and say, this is what I want to do. And then figuring out how does AI unlock new solutions for that business question. So I wish there would be more time spent on the business question and less time spent on the tool because as you know, the tools change every single day. So you can spend as much time chasing one thing or another. And it’s going to be yesterday’s news in very short order. So I really do wish that more people would think about, how do we do AI, but how do we solve this business question? I think that would be wonderful if we would spend more time doing that.
And I’ll give a very quick example. We had the opportunity to do a great project with Microsoft. I can say that because they presented at SMR, not this past year, but the year before. And they do a lot of pricing research. And one of the questions they had is how would you do pricing research with Gen AI? And one of the things that we came up with, do conjoint and discrete choice and all that fun stuff. But they also do Gabor Granger just to get these initial pricing curves.
And what we, the question that we kind of uncovered there is, well, Gabor Granger is a pretty artificial exercise. You never ask someone, you know, Priscilla, would you buy this box of Lucky Charms for $3? And if you say yes, I wouldn’t say, well, how about $4? How about $5? So another way that we thought about asking that is, well, let’s make that more of a conversational experience, but we can still extract their maximum willingness to pay from that.
And then another one that we started exploring with them and we haven’t implemented it yet, but how do you actually make that a negotiation? So saying Priscilla how much would you purchase this box of Lucky Charms for and when you give me an answer then I say I was thinking more, you said three dollars I say five dollars and then we can leverage AI to negotiate from there and you just get so much more rich contextual information by doing that. And we couldn’t do that. We were limited by Likert scale questions, you know, just a short time ago, but generative AI really does unlock those new solutions where if you start with what is the question, what am I trying to answer here instead of saying, let’s do AI, I think we would have a lot more opportunity to explore some of these fun applications.
Priscilla McKinney: I love it. And now I just want to go eat Lucky Charms. That’s all. I love it. I love it. Well, I’m telling you, excellent presentation. I think you’re providing value there. We’re all learning, even with the rigor and the expertise you have. It just shows that you’re still curious about it and that there’s always something new around the corner. You heard on this episode Mike is super approachable. Definitely try and find him. It’s Mike Deinlein and it is Mike D-E-I-N-L-E-I-N. If you want to find him on LinkedIn, he’s with Burke. So Mike, thank you so much for sharing your expertise.
Mike Deinlein: Absolutely can’t thank you enough for having me, Priscilla. This is wonderful.
