Boost Your Sales With Incrementality

[00:00:00] Taylor Holiday: Welcome back to another episode of the Ecommerce Playbook Podcast. I am excited today to be joined by the Chief Strategy Officer from Measured, Nick Stolz. Nick, it is a pleasure to have you on the show.

[00:00:13] Nick Stoltz: me. I'm excited to be here.

[00:00:15] Taylor Holiday: So we have spent many years. Talking about various forms of measurement. It has clearly and certainly been a hot topic in the e commerce world over the last let's say three to four years, these things tend to go in cycles.

I think some people would be shocked to find out that measurement is not a topic that is new to the advertising world, but it is certainly one that of late has been very top of mind. And we have been, I would say very careful About making a selection around partners and vendors in the space, if anything, I think most people would find us to be critics of a lot of the solutions more so for me in their attempt at presenting themselves as a panacea solution to all things, measurement problems.

In our journeys, we get the opportunity to work with brands who have a lot of these partners in place. And one of the ones that we have come to really appreciate for both the quality of the product from a technological standpoint, the UI, the information, but also the process. And I would say, Oh, in some cases, the humility with which the information is presented is measured.

And so we are excited to be announcing a larger partnership with you guys. And so, Nick, I wanted to bring you on and have a discussion around what is the current topic du jour around measurement, which is really about incrementality and measured role in that journey. So I'm excited to have you, man. Can you start out by just giving us a little bit about what measured is as a product and what it serves for its customers?

[00:01:43] Nick Stoltz: Yeah. I mean, you mentioned measurement is not a new topic. I've been here, you know, the, the CEO, Trevor, that, that I've been working with for a long time. We've been in this space for 15 years. All of these problems are not new, right now, what we saw, you know, Trevor started the company seven years ago.

I, I joined five years ago. We saw a bit of a gap. Out there, and it was around this concept of incrementality, right? Whether it's the challenges around user level attribution, whether it's the facts that like, look, mixed modeling is a great approach, but it has some gaps around agility around correlation is not causation, things like that. Incrementality solves a lot of those problems, right? So what measured is purpose built for is to allow marketers to. Understand and ultimately optimize their media mix net of incrementality. And we are trying to bring that into the broader measurement suite, right? It doesn't replace him in them. It, you know, we can talk about user level measurement that's got its own challenges, but it is a lens that is really important to bring to how you understand what your media is doing. Where you invest it, where you are potentially wasting dollars and how you can take those dollars and move them to do, you know, the more difficult work of, of driving awareness and driving new customers into the funnel and things like that. So, look, I'm really excited about this partnership too. I'm excited to talk about it today.

And I appreciate the kind words and, you know, look, I will say this because you make a very good point about a panacea. There's no product, there's no model. That is going to solve all your problems, right? Those are tools and they are extremely useful. They require your managerial expertise and they require you to build an organizational muscle around how you understand what your media is doing at all times, how you understand what it did yesterday and how you understand how that's changing. I think part of what measured goal is to do. And part of the thing that we're pretty eyes opened with, with brands is like, look, We're going to bring you some tools and then we're going to work with you to develop that muscle that you need to really kind of create a competitive advantage because everybody else out there is using a bunch of tools.

I'd like to think we're trying to do something different and I hope that comes through in our experience you know, working together on a few brands.

[00:04:01] Taylor Holiday: Yep, giving you a hammer doesn't make you a carpenter. It gives you the capacity to begin to learn how you might use it to become one. And I think What we want to do today is talk a little bit about what people can do with these tools and how they can begin to use them effectively. Cause I think there are so many different ways and I've seen so many different versions of these things playing out, but maybe let's start with establishing some shared ground around terminology, because I think it's important.

And these things can have wide ranging meaning. So I want to start with Incrementality, if we can, and just try to together define what do we mean when we say incrementality as it relates to specifically, let's narrow the term into the confines of e commerce measurement solutions. What are we saying when we say incrementality?

[00:04:46] Nick Stoltz: Yeah, fundamentally what we mean when we say incrementality, it is saying, you know, how many of the sales that I had in the last 30 days. Only fundamentally happened because of media that I serve to a consumer, right? So this is not some modeled estimation. This is not based on pixel level tracking. This is not based on the cappy information that you're sending to Facebook. This is based on, I set up a very careful test and control experiment. That control group got business as usual media. That test group was withheld some media, all media we can talk about what, and I saw a drop in sales. That drop in sales is what is being incrementally driven by that media. And that is a causal observation based on, you know, real world.

I turned off media and I saw my sales drop therefore that those sales must be being driven by that media. So that, that's what we mean by

incrementality. 

[00:05:45] Taylor Holiday: Yep. And so simplified down even further, it's purchases that wouldn't have happened otherwise. Right. And. Now, I want to push back a little bit because you said something there that I think it's important to understand the design of these things. I know this has been a learning process for me over the last few years around that it's not modeled, but what I would say is that the estimations of how many sales would have occurred otherwise are modeled, right?

So the most, the most common form of this is a geo holdout study. Is that fair to say that that's the most common test structure that's being used to determine 

[00:06:17] Nick Stoltz: I mean, look, ideally in a perfect world and I'll use the most obvious example of this, like, look, when people bring drugs to market, they're They create a randomized control trial. Thousands of people in this group, thousands of people in this group that, that look as identical as possible. This one gets the drug.

This one gets a placebo, right? That is ideally how we would like to do this. The reality is that those are near impossible to do in any marketing environment, right? You know, the, the, you could argue that maybe Facebook conversion lift study does that, but look, they're doing on the back end underneath the hood and who knows who opted out and opted in like. Maybe they're getting close to that. Maybe they're not. You as a marketer, if you want to own this, and I would argue that you do need the ability to own this. You don't have any reasonable ability to set up that randomized control trial, right? So what are we left with? We're left with what we're going to call is, is, you know, a reasonable approximation of that control. Okay. We're going to say, Hey, historically, if we look at our sales performance and we look at a basket of geographies, state DMA, you name it. Okay. We can find You know, baskets of these geographies that look, behave similarly, right? Which makes sense, right? There is obviously some variation and how customers respond in Florida versus Texas and things like that. But there is some consistency with how you are marketing to people. You're broadly, you're broadly marketing nationally. You're broadly marketing to the same folks. You're broadly pushing out similar messages. So what we do is we very, very, very carefully. Select a handful of test markets and a handful of control markets. We look at how those markets have behaved over the past few years. And we essentially build a model that says, Hey, if I know what's going on, if I know what sales you've got last week in Florida and Texas and California, New York, I can pretty accurately predict. And I can show you week by week for the past two years that I can predict sales in Mississippi within 1%. Okay. Now that's not perfect. But we show you that number, and that allows us to kind of say, okay, given that I know I have that accuracy historically, I can set up a test, I can do something different in Mississippi, I can leave California and Florida and Texas the same. And then I can predict what I should have seen in Mississippi and say, Hey, did I see a drop in sales?

Did I see an increase in sales? So you're right. This is not a perfectly randomized control trial, but it is about as close as we can get. And it is a powerful tool. And because you know how accurate you are historically, you can put a confidence bound on what you're seeing, right? If you know that you can predict sales within a percent and you see a 5 percent drop in sales, when you turn off Facebook prospecting, Look, we can attribute that amount to Facebook prospecting and we can put an error bar on it because we know how accurately we've been able to, to predict historically.

[00:09:13] Taylor Holiday: That's great. So the important things to understand here, and there's a lot to unpack about this. And I, when I talk about approaching these things with humility, I think it is really important to acknowledge any of the limitations that may exist in the process. So I appreciate you doing that. And so we have to depend on our ability, like you said, to accurately be able to predict the sales in a geographic region.

Now, one of the things that's hard about testing is a reality is that in many cases, The smaller brands suffer a lack of data that allows them to accurately and consistently progress, predict their themselves. And so there's sort of always an inverse relationship between the amount and the consistency of the data.

And the predictability of the data, or sorry, the positive correlation between those things. And so small brands sort of struggle because they, they may not be able to predict within a 1 percent in a region because the samples are pretty small. There's high variation, et cetera, et cetera. So there's one of the things I always tell brands when they start with us is that how accurately we're able to forecast your business is often related to.

The consistency of the data itself. And so there's some relationship here always within each brand. And this gets to a conversation that I think is really interesting, which is there's sort of this general idea out there that everybody should individually test their own data to get their own individualized results across all of the, the channels to get to what is uniquely true about you.

I, I, I, in some ways accept that that could be an optimal solution, but there are limitations to that reality in ways that I think small brands suffer from. And I think that there's more collective wisdom from my experience across these brands than we realize. So what I want to ask you is, is there any collective wisdom in your experience across doing this for so many brands that I'm not going to say are universally true, but are probabilistically true about what you've seen in terms of incremental results across the core media challenge channels that we all know of in e commerce.

[00:11:19] Nick Stoltz: Look, that's a great question. And to your point, I mean, look, the deck is stacked against smaller brands in a lot of ways, right? You don't have as much money to spend on this stuff. Your Facebook algorithm doesn't have as many observations to train on. You know, you don't have as much money to spend on, on media channels and other things, like you don't have as much historical data, like it is very important. Again, the smaller you are, the more you are kind of an up and coming business to leverage managerial expertise, right? Look, collect data. You want data. You want to build up a great data set. You ultimately want to give yourself the ability to get into a lot of this advanced stuff, but there is nothing that's going to serve you better than early in that journey than managerial expertise. And some of the things that I do think are, are pretty true across the mix, right? So let's kind of talk about this. Where do brands generally start with kind of measurement, right? They start with what they have. Okay. So generally speaking, brands are going to start and their media mix is going to look like it's going to start with probably ad affiliate. Email and Facebook, right? Like, those are kind of the layers that you start to track on, right? What is the kind of measurement or signal that you're going to get with that stuff? Well, typically you're going to have basically what's available in Google Analytics and you're going to have what the platforms report.

Facebook ads, right? Ads manager. You know, reporting from your affiliate or your email provider. You know, what you see in Google Ads and Google Analytics. Thanks. Now look, the smaller you are and the more condensed your media mix and the lower your national awareness, the more that stuff is going to be reasonably accurate, right? It's one, as you start to get bigger, as you start to have some baked in intent, as you start to have more returning customers, as you have to start more word of mouth, as you start to hit people across multiple channels, that's where this stuff starts to break down and become more complicated. And that's where you have to kind of bring in these more advanced techniques. To sort these things out right now, generally speaking, as you are going across that journey, okay. And think about as you're sort of putting this together through that, not all reporting in Google ads or Facebook ads manager or anywhere else. is apples to apples. Sometimes it's apples to oranges.

Sometimes it's apples to dump trucks. Okay. And the reality is that when you see attributed conversions within those platforms and you see one coming from a campaign against existing customers, you see one coming from a campaign that's targeted at website custom audiences. You see one that's targeted exclusively at prospecting with existing customers and site visitors excluded. Those are not all worth the same, right? And it makes sense intuitively because those things happen at different points in the funnel and different points of intent. Right. So what you can do is you can apply some discounting in your head based on funnel position, right? And I'm kind of talking Facebook, but the same thing applies for branded search versus non branded search, for example, right? Somebody who searches for your brand probably has more baked in intent than somebody who searched for t shirts and then ultimately bought Your brand of t shirts, right? You know, same for affiliates. If you offer someone a discount at checkout versus, you know, a piece of content that you're running that, that, you know, compares your shirts favorably to somebody else's, right?

Like you need to think about sort of baking in some fudge factors for that intent. And you know, look, there are general rules of thumb, right? You need to figure them out for your brand, right? But like, look, if you look at what Google analytics tells you about non branded search, that's probably a pretty accurate view of what it's doing, right? If you look at what it tells you for, for branded search, look, it's probably giving itself credit for like. two extra conversions for every one it drives, right? You know, there's a similar kind of logic around Facebook prospecting versus Facebook retargeting versus Facebook existing customers. So, you know, look early on, do you have to like really, really split hairs and say, Hey, should that discount factor be 33 percent or 25%? Probably not, but you do have to kind of take that into account. Okay. And sorry, let me, you go ahead.

[00:15:52] Taylor Holiday: No, well, so I want to stop because you, you said so much there. That's really important. Okay. There's a, there's two things that I want to focus on. One is I hear a lot of people talk about incrementality at a channel level. And I think this is really problematic because there are so many different ways in which you can structure these channels that would completely alter the outcomes of the results.

So you did the obvious one that we all know, which is Google. If we looked, if we said. The incrementality of Google as a channel. Well, that is like a nonsensical way to approach that problem. And you brought up a very specific distinction between non branded search and branded search which is like the classic place.

A lot of people start this journey is to understand the incrementality of branded search. And and I'd say even more recently, The bigger question that we run into on an organization that feels like an even more ambiguous solution is PMAX, right? Is trying to understand because they've sort of put brand into that in a way that now trying to understand the actual impact, it's been the hardest ad product to make a decision about performance on that I've ever experienced, but you said something here that I think is.

When I think about general rules and you have a baseball, I'm pointing down the logo there. So I'm going to use a metaphor that I learned about. I was, I got really deep into fantasy baseball for a long time and started really trying to understand how they forecast player performance and using predicted for stats.

And one of the things that you learn is that when a player is a rookie and they have very little stats about their own individual performance. We can't use that data to predict their performance in the MLB very well. And so what they do is they regress their predicted performance to league average.

Okay. And that is actually the best mechanism for project projecting a new player in a system.

[00:17:36] Nick Stoltz: player, 

[00:17:36] Taylor Holiday: So if that's right, exactly. So if we apply this logic, what, what I want to offer to some of these brands is to say, what is the league average outcome that we can trade on as a collective wisdom, not to assert that it will definitively be true for you, but to try to start from a foundation that we have to make decisions all the time in a predictive fashion, without the specific results.

We're doing this all the time and we should do it with good information. Okay. And so you said something there that I think is a, a way that I'm not going to hold you to it, but I, I, on, in my organization have to make decisions like this all the time, which is. Branded search, applying a 25 to 33 percent incremental value waiting as a default starting point to that consideration until I get better information, that kind of thing, I actually think is immensely valuable for people to begin to process, to help them make better decisions in the ad account, acknowledging full well, that it may not be the specific result that you end up with, but, and you guys do this when you start with brands, you apply weighted considerations.

For the tool generally, can you talk about a little bit about that? So a brand joins measured, I get some application of value based on historical learnings before I get my specific weightings of incrementality. Is that 

[00:18:48] Nick Stoltz: I mean, look, we actually use a very sophisticated way to kind of predict it before we go in. Now that comes from a couple of different things that, that I think brands early in the journey are going to have trouble doing, you know, look, we're, we're typically looking at a couple of years of history. And looking at spend and clicks and impressions and post click conversions and post impression conversions across all of your channels and all of your campaigns. Right. And we are working with brands that have. A couple year history minimum across probably five to 10 channels, right? That essentially allows you to build what is a media mix model, right?

How much of my spin moved? How much of those things move things together? Like, look, it gives you a quick out of the bottle box starting place. And that's a great place to essentially say, Hey, this is what I think is going on is where the data is pointing me. There's a second place that we look. You know, we run thousands of tests. Those tests are across all the places you would expect. I mean, most of them are on Facebook, Google, Pinterest, snap, Tik TOK, open web, CTV. Like that, that is 99 percent of our tests. We also work with a lot of brands that, that, you know, we're heavily concentrated in a few industries, right? So we have a lot of what you would call retail consumer brands. We have a lot of brands of different shapes and sizes. We have a lot of brands of different types of awareness. If we know those three things, like, look, you know, we see a lot of tests that here's what brands like you tend to see now, is that your answer? No, but to your point, It's probably a more informed data point to kind of start, right? So, you know, we're kind of doing what you talked about with, you know, the fantasy baseball projection. You know, we're probably working with, hey, but we have the luxury of saying, hey, here's a player in year seven. And we've got pretty good data on, on how he's performed for the last five or six years. That gives us a pretty good starting point of what we think he's going to do next year. And then the testing really allows you to kind of go in and really, like, observe that, right? And then, look, that observation gets fed back in. And so it

allows you to have a pretty sophisticated I'll get geeky here. It's like, look, I've got a lot of data and my hypothesis is that your brand search works that way. Let's go test that hypothesis and see if it's pretty close dead on or Hey, it's way off because you work differently.

[00:21:15] Taylor Holiday: That's right. That's right. So I love that. So, so I'm going to do something now that may make you uncomfortable in that. I'm going to assert some things based on my observation of incrementality. And I want you to tell me where you would agree or disagree relative. And I have, I have right now, the measured dashboard up for one of my customers that I think is a pretty clear indication of what I've seen.

Generally as a result across these channels. So. Number one the highest incrementality factor that I see repeatedly over and over is in meta prospecting with a click optimization setting. So in terms of the, where I see the highest incrementality percentage relative to platform reported result, where the platform ends up under reporting impact.

Most often is if you have a click based optimization setting on pure prospecting, that is the, that is the thing I've seen most often. How would you feel about that statement?

[00:22:12] Nick Stoltz: So let me, let me put it this way. So, so platform reporting, the way meta does platform reporting and Google does platform reporting and Tik TOK does platform reporting is very, very different. You know, Google has Cappy or Facebook has Cappy set up that they see more clicks and track those back better than other places that may not happen. Right. So when you look at like, look how accurate is platform reporting that there's some variants across those, but let me, let me tell you what I do think is true. You know, your Facebook prospecting program. Is probably one of your more efficient performers. It is also probably driving the, a pretty large incremental, like total contribution to your business, like from a volume and efficiency perspective for most of the brands we work with and where they are in the journey. Facebook prospecting is probably like your workhorse. Okay. And that much I would say is definitely true now. And this is where I think, you know, a lot of the problems we're, we're really lining up to solve for, for brands that are trying to hit the next phase of growth is like, you know, Is Facebook prospect going to get you there?

Well, no, not necessarily. And it has to be an important part of the mix, but you may need to work into other stuff to feed that workhorse. But, but generally speaking, I would say this, it is probably the, the across our portfolio, it is number one in media mix on average, across our portfolio. If you look at its efficiency, whether that's CPO or ROAS basis, it is like top 20th percentile consistently. And it is that put those two things together. And it's probably number one on increment, incremental contribution to sales. There's some variation in how accurate Facebook's reporting is relative to those numbers that, that can get pretty specific.

[00:24:08] Taylor Holiday: Well, yeah, and that, that for sure. And I think that that differs on a campaign level. It differs in lots of ways. I think the main thing I see repeatedly is that there's a pervasive narrative that meta over reports. And I, I would just contend primarily with people that that's just a function of the attribution setting they're choosing, which is inclusive of view.

But when it is click only, it actually more often than not under 

[00:24:30] Nick Stoltz: Yeah. If you're looking at one day, quick reporting meta is, is very like,

[00:24:35] Taylor Holiday: even seven day click, even 

[00:24:36] Nick Stoltz: yeah that, yes, that, that is, if you're looking at one or seven day click, especially for brands that do not have massive national awareness,

[00:24:45] Taylor Holiday: yes.

[00:24:46] Nick Stoltz: almost certainly nearly accurate or under reporting.

[00:24:50] Taylor Holiday: That's right. That's, that's what I see repeatedly. Okay. The second thing I'm going to read down the incrementality factors for this specific brand that I'm looking at again, this is one brand. This is, but I think it is directionally consistent from what I want to see. So by channel paid social incrementality percentage, which is for your guys's reporting the percentage of applied revenue relative to the platform reported number, correct?

That's, that's 

[00:25:16] Nick Stoltz: Usually on a seven day, 

[00:25:17] Taylor Holiday: important to 

[00:25:18] Nick Stoltz: usually on a seven day click, one day view. But

[00:25:21] Taylor Holiday: In this case, we have it set up at seven, eight clicks. So the percentage is applied against our setting, which again,

it's different. So paid social 175%. Okay. P max 50 percent branded search, 33%. Display ads, 32 percent direct mail, 29 percent SMS, 23 percent affiliate 15%.

Okay. This is, and, and what I, what I think this is directionally useful for, and this is a large eight figure brand is that There are channels by which the default dashboard is over reporting and those where it may or may be tighter or, or under reporting. And it's really important to understand the default setting of each channel.

So Google as an example is 31. 1, 30 day click, one day engaged view, or and then one day view. That's a much broader setting than meta at seven day click. So when people talk about channels over under reporting, a lot of times it's just like a failure to rel to understand the relative comparison of attribution.

But I want to focus in on PMAX because this is one that I think has become really confusing for people relative to the expectation. And I've seen results all over the place in terms of PMAX. Now, caveat that PMAX could be structured and set up a million different ways. So even saying PMAX, It's problematic to apply a generality to, and I acknowledge that.

But what are you seeing from that specific campaign structure and how are brands using your tool to help create more understanding in that space?

[00:26:51] Nick Stoltz: PMAX is really tough because number one, they don't necessarily tell you how to set it up. Number two, they don't necessarily tell you what it's doing. Now you get reporting, but you don't know where that reporting is coming from. You don't know what makes a PLAs, what makes a brand search, non brand search, open web display, impressions, YouTube, you name it.

You can control that somewhat, but you don't know. It is when you say results are all over the map, that is 100 percent how I would describe it because of all of those settings and because of the way that that algorithm can essentially place across all of those different properties at Google, we have done something like 200 P max tests. Now, again, I'm gonna get a little nerdy here, but when we do Facebook prospecting, we see a normal, nice distribution of test results. You know, if you're on the high end or the low end, I can probably make some guesses as to why, but it's a nice normal distribution, a nice curve. Put those 200 PMAX tests up, it looks like a scatter plot.

It's like chaos. So there is no relative consistency to what it is necessarily doing for you. And that's on the good side and the bad side. We see results where it looks fantastic and somewhere it looks You know, very mediocre to poor now based on having 200 of those and actually going in very, very deeply and trying to understand where and how do you drive success. I would generally say there are probably a couple ways that you can set this up to be successful and probably one big stay away. Okay. I would say this for smaller brands where you have the comfortability to kind of let Pmax do everything. Generally speaking, we do see that it's pretty successful at that. Now the problem is that is a lot of faith and that's a big black box. And you are probably going to need to test that to verify that it is indeed, you know, driving incremental conversions and incremental sales for you if you let it do everything. Okay. The second place where we see it effective is if you set it loose on very specific corners of the world, right? I'm going to give it assets and keywords that specifically go after non brand shopping and, and, you know, non brand shopping and non brand search terms. You know, I'm going to very specifically focus it on these product lines. So like basically kind of saying, look, I don't know what you're doing, but I'm going to put you in a box.

Cause I know if you're in that box, like, I know like what area you're generally playing. The third one, and this is where I think it's tough because I think this is the way a lot of folks set it up and this is where you will see results all over the map. But most of the time when you see results are poor, it fits this category. I already have non brand search, brand search, brand shopping, non brand shopping. I'm essentially using all the Google properties and I have all those campaigns and I set Pmax Loose on top of that to essentially compete with all of those things. And where it's competing and how it's competing and how those things are stepping on one another and where who's getting prioritized in the auction and how it's spending is incredibly difficult to discern. But when you set everything up in competition, when you look at Pmax as a fundamentally different thing and layer it on top of all those other things, it's very likely you are getting. Subpar outcomes. That is the one thing that we have seen pretty consistently. And that's look, I don't blame advertisers because that's how it was sold.

It's a new product. It's like, no, it's a new algorithm that is selling all the existing products. You have to be aware of that. And by the way, there are very complicated auction rules about who wins when you're two different campaigns are competing against one another.

[00:30:40] Taylor Holiday: That's great. So, so this brings up one of the, I think the questions that I've found hardest for brands to figure out the answer to, which is this. I run a test, an incrementality test. I get a result back. P max is 50 percent incremental. What now? Okay. And I'm going to give you a couple of the pathways that I see people take.

Oh, okay. We're going to apply that weighted factor To the consideration of the return in that channel and have an IRO S if you will, that just functionally takes the existing platform report and cuts it in half. And we're going to make decisions based on that metric. The second, so that's sort of like, Oh, okay, cool.

Now we have an, a weighted factor to our thing. The second path is, Oh, that's a problem with my campaign setup. I need to design a more incremental campaign. And they will play with the structure and settings. Of the campaign and rerun the test until they get to a place where the campaign is generating better incremental results.

Which action do you think is the better ones for brands to take in that scenario?

[00:31:46] Nick Stoltz: So. I'd like to take a step back and, and I think this will resonate with you, how many advertisers that, that you work with, if you say, Hey, what ro, forget what you need to see in platform. Let's just say you saw a real ROAS number. Like what is the target that you need to get right? Whether it's a P max campaign, Facebook prospecting existing customers. And look, I don't know if you have this experience, but I get a lot of blank stares and about like a third to a half the time when I asked that question. And I think it comes from the fact that not a lot of marketers have actually sorted through and talk through like, look, exactly what type of efficiency and return do I have to achieve?

What's that based on? Is that based on finance and marketing agreeing on an ACOS or a MER? Is that based on some sort of LTV to CAC framework and CAC payback? Is that based on, Hey, I'm looking at existing customers and I need to be 10 percent profitable per purchase versus, Hey, if I get a new customer, I'm willing to accept a cost per acquisition that gives me a three month LTV payback. I don't know that a lot of marketers have really sorted through that to actually be able to say like, look, what is good? What is bad? What is above target? And that's the first conversation I always want to have with marketers of like, If I tell you you're getting a 1. 73 incremental ROAS in PMAX, is that good or is that bad and why?

And let's talk through that because if you don't know the answer to that question, I'd argue the part that you get to next, which we should actually talk about, probably doesn't matter or certainly doesn't matter. Like you're putting the cart before the horse. Let's put it that way.

[00:33:33] Taylor Holiday: And so this is why I think our partnership is perfect. Cause that's exactly what we do, right? Is our entire profit system is designed around the idea that your marketing initiatives need to ladder up to a financial objective and that the financial objective that helps you determine, okay, in order to achieve this level of profitability as an organization, I need this many new customers at this margin, you This expectation of my returning customers to understand then what the expectation of me, your media dollars are.

So I wholeheartedly agree that the starting point is understanding of expectation and why, and how that expectation may change relative to volume trade offs and all sorts of different things that go into it. So check let's agree. Now I'm coming back to you going, okay, I've got, I've got my expectation. I know 1.

7 is good. Now, do I take my 50 percent incrementality and go after it? Set my T Ross at 3. 4 or do I go try and make the campaign more incremental?

[00:34:29] Nick Stoltz: Okay, so now that we know what our targets look like, now I think we can address the question that you're asking, right? And the answer is, I think it depends. Because it depends on the channel. And it depends on the ROAS that you're seeing. Now, let's say you're doing branded search. And let's say that, hey, it comes back that my branded search ROAS is about 15 percent below my target. Guess what? Easy. Let's just change the TROAS that we're doing. Let's lower the impression share. Let's get it to target. Great. Now let's say it's PMAX. Okay. And let's say we find that Pmax is not just 30 or 40 percent below my targets, but also behaving in a way, and this is where, you know, look, the measure brings a lot of expertise to the table. Hey, look, you're showing some signals of things that, you know, like I mentioned, you put it on top of a bunch of other things, it's competing with a bunch of existing campaigns. We have a hypothesis that if you change the strategy, you might see a big jump in that efficiency. So let's actually change the strategy and go back in and test. Right? Or maybe it's even simpler and dumber than that, right? Maybe we see a Facebook prospecting program that is underperforming norms. And we go in and we look at it and say, Hey, you don't have website visitor exclusions on this. Like, we strongly suggest that best practice is to put website visitor exclusions and to go in and retest. Now that presumes you are missing your targets, right? If you are missing your targets by a large amount, it's like, look, we should probably test a different strategy. If you're just missing your targets and everything else looks good. Look, let's just update the TROAS and let's just go, let's level set it so that it's bidding against incrementality. If you're above your targets, Look, that's another situation where it's like, look, you are not spending enough here. You can loosen the bids. You can get it to be more aggressive, like have it go out and pay more. So I, I think it depends, but in general, it, you know, the more it is a channel where you can alter strategy and it is missing those targets by a meaningful amount, the more we might go in and try something else.

[00:36:53] Taylor Holiday: I think that's great. And I just, I would encourage advertisers to explore the why when they get an incrementality result is to rather than just applying broadly the factor and continuing as, You are to ask yourself, why is the incrementality so low? What is happening in this scenario? And could there be ways in which I could create a more impactful set of advertising, right?

Like in a lot of ways, incrementality is a function of impact. And so rather than just going, well, this campaign that I'm running is only 10 percent incremental. I'm just going to. 10 X the Ross expectation, there may be a way in which you go, okay, what is the structure of this campaign that's leading to that kind of result?

And what could I change that would make it such that more dollars were more impactful.

[00:37:35] Nick Stoltz: where you get the most interesting stuff. If you always say, I know what I got. And if I just take the letter of the law and look in a lot of places, you're going to take the letter of the law and go adjust the TROAS because everything's working as you want. But where

you really unpack and peel back the layer of the onion and say, why did I get what I did? Because look,

a measurement, a test and control, like they're great. Cause it'll give you something. It's not going to tell you why it happened. Like

I saw a test where we turned off branded search. Okay. And sales jumped by 4%. And we went through this, it's got to be wrong. There's something off with the data.

We went through it over and over and over again. Finally, we started really looking at it. You know what happened when they stopped spending on branded search? This very cool. It was a home workout product is very cool. Sponsored content ad that compared their product to their competitors. Very favorably, all of a sudden shot to the top. And guess what? People who clicked on that converted at a much, much better rate. And so when they

turned off branded search, not just do they save money, they saw more sales. Now that's a very specific example. But if you just took the result and moved on, you would never get to that. Like really awesome business insight.

[00:38:47] Taylor Holiday: That's it. Yeah. So that's exactly it. The other, the other thing I think it's really important to understand is that when I think about my capacity to win auctions, win placement, get more volume, It's always going to offer me access to more placement if I can win at a lower ROAS. So the higher I have to set my T ROAS, the more limited my ability to win delivery becomes.

And so if I can reconcile the incrementality closer to the platform expected ROAS, I get to be more aggressive in auction. And so I think that it's just like the more that you can actually try to create campaigns that are incremental. The more potential volume you can go and create. And so that's why, like, what's exciting about what we see in metal a lot is that when we see a 175 percent incrementality factor, what that means is that we get to lower the target ROAS bid more aggressively, go win more placement, get more volume.

And so I think those things are those are great. So, all right, Nick, last thing. Let's imagine someone comes to you. Let's talk about roadmap. So I come in. There's an infinite number of tests I could possibly run. How do you suggest I walk through a process of test experiment application? And what is the roadmap that you're walking those brands through?

And how should they 

[00:39:59] Nick Stoltz: Yeah. I'm glad we came here because we really put a lot of thought into this. We want this to resonate and make sense for advertisers. Cause guess what? We're going to have a very detailed methodology and testing all that stuff. Like, look, I want you to understand that, but let me lay it out in simple terms. Most advertisers that come to us. Are spending 90, 95, maybe even 100 percent on what I would typically call the mid and lower funnel. Right. The performance engine. Okay. This is all your social media. This is your open web. This is your search, your affiliate. This is stuff that is conversion optimized bottom and middle of the funnel, it is fundamentally your growth engine.

It is the thing you have to get right to be able to be successful and ultimately scale, right? The whole system is set up for you to waste some money there. That's the way the system's set up. It's set up because the algorithms are built to chase clicks. Okay? A lot of those clicks are real. Some of them aren't. They chase clicks because that signals intent. It doesn't necessarily signal incrementality. It's built in a way that none of them see what the others are doing. So they are all in some way probably hitting some of the same people. Okay? Add up all of the last touch conversions that every platform came, claims in the last 30 days, it's probably more than you sold. Look at it over the past two years and in this economic environment and all the challenges out there, it's probably growing faster than your sales are. That's a dead giveaway that you've got some waste. And that, look, that does not mean you are a bad person or a bad marketer. It means that you exist in the system that we all set up. Now, I don't know where your waste is. hypotheses. I don't know how much waste you have. I don't know if it's 5%. I don't know if it's 30%, but it's there. And if it's not there, you are the first marketer in the history of the world to be perfect in spite of everything being set up against you, right? Our job in the first six months is to find that waste because every dollar you find in waste Is pure profit to the bottom line that is ultimately look that's not going to be your growth engine That's the next job, but that pure profit to the bottom line Is really hard to find it should be viewed as a win Okay, any marketer that looks at that and says I must be doing a bad job because I wasted a dollar Is thinking about it wrong. You should celebrate the fact that you found a dollar you didn't need to spend Okay, that is what we are doing. Now we're going to come with some hypotheses. You know, we mentioned the model and the fantasy baseball predictions and things like that. We're going to have some hypotheses where it is. I want to hear your managerial expertise about where you think it is. You come to the table, you give me the top three places you think you could be wasting money. We're going to go through that process and we're going to find however many dollars are out there. We're going to find it. And why is that important? Because number one, you know, you're, you're a finance guy, right?

Pure profit. That's important. Number two, this is the easy part of the job. Okay. Like, look, identifying waste is fundamentally easier than driving new growth. That's harder work in the first six months that we work together. We're going to get to know your business. Okay. You're going to get to know how this stuff works.

It's complicated. I've been doing this for 15 years, right? You're going to get to understand how does incrementality tests work? How do I interpret them? What do I do with them? We're going to use it. We're going to get our reps. Okay? And we're going to get our reps on what I call, we'll keep the sports analogies going.

This is shooting free throws. We can hit these at 85, 90%. We can find that waste. If we see a media strategy that's not working, we can devise it. Great. Okay. That's six months. We're going to find some dollars and we're going to get this thing tuned up. And then after that period, we're going to go out together and do the hard work. We're going to figure out how we reach new audiences, how we drive new customer growth, how we do that with customers that are brand loyal and not just hunting around for promos. We're going to do that by trying out new media channels and new strategies that you probably haven't done yet. And all of that stuff is going to have a better chance to succeed because we're putting it into that performance marketing engine that has been souped up by that first six months we worked together. And because now the work that we're doing, it's not free throws anymore. We're hitting 99 mile an hour fastballs, right? And if we bat 350, we're leading the league. We're going to have some misses. There's going to be some strategies that don't work. There are going to be some things that don't give us lift in awareness or new customers or things like that, but we're going to tweak it or we're going to move on and try the next thing.

And we're ultimately going to build together. You know, the engine that drives your longterm growth and your longterm scale, and that's the hard work that we have to do together. And we're going to be successful because we spent that first six months, like getting ourselves in really good collective shape with that performance. That, to me, is like the journey that we go on together. And if we can get through that journey, look, what I just talked about, that is like an 18, 24, maybe longer journey. Because like, look, you are not hitting the easy button. There's nothing here that's the easy button. You are ultimately like, we are your partner.

Just like you are the brand's partner in helping to build that long term success. And it doesn't come easy. There are no fad diets in this business. There are no, you know, magic panacea silver bullet models that are going to do this for you. But if you sign up, if you do that work, if you promise to do the training, like, look, I think we can turn you into an Olympic level athlete. And that journey is really important. And, you know, the dividends that it is going to pay down the road, I think are game changing.

[00:45:57] Taylor Holiday: So I love it, Nick. Thank you again for the candor of what it actually requires to get there. And this is the part now where I'm going to tell you about the exciting thing that we're doing together is that one of the reasons I wanted to create this partnership at this exact moment in time is because that journey needs to start today in order to prepare you for what is upcoming, which is the biggest spend moment of your entire year.

So we are about four months out from Black Friday, Cyber Monday and holiday where likely 25 to 50 percent of your budget is going to be spent. And that means these next three, four months are a critical part of establishing a deeper understanding of where that waste is occurring, where the greatest incremental impact is happening, so that you aren't asking questions.

The week of Black Friday, Cyber Monday, you are deploying action against insights. And so we have developed an offering together with measured that combines CTC's profit system, which is going to give you P and L level forecasting, daily contribution, margin goals, budget allocation across challenge, marketing calendar, creative backlog, all those things with a measurement testing roadmap.

And combining those things together to ensure not only do you know what your target should be from a financial standpoint, but we know which channels are making the greatest impact. And so we're, we have 5 very special limited spots in the month of July that we're putting together with measured for this new what we're calling profit system.

Plus, that's going to combine measurement road map testing and our services combined in 1. And I think it's going to be an incredible solution for brands to walk into Q4, better prepared with more clarity of what is making an impact than ever before. So as Nick stated, it's a journey that journey needs to start now in order to help you drive the best results you can in Q4.

So we're excited, hit us up. We'd love to talk to you about it. We'd love to walk you through how we're going to bring the measured tool into our system, walk you through a testing roadmap and go out and make it happen. So Nick, man, I'm excited to get through these reps today with you. We already have one brand that's taken up one of those five spots.

I'm headed out to Utah on Monday to work with them. I'm excited to get them up and rolling. They're going to, there's going to be hundreds of thousands of dollars saved in this specific case. I can't wait to do it. So man any last notes you want to leave people with?

[00:48:04] Nick Stoltz: go. I'm ready to do this. I'm excited too.

[00:48:07] Taylor Holiday: All right. Appreciate you, buddy. Thanks for coming on. Thanks for chatting with us. And we'll talk soon.