September 15, 2023

Beyond attribution: understanding the incremental impact of your marketing

Eric Sloan

TL;DR

The world of marketing measurement is complex and traditional attribution methods often fail to share the whole story. We believe that incrementality is a more reliable determiner of revenue outcomes than many attribution models. Incrementality is all about measuring the real impact of our ads – the stuff that wouldn't have happened without them. We measure incrementality in two ways:

experiments and testing

media mix modeling

To get started, we suggest running small experiments and begin building an understanding of how different media channels contribute to top-line revenue.

Listen, measuring marketing has never been easy and that’s coming from an agency that offers it as a service. We’re always plagued by so many questions.

How do we measure our marketing programs effectively and at scale?
How do we get to a single source of truth for marketing so we can measure the return for every dollar spent?
How do we decide where to make investments in our marketing programs - where are we over spending or where is there opportunity to scale?
How do we measure the impact of our top of funnel marketing campaigns so we can report on this effect to the executive team?

And it’s not getting easier. As we see our ability to track most metrics and touch points decrease (goodbye cookies) these questions are getting harder to answer. To be totally honest, there’s really no single solution for every organization BUT it is possible to get to a place where we can make data-informed decisions for our marketing programs focusing on business outcomes and returns. That’s what we’re here to talk about today!

What is attribution?

We define attribution as the identification and measurement of a set of user actions contributing to a desired outcome. These outcomes could include a purchase, a click, brand awareness, sign-ups, etc. Basically attribution is about assigning credit to digital marketing efforts based on a customer or prospects path to some action (typically a purchase or other high value action).

There are lots of types of attribution as you may already know but one that tends to get a whole lot of credit is last-click attribution. This is the simplest form of attribution and it’s where we assign all of the credit for a purchase to the most recent touch point. For example, if a user clicked on a Google Search Ad - and then made a purchase - 100% of the value of that purchase is attributed back to the ad.

More complex applications of attribution take other factors into account like first and last touch as well as other user actions. Some even go so far as using machine learning techniques to build complex models that can assign credit across many channels.

But all of these forms of attribution don’t really tell us what we need to know, they fail to tell us the true impact from marketing. They fail to measure incrementality - the true impact of your marketing.

What is incrementality?

Incrementality is the real meat in the attribution sandwich. It’s the value driven from your advertising program that wouldn’t have been driven without the advertising. It’s the money and exposure you earned that you wouldn’t have without your ads.

Let’s look at a simple example. A business currently generating $100k in monthly revenue starts a new advertising campaign with $20k in spend. The next month, sales increased to $160k in revenue. The incremental lift to revenue was $60k.

The difference between this lift and an attribution report can be sizable. An attribution report might say the advertising campaign drove $85k in revenue by association, even if the real lift to the business was only $60k.

Why attribution doesn’t measure incrementality?

The key thing to understand about attribution is that it’s assigning credit based on association. It’s correlation. Someone looked at an ad and they bought something – that’s attribution. But because of this, attribution does a poor job of explaining what CAUSED a purchase to happen. That’s why we like to turn to incrementality.

Let’s look at a quick user path example using last-click attribution:

You’re going on a hike this weekend and you need a big back for all your supplies (and by supplies we mean snacks and a few beers)
You talk to your friend about hiking bags and they recommend a brand they like
You go to the brand’s website through a Google search on your desktop, add a bag to your shopping cart but you get distracted and don’t buy it
Two days later you’re on Instagram and an ad for that bag is in your feed so you click the ad and buy it.

In this example, if we used last-click attribution, 100% of the credit would be assigned to the Instagram ad for the purchase. Even multi-touch attribution would likely only see the Instagram ad (because we used two different devices) and attribute all of the value in the same way.

Is that the best way to attribute the value of this purchase? We say no. Would you have purchased the bag regardless of seeing the ad? Maybe you would, maybe you wouldn’t, but in either case, 100% of the credit shouldn’t be going to this ad. There were way more brand touch points unaccounted for.

How do you measure incremental impact?

Ok, so now that we know incrementality is the superior form of measurement, how do we find it? Well there are two common ways to measure the incremental impact of your advertising:

Experiments and testing
Media mix modeling (or MMM as you often see it referred to)

The precise details of each of these approaches would be a whole other blog post (which you can sign up to get straight to your mailbox if you leave your email here) but here is a brief overview of each, including pros and cons.

Experiments & testing

1. What? - A structured test (like an A/B test) to understand the real lift from your ad campaign.
2. How? - Split an audience into two groups, serve ads to only one of the groups and measure the difference in outcomes.
3. Pros - Randomized control testing is the gold standard for understanding incrementality.
4. Cons - Tests give you results for a point in time and the results of a test may not translate into the future. Testing also requires quite a bit of time and it can be difficult or impractical to build a proper control for measurement.

Media Mix Modelling

1. What? - A statistical model looking at historical patterns to estimate ad effectiveness, saturation points, and lag effects.
2. How? - Use historical data to model what has happened and use the outputs of that model to estimate the effect of each of your advertising channels.
3. Pros - Some well implemented models can measure incrementality faster than testing and these models estimate values across your entire advertising program instead of just one channel at a time.
4. Cons - Can be quite complex and expensive to set up - these models are also easy to misinterpret. A well designed model will be thoroughly backtested to be defensible.

Bringing it all together - how to understand incremental impact

So now that we know why we need incrementality, and how to measure it, let’s talk about exactly how to understand and interpret this big topic. Specifically, let’s talk about how to incorporate it into your day-to-day using these four steps.

1. Incrementality measurement
2. Calibrate your attribution reports
3. Collect qualitative information
4. Align with business outcomes

Incrementality measurement

The best time to start trying to understand the incrementality of your marketing is right now. Besides, running small experiments isn’t too hard and the results might surprise you (maybe a campaign is driving 3x the value you thought it was… or maybe none at all). In any case, it’s better to know than not.

We strongly recommend starting with some experimentation even if you want to build an MMM model. You need some baseline prior understanding of incrementality to validate and calibrate those models.

Finally, try to test the biggest and most impactful questions you have first. We often recommend testing your entire digital advertising program all at once as a starting point. This will give you a good benchmark to start improving against.

Once you’ve established a baseline, you can start to test individual channels to understand what channels are driving the most value and which ones you can invest less in.

Calibrate your attribution reports

Understanding incremental impact is just one part of the puzzle - the next step is putting it into action. The simplest approach is to use your incrementality results as a multiplier for day-to-day decision making.

Let’s look at an example campaign:

Blog #2_ Example Campaign 1

Now that you better understand your incrementality, you can update your assumption about this campaign because you know it’s driving 1.43x the value of what you see in the advertising platform. This can help you make better media investment decisions on a day-to-day basis without having to wait for the results of your next test.

Collect qualitative information

The success of a marketing program can’t (and probably shouldn’t) be entirely measured by quantitative data. The numbers can’t tell you everything.

Gathering qualitative information from prospects and customers can help you better understand how your marketing is being perceived and help you make strategic decisions. Asking customers how they heard about your product will often tell you a completely different story than your attribution reports. This information can help you understand word of mouth, social media impact, referrals, and more.

Besides, regularly speaking to customers and reading reviews can help you be a better overall marketer. You can use the information you gather to create more accurate messaging in both content and tone of voice.

Align with business outcomes

No matter how good the metrics on your marketing campaigns look, the most important factor is how they contribute to overall business goals and outcomes (your CFO likely doesn’t care about impressions). It's possible to improve your reported marketing return on ad spend (or ROAS) without actually driving more sales. This is why, no matter how you measure your marketing, a clear focus on high level business results is imperative to keep you grounded.

Wrapping up

Sure, attribution is great and all, but it’s not 100% reliable and it’s not necessarily measuring the real outcomes of your hard work. Implementing incrementality into your marketing measurement will help you ensure you’re driving real business results instead of padding an attribution report. This type of measurement is harder to execute, but it’s the only way that you can really know what value advertising is driving, and where to wisely invest your marketing dollars.

If you’re struggling to get a hold of your data and reporting, we can help. Check out our data & reporting page to learn how Thrive can improve (and provide clarity on) your ad spend effectiveness.