This year’s Super Bowl LVIII was watched by a record 123.4 million viewers. The Kansas City Chiefs beat the San Francisco 49ers in a thrilling game with a final score of 25-22.
But beyond the game itself, another competition was happening – determining which of the 59 commercials that aired was the most effective advertisement.
Analyzing ad effectiveness is a complex endeavor that requires looking beyond just viewership numbers to metrics that measure real business outcomes.
This article explores different methodologies and key performance indicators used to evaluate Super Bowl commercials, making the case for metrics like engaged-view conversions that tie advertising directly to customer actions and sales.
Measuring ad effectiveness: The top 2024 Super Bowl commercials
Dome ad agencies may tell their clients that more than 100 million people watched last year’s Super Bowl, so we can expect that even more people may have tuned in to see Taylor Swift kiss her boyfriend, Travis Kelce, after the big game.
OK, maybe a percentage of this audience missed a brand’s TV commercial because they were in the kitchen getting more snacks. But even if a substantial portion of the Super Bowl audience saw a brand’s spot, which cost roughly $7 million for 30 seconds of air time on CBS, that doesn’t mean it was effective.
This year’s big game was played in between 59 ads. If you used a brand lift survey to measure metrics such as ad recall, brand awareness, and consideration, then you’d find that only a fraction of the Super Bowl 2024 ads would be considered effective.
That’s why showing up is necessary but insufficient to “winning.”
Nevertheless, I suspect that 10 ad agencies told their clients that they “won” because they were among “The top 10 Super Bowl 58 commercials,” according to USA TODAY Ad Meter.
Well, kudos to them. Here are the top eight (which got an average rating of at least 6 out of 10):
“Like a Good Neighbaaa (:60) | feat. Arnold Schwarzenegger & Danny DeVito | State Farm® Commercial,” (6.68).
“Dunkin’ ‘The DunKings’ ft Ben Affleck, Matt Damon, Tom Brady, Jack Harlow, Jennifer Lopez, & Fat Joe,” (6.52).
“Perfect 10: The Kia big game commercial featuring the 2024 Kia EV9,” (6.36).
“DON’T FORGET UBER EATS,” (6.26).
“NFL Super Bowl LVIII || Born to Play,” (6.23).
“Hard Knocks: A Dove Big Game Film | #KeepHerConfident,” (6.18).
“BMW USA | Christopher Walken in “Talkin Like Walken” (ft. Usher),” (6.10).
“Budweiser | Super Bowl LVIII Old School Delivery,” (6.08).
But digital marketers who know how to measure what matters should remind their brands and ad agencies that USA Today created the Ad Meter’s methodology back in 1989. And in 2024, it said:
Participants: Any U.S. citizen 18 and older can participate as an Ad Meter panelist.
Rating System: Each participant rates each commercial on a scale of 1 to 10, with 10 being the highest score.
Categories: Some ads are not eligible for Ad Meter, including ads that only run on digital platforms.
So, Ad Meter may not be an accurate “gauge of consumers’ opinions about television’s most expensive commercials.” USA Today says, “We fully expect brands to use social media outlets as a way to promote a campaign.”
Are there better ways of measuring the most effective Super Bowl 2024 ads? Yes, there are.
For example, Kantar used LINK AI tool, which predicts creative effectiveness, to rank this year’s Big Game ads. However, only 12 of them reached “top performer” status based on their priority metrics: Impact, Power, and Enjoyment. Here are Kantar’s top five ads:
“Tina Fey books whoever she wants to be | Booking.com 2024 Big Game ad.”
“REESE’S Cups Big Game Commercial 2024 | Yes! :30.”
“Hello Down There (Extended) – Directed by Martin Scorsese | Big Game Commercial 2024 | Squarespace.”
“‘Dareful Handle’ | The all-new 2024 Tacoma | Toyota Super Bowl LVIII Commercial.”
Doritos Dinamita’s “Dina & Mita (Extended Cut).”
Finally, DAIVID used its content testing platform, which includes facial coding, eye tracking, and survey responses, to measure the Super Bowl 2024 ads with the biggest emotional and business impact on viewers. A total of 4,950 U.S. respondents took part in the study, which didn’t include movie trailers.
Here are the 10 most effective Super Bowl 2024 ads:
“NFL Super Bowl LVIII || Born to Play,” (7.45).
Popeye’s “The Wait Is Over” :60,” (7.29).
“DON’T FORGET UBER EATS,” (7.23).
“CeraVe with Michael Cera…Ve,” (7.11).
“Paramount ‘Patrick Stewart and Hey Arnold’ Super Bowl LVIII (58) 2024 Commercial,” (7.10).
“Auditions | 2024 Big Game Day Commercial | T-Mobile,” (7.07).
“MTN DEW | Aubrey Plaza Having a Blast Super Bowl Ad,” (7:07).
“That T-Mobile Home Internet Feeling | 2024 Big Game Day Commercial | T-Mobile Home Internet,” (7.04).
“2024 BUD LIGHT SUPER BOWL COMMERCIAL | EASY NIGHT OUT | EASY TO DRINK EASY TO ENJOY :60,” (7.03).
“Pringles | Mr. P,” (7.01).
Ian Forrester, CEO and founder of DAIVID, said:
“The NFL’s Big Game spot evoked intense positive emotions among 66.5% of the audience (+29.1% vs. US norm). With the vast majority of ads this year seeking to amuse viewers, the ad stood out by evoking inspiration (29.7%), admiration (24.1%), hope (21.4%), and warmth (21.4%). This outstanding emotional response, combined with strong attention (74.5%, +4.9% vs. norm) and excellent brand recall (83.5%, +20.0% vs. norm), won the ad top honours.”
So, which metrics should digital marketers use?
Or, as Alice asks in Lewis Carroll’s Alice’s Adventures in Wonderland, “Would you tell me, please, which way I ought to go from here?” And, as the Cheshire Cat responds, “That depends a good deal on where you want to get to.”
But clients increasingly want to get farther than the so-called top of the funnel. With Google Analytics 4 (GA4) enabling brands and businesses to measure engaged-view conversions (EVCs), I would advise ad agencies to learn how to track when someone has watched a YouTube video for at least 10 seconds and then converted on their client’s website within three days of viewing the video.
Why adopt this new metric as a KPI of ad effectiveness?
According to Google Ads Help:
“YouTube video viewers have strong intent to watch the content they’ve come to YouTube for and because of that, are very likely to stay on the platform when they encounter an ad as part of their viewing experience.” This explains why people often don’t take action directly after viewing a video ad.
However, they often take action after they’ve finished their full viewing session. This explains us why “70% of YouTube viewers say that they bought a brand as a result of finding the brand on YouTube.”
This is where EVCs come in. They are counted when a user watches at least 10 seconds of a skippable in-stream ad (or watches the entire skippable in-stream ad, if it’s shorter than 10 seconds) and then converts within the EVC window.
Brands and their ad agencies couldn’t measure that back in 1989 when the Ad Meter’s methodology was created. If they could have, then I’m confident that my father, who was director of marketing at Oldsmobile then, would have used this KPI to measure the impact of his classic campaign, which proclaimed, “This is NOT your father’s Oldsmobile.”
Ironically, it was my father’s Oldsmobile. I learned a painful lesson from my dad that has dramatically influenced my choice of marketing KPIs for more than 35 years:
Measure outcomes, not outputs.
The typical business model for ad agencies in the late 1980s was built on how much media they bought on behalf of their clients.
So, most ad agencies used gross rating points (GRPs) to measure the size of an audience reached by a specific ad campaign.
GRPs were often used in the context of television advertising, where they represented the total number of impressions (the number of times an ad was shown) generated by a specific television schedule.
Now, automotive marketers need consumers to:
Be aware of their brand.
Have a favorable opinion of their brand.
Consider buying their brand.
Intend to purchase the brand the next time they’re in the market.
So, sometimes, the agency’s focus on metrics like GRPs is aligned with the client’s goal of brand awareness, favorability, consideration, and purchase intent.
But sometimes it’s not. Why? Because it is a myth that all advertising increases sales to some degree. It doesn’t.
As David Ogilvy observed in his classic book “Ogilvy on Advertising,” the wrong type of advertising can hurt sales. He referenced a survey showing lower beer consumption among people who recalled its ads than those who didn’t. In effect, “the brewer had spent millions of dollars on advertising which un-sold his beer.”
My father also learned this painful lesson when his classic ad campaign, which proclaimed, “This is not your father’s Oldsmobile,” un-sold 62.5% of his target audience in less than five years.
Before the ad campaign was launched, Oldsmobile had sold 1.2 million cars in 1986, making it the third leading car brand in the U.S., behind only Chevrolet and Ford.
His ad campaign featured celebrities like William Shatner and Ringo Starr driving a new Oldsmobile with their daughters.
The ad campaign made the phrase, “This is not your father’s Oldsmobile,” a meme that many older consumers still recall today. But Oldsmobile’s sales fell to 900,000 in 1987, 800,000 in 1988, 700,000 in 1989, 600,000 in 1990, and 450,000 in 1991.
During this time, my father asked his ad agency, Leo Burnette, how they measured the ad campaign results. They answered, “GRPs.” This prompted my father to ask them, “How many GRPs do we need to sell a car?”
Now, General Motors had many other problems, so it filed for bankruptcy in 2009 and closed several brands, including Pontiac, Saturn, and Hummer.
But Oldsmobile’s 107-year history ended five years earlier in 2004. So, I’d argue that measuring the results of an advertising campaign using GRPs knocked about five years off the life of GM’s second most popular brand.
So, the lesson that I learned from my dad is to measure outcomes, not outputs. And it’s one that I’d like to share with digital marketers in 2024. I’m just paying it forward.
Dig deeper: SEO outcomes vs. SEO outputs: Understanding the difference
