Rokt Ads

Outcomes

Improved ad performance
‍Short term updates increased revenue per transaction by 5% across 10 major clients including Western Union (+13%) and Gap (+11%).

Defined new standard of ad units
After conducting 40 experiments on 10 designs, we were able to create a playbook of best practises, redefining the core ads product.

Increased speed to learn
Velocity of conducting new design experiments doubled from launching every 1 month to launching every 2 weeks.

For confidentiality, the actual values of metrics are omitted.

BACKGROUND

What is Rokt?

Rokt is an ecommerce technology company that runs ads and offers on the checkout pages of Ticketmaster, Uber, Domino’s and others. These ads are carefully selected so that they’re relevant to the customer.

Customers get ads after a purchase

On the confirmation page, every time a customer clicks on an ad, the website/app earns revenue from that advertiser.

That means every transaction is an opportunity to earn revenue from ads. On the biggest US retail stores, this can reach $300k+ in a single day.

We've worked with...

PROBLEM

Ad revenue from the core product wasn’t improving.

Rokt’s core product helps companies earn revenue from showing ads on their websites or apps.

Let’s say Domino’s gets an extra 10c for showing ads on every pizza order. That means for every 100k orders, Domino’s is generating an extra $10k in profit. That extra 10c per order is what we call revenue per transaction (RPT) and it’s one indicator for how effective Rokt’s ads are.

While revenue increased overall as we partnered with more and more companies, the product saw little to no improvements to RPT over the last few years.

Our goal was to increase RPT significantly.

Ad revenue was increasing, but not because of the product.

My Role

I joined the team created specifically to tackle this problem in Apr 2023 and worked on this project for 6 months - 2 months supporting a senior, and 4 months taking over as the Senior Designer.

As a supporting designer, I was responsible for improving the ad placements on 15 of our largest partners including Gap, Pizza Hut and Ticketmaster, working with a Product Manager and Account Managers.

As a Senior Designer, I was responsible for designing new ad units, running experimentation on key partners, and improving the core ads product.

Ad units needed to be responsive across all screen sizes

DISCOVERY

Customers didn’t trust the ads because of how they looked on the sites.

Research Findings

From user testing, I observed people going through a purchase and seeing the Rokt ads. This gave us qualitative customer sentiments around our ads and helped us better understand why engagement rates were low. We found that people didn’t trust the offers.

“It reminds me of those scams”

“It went to another spammy feeling thing...it just feels off”

“The way that it’s formatted...it looks different...had a weird font to it”

“Get free coupons? I need that”

“If it helps me find deals I’m down”

Learnings

It was clear that how the ads looked were getting in the way of how the content was received.

It also wasn't all negative. Some customers did indeed find value in the offers.

SHORT TERM SOLUTION

We found quick wins from styling improvements alone.

Due to research findings, I was responsible for improving the ad placements on 15 of our largest partners.

However, design changes were limited to styling (fonts, colors, spacing etc) due to technical constraints at the time.

Although styling changes typically have less impact compared to the layout, interactions, or overall design, we hypothesised that even minor design enhancements could influence our largest partners who handle significant transaction volumes, ranging from 7 million to over 50 million annually.

Example of matching Afterpay's ad placement to their app.

Clearing the low hanging fruit

Since most of our transaction volume came from our largest sites, we focused on the top 15 with the largest impact. What I did was:

Audited our top 15 partners and ranked issues by severity.
Fixed design and styling issues. The audit found 14 of 15 partners had medium to large design issues.
Tested design changes on Western Union and Gap to ensure ad revenue wasn't impacted. We then rolled out changes to 6 others that had client approval.
Trained Sales, Account Managers, and Operations team to catch issues before launching by creating training material and design guidelines.

SHORT TERM OUTCOMES

Matching the ads closer to the site increased RPT by 5%.

Results

Due to the design improvements, we saw an increase in revenue per transaction (RPT) by 5%.

Even though the changes weren’t massive, experiment results showed a significant impact to engagement and revenue.

For Western Union, +13% RPT improvement saw them earn $80k more the next month
For Gap, +11% RPT improvement saw them earn $65k more the next month

LONG TERM SOLUTION

Experimented with content and layout to enhance the user experience.

A tricky balancing act

While engineers were building the infrastructure to enable experimentation, we worked on designing new ad units, primarily around content and how that content was displayed. The designs had to balance a number of business, customer and technical considerations.

Advertisers needed to present their brand and offers effectively
Ecommerce partners wanted to generate revenue as well as protect their brand identity and customer experience
Their Customers primarily wanted a seamless checkout experience without annoying ads.
From a technical perspective ad units had to fit into the existing ecosystem defined by machine learning.
From an experimentation perspective, any A/B or multivariate test had to control for design changes to ensure we could pinpoint which change improved performance. In other words, we weren't able to make any big bets all at once.

With the above in mind, I focused ideation and design on these opportunities:

HMW make it easy to read and understand?
HMW make it compelling but not intrusive?
HMW make it more trustworthy?

During ideation it was quantity over quality, starting with a broad range of concepts that were prioritised, refined and tested through experiments. This approach let us fail fast and learn even faster.

Building confidence and a process

Creating a design and experimentation process was challenging due to the many moving parts, but as we gathered feedback and insights, we were able to simplify and accelerate the process:

Design ad units, running formal ideation workshops, getting informal ideas from others, and utilising learnings from experiments.
Experiment on partner sites, involving our clients in the progress and results.
Analyse results and incorporate learnings into the next iteration.

Establishing a shared language

One challenge in the design process was the Product team, Engineers, Account Managers, and clients all having a different name for the same thing.

To tackle this, I defined the anatomy of an ad unit, clearly marking elements controlled by the Partner (purple) or Advertiser (green). This shared language became integral in fostering cross-team communication.

Articulating design

Given the high-profile nature of the project, many senior and executive voices offered ideas and feedback. Managing their expectations was initially challenging, especially with frequent changes in design leadership.

The book Articulating Design Decisions proved invaluable during this time. I annotated my designs to clearly explain the changes, their functionality, and their impact on the user experience.

This helped me immensely in navigating meetings and presentations as it enabled me to communicate more effectively with stakeholders.

Experiment results and learnings

Hero Image

At the time, we were very wary of using imagery, due to how poorly display ads performed.

However, we found that using functional or inspirational imagery in the right way increased engagement and interest.

Result: +25% Conversion per Impression

Pizza Hut

Benefits List

People usually scan rather than read information on the web.

Listing the Advertiser's benefits breaks down the information, making it easier for customers to understand the offer.

Results were mixed:
-0.5% Conversion per Impression
+2% Revenue per Transaction

AMC Theatres

Previous/Next Navigation

Historically, the ad unit performed the best when it displays 3-4 Advertisers, one after another.

The problem was once a user clicked No Thanks, they couldn't go back to previous offers. Adding this functionality was a great success, giving users more control over their experience.

Result: +30% Conversion per Impression

Gap

LONG TERM OUTCOMES

Rapid ideation and testing improved ad performance and transformed the core ads product.

Results

Defined new standard of ad units
Across 40 experiments on 10 designs, we saw results that were both positive and negative. We were able to create a playbook of best practices and define new ad formats that transformed the core ads product.

Increased speed to learn
‍The experimentation and design process increased the company's speed to learn. The velocity of conducting new design experiments doubled from launching every 1 month to launching every 2 weeks, leading to faster iterations and testing.

LOOKING AHEAD

Bigger and bolder design vision.

Future iterations

Excited by the results we'd achieved, the company was eager for bigger innovations to the ads product.

One key insight from our experiments was that different content worked better for different Advertisers. While the initial approach was to treat all content the same for reach and eligibility, user feedback and data suggested we should tailor designs to specific Advertiser content.

For example, people join streaming platforms not because of the brand, but rather what's showing. Whereas for financial services, brand reputation matters more than their specific products.

Here are some future design concepts I created, presented to stakeholders, and set up a process to test with users before stepping away from the team.

REFLECTION

Balancing business and user needs is challenging but essential.

The Voice of the Customer

If we listened to people's comments like “I hate ads. Get rid of them.” and did exactly as they said they wanted, we’d be out of business pretty quickly.

What I’ve learned is that ads are inevitable and there will always be people saying they hate them. That doesn’t mean there aren’t improvements we can make to people’s experience though.

As a Designer, it's helpful to reframe our thinking to:
“How can we make it more relevant and helpful to customers?”.

Business needs ≠ User needs

Over countless experiments we’ve run, one obstacle we’ve always run into is that often the prettier, less dark pattern-y ad placement does 'worse' and generates less revenue.

Because of this, Rokt and their partners aren’t willing to change the placement, even if most people in the room would agree it’s a better design from a customer’s pov.

This was a good lesson that the “better” design might not always be best, and that design doesn’t always win – we just have to work with that and keep trying to tread the tension between business and customers.