EA boosts Sim City downloads with smart A/B testing

Digital marketing industry case study library

Back in 2013, Electronic Arts used an A/B testing strategy to boost digital downloads of its SimCity 5 game. This case study looks at how the games maker made smart use of website optimisation to sell 1.1 million copies in the first two weeks, with 50% coming from direct online downloads.

sim%20city%205.jpg

Case study summary

• Games maker wants to drive download sales of new game
• Used split testing based on several hypotheses
• Tested 100% of traffic to find optimised solution quickly
• Winning variation was a big surprise… driving 43.4% more purchases

The challenge

As EA prepared to release the new version of SimCity, they released a promotional offer to drive more game pre-orders. The offer was displayed as a banner across the top of the pre-order page. But according to the team, the promotion was not driving the increase in pre-orders they had expected. The team decided to test some other options to see what design or layout would drive more revenue.

The solution

Working with marketing tech firm Maxis, EA set up an A/B test pitting the original SimCity pre-order landing page against several variations that displayed the promotional offer in different ways.

Each variation corresponded with a hypothesis the team had set beforehand regarding placement, colour, and display of the promotional offer. One variation removed the promotional offer from the page altogether.

After building each variation in Optimizely’s visual editor, the team set their experiment goal — clicks on the “Pre-Order” button — to track purchases generated from the test.

They allocated 100% of site traffic to the experiment in order to find an optimized solution as quickly as possible during the high-traffic SimCity launch period.


The results

The Maxis digital marketing team quickly discovered shocking results – the variation with no offer messaging whatsoever drove 43.4% more purchases. This suggested people really just wanted to buy the game, no extra incentive necessary.

sim%20city%20control.jpg

sim%20city%205%20results.jpg

“We found that the variations with the promotional offer actually performed worse than just a clean version of the page. This was really surprising to us, but it had a pretty significant impact,” Burk said.

Most people believe that direct promotions drive purchases, but for EA, this turned out to be totally false. Testing gave them the information needed to maximize revenue in a way that would not have been otherwise possible.

Copyright ©2000-2018 Digital Strategy Consulting Limited | All rights reserved | This material is for your personal use only | Using this site constitutes acceptance of our user agreement and privacy policy