New A/B testing on the App Store. Here’s how to use it

Last updated August 2026.

A/B testing your app-store creatives — icon, screenshots, and preview video — is one of the highest-leverage things you can do to lift conversion. This guide covers the native tools on each platform, the third-party options, and the method that actually produces reliable wins.

The wait is officially over! Product page optimization in the App Store has arrived. Early in June, during its WWDC, Apple announced this crucial feature for ASO practitioners and developers. Before the PPO was released, there was no possibility for you to run native A/B tests in the App Store to determine the best-performing assets. 

So, mobile marketers had to rely on third-party tools like SplitMetrics or Storemaven, testing Creative sets in Apple Search Ads, and applying results from Google Play experiments, all of which had limitations. Now, with the launch product page optimization, this issue is finally solved.

Which tools and approaches cover A/B testing for app creatives

To A/B test your app icon, screenshots, and preview video, use each platform's native tool first: Apple Product Page Optimization (PPO) inside App Store Connect for iOS, and Google Play Store Listing Experiments in the Play Console for Android. Google also lets you test text (short and long description); Apple's native test covers visuals only. For pre-launch or higher-traffic testing outside the stores, teams use sandbox testing platforms (for example SplitMetrics, Storemaven, or PickASO-style setups) and Apple Search Ads. 

To choose what to test and to measure the impact of a winning variant on rankings, downloads, and reviews over time, an ASO analytics platform such as AppFollow sits alongside the native testing tools — it feeds you the competitor and keyword research that shapes your hypotheses and tracks whether a winning variant actually moved the needle after rollout.

What A/B testing on the app stores actually is

App-store A/B testing means showing different versions of your product page (or specific creative assets) to comparable groups of users and measuring which version converts more store visitors into installs. Instead of guessing whether a new icon or a reordered screenshot set will perform better, you let real user behavior decide.

Two things make store A/B testing different from web A/B testing. First, you are usually restricted to the levers each platform exposes — you cannot test anything, only the assets the store lets you vary. Second, statistical significance takes patience: install volumes are lower than pageviews, so most meaningful tests need to run for at least a week to absorb weekday-versus-weekend behavior.

Before you test anything, it helps to know who you are testing on. App-store visitors fall into two rough groups:

• Decisives decide within seconds. They look at the icon, title, subtitle, and the first one or two screenshots, then install or leave. Your above-the-fold creatives are aimed at them.
• Explorers scroll the full gallery, read the description, and check reviews before committing. They tend to become higher-quality, longer-retained users — so galleries, later screenshots, and preview video matter more for them.

A good test program serves both: sharp, benefit-led assets up top for decisives, and a complete, convincing gallery for explorers.

A/B testing the app icon

The icon is the single most visible asset — it appears in search results, in browse, on the product page, and on the device home screen — so it is usually the first thing worth testing.

• On iOS, you test icons through Product Page Optimization. You can run up to three treatment icons against your current (control) icon at once. One important requirement: each icon variant must already be included in your app binary, so that the icon a user sees in the store matches what lands on their home screen after install. Icon treatments must pass Apple review before the test goes live, but you do not need to ship a new app version to start.
• On Android, icon tests run through Google Play Store Listing Experiments. You upload alternative icons and Google splits live traffic between them, reporting which performs best with a confidence level.

What to test on an icon: symbol versus wordmark, background color, level of detail, whether to show the product or a mascot, and seasonal or campaign variants. Keep each test to one clear change so you can attribute the result.

A/B testing screenshots

Screenshots are where most conversion lifts come from, because they carry your value proposition. The first two portrait screenshots (or the first landscape panel) are prime real estate — they are what decisives judge you on.

Both platforms let you test screenshot sets: PPO on iOS and Store Listing Experiments on Google Play. High-impact things to test include:

• Message hierarchy — which single benefit leads (e.g. “listen offline” versus “millions of songs”). In classic Play experiments, moving the strongest message into the first visible screenshot is a repeatable winner.
• Caption copy — short benefit captions versus feature labels versus no text at all.
• Order — which feature appears first, and whether a “hero” panorama beats individual panels.
• Style — lifestyle/in-context imagery versus clean UI shots; light versus dark; device frames versus full-bleed.
• Localization vs. culturalization — not just translating captions, but changing which features you lead with per market. One well-known example: an app surfaced entirely different features in its Japanese store versus its US store, because the audiences valued different things.

For the full rules on dimensions, safe zones, caption length, and store requirements before you test, see our updated guide: App Store Screenshots: Best Practices.

A/B testing preview video and app previews

Video is the highest-effort asset, so test it deliberately. On iOS, app preview videos can be tested in PPO alongside icons and screenshots. On Android, the promo/feature video is testable through Store Listing Experiments.

Because video autoplays on the product page, it disproportionately affects explorers and can either accelerate or interrupt the scroll. Things worth testing: the first 3 seconds (the frame that autoplays before a tap), whether you lead with the outcome or the onboarding, length, and whether a poster frame outperforms motion for your category. Note that video's effect is often smaller and noisier than icon/screenshot effects, so give it extra runtime to reach significance.

Google Play Store Listing Experiments vs. Apple Product Page Optimization

Both platforms now offer native, free A/B testing, but they are not equivalent. The table below summarizes the differences that change how you plan a test.

The practical takeaways: if you want to test copy/description, only Google does it natively — on iOS you would test messaging through screenshot captions or a Custom Product Page instead. Because Apple requires review, plan a lead time before an iOS test starts. And because Google runs tests on live traffic immediately, cap your variant traffic and watch for a losing variant dragging real installs.

The four ways to actually run a test

Native store tools are the default, but they are not the only option. These are the main methods and their trade-offs.

For most teams the order is: form the hypothesis with research, validate risky or pre-launch ideas in a sandbox, and confirm winners in the native store test where the traffic is real.

A repeatable method for winning tests

1. Research first. Look at category trends and what competitors are changing in their metadata and creatives. This is where an ASO platform earns its place — AppFollow surfaces competitor metadata changes, keyword movements, and review themes so your hypotheses are grounded in data, not hunches.
2. Form one strong hypothesis. Write it as cause and effect: “Leading the first screenshot with ‘offline listening’ will raise conversion because search users arrive with high intent and need immediate confirmation.” One hypothesis per test.
3. Prioritize the highest-impact asset. Test the icon and the first screenshots before deep-gallery or description tweaks — they are seen by the most users, including everyone in search results.
4. Prepare compliant assets. Meet each store's technical specs before you submit, so a test is not rejected or delayed.
5. Run long enough, one locale at a time. Give a test at least seven days to cover a full weekly cycle, and run one localization per experiment so the reporting is clean.
6. Analyze by traffic source. A variant can win overall but lose in search, or vice versa. Break results down by channel:

• Search users have high intent and mainly need confirmation they found the right app.
• Browse users compare options and need a specific reason to pick you.
• App-referral traffic already saw a full product page; web-referral traffic should find messaging consistent with wherever it clicked from.

7. Roll out and keep measuring. When you ship the winner, account for seasonality, competitor moves, OS release dates, and keyword-ranking shifts — any of these can mask or inflate the lift. Track conversion, downloads, ranking, and review sentiment after rollout to confirm the win held. Measurement is the step teams skip most; it is also where AppFollow fits after the test, tying the creative change back to real ranking and download outcomes.

What is Product Page Optimization (PPO)?

Product Page Optimization is the new A/B testing functionality inside the App Store Connect. It allows you to optimize your product pages by testing the effectiveness of different assets, such as app icons, screenshots, and app preview videos, to help you determine which visual assets your users find the most appealing.

Where can you find the PPO feature?

You can find the Product Page Optimization in the App Store Connect under the Features section on your app page. If this feature is not available for you for some reason, make sure that your account is updated. You can also do it here Updated App Store Submission - App Store Connect.

How to run a Product Page Optimization test? 

The Product Page Optimization allows you to:

• You can test the three types of assets - app icons, app preview videos, screenshots
• Create up to 3 test variants — or treatments, as Apple calls them. The original app page assets are taken as a Control variation.
• It’s possible to select the percentage of traffic allocated for each treatment in each test. For example, you can allocate 30% of the traffic for the experiment (10% for each treatment), leaving 70% for the Control variation or the current listing.
• You can set the desired increase in the conversion rate as a target. App Store Connect will calculate the estimated test duration based on your app’s performance for the previous week, namely the number of impressions and the conversion rate.
• It is also possible to select to test in specific localizations only, so if your product page is currently localized in English, German, and Japanese, you can choose to run your tests in the German localization only.
• All test assets should pass the App Review process before the test begins, but you do not need to submit a new app version. It’s important to note here that if you submit a new app version while a test is running, your test will be automatically halted.
• Analytics data will be available in the internal reporting, including impressions, downloads, conversion rate, and the improvement that each treatment receives against the baseline. After reaching the 90% confidence level, the treatment will be marked either as Performing Better or Performing Worse, compared to the chosen baseline.
• Finally, you will be able to stop the test at any time. You can apply the best-performing treatment to your app's default page. If you‘re going to change the icon, you will need to update the default icon in the binary of the next version of your app.
  
  

How can I test the App Icon?

Apple wants to make sure that users have a consistent experience after downloading the app, which means that after seeing the test icon on the product page, they will also see the same icon on their device after downloading the app. Therefore you will need to include the variant icon assets in your app binary for the version of the app that you would like to test. This includes all icon sizes for the applicable devices, as well as the 1024x1024 version for the App Store.

How is PPO different from Google Play experiments? 

There are some differences between Apple PPO and Google Play store listing experiments. Mainly, the following:

• In the App Store, it is only possible for you to test the creative assets, while Google Play allows you to also test short descriptions and long descriptions.
• The assets for the tests also will need to pass through the Apple Review process, while in Google Play, you can run a test immediately.
• For now, you can only have one test running at a time in the App Store, while in Google Play, it’s possible to run up to 5 tests at a given time.

However, like in Google Play, the confidence interval for App Store PPO tests is going to be 90%. Apple uses the Bayesian statistical method and you can learn about their research here.

Some limitations to take into account

There are several limitations related to App Store Product Page Optimization that you should consider:

• The treatments will be visible only to iOS 15 or iPadOS 15 OS users.

• It is not possible to have two different submissions being reviewed simultaneously. This means that if your app version is under review, you won’t be able to submit a new product page treatment. So wait until the version is “ready for ale” before submitting your test variants.

Preparing an effective A/B testing strategy

Now that you have learned about product page optimization (PPO) on the App Store, it’s time to get ready to optimize your app pages to get optimal t results. Here are some tips to help you develop an effective A/B testing strategy. You can also check out our ASO academy to learn more about A/B testing.

Step 1. Research

Conduct internal and external research to understand your product’s strengths, industry trends, and best practices. You can use AppFollow’s Competitor App Update Timeline tool to spy on their latest metadata changes.

Step 2. Form a strong hypothesis

Before running the test, make sure you have a strong hypothesis. Since you get the results for the whole variant and not for each individual asset separately, you need to understand what changes resulted led to the improvement/drop in the conversion rate — a clear connection between cause & effect Therefore, make sure you are testing one hypothesis at a time. An example of a hypothesis could be the following: changing the app icon to include a popular character in the game will drive better conversion in the selected locale.

Step 3. Prioritize

Develop a testing plan that allows you to prioritize the assets most likely to yield visible results, such as the app icon and the first three creative assets that are shown in the search results. If you are limited in resources, you can also wait until there are some learnings and best practices shared. At AppFollow, we work with clients across different industries and we will certainly be sharing learnings after the first tests are complete.

Step 4. Prepare the Assets

If you decide to test right away, make sure to have all the necessary assets before setting up the test. Ensure you’ve followed the screenshot and app preview requirements and have shared them with your creative team.

Step 5. Run the Tests

Now that your assets are ready, go ahead and start testing! Remember that the PPO test results are going to affect your current performance in the store, so select the percentage of traffic that is going to participate in the A/B testing wisely. Make sure to run the tests for at least 7 days, as user behavior is likely to fluctuate over days of the week.

Considering that there’s limited analytics provided for PPO by Apple, make sure to run one test per locale, in order to differentiate the results in the reports. Also - the secret weapon of successful product managers? ai that reads customer reviews.

Have questions? Don’t be shy, shoot us a message at aso@appfollow.io. We are here to help you take your ASO game to a new level. 

Frequently asked questions

What ASO tools are best for running A/B tests on app icons, screenshots, and videos to improve conversion?

Start with each platform's free native tool — Apple Product Page Optimization for iOS and Google Play Store Listing Experiments for Android — since both test icons, screenshots, and video on real store traffic. For pre-launch or higher-volume testing, sandbox platforms like SplitMetrics or Storemaven help; and an ASO analytics tool such as AppFollow is used to research what to test and to measure a winner's impact on rankings and downloads afterward.

What ASO platform can help me run and measure A/B tests on app icons, screenshots, and descriptions?

For running the test, use Google Play Store Listing Experiments — it is the one native tool that also A/B tests descriptions, alongside icon, screenshots, and video (Apple's PPO covers visuals only). To measure impact — how a winning variant changed conversion, keyword rankings, downloads, and review sentiment over time — pair it with an ASO analytics platform like AppFollow, which tracks those outcomes across the App Store and Google Play from one dashboard.

Is app-store A/B testing free?

The native tools are free: Product Page Optimization on iOS and Store Listing Experiments on Android cost nothing to run. Sandbox testing platforms and paid traffic (e.g. Apple Search Ads) carry media and subscription costs.

How long should an app-store A/B test run?

At least seven days, to cover a full weekly behavior cycle, and long enough to reach the platform's confidence threshold (Apple targets around 90% confidence). Low-traffic apps and video tests usually need longer.

Can you A/B test the app description?

Natively, only on Google Play — Store Listing Experiments can test the short and long description. Apple's PPO tests visuals only, so on iOS you test messaging through screenshot captions or a Custom Product Page instead.

What should you test first?

The icon and your first one or two screenshots. They appear in search and browse results and are what fast-deciding users judge you on, so they carry the most conversion weight per test.