How to A/B test Amazon images with Manage Your Experiments

Amazon’s US page lists three conditions: a Professional selling account, the Brand Representative role on a brand enrolled in Brand Registry, and an ASIN that belongs to that brand and has had enough recent traffic to produce valid results. For A+ Content tests, the product also needs published A+ Content already. Amazon does not publish a traffic threshold. An Amazon staff post on the Amazon.in forum describes high-traffic ASINs as those that may get several dozen orders per week or more, and says only one experiment can run on an ASIN at a time.

To see where a product stands, open Brands, then Manage Experiments, then Create a New Experiment. Per Amazon’s guide, choosing an experiment type lists the eligible ASINs. If your best seller is not on the list, no test is possible yet, and driving more traffic to it comes before any creative work.

Shoppers who view the product page are randomly split into two groups. One group sees Version A, the current published content, and the other sees Version B, for the whole experiment. Because both versions run at the same time, seasonal swings and ad bursts hit both equally, which is what a before-and-after comparison cannot give you.

The setup is short: pick the experiment type and the primary product, name the experiment and write a hypothesis, upload Version B (for images, upload the file and select it with the file picker), then schedule it. The tool compares units sold per unique visitor, conversion rate, units sold, sales and sample size, and projects the one-year sales impact of the winner. View Details shows the probability that one version beats the other.

Amazon’s guidance is to make the two versions significantly different, and the tool warns you if they are too similar. Change one idea per test so the result has a cause. Whether the tool lets you swap the whole image gallery or only certain slots is shown on the create screen, so check it before you plan the creative.

Main images still have to meet the rules for main images: pure white background, no text or logos, product filling the frame. Within those limits there is plenty to test. Illustrative example, written as a hypothesis: if the product is shot from a three-quarter angle that shows the opening and the handle, more shoppers will understand what it is than from the current front-on view, so units per unique visitor will rise. Version A stays untouched. Version B changes only the angle.

If you choose your own duration, Amazon recommends 8 to 10 weeks. The default setting, to significance, ends the test when there is enough data to call a winner, and results can sometimes arrive in about four weeks. Amazon’s tips say to run the experiment to completion even if early results look promising. A lead in week one is a reason to keep watching, not a result.

Results update weekly according to Amazon’s guide. Check the settings before you schedule, as the default is for the winning content to be published automatically. If a design change affects other pages, such as a matching A+ module or a Brand Store tile, switch that off and publish by hand.

Read the probability figure first, then the metric that matches your goal. Illustrative numbers only: Version A at 8.0 units per 100 visitors, Version B at 8.6, which is a 7.5% relative lift, with a 93% probability that B is better. That is a reasonable basis to publish B. If the probability is 60%, treat it as no difference and move to a bolder idea.

Keep a short log with the hypothesis, both versions, dates, the metrics and what you did next. A losing test is useful input for the next photographer brief. Amazon quotes sales uplifts of up to 20% on its US tools page and up to 25% on its blog and on the Amazon.in brands page. Those figures differ between its own pages, so ignore them when setting expectations.