Automatic winner selection for landing-page tests

Automatic winner selection should change a landing-page test’s traffic allocation only when one variation has enough trustworthy evidence to outperform the others. It is more than choosing the page with the highest conversion rate at the current moment.

Before enabling a selector, a buyer should know which outcome it uses, how it handles delayed conversions, how much evidence it requires, and how to reverse a change.

Why a simple conversion-rate comparison fails

Imagine two pages:

• Page A: 1 conversion from 10 visits;
• Page B: 80 conversions from 1,000 visits.

Page A has the higher observed conversion rate. It also has too little evidence to justify moving all traffic there.

A useful selector must account for:

• sample size;
• conversion delay;
• revenue or profit, not only conversion count;
• traffic-source and audience differences;
• bot and suspicious-traffic signals;
• refund or reversal events;
• the cost of making the wrong change.

“Pick the current leader” is not an optimization strategy.

Start with a clear objective

The system needs one primary decision metric for each test.

Possible objectives include:

• conversion rate;
• revenue per visit;
• profit per visit;
• approved-lead rate;
• cost per acquisition;
• return on ad spend.

The metric should match the business event the campaign can verify. Optimizing for form submissions is dangerous when the buyer is paid only for approved leads.

Secondary metrics can act as guardrails. A page might improve conversion rate while reducing average payout or increasing suspicious traffic.

Respect conversion delay

The winning page cannot be judged only from conversions already reported.

If conversions commonly arrive hours or days after the click, recent traffic has incomplete outcomes. A selector needs an attribution window or maturity rule before treating those visits as losses.

Otherwise, the system will favor pages whose conversions report quickly rather than pages that produce more value.

Use minimum evidence rules

Automatic selection should remain inactive until the test reaches a defined evidence floor.

That floor may include:

• a minimum number of eligible visits per variation;
• a minimum number of conversions;
• a minimum test duration;
• enough mature clicks to cover the conversion-delay window;
• a confidence or probability threshold;
• a minimum practical improvement.

Statistical confidence alone is not enough. A tiny improvement can be statistically credible and still be operationally irrelevant.

Keep an exploration floor

Sending 100% of traffic to the current winner can make the decision impossible to challenge. Audience mix, ad creative, device distribution, and offer behavior can change.

An exploration floor keeps some traffic flowing to other eligible variations. It lets the system detect when the previous winner stops winning.

The acceptable floor depends on campaign volume and risk. A high-volume test can learn with a smaller percentage. A low-volume campaign may need a larger fixed split or manual review.

Protect against bad inputs

Automatic reweighting amplifies whatever data it receives. That makes data quality part of the routing decision.

Before moving traffic, verify:

• duplicate conversions are controlled;
• test events are excluded;
• click and conversion identifiers join correctly;
• traffic-quality exclusions are consistent across variations;
• spend is available for profit or ROAS objectives;
• payouts and currencies are normalized;
• the route did not change during the measurement window.

A fast algorithm cannot rescue a broken attribution path.

Make every decision reversible

The buyer should be able to see:

• which metric drove the change;
• the evidence available at decision time;
• the old and new weights;
• the guardrails that passed;
• when the next evaluation will run;
• how to pause or override automation.

Automatic does not have to mean opaque. A visible decision record makes the system easier to trust and debug.

A checklist for evaluating winner selection

Before automating a test, confirm that the product lets you:

1. choose one documented objective;
2. wait for minimum mature evidence;
3. exclude test and ineligible events;
4. update only the intended route weights;
5. preserve an exploration floor;
6. record the reason and evidence;
7. honor pause and manual override;
8. roll back safely.

These controls make each decision inspectable and reversible instead of hiding it behind an “AI optimized” badge.

Where Tracked is today

Tracked supports user-configured weights for pages, offers, destinations, and eligible rules. Automatic winner selection is in development; Tracked does not currently reallocate traffic on a buyer’s behalf. Set and review weights yourself, then inspect the resulting clicks and conversions in the campaign report.

The practical rule

Automate a decision only after the objective, evidence floor, attribution window, guardrails, and rollback path are explicit. Until then, keep traffic weights under the buyer’s control.

Tracked's current campaign workflow keeps hosted landing pages, routing, click and conversion reports, and visual editing together. Set traffic weights yourself today, then use the report to decide which page to change.

Frequently asked questions

Is automatic winner selection available in Tracked?

Not yet. Automatic winner selection is in development. Tracked’s current weighted routing uses weights configured by the campaign owner.

How many conversions are needed before choosing a winner?

There is no reliable universal number. The decision depends on traffic volume, conversion delay, the size of the observed difference, the chosen objective, and the minimum evidence rules defined before the test starts.

What should happen after a page wins?

Keep a documented decision record, preserve a small exploration path when appropriate, and make rollback possible. A winning result should remain traceable to the page, route, attribution window, and evidence that produced it.

Start free and configure landing-page traffic weights.