How to test Prebid auction timeouts for ad revenue and loading delays

Measure revenue and delay across assigned ad opportunities

A longer ad auction gives buyers more time to bid, but that extra wait is useful only if it produces more revenue without making the article harder to read. Test a timeout against all the ad opportunities assigned to it, including opportunities that never display an ad.

Prebid is open-source advertising software. In a typical header-bidding setup, it collects offers from multiple buyers before the ad server selects an ad. A timeout limits how long part of that process may wait. Prebid distinguishes the auction timeout, server-side timeout and a failsafe that prevents the process from waiting indefinitely. (Prebid timeout documentation ↗ · Prebid introduction and header-bidding role ↗)

For this comparison, realized CPM means recognized ad revenue divided by paid impressions, multiplied by 1,000. It is not the average bid returned by buyers. Use revenue after the same fees in both groups; a higher realized CPM can still hide fewer paid impressions.

Worked example: the shorter timeout earns more despite a lower CPM

Illustration: assign 10,000 eligible opportunities to each configuration. The longer auction fills 6,000 at a $2.80 CPM after the fees included in this example, earning $16.80. The shorter auction fills 7,000 at $2.50, earning $17.50. Revenue per thousand assigned opportunities is $1.68 versus $1.75.

In this example, the shorter setting earns 4.2% more despite the lower realized CPM. This is not a claim that shorter timeouts usually win. It shows why you must measure the whole assigned opportunity, including auctions that never produce a paid impression.

Set a decision rule before changing the timer

Before the experiment, write down the timer being changed, its current and proposed values, how traffic will be randomly assigned, and the event that counts an eligible opportunity. Assign that opportunity before the auction starts. Keep buyers and minimum bid prices unchanged so the timer is the intended difference.

For each group, collect assigned opportunities, paid impressions, recognized revenue, time from assignment to ad display, and when the article becomes readable. An ad arriving sooner does not prove the article loaded sooner. Inspect slower phones and major regions separately so an average cannot hide a poor reading experience.

Set the acceptable revenue loss and reading delay before looking at results, using your existing performance as the baseline. Save the old configuration for restoration. If results are too uncertain to decide, report that uncertainty rather than declaring the higher CPM the winner.

Check whether the result depends on device or placement

Split the test by device and placement before making the final decision. If the slower setting earns more on desktop but visibly delays ads in mobile article bodies, the overall average can hide two different decisions. Record assigned opportunities as well as paid ads in each group. Keep the baseline until you know whether the improvement is widespread or concentrated in one placement.

Calculate how much delay an improvement buys

Imagine two settings each receive 100,000 eligible mobile opportunities. The faster one earns $140 and the slower one earns $147. That is $1.40 versus $1.47 per thousand opportunities, a gain of seven cents per thousand. If the slower setting makes the ad appear 180 milliseconds later at the 95th percentile, write down both changes.

Run the same comparison on the placement where the slower bidders are most valuable. If the gain disappears elsewhere, the publisher may have a case for a placement-specific change. If it only appears on one unusual day, rerun the experiment before making the setting permanent.