Recommendations the merchants cannot buy

A discovery platform has two customers who want opposite things. People want the most relevant answer. Merchants want to be the answer, and are willing to pay.

Both are legitimate, and the platform needs both. But there is an order of operations that cannot be reversed: the moment ranking becomes purchasable, results get worse, users leave, and the visibility the merchants bought becomes worthless. The revenue that looked like a business model destroys the thing being sold.

Paid placement has to be structurally separate

The distinction that holds is not a labelling convention. It is a structural one: paid placement occupies its own slots, drawn from its own selection process, marked as such — and the organic ranking has no input that money can move.

Blended designs, where paying merchants receive a scoring boost inside organic results, fail predictably. The boost starts small and defensible. It grows, because there is always a quarter where it is the easiest lever. Nobody can point to the moment quality broke, because the mechanism was designed so that no such moment exists.

Keeping the two separate means the question did we make results worse this quarter has an answer, which is the only reason it stays answerable.

Behaviour is a compromised signal

The obvious relevance signal is what users click and book. It is also shaped by what was shown, which was decided by the ranking being trained.

Left alone this closes into a loop. Whatever ranked well accumulates the engagement that keeps it ranking well, and a genuinely better merchant listed last month cannot climb because nobody saw it. The system converges on being confident about the past.

Mitigations are all variations on deliberately spending some traffic: reserve a share of impressions for items with little data, discount engagement by the position it was shown in, and treat a new listing's early performance as provisional. Each costs something measurable today against something that only shows up as gradual staleness — which is why they are usually argued about and rarely implemented.

Feedback needs to be captured against what was shown

If recommendation quality is going to be improved, the record has to include the state at the time: what was returned, in what order, and what the person did next.

Reconstructing that later from current data is impossible, because the catalogue, the ranking and the merchant set have all moved. Capture is cheap at request time and unavailable afterwards, which makes it one of the few things worth building before there is a clear use for it.

Metrics that quietly reward the wrong thing

Click-through rate rewards whatever provokes a click, which is not the same as whatever satisfies the person. Optimise it hard enough and the results fill with overstated listings and prices that turn out to be conditional.

Better signals sit further down: did the booking complete, was it honoured, did the person come back. They are sparser, slower and noisier — and they are the ones that correspond to the product working. A platform that optimises the fast metric because it is the measurable one usually finds out from its retention curve rather than its dashboard.

Governance of overrides

There will be overrides. A merchant is promoted for a campaign, another suppressed after complaints, a category reordered for a season. This is normal and necessary.

What matters is that each is an explicit record with an author and a reason, not a value edited in a table. Two years on, the useful question is why a particular merchant ranks where it does, and the honest answer has to be reconstructible. Systems that allow silent manual adjustment lose the ability to tell an intervention from a bug, and eventually the team stops trusting its own ranking — which is a worse outcome than any single bad result.

The tension does not resolve

There is no design that makes these two audiences want the same thing. The platform serves both by keeping one of them out of a specific mechanism, and by being able to demonstrate that it did.

That demonstrability is the whole point. Merchants accept that ranking is not for sale far more readily when the separation is visible and consistent than when they are told it is fair.