A bidding tool cut our cost per install by 23% and added more than eleven thousand installs on the same money. Then I put its own invoice into the table as a line item, and the year came out about $2,400 behind.
In 2020 I ran Apple Search Ads for Playtika, on an agency engagement subcontracted through a mobile-marketing agency called Incipia. Apple Search Ads, ASA in the trade, is the paid placement at the top of App Store search results.
Six of Playtika's mobile game titles were under management at once, thousands of keywords across dozens of country storefronts. One number ran everything: cost per install, or CPI, the media spend divided by the installs it bought.
A vendor pitched us a platform that promised to push that number down. The evaluation that followed is the most useful thing I took out of that year, and it had almost nothing to do with mobile games.
What the platform actually did
The job was bid management at a scale a person cannot keep up with. Every App Store search runs as an auction, and you set a maximum cost-per-tap bid on each keyword. Apple's own documentation calls that bid a price ceiling, and says the price you end up paying may come in below it.
So the lever is the bid, keyword by keyword, day after day, across thousands of terms in dozens of storefronts. Software is genuinely better at that grind than a human with a pivot table, and this software was good at it.
It delivered exactly what the pitch said
Modeled over a year with media spend pinned at $140,000 either way, every metric moved the right direction. The install price fell from $3.73 to $2.87. Installs climbed from about 37,500 to 48,800. Purchases rose from 375 to 488.
Nothing in that is in dispute. The platform was hired to bring the install price down, and it brought the install price down, with the volume and the buyers moving behind it.
On that evidence renewal is a formality, and the conversation moves along to whatever else the vendor sells.
The row that changed the answer
So I built the version of the table the vendor would never build. The same year side by side, with and without, and then one more line underneath: what the thing cost.
The extra revenue attributable to the platform came to about $5,600. The platform's price for the year was $8,000. The account finished about $2,400 down.
The tool did everything it promised. Cheaper installs, more of them, more purchases. It still lost money, because its invoice is part of the funnel.
Why the fee never makes it into the evaluation
This is easy to miss for a structural reason rather than a careless one. The metric lives in the ad platform, where the effect is visible every morning and reads as a win. The fee lives in accounts payable, where nobody is grading channel performance.
Two systems, two owners, no report that puts them on the same page.
The vendor's case study measures the metric. Your ledger measures the tool.
Two things the table cannot see
Both of them cut against the tool rather than for it, which is why they belong in the piece. The comparison models a year at held-constant spend instead of a live holdout test, and it assumes the extra installs monetize exactly like the ones already coming in, at roughly fifty cents of revenue each.
That assumption is generous. A lower install price usually comes from broader, lower-intent keywords, and lower intent tends to mean less revenue per install. The revenue column is also gross in-app purchase revenue, before the app store's cut and before any cost of running the game.
Correct for either one and the hole gets deeper. The direction of the finding was never in question; only its size was.
The software was fine. The account was too small for it.
Here is the part that changed how I price anything sold on a flat annual fee. The benefit scaled with spend. The fee did not.
The lift worked out to about four cents of extra revenue for every dollar of media running through it. Covering $8,000 on that rate takes somewhere near $200,000 of annual spend. At $140,000 the account sat under the line. At $250,000 the identical product, performing identically, is an easy yes.
That reframes the question. Stop asking whether the tool works and start asking how much volume you are pointing through it, because a flat fee against a proportional benefit always has a crossover somewhere.
Judge a tool on incremental profit after its own fee, on your account, over a defined window. Everything else is a testimonial.
Running the test on your own stack
None of this needs a data team. It needs one column added to a comparison you are probably already half building.
- Fix the window before you start, a quarter or a year, and write down the single metric the vendor claims to move.
- Measure the effect on your own account, with spend held as steady as you can hold it, or with a holdout if you can run one.
- Convert the effect into revenue, then into gross profit, so a margin-thin lift cannot pass itself off as a win.
- Subtract every cost the tool creates: its fee, any extra media it pulls, and the analyst hours it takes to operate.
- Solve for the spend level where a flat fee would break even, then check which side of that number you are on.
- Repeat once a year on every subscription line, because the account keeps changing and the fee usually does not.
The annual pass is the step everyone skips. A purchase that cleared the bar comfortably at last year's volume can slide under it after a budget cut or a channel shift, and nothing about the renewal notice will mention that.
This is lens four of my 30-day growth audit, the paid-accounts pass, pointed at the software sitting on top of the account rather than the campaigns inside it. The same arithmetic decides whether to keep renting a capability or own it outright, a call I work through in the marketer who ships.
Two habits keep the number honest. Grade whatever the tool optimizes against the same revenue-connected events every time, which is what my creative scoring system does. And put gross profit on top instead of revenue, the case I make in LTGP:CAC.
What this costs you if you skip it
Every stack I audit has at least one of these in it. A tool that works, on a metric that is real, priced above what its work is worth at this account's size. Sometimes several.
They rarely get caught because each one passes the test it is given. Nobody is lying. The performance is genuine, the reporting is accurate, and the money still leaves.
What I took from that year is a habit I have used on martech ever since: before renewing anything, write the fee into the same table as the benefit, in the same currency, over the same window. Most of the stack survives it. The ones that do not were never earning their place.
If nobody has priced your martech subscriptions against what they actually return, that is a morning of work with real money sitting in it. Let's talk.