Two simulated CVM platforms · one unchanged policy
| Platform | Method | Mean propensity | Holdout export |
|---|---|---|---|
| mock_cots 3.4.1 | Commercial CVM platform (simulated). Gradient-boosted propensity. | 0.0495 | yes |
| legacy_scorecard 1.2 | Legacy rules-based retention scorecard (simulated). | 0.0742 | no |
Correlation between the two platforms' scores: 0.654 — similar enough to be plausible substitutes, different enough that the swap is a real test. The operator's own control-trained risk averages 0.0448; both vendors train on the full base, which is the realistic default and encodes a churn rate last quarter's campaign already altered. “Can we define holdouts and get the assignments back?” is the question whose answer should worry you — without it there is no counterfactual and no uplift model, ever.
| Whose ranking decides | Audience overlap | Net margin, platform A | Net margin, platform B |
|---|---|---|---|
| Operator — expected value from the operator's own uplift model | 100.0% | €-8,204 | €-8,204 |
| Vendor — the platform's own propensity ranking | 59.6% | €-7,951 | €-10,365 |
Policy fingerprint cb42f4595a2b before the swap and
cb42f4595a2b after — the objective function, guardrails and
suppression rules are byte-identical. The operator's uplift column is unchanged, because it
is computed from the operator's own copy of features, treatment and outcome and does not
depend on who supplies the propensity.
| Of the top 1,800 by the platform's own ranking… | Would breach an operator suppression rule |
|---|---|
| mock_cots | 983 (55%) |
| legacy_scorecard | 1,403 (78%) |
These are customers with an unresolved complaint — where a discount offered before the fault is fixed reads as a bribe — or with an active port-out enquiry, where an automated offer is inappropriate and the case belongs with a trained agent. A platform ranking on churn probability alone finds them extremely attractive. Suppression is a conduct decision, and it is never the vendor's to make.
Let the operator's policy decide and the campaign is vendor-independent: identical audience, identical outcome, across two platforms built on different methods. Let the platform's ranking decide and the campaign changes with the supplier — a third of the audience differs, the outcome moves, and most of the list breaches rules the operator wrote.
Nothing here required renegotiating a contract. The separation is architectural: compute features in your own warehouse and push them in, apply policy after the score, write the decision log to your own storage. Exportable is the weak form of sovereignty; never having put it in is the strong one.