We've updated how model performance is reported
Friday 28th August 2026
We've made two changes to give you a more realistic picture of how your models perform. Your models themselves haven't changed — only how we measure and report them. The first change applies to every model, including ones you have already trained. The second applies to models you train from now on.
On every model: Spearman's ρ replaces R²
R² measures how close each prediction lands to the measured value. Spearman's ρ measures how well the model ranks molecules against each other — which is what you rely on when deciding which designs to make next. A model whose predictions are all shifted by half a log unit still ranks molecules perfectly, but R² penalises that shift heavily. R² can also fall below zero, which looks like a broken model when it usually means the molecules it was scored on covered a narrow range.
Your existing models are not retrained or re-evaluated on different data. We read back the cross-validation results they already had and calculate ρ from them, so the score you see is measured on exactly the same molecules as before. Because ρ is not penalised by a constant offset, it is almost always the higher of the two numbers — so most existing models will show a better score than before. A small number of much older models whose training records are no longer available cannot be recalculated, and will show a dash until you next retrain them.
On new models: we validate against your newest chemistry
For models you build from now on, when your dataset carries a numeric molecule ID we sort by that ID and hold back your most recent molecules, then score the model only against those. The more measurements a property has, the more we can afford to hold back:
| Molecules measured for that property | Held back to score the model |
|---|---|
| 1,000 or more | 200 newest |
| 500 – 999 | 100 newest |
| 100 – 499 | 50 newest |
| Fewer than 100 | none — cross-validated instead |
Counted per property, so one dataset can have a newest-molecule score on potency and a cross-validated score on a property you have measured less often.
Cross-validation mixes old and new molecules together, so the model is always tested on compounds that have close analogues in its training set. Holding back your newest molecules asks the question that matters: how well does this do on chemistry it genuinely has not seen? Datasets without a numeric molecule ID keep being cross-validated, and we label which method produced each score.
What this looks like in practice
Here is one newly built 155-molecule dataset with three endpoints, scored both ways, with the 50 newest molecules held back. Two things are worth noticing. On potency, holding out the newest molecules costs R² 0.18 but ρ only 0.04: the predictions drifted in absolute terms on newer chemistry, yet the model still puts molecules in nearly the right order. On LogD both metrics fall sharply — and that is the more useful signal, because it tells you cross-validation had been concealing how this model handles unfamiliar chemistry.
| Endpoint | Cross-validation R² | Cross-validation ρ | 50 newest held out R² | 50 newest held out ρ |
|---|---|---|---|---|
| Potency (pIC50) | 0.57 | 0.66 | 0.39 | 0.62 |
| LogD | 0.67 | 0.81 | 0.14 | 0.43 |
| Metabolic stability | 0.28 | 0.56 | −0.18 | 0.11 |
Your models are unchanged
Once the metrics are calculated, the model you use for predictions is refit on all of your data, including the held-out molecules. The evaluation is a measurement, not a restriction on training.
We checked this directly. In a separate test we built models both with and without a held-out set, then ran both over 24 molecules that neither had seen — taken from a public blind-challenge dataset with independently measured values, so nothing about the comparison was ours to influence.
| Endpoint | Without held-out set ρ | Without held-out set R² | With held-out set ρ | With held-out set R² |
|---|---|---|---|---|
| Potency (pIC50) | 0.715 | 0.768 | 0.706 | 0.757 |
| LogD | 0.874 | 0.726 | 0.874 | 0.726 |
| Metabolic stability | 0.805 | 0.575 | 0.786 | 0.530 |
The same 24 unseen molecules, scored by both. The largest difference is 0.019 ρ; on LogD the two agree to five decimal places. Reference values from the ASAP–Polaris–OpenADMET blind challenge.
On truly unseen molecules the two perform the same. Holding molecules out to measure a model does not cost you anything in the model you end up using.
