We've updated how model performance is reported
Tuesday 1st September 2026
We've made two changes to give you a more realistic picture of how your models perform. Your models themselves haven't changed — only how we measure and report them. The first change applies to every model, including ones you have already trained. The second applies to models you train from now on.
On every model: Spearman's ρ replaces R²
R² measures how close each prediction lands to the measured value. Spearman's ρ measures how well the model ranks molecules against each other — usually what you rely on when choosing between designs. A model whose predictions are all shifted by half a log unit still ranks molecules perfectly, but R² penalises that shift heavily. R² can also fall below zero, which looks like a broken model when it usually means the molecules it was scored on covered a narrow range.
Ranking isn't the whole story, though. If every compound is correctly predicted to be poor, you need new designs, not the best of a bad lot. That's why we show ρ alongside MAE (mean absolute error), which tells you how far predictions typically sit from the measured values. MAE is reported in the units your property is measured in, so there is no conversion to do. The one exception is a property whose values span more than two orders of magnitude (a raw IC50 in nM, say): we model those on a log scale, and its MAE is reported on that log scale too.
Your existing models are not retrained or re-evaluated on different data. We read back the cross-validation results they already had and calculate ρ from them, so the score you see is measured on exactly the same molecules as before. Because ρ is not penalised by a constant offset, it is almost always the higher of the two numbers — so most existing models will show a better score than before. A small number of much older models whose training records are no longer available cannot be recalculated, and will show a dash until you next retrain them.
On new models: we validate against your newest chemistry
For models you build from now on, when your dataset carries a numeric molecule ID we sort by that ID and hold back your most recent molecules, then score the model only against those. The more measurements a property has, the more we can afford to hold back:
| Molecules measured for that property | Held back to score the model |
|---|---|
| 1,000 or more | 200 newest |
| 500 – 999 | 100 newest |
| 100 – 499 | 50 newest |
| Fewer than 100 | none — cross-validated instead |
Counted per property, so one dataset can have a newest-molecule score on potency and a cross-validated score on a property you have measured less often.
The smaller the held-out set, the noisier the score. A ρ measured on 50 molecules will move around more than one measured on 200, so treat small differences on small sets with caution. This is also why properties with fewer than 100 measurements stay cross-validated rather than getting a tiny held-out set.
Cross-validation mixes old and new molecules together, so the model is always tested on compounds that have close analogues in its training set. Holding back your newest molecules asks the question that matters: how well does this do on chemistry it genuinely has not seen? Datasets without a numeric molecule ID keep being cross-validated, and we label which method produced each score. Cross-validation is 5-fold with scaffold-grouped splits, so molecules sharing a core scaffold are always kept in the same fold. See How models are evaluated for the full method.
Tip: if your IDs don't reflect chronology. Purchased libraries and merged datasets often have IDs that say nothing about the order compounds were made or acquired. You can assign your own sequential IDs to define a chronology — keep a lookup table mapping back to the originals. Any ID column works as long as every value contains digits; we strip other characters and sort on the number that remains, so CMP-00123 sorts as 123.
What this looks like in practice
Here is one newly built 155-molecule dataset with three endpoints, scored both ways, with the 50 newest molecules held back. Two things are worth noticing. On potency, holding out the newest molecules costs R² 0.18 but ρ only 0.04: the predictions drifted in absolute terms on newer chemistry, yet the model still puts molecules in nearly the right order. On LogD both metrics fall sharply — and that is the more useful signal, because it tells you cross-validation had been concealing how this model handles unfamiliar chemistry.
| Endpoint | Cross-validation R² | Cross-validation ρ | 50 newest held out R² | 50 newest held out ρ |
|---|---|---|---|---|
| Potency (pIC50) | 0.57 | 0.66 | 0.39 | 0.62 |
| LogD | 0.67 | 0.81 | 0.14 | 0.43 |
| Metabolic stability | 0.28 | 0.56 | −0.18 | 0.11 |
R² here is the standard coefficient of determination (1 − residual sum of squares ÷ total sum of squares) computed on the evaluation predictions. It appears in this post for comparison only and is no longer shown in the app.
Your models are unchanged
Once the metrics are calculated, the model you use for predictions is refit on all of your data, including the held-out molecules. The evaluation is a measurement, not a restriction on training.
We checked this directly. In a separate test we built models both with and without a held-out set, then ran both over 24 molecules that neither had seen — taken from a public blind-challenge dataset with independently measured values, so nothing about the comparison was ours to influence.
| Endpoint | Without held-out set ρ | Without held-out set R² | With held-out set ρ | With held-out set R² |
|---|---|---|---|---|
| Potency (pIC50) | 0.715 | 0.768 | 0.706 | 0.757 |
| LogD | 0.874 | 0.726 | 0.874 | 0.726 |
| Metabolic stability | 0.805 | 0.575 | 0.786 | 0.530 |
The same 24 unseen molecules, scored by both. The largest difference is 0.019 ρ; on LogD the two agree to five decimal places. Reference values from the ASAP–Polaris–OpenADMET blind challenge.
On truly unseen molecules the two perform the same. Holding molecules out to measure a model does not cost you anything in the model you end up using.