We've updated how model performance is reported

Tuesday 1st September 2026

We've made two changes to give you a more realistic picture of how your models perform. Your models themselves haven't changed — only how we measure and report them. The first change applies to every model, including ones you have already trained. The second applies to models you train from now on.

On every model: Spearman's ρ replaces R²

R² measures how close each prediction lands to the measured value. Spearman's ρ measures how well the model ranks molecules against each other — usually what you rely on when choosing between designs. A model whose predictions are all shifted by half a log unit still ranks molecules perfectly, but R² penalises that shift heavily. R² can also fall below zero, which looks like a broken model when it usually means the molecules it was scored on covered a narrow range.

Ranking isn't the whole story, though. If every compound is correctly predicted to be poor, you need new designs, not the best of a bad lot. That's why we show ρ alongside MAE (mean absolute error), which tells you how far predictions typically sit from the measured values. MAE is reported in the units your property is measured in, so there is no conversion to do. The one exception is a property whose values span more than two orders of magnitude (a raw IC50 in nM, say): we model those on a log scale, and its MAE is reported on that log scale too.

Your existing models are not retrained or re-evaluated on different data. We read back the cross-validation results they already had and calculate ρ from them, so the score you see is measured on exactly the same molecules as before. Because ρ is not penalised by a constant offset, it is almost always the higher of the two numbers — so most existing models will show a better score than before. A small number of much older models whose training records are no longer available cannot be recalculated, and will show a dash until you next retrain them.

On new models: we validate against your newest chemistry

For models you build from now on, when your dataset carries a numeric molecule ID we sort by that ID and hold back your most recent molecules, then score the model only against those. The more measurements a property has, the more we can afford to hold back:

Molecules measured for that property Held back to score the model
1,000 or more 200 newest
500 – 999 100 newest
100 – 499 50 newest
Fewer than 100 none — cross-validated instead

Counted per property, so one dataset can have a newest-molecule score on potency and a cross-validated score on a property you have measured less often.

The smaller the held-out set, the noisier the score. A ρ measured on 50 molecules will move around more than one measured on 200, so treat small differences on small sets with caution. This is also why properties with fewer than 100 measurements stay cross-validated rather than getting a tiny held-out set.

Cross-validation mixes old and new molecules together, so the model is always tested on compounds that have close analogues in its training set. Holding back your newest molecules asks the question that matters: how well does this do on chemistry it genuinely has not seen? Datasets without a numeric molecule ID keep being cross-validated, and we label which method produced each score. Cross-validation is 5-fold with scaffold-grouped splits, so molecules sharing a core scaffold are always kept in the same fold. See How models are evaluated for the full method.

Tip: if your IDs don't reflect chronology. Purchased libraries and merged datasets often have IDs that say nothing about the order compounds were made or acquired. You can assign your own sequential IDs to define a chronology — keep a lookup table mapping back to the originals. Any ID column works as long as every value contains digits; we strip other characters and sort on the number that remains, so CMP-00123 sorts as 123.

What this looks like in practice

Here is one newly built 155-molecule dataset with three endpoints, scored both ways, with the 50 newest molecules held back. Two things are worth noticing. On potency, holding out the newest molecules costs R² 0.18 but ρ only 0.04: the predictions drifted in absolute terms on newer chemistry, yet the model still puts molecules in nearly the right order. On LogD both metrics fall sharply — and that is the more useful signal, because it tells you cross-validation had been concealing how this model handles unfamiliar chemistry.

Endpoint Cross-validation R² Cross-validation ρ 50 newest held out R² 50 newest held out ρ
Potency (pIC50) 0.57 0.66 0.39 0.62
LogD 0.67 0.81 0.14 0.43
Metabolic stability 0.28 0.56 −0.18 0.11

R² here is the standard coefficient of determination (1 − residual sum of squares ÷ total sum of squares) computed on the evaluation predictions. It appears in this post for comparison only and is no longer shown in the app.

Your models are unchanged

Once the metrics are calculated, the model you use for predictions is refit on all of your data, including the held-out molecules. The evaluation is a measurement, not a restriction on training.

We checked this directly. In a separate test we built models both with and without a held-out set, then ran both over 24 molecules that neither had seen — taken from a public blind-challenge dataset with independently measured values, so nothing about the comparison was ours to influence.

Endpoint Without held-out set ρ Without held-out set R² With held-out set ρ With held-out set R²
Potency (pIC50) 0.715 0.768 0.706 0.757
LogD 0.874 0.726 0.874 0.726
Metabolic stability 0.805 0.575 0.786 0.530

The same 24 unseen molecules, scored by both. The largest difference is 0.019 ρ; on LogD the two agree to five decimal places. Reference values from the ASAP–Polaris–OpenADMET blind challenge.

On truly unseen molecules the two perform the same. Holding molecules out to measure a model does not cost you anything in the model you end up using.