How it works

A model published in 1998 was rebuilt from its description, then the same search was run again on twenty eight years of additional data and a descriptor pool that did not exist at the time. This page is the whole account, including the parts that did not reproduce.

1. The 1998 model

Wessel, Jurs, Tolan and Muskal fitted percent human intestinal absorption to 86 compounds. The structures were encoded with 728 descriptors, reduced to 127 after correlated and near-constant columns were dropped. A genetic algorithm searched that pool and chose six descriptors, which were fed to a 6-4-1 feed forward neural network: six inputs, four hidden units, one output. The reported errors were 9.40 percent absorption units on the training compounds and 16.00 on an external set of 10 compounds.

The six chosen descriptors were dominated by hydrogen bonding and molecular geometry: how much of the surface could donate or accept a hydrogen bond, and how that surface was arranged. The model was small because the data was small, and the paper said so.

2. The reproduction

The descriptor software of 1998 is gone. ADAPT computed the 728 descriptors on hardware and in a language that no longer runs, and no archived build produces the same numbers today. Reproducing the model therefore meant reimplementing the six published descriptors from their written definitions rather than recovering their original code, and accepting that a reimplementation from a definition is not the same object as the original implementation.

With the six descriptors reimplemented and a 6-4-1 network fitted to the same 86 compounds, the reproduction was measured the way the rest of this work is measured: over 250 stratified extractions rather than one split. It gives 13.43 percent absorption units on the training compounds, 29.37 in cross validation and 30.39 on the extracted compounds, against the published 9.40, 19.70 and 16.00.

The reproduction is therefore worse than the 1998 report. A single favorable split can land near the published figure, and one of these extractions does, but the distribution sits well above it. The method carries over; the published error belongs to the tail of what repeated extraction shows rather than to its middle.

3. How the models are built

Every endpoint is built the same way, and the shape of it is what makes the numbers mean anything.

The pool. Each model starts from a large set of calculated molecular descriptors, 1,301 for absorption and the Caco-2 small molecules, and 1,257 for the peptides. Inside every extraction that pool is cut by two objective rules: drop any descriptor whose values are more than 80 percent identical, then drop one of any pair correlating above 0.90. Between 127 and 184 survive, depending on the endpoint: about 148 for absorption, 181 for the Caco-2 small molecules, 142 for the Caco-2 peptides and 131 for the artificial-membrane peptides.

The search. A genetic algorithm evolves descriptor sets of 5 to 40 members in steps of 5, 5 to 55 for the Caco-2 small molecules and 5 to 60 for the Caco-2 peptides, for up to 40 generations with a population of 60. Absorption and the artificial-membrane peptides settle at a median of 25 descriptors, the Caco-2 small molecules at 55 and the Caco-2 peptides at 50. Each candidate is scored by a random forest out of bag, so the search never reads a compound outside its training group and the extracted group stays untouched until the search has ended.

The repetition. All of it runs again on 250 independent stratified extractions. That gives a distribution of set sizes, a recurrence count for every descriptor, and a distribution of errors rather than a single number that happened once.

The set that ships. The 250 selected sets are not identical, and they are not expected to be: the descriptor space is redundant, so many different sets do a similar job. What ships is the consensus, the most recurrent descriptors taken to the median selected set size. That union is not any one of the selected sets, so it has no error yet. It is fixed and evaluated over the same 250 extractions, and those numbers are the ones reported on this site.

The model that ships. One random forest, 300 trees, at least 2 samples per leaf, fitted on every compound in the dataset. Nothing is withheld, because the evaluation above is what states the performance, not a leftover test set.

3b. What the absorption search chose

The deployed absorption model reads 25 descriptors. Every one of them is listed below, read from the bundle rather than typed. The recurrence beside each is a property of the search that chose it, not of the model.

These descriptors are what the absorption model reads. Absorption is led by charge and hydrogen bond donation: the mean partial charge was chosen in every single extraction, and carboxylate count, donor count and donor surface fraction follow it.

The kfm14 prefix names a block of fourteen size and composition terms: molecular weight, heavy atom and bond counts, rotatable bonds, ring count and element counts. They are borrowed from the ligand featurization of a kinase foundation model, and every one of them is a count or a sum computed directly from the structure.

The 1998 model, searching an unrelated pool with software that no longer runs, chose hydrogen bonding and geometric descriptors. The modern search, over a pool assembled from a different toolkit, arrived at the same physics by a different route.

4. The ceiling is in the data

Published absorption values for the same drug disagree with each other, and the disagreement is not small. Of the 780 compounds modeled, 262 carry more than one published value. Sixty one of those disagree by more than 20 points. Thirty one disagree by more than 40.

Methotrexate appears in the literature at 20, 59, 65, 70 and 100.

A model fitted to labels that disagree by 40 points cannot be more precise than 40 points on those compounds, whatever it does internally. The error below is therefore close to the resolution the published record currently supports, rather than a shortfall against some cleaner achievable number. Reporting a tighter interval would mean reporting a precision the measurements do not have.

Fitted on all 780 compounds, with 25 descriptors. Nothing is withheld from the shipped model, so none of the numbers below comes from its own fit: they are measured over 250 extractions of the same fixed descriptor set.

Error depends on where the compound sits, so a single overall figure would flatter the hard cases. The bands are below.

5. Applicability

The model fails on molecules unlike its training set, and the important part is that it fails quietly. A compound can break no descriptor range and still sit where these descriptors do not resolve it, so the failure looks like an ordinary answer.

So every prediction carries a distance to its five nearest training compounds in the scaled descriptor space, and that distance is compared against the distribution of the same quantity inside the training set. Above the 95th percentile the region is called sparse. Above the 99th it is called far. A prediction from either region is flagged in a visible box above the result rather than in a footnote under it. The prediction is still shown, because hiding an answer teaches a reader nothing, and a flag is itself an answer: it says the model has nothing reliable to offer for this structure.

6. The peptides

The two cyclic peptide models were built the same way, from a peptide pool of 1,257 descriptors: Morgan bits, two-dimensional RDKit descriptors and a peptide-specific block. For the Caco-2 peptides the search chose Balaban's J, a topological index, in 84.40 percent of extractions, the minimum partial charge in 83.60 percent and the maximum in 79.60 percent. For the artificial-membrane peptides it chose calculated logP in 79.60 percent, that same balance term in 43.20 percent, and the fraction of N-methylated amides in 23.20 percent.

Two assays, two models, never pooled. Both are served here and each answers for itself. The membrane model is passive diffusion through lipid, with no cells present. The Caco-2 model is transport across a living monolayer of a human intestinal cell line, where transporters and efflux are active. On the 229 peptides a single study measured in both, the assays correlate 0.62, with the membrane value about half a log unit below the Caco-2 value. A relationship that loose between two assays on the same peptides is the reason this site keeps two models rather than one.

The two released files share 588 peptides, and comparing those two files row by row is the wrong comparison: most of those pairs come from different studies, and across studies the two assays carry no usable relationship. The figures above come from the single-study records instead.

A value of exactly -10.00 is a below-detection placeholder, not a measurement. It stands for a peptide whose permeability was under the limit of the assay, and it appears for 21 peptides in the Caco-2 file and 269 in the membrane file. Treat it as a floor, and censor or filter it before fitting anything to these files.

No combined number is offered, no conversion between them is valid, and a query is never answered by whichever model happens to be available. Falling back would return a confident number measuring something you did not ask about.

A structure is scored by both models in one request, and each result is labeled with the assay it belongs to. Two answers that differ are two measurements that differ, not a disagreement to resolve.

The two Caco-2 panels of that figure, left and middle, are the two models below: the same measurement on different chemistry.

6b. What the cyclic peptide Caco-2 model achieves and reads

Fitted on all 1,281 peptides, with 50 descriptors, measured over 250 extractions of the fixed consensus set.

An error of 0.49 log units is a factor of 3.10 in the permeability rate. A prediction of one rate could reasonably be that rate multiplied or divided by about 3.10.

The deployed Caco-2 peptide model reads 50 descriptors. Every one of them is listed below, read from the bundle rather than typed. The recurrence beside each is a property of the search that chose it, not of the model.

It reads twice as many as the membrane model. The Caco-2 peptide search was run on a grid of 5 to 60 descriptors and settled at a median of 50. Its inputs are not those of the membrane model and the two are not interchangeable.

6c. What the artificial membrane model achieves and reads

Fitted on all 7,298 peptides, with 25 descriptors, measured over 250 extractions of the fixed consensus set.

An error of 0.82 log units is a factor of 6.68 in the permeability rate. A prediction of one rate could reasonably be that rate multiplied or divided by about 6.68.

The deployed membrane model reads 25 descriptors. Every one of them is listed below, read from the bundle rather than typed. The recurrence beside each is a property of the search that chose it, not of the model.

These descriptors are what the membrane model reads. Passive permeability is led by size, shape, partial charge and lipophilicity, with polarity entering only per heavy atom. It reads a different set from the absorption model: these are different models of different physics and their inputs are not interchangeable.

6d. What the small molecule Caco-2 model achieves and reads

The same procedure, applied to a fourth dataset: apparent permeability across a Caco-2 monolayer measured on small molecules from lead optimization, rather than on cyclic peptides. It predicts the same quantity in the same units as the peptide Caco-2 model, and the two are not interchangeable.

Fitted on all 3,771 small molecules, with 55 descriptors, measured over 250 extractions of the fixed consensus set.

An error of 0.35 log units is a factor of 2.25 in the permeability rate. A prediction of one rate could reasonably be that rate multiplied or divided by about 2.25.

The deployed small-molecule Caco-2 model reads 55 descriptors. Every one of them is listed below, read from the bundle rather than typed. The recurrence beside each is a property of the search that chose it, not of the model.

The kfm14 prefix names a block of fourteen size and composition terms: molecular weight, heavy atom and bond counts, rotatable bonds, ring count and element counts. They are borrowed from the ligand featurization of a kinase foundation model, and every one of them is a count or a sum computed directly from the structure.

What the numbers are

Every figure above is measured over 250 extractions of the descriptor set that ships, not over the search that found it, and not on the shipped model's own training data.

No comparison against a model that answers the training mean appears here or anywhere on this site. Such a reference makes any model look good and tells a reader nothing about whether an answer can be used. What an error means in practice is in the bands and in the decision accuracy.

Predict absorption See the training data