R1Signal
Rat expression adds a small, stress-response-specific benefit.
Three models, identical except the feature block. Gene expression alone already
beats chance — proof the rat transcriptome carries human-toxicity signal.
Why. Expression helps stress-response and the two transcriptionally active
receptors (PPARγ, ER are ligand-activated transcription factors), and does nothing for the pure
ligand-binding assays (AR-LBD, ER-LBD) where structure already hits ~0.90. The SR > NR prediction holds.
R2Gotcha caught
The benefit only appears once the two feature blocks are balanced.
Before z-scoring the PCA blocks, structure's high-variance components drown out the
100 expression dimensions and fusion loses. Documented as a before/after so the win can't be dismissed.
Why. An L2 head penalizes all coefficients equally, so an unscaled block with larger
variance is effectively ignored. This is exactly the "expression drowned out" artifact — and why the
effect is real but fragile.
R3 · R4Holds under stats
100 components is optimal, and SR > NR is significant at the repeat level.
The stress-response benefit grows monotonically with the number of expression components —
it's distributed signal, not overfit noise. And the panel contrast survives a formal test.
Why. The repeat-level test is significant; the assay-level test is a strong trend
(only 12 assays, and PPARγ/ER break the clean split for a good biological reason). The real divide is
"acts through gene regulation" vs "acts through binding."
R5 · R5bFragile
The benefit is model- and encoder-dependent — and ECFP beat ChemBERT.
Two robustness probes. Gradient-boosted trees overfit at N=177 and erase the gain.
A frozen ChemBERT structure encoder turned out weaker than plain ECFP fingerprints.
Why ChemBERT loses — a coordinate mismatch, not a knowledge gap. ECFP is a sparse,
axis-aligned basis where each bit is a substructure, so a linear head reads "has reactive group
X → active" straight off — exactly how these assays fire. A frozen transformer stores the same facts in a
dense basis rotated for a different objective (reconstructing SMILES tokens); re-isolating "has a
nitroaromatic" from 768 entangled dimensions is a learning problem you can't solve at N=177. So the loss
is expected — and it strengthens the study: "does GE add?" ran against a genuinely strong,
literature-appropriate baseline, not a weak one. It fires especially cleanly here because SR toxicity is
driven by reactive structural alerts — the very substructures ECFP enumerates and a fingerprint
was built for.
Ran it → next card
That "cheap next try" (ECFP-counts + physchem + L1) and the deferred neural net were both
run — see the next card. Short version: N is the ceiling, not the encoder or the head.
R5c · R5dChoice validated
We gave structure its best shot — and finally ran the deferred neural net. Both confirm the setup.
Two "did we under-model this?" probes: a richer counts+physchem structure baseline, and
the regularized multitask MLP the torch env had blocked. Neither overturns the result — and the MLP is
decisively worse.
Why it all points one way. A richer representation lifts structure just +0.006
(inside the CI); L1 on 2066 raw features is worse, because N=177 can't estimate that many
sparse coefficients; and the MLP — small and heavily regularized — still loses on every arm, overfitting
generic variance. Tellingly the MLP's SR-specificity collapses (SR +0.060 ≈ NR +0.057): the linear head
isn't just stronger here, it's the one that keeps the mechanism legible. N is the ceiling,
not the encoder or the head — which points the lever back at data comparability and the rat→human layer.
R6 · R7The wall
More data didn't help — matched data helped, in proportion to how comparable it was.
Pooling a second rat dataset (TG-GATEs, +79 compounds) with a mismatched exposure
window (hours) washed the signal out. The time-matched version (days) recovered it partway.
The recovered signal tracks cross-dataset agreement.
N=177 · clean
+.025
SR ΔAUC
single protocol
+256 · hours
0.38
SR → +.001
ComBat agreement r · washed out
+256 · days
0.44
SR → +.013
ComBat agreement r · recovered
Why. Even same-platform, same-organ rat liver signatures for identical molecules agree
only r ≈ 0.44 after batch correction — a 24-hour liver and a 5-day liver are
different biological states. The bottleneck is comparability, not sample size. It's the measurable,
in-rat preview of the animal→human gap.
Val AReframed
Held-out Tox21 is infeasible by construction — the cross-validation already is that test.
Every rat compound with a Tox21 label is, by definition, in the supervised set (it's the
intersection). There is no external hold-out — so the out-of-fold CV already answers it.
Why. The supervised set is (TG-GATEs ∪ DrugMatrix) ∩ Tox21. Honest to
say so rather than manufacture a set that doesn't exist.
Val BStructure wins
For real human liver injury (DILIrank), the model adds nothing over structure.
A leakage-safe ladder on 134 shared drugs. The sanity gate passed first — measured Tox21 →
DILI ≈ chance, reproducing the published near-random result — then structure beat everything.
Fused representation
0.600
Predicted Tox21 · cross-fit
0.465
The decisive check. The chained pipeline (predicted Tox21 → DILI) looked promising at
0.646 — but that was leakage. Regenerating the
Tox21 predictions out-of-fold collapsed it to 0.465 ≈ chance. A ~0.12-AUC
illusion only cross-fitting exposes.
Val B · 2ndBiology edges it — noisy
For market withdrawal, the pattern flips: biology beats structure.
A separate, noisier target (ChEMBL withdrawal status). Here structure is near-chance —
withdrawal is driven by in-vivo/clinical effects, not molecular shape — and Tox21/expression features win.
Treat as a lead, not a headline. Only 35 withdrawn positives, a heterogeneous label
(drugs are pulled for efficacy and commercial reasons too), and the predicted-Tox21 number swings between
encoders — so it's a genuine, honest contrast worth chasing with a larger tox-specific set.