Case study
Cross-Reactivity-Aware TCR–pMHC Binding Prediction
Jun 2026 – Present
The problem
Binding datasets record which T-cell receptor and peptide-MHC pairs were tested and found to bind. They rarely record which pairs were tested and found not to, because that is not what an assay is usually run to establish, so the negative class gets fabricated by pairing sequences at random.
That fabrication has a specific and damaging consequence. Random pairing leaves the binder and non-binder CDR3β pools nearly disjoint, so a model can separate the two classes by memorising the receptor and never looking at the peptide at all. It then scores beautifully on a random split and collapses on unseen peptides — the only split that measures what anyone wants to know. Cross-reactivity is what makes that shortcut expensive rather than merely untidy: one receptor recognises many peptides, so an untested pair is not weak evidence of non-binding but no evidence at all.
What I did
Before proposing anything I rebuilt the ground under the claim, because “positive-unlabeled learning helps” is only meaningful against a baseline you have measured rather than quoted. That meant a data loader reproducing the published tuple counts to within 0.005%, the hard split regenerated to its exact candidate-peptide counts across five seeds, and an evaluation protocol frozen in place so that nothing downstream could be compared against a moving ruler.
I also built the diagnostic the rest of the project depends on. The epitope-shuffle test re-pairs every receptor with a different peptide and rescores: a model that learned the interaction should fall to chance, while one exploiting the receptor-only shortcut keeps its score. A high hard-split number that survives shuffling is a red flag rather than a result.
What came out
The collapse reproduced, and the contrast is stark:
| Model | Split | AUROC |
|---|---|---|
| supervised-BCE | random + assay negatives | 0.973 ± 0.001 |
| supervised control | hard + random negatives | 0.523 ± 0.013 |
| nnPU | hard + random negatives | 0.493 ± 0.006 |
The confirmed-negative reference also reproduced, at 0.901 ± 0.023 balanced accuracy against a published 0.92, which is what tells me the pipeline is sound rather than merely pessimistic.
The shuffle diagnostic then produced something I had not planned for. Both models fall from roughly 0.97 to about 0.60 — so they are genuinely using the peptide — but they plateau there rather than at chance, and that residual tenth of a point is a receptor-only shortcut with a traceable cause. The assay negatives have a CDR3β pool about 99% disjoint from the binders’, so the receptor side alone carries real separability. The published reference model goes below chance on the same test, because its negatives share 14.7% of their receptor sequences with positives, leaving no shortcut to exploit and actively misleading it when fed wrong peptides.
That comparison is the useful result: how much exploitable shortcut a dataset contains is a property of how its negatives were built, and it can be measured rather than argued about. I went looking for a yardstick and found a diagnostic instead.
What’s next
Phase one drops fabricated negatives entirely, treating each peptide as its own positive-unlabeled problem over a shared encoder and propagating cross-reactivity by reweighting likely hidden positives through peptide similarity. It has to clear two bars, not one: beat the random-negative baseline on unseen peptides, and fail the shuffle test. A model that wins the first without the second has just found a better shortcut.
One constraint from this phase carries forward and is worth stating plainly. The assay negatives cover 17 peptides, all among the most frequent in the corpus, so the hard split excludes every one of them by construction — none survived in any of the five seeds. Hard-split evaluation therefore has to use fabricated negatives even though training will not, and pretending otherwise would undo the point of the exercise.