Case study
Research Intern — Machine Learning for Healthcare & Biomedicine
Nov 2025 – Present
The problem
A T-cell receptor recognises a peptide presented on an MHC molecule, and predicting which pairs bind is the computational bottleneck in designing a cancer vaccine. Plenty of published methods report strong numbers on this task. The lab could not use any of them as a reference point, because a number from someone else’s paper describes their data, their negative sampling and their train/test split, and changing any of those changes the number.
This is not scepticism about the field so much as a fact about the task. Binding data is heavily positive-biased, negatives are usually generated rather than measured, and how you generate them determines how hard the problem is. A model that looks strong against randomly paired negatives may be separating plausible sequences from implausible ones rather than binders from non-binders. So the lab needed baselines of its own: one protocol, its own data, measured in house. Without them there was no way to tell whether a new architecture actually helped.
What I did
I benchmarked several published TCR–pMHC binding predictors against an in-house dataset, which meant standardising everything around them first: one preprocessing path, one splitting strategy, one set of evaluation metrics, so that differences between methods were differences between methods rather than artefacts of how each paper had set up its own experiment. Alongside that I trained and tuned deep learning models on the lab’s curated data, iterating on those same three choices, and read through the current modelling literature to work out which architectural and objective-function changes were worth trying against the distribution we actually had.
What came back was reassuring in one direction and sobering in the other. Absolute performance fell across the board once methods were run on the lab’s data under a common protocol, which is what you would expect when negatives are constructed differently and the split is no longer the one a method was tuned against. The ordering, though, held: the methods that led on their own benchmarks still led on ours. Published rankings turned out to be portable in a way published numbers were not, and that distinction is what made the baselines usable — we could trust them to say which approach was better without trusting them to say by how much.
What came out
The benchmarking work fed into PIT-GCL, a protein interaction model I helped develop and the lab’s NeurIPS submission. Its argument is that a protein’s sequence, its local geometry and its global shape are three different kinds of evidence, and that collapsing them into one representation throws away the distinctions. So the model keeps them apart and combines them late: ESM-2 residue embeddings carry sequence semantics, a learned RBF attention bias over pairwise Cα distances carries local geometry, and persistence landscapes computed from a Vietoris–Rips filtration in dimensions zero and one carry topology. Two towers encode the pair separately and meet through latent cross-attention, trained under a contrastive objective alongside binary cross-entropy.
The published results, from the public repository:
| Dataset | AUROC | AUPRC | Accuracy | F1 |
|---|---|---|---|---|
| PPIRef | 0.933 | 0.938 | 0.848 | 0.847 |
| STAG | 0.873 | 0.716 | 0.863 | 0.700 |
The gap between the two rows is worth noting. On PPIRef the model separates interfaces cleanly and precision tracks recall closely, while on the STAG TCR–pMHC benchmark AUROC stays respectable but AUPRC falls to 0.716 against an accuracy of 0.863. Ranking and thresholded accuracy can both look healthy while precision under class imbalance does not, and TCR–pMHC is where that shows.
What’s next
The benchmarking is the part I expect to keep paying off. Having measured baselines under one protocol means the next architecture can be judged against something the lab controls rather than against numbers borrowed from papers that used different data, and the ordering result makes those baselines cheaper to maintain — new methods can be screened on their published rankings before anyone spends time reimplementing them. That is the unglamorous half of the work, and it is the half that determines whether the interesting half is measurable at all.