Undergraduate Research Assistant — LLM Safety & Optimization Research

The problem

Unlearning promises that a model can be made to forget something specific — a memorised training example, a harmful capability, a fact it should never have absorbed — without retraining it from scratch. The promise is attractive and the claim is unusually hard to check, because the obvious test does not work. Asking the model about the thing it was supposed to forget tells you only that this prompt no longer produces that answer, and a model can be steered away from a phrasing while retaining everything underneath it.

The harder problem is the collateral. A procedure aggressive enough to remove a capability is aggressive enough to damage capabilities nobody intended to touch, and that damage is invisible if you only measure whether the target was forgotten. Verifying an unlearning method therefore means measuring what it cost everywhere else, which is a question about the whole model rather than about the thing removed.

What I did

I investigated a set of LLM unlearning techniques, aimed at reducing hallucination and at removing capabilities that made models vulnerable to adversarial prompting, and built the evaluation programs that measured what those procedures did to the rest of the model.

The measurements were chosen to catch different failure modes. Diversity detects collapse toward a narrow output distribution, which is what a model does when it has been pushed hard away from a region of its output space. Fluency catches degradation that shows up as degraded language before it shows up as degraded content. Utility preservation asks the plainest version of the question: can the model still do the things it could do before, on tasks unrelated to what was unlearned. Since none of those catches meaning that has quietly shifted while the surface stays intact, I used BLEURT to compare unlearned outputs against their originals semantically, looking for cases where a response still read well and no longer said the same thing.

What the evaluation was built to catch

I no longer have the results from this work, so I will describe what the instrument was designed to detect rather than claim findings I cannot check.

The design rests on one assumption worth stating plainly: that the interesting failure of unlearning is not incomplete forgetting but collateral damage, and that the two are measured in opposite places. Incomplete forgetting shows up when you probe the target, which is where everyone looks. Collateral damage shows up everywhere else, on tasks chosen precisely because they have nothing to do with what was removed, and it is invisible to any evaluation that only asks whether the target is gone.

That is why the measurements are spread across output distribution, surface fluency, unrelated task performance and semantic drift rather than concentrated on the forgotten content. Each catches a different way a model can be damaged while still answering the one question the procedure was optimising for. Building the evaluation that way was the part of the project I would defend; whether the methods we tested survived it is something I would want to re-run before asserting.

What I’d do differently

The evaluation leaned on one semantic metric, and I would not build it that way again. BLEURT is a learned metric with its own training distribution and its own failure modes, so using it as the sole check on semantic preservation means any degradation it is insensitive to goes unrecorded — and there is no way to detect that from inside the evaluation, because the metric agreeing with itself looks identical to nothing having gone wrong. Two or three metrics that fail differently, with the disagreements between them treated as the signal worth investigating, would have been a more honest instrument.

I would also fix the target now rather than then. We measured whether capability survived and whether meaning survived, but never pinned down what successful forgetting would look like independent of the prompts we happened to test, which left the central claim resting on the absence of evidence rather than on evidence of absence.