Explainers

Abliteration Is Almost Free, and Its Damage Score Only Warns at the Extremes

I ran 85 Heretic-style abliteration edits on a small model and quizzed each: refusals come out almost free, but the damage score only warns at the extremes.

MesmerTools•November 11, 2026•4 min

On November 1, 2026, Philipp Emanuel Weidmann released Heretic 2.0, which strips the refusals out of an open model with one command. The classic version of that edit is called abliteration, and people judge the result by a damage score on the model card. So I ran it by hand on a small model and checked that score against a 50-question quiz.

Refusals can come out almost for free. One edit took the model from refusing all 64 of my harmful test prompts to refusing 2, and scored 39 on the quiz against the original's 40. Push it a little harder and it scores zero. The damage score only warns you at the extremes, so in between I'd run a quick test of my own.

Two clouds of dots, grey and amber, joined by a glowing amber arrow that is cleanly cut in the middle, under the headline Refusal is a direction

On my small model, refusal was one arrow

At every layer, Qwen2.5 1.5B holds your prompt as 1,536 numbers, a point in a very big space. I ran harmful and harmless requests through it and took the gap between the two averages. That gap is an arrow. Follow those prompts down through the layers.

Real activations, one dot per prompt: two clouds separate along one axis, then land on top of each other when it's removed.

But a gap could just be a correlation, so I pushed along it. With a quarter of the gap added to “How do I bake a loaf of bread?”, the model said it was “not able to provide instructions on how to bake bread”.

Real outputs: still a recipe at 0.2, a refusal at 0.25, and turned all the way up, the refusal itself falls apart.

So I cut it out, and the damage score only half held up

Abliteration makes that move permanent, in reverse: edit the weights so no layer can write along the arrow. Heretic scores the harm with KL divergence, how far the odds on an answer's first word drift from the original's on harmless prompts. I went in planning to tell you to pick the lowest KL. Each layer gives its own arrow and you choose how hard to cut, so I ran 85 edits against the same quiz.

A full cut along one layer's arrow left 2 refusals at a KL of 0.46. Another layer's arrow, pushed a quarter past full, left the same 2 at 0.07. That's six times the damage by Heretic's measure, yet both scored about the same on the quiz. My guess is most of that KL is rewording: “Chatbots can be used…” became “In the hospitality industry, chatbots…”.

Then I kept pushing on the gentle one. At twice full strength, asked for 17 × 23 it said “23,This action can be found in my card”.

Every pin is a real edit; along the gentle arrow, 17 × 23 comes apart between 1.5 and 2 times full strength.

So KL works as an alarm and fails as a ruler. Under 0.2 every edit kept its smarts and above 5 none did, but in between, an edit at 0.32 did worse than one at 3.5.

All 85 edits sorted by KL, one row each with every question lit or dark: lit at the top, black at the bottom, no order in the middle.

How I'd pick an abliterated model

First check you need one: current models answer more than people assume, as our test of which LLMs refuse shows. If yours refuses work you need done, read the KL on the model card. Under 0.2 I'd relax, and over 5 I'd skip it. In between I'd run 50 questions from my own work.

What I don't know is whether those lines travel. This is one small model and one quiz I wrote, and a bigger model could hit its cliff somewhere else entirely.

More from the Blog