neuron zoo

A small language model has 86,016 neurons. Hold one of them down, and the model starts talking about cats.

Every neuron in Qwen3.5-0.8B, one row per layer, input at the top. The bright marks are the specimens below. Tap one.

push it down ← · → push it up
what we ask

what's actually happening

  1. A neuron here is just one number. Each of the model's 24 layers has a middle step where 3,584 numbers light up for every word it reads. That's 86,016 in total.
  2. Finding a specimen. We fed the model eight sentences about cats, eight about the ocean, and so on, plus some boring office sentences. (A second hunt did the same for dogs, rain, light, memory, praise, love, maths and mothers.) A neuron makes the zoo if it fires far more on one topic than on all the others.
  3. Holding it down. While the model writes, we add a fixed amount to that one number, at every word. Nothing in the model's weights changes. Let go and it's back to normal.
  4. Pushing down. The minus doses subtract instead of add. Mostly that does very little until the model breaks, so these switches mostly work one way. But a few opposites are wonderful: anti-joy talks in timesheets and shifts, anti-sleep becomes a hyperactive camp counsellor, anti-music announces it is “the CEO of your company”, and anti-laude just chants “root”.
  5. The dose. 1 means "push it about as hard as the whole layer usually pushes". Too little and nothing happens. Too much and the model falls apart into "how how how how".

Most of the neurons we tried did nothing useful, or just broke the model. The ones in the zoo are the few that held a theme and stayed (mostly) coherent. It's not that every idea has a neat switch. Some just happen to.