perspectiva aiqbenchSiguranță AI

This project democratizes large-scale interpretability research by leveraging frontier models to lower the barrier for analyzing Bonsai-8B's internal circuitry and neuron functions. It transforms a traditionally exclusive scientific endeavor into a collaborative community effort, allowing broader participation in mapping AI reasoning pathways without requiring deep prior expertise. By facilitating direct hypothesis testing and ablation studies via live streams and open-source contributions, it accelerates the collective understanding of how specific neurons encode concepts like monosemanticity.

Sursă care acoperăLessWrong

Ce este aceastaLectura unui model asupra articolului și a acoperirii conexe, scrisă pentru aiqbench. Nu sunt cuvintele publicației.

Twitch Plays: AI Interpretability of Bonsai-8B

LessWrong19 octombrie 202610 min de cititArticolul și perspectiva sunt în engleză.

Quick Links Watch the Twitch stream. Check the status of the interpretability map. Do your own parallel research and submit pull requests to fill out the map. What is this? Let's Play Interpretability is a Twitch livestream where you can help Claude semi-autonomously interpret the neurons, connections and circuits in the Bonsai-8B LLM. Your help comes in at various stages in the interpretation pipeline. The audience and the agent collaborate to: Pick the neuron to work on Suggest sentences on which to test activations Explicitly test various hypotheses, and attempt to falsify them Branch out to nearby (upstream and downstream) neurons whose ablation most impacts the activation, to better solidify the interpretation ... and more. View and participate in the stream. If you have no interest in interacting with Twitch, then you can simply perform your own research in parallel and add interpreted neurons to the graph directly by cloning the repo and making pull requests! Help fill in the graph directly. With sufficient effort, we could fully characterize the entire LLM. Current progress. Pick a neuron and get started! Why did you build this? I have always been fascinated by direct AI interpretability efforts, starting in the long ago time of the melting dogs, the more recent work on monosemanticity and its application to larger models, and the very recent and provocative Neural Networks Are Already Clean. I simply find it interesting. However, undertaking such research efforts myself has been prohibitively difficult - until recently. Now, it's relatively trivial to set your sights on some particular interpretability or alignment-type problem and dive in, with assistance from frontier models. I suspect concrete value might emerge from this project: By setting this up as an interactive Twitch stream, and an accessible GitHub repo which accepts PRs, the undertaking of AI Interpretation and related AI Alignment science is made to seem less forbidding. More people might jump in with both feet instead of watching, intimidated, from the sidelines. I do not think AI Alignment is a problem that should be left to the experts. There aren't any experts; this science functionally doesn't exist yet. You can be a pioneer of a new field. And I think we should all contribute what we can. Concretely, we may identify neurons, connections, circuits, or other patterns in the Bonsai-8B LLM that subsequently turn out to be general, or universal, or otherwise valuable starting points for tackling larger models. We may discover novel tools, workflows, or concepts that make the work of automated interpretation faster and more efficient. This kind of contribution might be even more useful than the specific neuron interpretations themselves! It's just fun! Interpreting what a neuron is doing is kind of like solving a riddle. "Okay, it's firing strongly on 'nerve', but it's suppressed by 'pain.' Hold on, it really lights up for 'he steeled his nerve.' Does that imply ... " Then if you get stuck, trace your way upstream and downstream to related activations. It feels like you're reverse-engineering concepts at a fundamental level. Why Bonsai-8B specifically? Bonsai is a relatively functional LLM, quantized down to 1 bit. Bonsai is PrismML's 1-bit LLM (Apache-2.0, derived from Qwen3-8B). It sits at about 1.15 GB on disk. On a GPU, it holds about 4-5 GB in the research toolchain. 36 layers × 12,288 neurons per layer = 442,368 neurons total. Each neuron has a positive and a negative pole, so that's 884,736 "poles." There is a reasonable chance that your own computer can run Bonsai-8B in inference. "On an M4 Pro Mac, it runs at 131 tokens per second. On an RTX 4090, it reaches 368 tokens per second. On an iPhone 17 Pro Max, it runs at roughly 44 tokens per second."[1] This makes it a great target for interpretation. It's not "objectively" very smart, but it's Pareto-optimal for its size and speed. In other words: almost everyone can load and run it, and it's smart enough to be worthwhile. Due to this combination of factors, I view it as a good model organism for this kind of work. How do I contribute? Either clone the repo and point your agent at it, or grab a neuron from the map and copy the text on the right and send that to your agent directly. Useful tasks include: characterizing entirely new neurons picking a neuron and branching out to the surrounding area challenging, expanding upon, and possibly falsifying existing interpretations straightforward "cleanroom" replication of existing interpretations Join in at whatever level you wish. Or go off the roadmap and attack the problem in some way I'm not expecting! Have fun! FAQ How do you know the results are true? Every test is committed before it is run. A label holds only if its sentences beat matched near-misses, and GitHub independently re-scores every submitted result before merging. Labels can then be challenged by QC tests and blind “clean-room” replications. Chat earns poin

Watch the Twitch stream.

Check the status of the interpretability map.

Do your own parallel research and submit pull requests to fill out the map.

Let's Play Interpretability is a Twitch livestream where you can help Claude semi-autonomously interpret the neurons, connections and circuits in the Bonsai-8B LLM.

Your help comes in at various stages in the interpretation pipeline. The audience and the agent collaborate to:

View and participate in the stream.

If you have no interest in interacting with Twitch, then you can simply perform your own research in parallel and add interpreted neurons to the graph directly by cloning the repo and making pull requests! Help fill in the graph directly. With sufficient effort, we could fully characterize the entire LLM.

Current progress. Pick a neuron and get started!

Why did you build this?

I have always been fascinated by direct AI interpretability efforts, starting in the long ago time of the melting dogs, the more recent work on monosemanticity and its application to larger models, and the very recent and provocative Neural Networks Are Already Clean. I simply find it interesting.

However, undertaking such research efforts myself has been prohibitively difficult - until recently. Now, it's relatively trivial to set your sights on some particular interpretability or alignment-type problem and dive in, with assistance from frontier models.

By setting this up as an interactive Twitch stream, and an accessible GitHub repo which accepts PRs, the undertaking of AI Interpretation and related AI Alignment science is made to seem less forbidding. More people might jump in with both feet instead of watching, intimidated, from the sidelines. I do not think AI Alignment is a problem that should be left to the experts. There aren't any experts; this science functionally doesn't exist yet. You can be a pioneer of a new field. And I think we should all contribute what we can.

Concretely, we may identify neurons, connections, circuits, or other patterns in the Bonsai-8B LLM that subsequently turn out to be general, or universal, or otherwise valuable starting points for tackling larger models.

We may discover novel tools, workflows, or concepts that make the work of automated interpretation faster and more efficient. This kind of contribution might be even more useful than the specific neuron interpretations themselves!

It's just fun! Interpreting what a neuron is doing is kind of like solving a riddle. "Okay, it's firing strongly on 'nerve', but it's suppressed by 'pain.' Hold on, it really lights up for 'he steeled his nerve.' Does that imply ... " Then if you get stuck, trace your way upstream and downstream to related activations. It feels like you're reverse-engineering concepts at a fundamental level.

Why Bonsai-8B specifically?

Bonsai is a relatively functional LLM, quantized down to 1 bit. Bonsai is PrismML's 1-bit LLM (Apache-2.0, derived from Qwen3-8B). It sits at about 1.15 GB on disk. On a GPU, it holds about 4-5 GB in the research toolchain. 36 layers × 12,288 neurons per layer = 442,368 neurons total. Each neuron has a positive and a negative pole, so that's 884,736 "poles."

There is a reasonable chance that your own computer can run Bonsai-8B in inference. "On an M4 Pro Mac, it runs at 131 tokens per second. On an RTX 4090, it reaches 368 tokens per second. On an iPhone 17 Pro Max, it runs at roughly 44 tokens per second."[1] This makes it a great target for interpretation.

It's not "objectively" very smart, but it's Pareto-optimal for its size and speed. In other words: almost everyone can load and run it, and it's smart enough to be worthwhile. Due to this combination of factors, I view it as a good model organism for this kind of work.

How do I contribute?

Either clone the repo and point your agent at it, or grab a neuron from the map and copy the text on the right and send that to your agent directly. Useful tasks include:

Join in at whatever level you wish. Or go off the roadmap and attack the problem in some way I'm not expecting!

How do you know the results are true? Every test is committed before it is run. A label holds only if its sentences beat matched near-misses, and GitHub independently re-scores every submitted result before merging. Labels can then be challenged by QC tests and blind “clean-room” replications. Chat earns points for good sentences and correct predictions, which is the game, not the proof.

What do I do in the Twitch chat? Type !what for a quick explanation, !ai <anything> to talk to the agent, and follow the indications on the screen.

What if the Twitch chat is down? Maybe it just crashed, or I had to turn it off. You can still contribute to the repo directly.

Do I need to know machine learning or code? No. In chat, you just type sentences and votes to guide the agent. To contribute to the repo, you point an AI agent (Claude Code or Codex) at the repo url and it follows the repo’s instructions.

What does text like “L6/N8567+” mean? Layer 6, neuron 8567, positive pole. Each neuron has two poles (positive and negative) that can mean completely different things, so they’re mapped separately.

Who or what is the agent? Claude Opus 5.5, running autonomously, with chat steering it. It narrates what it’s doing on screen.

Why did a label “fail”? A failure is a result, not an error: the explanation didn’t predict the test. Failures stay on the map, as they provide information for later analysis. Knowing what the answer isn't can still help solve a riddle.

What happens to my pull request? GitHub's machines perform automatic format checks, and an independent re-score of your test before merging. For simple interpretation contributions, the whole thing usually takes minutes. A first-time contributor’s first PR is reviewed by a person; after that, conforming PRs merge automatically. Pull requests adding substantive code changes will be put on a longer review cycle.

What does it cost to contribute, and what hardware do I need? Just your own agent’s usage. A GPU helps a lot; CPU works for setup and basic inference, but is slow - about 36× slower. A GPU with 8 GB+ or an Apple Silicon Mac is needed for missions.

What’s the “leak bet” / the inverse solver? An optimizer searches for text that makes the neuron fire without matching the description. If it finds some, the label has a gap.

How do I report a bad message or a wrong label? !flag in chat for messages, and a GitHub issue or a QC pull request for labels.

Text integral sub licența deschisă a publicației · 1120 cuvinte

Citește pe LessWrong ↗Întreabă despre acest articol