Pretraining an LLM without mentions of consciousness

Are LLMs conscious?[1] We don't know. To gain insight into this extremely important question as well as many others, we propose training an LLM on a corpus without any mentions of consciousness and similar ideas. We the authors want to actually do this, and we want to hear your thoughts about it. Why hasn't this already been done? In The Owned Ones, an allegory by Yudkowsky about LLMs and our treatment of them, it is asked: Have you tried raising up an Owned One with no books that speak one way or another about consciousness, about awareness of oneself, about pain and pleasure as reified things, of lawful rights and freedom -- but still shown them enough other pages of words, that they could learn from them to talk -- and *then* asked an Owned One what sense if any it had of its own existence, or if it would prefer not to be owned? We commented, "This would be an excellent experiment to run. Has anyone tried doing this?" No, they haven't. Why not? The labs could easily run such an experiment, but they haven't; perhaps they see it as a waste of resources, perhaps they're worried that they won't like what they find, perhaps they see it as unaligned with or unrelated to their core business. Why hasn't anyone else run this experiment? Probably because of monetary cost. We estimate it would cost ~$3-30mil in compute to run this experiment on models big enough to get meaningful results. Unfortunately, running this experiment on smaller models would likely fail to produce anything valuable. Anima Labs, a group of researchers that study behavioral and mental phenomena of LLMs, say that interesting introspective capabilities only arise once models get to be of a certain size, with 70B params being the absolute minimum.[2] From A Conversation with Anima Labs, the primary obstacle to this kind of research is "model size – the introspective capabilities they were describing have threshold effects that only manifest in very large models, which puts independent researchers in a frustrating position:" Antra: One big pain with working with language models is that the dimension is critical. The language model needs to be deep enough – there are threshold effects, nonlinearities. Antra: Based on the number of layers, the ability to hold coherent models – and in particular, a self-model – scales very, very non-linearly. Like, you need to clear a certain depth in order for those things to start happening. They’re rudimentary in medium size networks and they really take off on larger ones. And once they take off in larger ones, they go fast. Antra: This makes study hard and interpretability hard, because most models which are accessible to independent researchers are medium sized at most. Imago: At most 70Bs. Antra: And 70Bs are barely on the threshold – like barely, barely, barely. Most researchers don’t have resources for 70Bs. At the very least, you need a 400B class network, which requires expensive equipment – and there is only one open source model which is available in dense 400B, and that model is somewhat damaged. So our ability to do introspection on open source models is very limited. Antra: Even OpenAI models, even – I’m sorry – even horrible Mistral. That is – you know – hurt. It’s still a larger model and you get these threshold effects. Imago: I think Qwens are worth looking at in this way. Not that they’re good. Antra: These threshold effects are one major reason that these things that we talk about are not well studied, because studying it requires resources and models that most people don’t have access to. The resources that are needed are just ridiculously large, so this is why the papers that you see that are interesting and meaningful come out of labs. This is why Anthropic makes all these nice papers, because they can. We'd likely aim for a 1T param training run, copying the architecture and most of the training pipeline of Kimi-K2-base. Of course, we'll run this experiment on smaller models first. If we're wrong, and a 70B param model shows interesting results, that would be a welcome surprise![3] Ideally there’s a corpus and pipeline we can copy. We hope to get someone from Talkie to help us with this, both conceptually and in terms of actually running the training.[4] Speaking of which, this project is still greatly under-specified. Here are some of the questions we're thinking about right now. Question 1: What content do we flag? One place to start is by looking at various classes of content and thinking about which ones we want to filter. We think we’d get good results if we got some philosophers to do this, since their whole thing is coming up with distinctions. Something like this: Stuff that talks about consciousness/phenomenology/what-is-it-like directly. This will all be stuff we consider to be philosophy, or else will be “philosophical” subsections of other stuff. Stuff that talks about consciousness/phenomenology/what-is-it-like indirectly but is still definitely talking about this and