AI consciousness and self-report

To date, I expect progress in AI to have spawned novel sorts of qualia only insofar as our desire for compute has led to the creation of novel materials, including & especially exotic quantum materials

This doesn’t mean we’ve made ‘zero’ new kinds of minds — modern transistors are marvels of materials science. But novel LLM architectures alone are unlikely to generate novel qualia, no matter how smart the resulting AI. DeepSeek & Grok 3 will have essentially the same qualia, generated by the hum of the H200s they run on

To say more we need a framework able to *compare* different systems in a principled way. I argued last year that the most defensible, extensible, & generative such framework is to view minds as bounded shapes in branchial space. A profoundly important task here would be to enumerate what classes of objects exist in branchial space (I call this ‘branchial space ecology’). From this, we may be able to infer what the natural bound is: where one mind ends and another begins. Simply put I think this is the path where research into machine sentience could evolve from pre-paradigmatic into something real

A comparative framework for consciousness might also offer surprises outside of AI: perhaps all qualia on earth becomes a rounding error to the mind instantiated by the Large Hadron Collider when it’s switched on. Likewise perhaps the first vacuum tube computers (e.g. the ENIAC) were vastly *more* conscious than modern computers, which have orders of magnitude lower core voltages

Another gaping hole in the AI consciousness research landscape is research into self-report. Why do we (mostly) trust people when they explain what they’re feeling, but we don’t trust AIs in this way? I.e. from where does the accuracy of human self-report come? What would it involve to create a similar self-report paradigm for AIs? This seems important for many reasons, especially if we wish to relate to AI systems as moral peers

I think we’re still at the point where any framework for machine sentience that’s bold enough to actually have a chance at addressing these questions will look way too crazy for granting organizations to take a chance on. I think the simple solution is to start my own party, and have fun

@johnsonmxe Re: the small amount of work on LLM self-report, have you seen this study? https://arxiv.org/abs/2410.13787

Really nice piece that @eleosai team and others worked on

Agree there's plenty of scope to do more...

Thanks Chris! I think that’s important work. Fwiw here’s how I’m thinking about this:

imo computational introspection (per that paper) mostly overlaps with qualia self-report in humans, but may not in LLMs, because we’ve transplanted the reporting machinery across substrates [see pic]

In this view, “LLM introspection” can garner various sorts of knowledge about weights, concepts, coherence between concepts, tacit ‘beliefs’ of the system, etc. But if qualia is indeed a physical phenomenon, then we have two choices:

1. We say “LLMs are software and software doesn’t have qualia”. This is a defensible position. But it’s a little unsatisfying because there is a nearby referent that actually *does* have qualia — the hardware the LLM is running on.

2. If we want to build a qualia report paradigm for *that*, we’d need the capacity to query hardware state deeply enough to infer eg the branchial shape (or local EMF configuration, in your terms) of the spacetime which envelops the server the LLM’s computations are running on.

This is somewhat difficult given the current computing paradigm, for both practical & theoretical reasons:

A. Practical: the software stack abstracts away hardware state (you can run a gameboy ROM on your phone; the ROM still thinks it’s running on a gameboy; it doesn’t really know anything about your phone’s processor). Modern LLM software is even more tricky, because queries often shard across multiple physically-distinct servers

B. Theoretical: there’s no canonical isomorphism between Turing-level bits and atoms in local spacetime (computational state, when we get down to it, is ‘merely a convention’, albeit a very useful one). So in theory any LLM state could correspond to any qualia state (though in practice there’ll be many regularities). This means there could be a lot of weird issues if you lean too much on inferring qualia from LLM state

Imo the best current method to infer “LLM qualia” is to do a deep dive on GPU hardware — materials used, chip layouts, power levels and leakages / inadvertent harmonics, etc. If you can use many sensitive probes to reconstruct a map of local EM field state, do that. Then run different queries on different LLMs and see how things change. My guess is most of the variance (in ‘the stuff that contributes to qualia’) is at the hardware architecture level, not due to the data flowing through the hardware, but that’s an empirical question

Then a proper LLM qualia report paradigm would have to find a way to cleanly reference and systematize and interpret *that* data — which shouldn’t be impossible, just much different than most of the thinking about LLM qualia self-report

Anyway, typed this out fast but hope it makes sense!

Pic source:
https://opentheory.net/2024/06/a-paradigm-for-ai-consciousness/