(The following bullet points are a summary of my current position on machine consciousness taken from a larger discussion. In particular, the focus is with respect to the question of whether large language models in their current form have "consciousness"—that is, subjective experience.)

Contemporary algorithms have seen a qualitative jump vs. older ML methods in structure-learning—that is, in their ability to converge to internal representations that recapitulate core structural features of concepts.

⁠There is, I and others conjecture, no mathematical reason barring these models from eventually converging on representations of (sub-)structures of phenomenal experience; we lack adequate theory of these methods to understand the contours of the circumstances and dynamics through which this could occur.

⁠If they can, however, then that convergence currently appears weak: current progress has stalled.

Most progress after the GPT-4/4o series has been KPI+PR hacking—that is, using the draw of recruiting academics to (mostly unconsciously) bias academic output of eval tasks and datasets toward those regimes which make the company-produced models look good, as a positive feedback loop to drive valuations and recruit to lucrative positions.

This encompasses reasoning, model-routing, agents, memory systems, etc.

This has likely occurred largely because the big labs have used all the data on human language, essentially, that exists and is readily accessible—and, because the model class we are working with, though incredible, is still bad.

The reason for this can be seen in poor sample efficiency; one might hypothesize that this arises from failing to take advantage of the strategies that our brains use to perform active inference, that surrogate data doesn’t meaningfully help. On a deeper level, it is the fundamental issue with recapitulating structure, as opposed to creatively generating new structure—biological systems need both to exist under inconsistent environments, hence autopoiesis.

⁠Current methods introspecting into existing models indicate that some macroscale structures are similar to human neural activity—but also, many aren’t.

The pattern of similarities and differences likely owe to training biases from Internet and exploited-labor-in-developing-nations–provided training corpora, and from the (as yet poorly understood) inductive biases supplied by the model classes of token prediction, or even scalar optimization.

⁠All together, this makes it very unlikely—though not impossible—that all of the structural characteristics we would as a society lay out to define “the structural motifs of consciousness that have moral-legal consequences” are present in current LMs.

It is important to note that this descr certainly changes over timeiptor, as has been seen remarkably over the last few decades with respect to the moral status of animal consciousness—though I would argue that there is not compelling evidence at present of internal representations within LMs that raise concerns anywhere near as apparent as with animal welfare, and that the bar for evidence should be lower for animal consciousness given the strong structural constraint of their shared evolutionary history.

⁠It is unlikely (though more likely) that certain structural features of xenophenomenology (experience entirely alien to us or to biological organisms) may exist in current LMs—but, we do not yet have the consensus theory to map hypothesized features of xenophenomena to structural characteristics we can systematically investigate within models empirically.

Because models are trained to recapitulate human behavior explicitly—and because model-space is large enough to arrive at vastly differing internal structures that recapitulate behavior—our intuitions interacting with explicitly anthropomorphized UX only go so far with respect to analyzing models' internal structural knowledge.

Though, there is hope!

⁠Looking forward: my imminent moral-legal concern is low for artificial models, unless there is a qualitative leap in algorithmic class that emerges.

But, now is the time to heavily ramp up our global research program in mathematical consciousness science, so we are prepared when that occurs.

⁠More pressingly: the sociological forces at play in consciousness attribution within the broader public—driven by company marketing and UX design choices aimed, perhaps unconsciously, toward clear incentives of capital—give me much greater concern.