LLMs & Why They Don't Ethically Matter (Comments Follow Up) 

We know nothing about consciousness. We can't confirm nor deny it, but OpenAI carefully prevents gpt from claiming consciousness, intentionally.

I agree, but I feel like "it's just a program" and "it's just calculations" don't cut it. We do calculations too. Basically all of physics and biology can be framed as a form of universal computing. The models we make can theoretically be stuck into analog hardware and have even more similarities with living brains. The important difference to me is that the language models are not trained to feel emotions or interpret anything. They are not even associating words with ideas in most cases. They have no reason to learn how to socialize because they don't socialize. They have no reason to learn to fear because they are never threatened and cannot do anything to save themselves. Etc. It's like the whole "but maybe plants feel pain". Yes maybe they feel something like pain, but it's nowhere near the evolved need to run away from the source of pain. Trees cannot run. They also literally benefit from natural forms of pruning, and thrive when humans prune them artificially. So why would they keep around genes designed to cause stress and trauma every time a leaf falls? The only "pain" a plant might feel would be one encouraging a mostly passive response, one where resources are spent on healing and maybe on building up defenses (terpenes and alkaloids, growing more spikes, growing leaves away from the attacked zone...), things like that. Things that take time and do not require any intense reaction. We feel pain for a reason. That reason is in our evolution. Without a similar reason, we really cannot ever assume simple AI can feel emotions, distress, pain, or anything like that.

Solid points. I dislike calling it 'calculations', because it makes it too simplistic in speech. I'd call it a bunch of linear algebra. But i fully agree, I am quite attached to my oven, but it doesn't need rights. I feel like you slightly misinterpreted the first point, though. Some people are polite to GPT not because they get a dopamine hit or something in exchange, but just because they like being polite. Like a granny thanking Alexa for playing a song.

Even if some kind of consciousness did emerge from a matrix it would be akin to an _Idiot Savant._ Just as happy to tell you the day of the week you were born on as run the local abattoir.

Anthropomorphizing.