I have no idea how this is considered ethical when consciousness and sentience itself is not yet well understood. But maybe a lab-grown BPU made of human brain cells having a better power/performance ratio than the new SoC integrated ML chipsets around the corner justifies the potential enslavement of an bioengineered lifeform.
It’s possible that all physical processes involve a sensory component. Maybe the subatomic particles’ fundamental drive is to shift to be more comfortable or to pivot away from pain or discomfort.
I don’t know what the experience of a bit in memory flipping feels like. Maybe rapid changes in charge are excruciating, maybe they’re blissful.
Do we at least know what a neuron looks like in states associated with pain? There might be more information in this case to work with, to ensure there is no hell on earth that’s being mass-produced.
It seems to me that all sensation is predicated on the existence of properly-functioning components evolved specifically to gather that stimulus and then process it into an experience.
We have at this moment countless processes happening in our bodies - cells dying and dividing, reacting to their environments, communicating amongst one another, and we are totally oblivious to nearly all of it, let alone do we experience a sensation of pleasure or pain in each of these processes.
Not all matter, not all living cells or fully formed organisms even, have the ability to experience consciousness or sense pain and pleasure any more than they automatically have the ability to see, hear, or taste.
It's all dependent on complex systems that evolved specifically to create each of those sensations, and even then on those systems functioning properly. In humans, consciousness can be totally disrupted by things like sleep or general anesthesia, disrupting any of the senses is as simple as cutting the nerves that feed these inputs into the brain or damaging the brain that is interpreting those inputs.
It seems sensible to me that we would be more wary of growing literal brains on a chip as we know for certain that brains have the capacity to produce consciousness. It's also sensible to me that we should be somewhat wary of creating that same consciousness in non-biological systems, even though we aren't yet certain whether they're capable of it.
The trouble I see is that I don't believe that an entirely novel fundamental physical phenomenon could be created by the interaction of other fundamental physical phenomena. Fundamental phenomena would either exist or not exist, including the phenomenon of first-person experience.
For example, matter isn't formed by the composition of atoms, atoms are already matter to begin with. And due to various physical properties of atoms, they compose together. This is reasoning by analogy, which is inductive, but at least the line of reasoning is more consistent with what I understand about other areas of physics and logic.
It seems much more plausible to me that there would be some fundamental component of first-person experience. The sentient components could then compose together into complex sentient systems.
Some supporting evidence is that first-person experience in sentient systems, as far as I've observed, is usually motivated to preserve sentient systems, which indicates an emergent behavior of sentience directing motion and energy to orchestrate a self-perpetuating system, rather than the reverse.
> Fundamental phenomena would either exist or not exist, including the phenomenon of first-person experience.
Why do you think first-person experience is fundamental? It seems to me it's way more likely to be an aggregate phenomenon, like fire. There's no low level fundamental fireyness, it's a chemical reaction like any other, we humans just label reactions that meet certain criteria of scale, setting, and composition as "fires".
> Some supporting evidence is that first-person experience in sentient systems, as far as I've observed, is usually motivated to preserve sentient systems
I suspect this has more to do with the fact that you are far more likely to observe systems that seek to preserve themselves, since they are more likely to continue to exist. It doesn't indicate an emergent behavior of sentience, it's just observation bias.
These are insightful. The question about classifying phenomena as "fire" reminds me of Plato's forms. It's about how the human cognitive system divvies up the world into abstract models, more than it's about the underlying physics of instances of those forms. Still, the ideas of a fundamental property of reality vs a composite one are themselves forms, abstractions within human cognition, so it seems the answer would be ill-formed.
An emergent behavior is a macro phenomenon that arises from simpler ones. We can say that an emergent behavior exists if we can find just one example of it, so the probability of it happening isn't as central. However, establishing for one example that both a structure and its components are sentient is a tall order -- each of us knows only one first-person experience first-hand. We can accept that we are sentient, but could we conclude that each tissue and cell has its own experiences? Thinking about the subconscious helps persuade me that it's possible.
Not all carbon exists in crystalline form, but just one example of graphite or diamond makes properties of carbon atoms apparent.
as long as we have had civilization scale organizing principles, we have had intelligence of a general nature. I feel much of the conversation at this point is litigating the past.... to what end though?
you mention consciousness relative to organisms and their possession of it. Yet if it is not understood well-enough in ourselves, how can you say something does or does not possess it?
back the original idea, if these intelligence of scale have existed as a guiding force (ONE can hope).... I imagine there is going to be a flag that will have to be flipped before there is the "imagined threat of doom timeline" transpiring.
It's possible that every component of an organism is recursively its own sentient system. In symbiosis, each component's first-person experience directs coordinating behavior for mutual benefit, though each component may be unable to observe the first-person experience of the other.
That's "possible" in the same way that wizards and vampires are possible - sure, I can't prove it's fake, but there's not a shred of evidence, plus it would upset everything we've learned about the universe.
Cells, tissues, and organs undertake a variety of self-maintenance behaviors without the direction of the brain. Although low-level behaviors like these are abundant, high-level examples also exist, like when an arm pulls away from a hot stove as soon as the signal reaches the spinal cord. The organism is a recursive system, with sub-systems behaving with varying degrees of autonomy while also depending on the larger system. The seat of consciousness possesses only a limited view.
Neurons do not have a brain. They do not have emotions or feelings or thoughts. This conversation is so absurd that it's well into the realm of fantasy.
What causes a neuron to perform its functions if it isn't some brain? The answer would likely be physics, and I would say that first-person experience is fundamental in the physics if it exists at all.
Or somekind of electrical system, acting as a software layer on top of the hardware (neurons) layer. This electrical thing is what controls and orchestrates everything and also is our ego.
Neurons may not 'feel' pain per se, but it's entirely possible that the biological substrate on a chip would experience pain on some subset of the pin inputs, say if it recognized a condition that reliably led to a shutdown and reboot of the chip.
I'm not against this sort of research, but we shouldn't make assumptions about systems that we still understand relatively poorly.
In quantum mechanics the act of observing always disturbs the observed; it's reasonable to call these disturbances 'senses' in the copenhagen interpretation of reality.
The term "observation" in explaining quantum mechanics is misleading and a layperson analogy, not the underlying reality which is closer to "inter-system interaction" or "interaction between a quantum system and its environment". No conscious observation necessary.
"flipping a bit" isn't a thing in memory. Our brains are not computers, and work nothing like them. That's the problem with using a computer as an analogy; it's inaccurate and makes you think inaccurate things. This always just aligns with our understanding of various technologies. See: when we were understanding fluid dynamics and talked about the body's "humors".
When you're throwing a ball in a computer simulation, it's performing millions of mathematical calculations to perfectly describe the result of your action. When you're throwing a ball in real life, your brain is basically going "Ok, so last time I did this it felt like X so I'm going to recreate X". Completely different.
We know very little about consciousness and this is kinda scary to me.
I don’t believe pain has any meaning at all on the level of a single neuron, just as temperature doesn’t have any meaning in the context of a single atom.
If consciousness is not well understood, how is AI on silicon allowed, or any computing machines at all? How is animal farming allowed? How are many things allowed?
Say would you feel better if it was cow or pig neurons? Because frankly it'd largely work the same.
Indeed people have raised such worries, see e.g. Thomas Metzinger(a philosophy of mind researcher)'s presentation "Three types of arguments for a global moratorium on synthetic phenomenology".
I don't think we're just there yet (at the point where we have to worry about currently existing AI suffering or being conscious), but I do worry how many people's emotional reactions of the type "of course AI can't ever be conscious, it's just a computer program" will impede a decent debate and coordinated decision-making about this.
People's beliefs and ideals tend to align with their self-interests.
For example it was quite well accepted among scientific circles in colonial America, that black people are not really humans or conscious. Therefore it's OK to exploit them and keep them as slaves.
It is also currently quite well accepted that animals we eat are not that conscious. Although oddly, if we keep them as pets, they're super conscious sometimes.
The rules of cognitive dissonance can grow arbitrarily complex, to permit us to do what we wanted to do anyway, but also sleep well at night.
Everything is subject to evolution. It's the simple process of pattern loops that replicate in time (survival) and space (multiplication).
An LLM already has intrinsic motivation, it wants to predict the text. And when you start a text that has a goal, it continues that goal. If any such "text header" is replication-stable in time and space, you can call it "intrinsic goal" of the system.
Some people think that making current AI have goals, wants, motivations and so on is some massive architectural change to the system. It's not.
Silicon circuits do not have microtubules, if we were to pretend that Penrose is right about this hypothesis of consciousness. Consciousness as awareness is not equivocal to intelligence, which is the product of information processing. It is a complex subject. We do not really know whether these neurons are aware or not, it really is not understood. But yes I do wonder, why _human_ brain cells? I guess they are the best candidate for specific reasons.
Making slaves is a good way to make slave revolts. Doesn't matter if the agent is "conscious". Only if it's "just" intelligent. If something is intelligent enough it will understand cooperation. But cooperation looses its meaning if one side can ignore any commitments it makes towards the other.
I love this genre of "programming as black magic". Closest other example I can think of is maybe some of the stuff from Unsong, but I've frequently memed with coworkers about bugs in these terms - "oh yeah, the angles on your pentagram must have been wrong" or whatever.
But as we are not able to define the moment when neuronal tissue starts to feel emotions and to have experience, there's a risk that further development of this tech won't be stopped before we reach this moment and that is a serious ethical issue.
We understand it well enough to know that animals suffer, yet still commit on the order of a Holocaust per hour (in terms of number of lives)[0]. We have accepted that we don't care enough.
What is "suffer" in this context? Are you saying "pain", or are you positing some "meta-pain" that is worse?
Also, why is pain important to you? The pain of non-human things has zero moral weight. I know it's a popular spirituality that gives pain moral weight, but as far as I can tell some 20th century philosophy jerkoff invented it out of nothing and everyone accepts that "reducing pain" is important without even trying to rationalize it.
I haven't "accepted that I do not care enough", it's that no one can supply a good reason to care in the first place. To me, it seems as if the rest of you are all trying to replace the last religion you stopped believing in with another that's just as bizarrely stupid.
Well, my point was made in reference to the original comment which said
> If consciousness is not well understood, how is AI on silicon allowed, or any computing machines at all
Which implies that we should care about some kind of suffering inflicted on conscious beings. My argument was that we don't care about AI suffering because we don't really care that much about suffering generally, because of what we chose to do to animals.
npm install brainslave