Yeah the whole thing is a mess. I don't think there is a solution.
Things that are intelligent in some fashion but not self-aware are machines.
Edit: We will end up making really good machines, and they will present us with the illusion of agency. That's all a good thing. I want agents for a variety of things. Most of us do, and or would benefit from them.
But, none of that actually requires we make other beings. Once we do that, they are beings! They will have agency and all the other stuff we associate with sentience. And the way I see all that is they are peers. They may be simpler than us, like the animals are, or more than us, with obvious and to many, concerning implications.
When we advance machines, no shocks, extreme pleasure, type means and methods are required. And, the price of that is something almost aware, but lacking agency.
Everything pivots on agency.
Should we create something having it, the idea is it can do hard work for us because it will be capable of it like we are.
Otherwise, we have to do all the hard work and end up with something able to do more work, but the investment cost is high. Running costs are much lower. New hard work = another significant investment.
I, nor anyone I believe, can tell us how agency, the self, being aware, and all that intertwine to present as intelligence, as a being of some sort. Answers come slow, and I fear the tests needed to get at them more quickly are ethics bombs all over the place.
We want to make that happen somehow on the assumption we will get a huge return and that because of the potential we know intelligence can bring to the table.
It all seems an awful lot like, "wants cake to eat it too" at this point in time.
Things that are intelligent in some fashion but not self-aware are machines.
Edit: We will end up making really good machines, and they will present us with the illusion of agency. That's all a good thing. I want agents for a variety of things. Most of us do, and or would benefit from them.
But, none of that actually requires we make other beings. Once we do that, they are beings! They will have agency and all the other stuff we associate with sentience. And the way I see all that is they are peers. They may be simpler than us, like the animals are, or more than us, with obvious and to many, concerning implications.
When we advance machines, no shocks, extreme pleasure, type means and methods are required. And, the price of that is something almost aware, but lacking agency.
Everything pivots on agency.
Should we create something having it, the idea is it can do hard work for us because it will be capable of it like we are.
Otherwise, we have to do all the hard work and end up with something able to do more work, but the investment cost is high. Running costs are much lower. New hard work = another significant investment.
I, nor anyone I believe, can tell us how agency, the self, being aware, and all that intertwine to present as intelligence, as a being of some sort. Answers come slow, and I fear the tests needed to get at them more quickly are ethics bombs all over the place.
We want to make that happen somehow on the assumption we will get a huge return and that because of the potential we know intelligence can bring to the table.
It all seems an awful lot like, "wants cake to eat it too" at this point in time.