If the trade-off between energy-cheap human-like processing and our computers is reliability, it's definitely worth doing.
If we had unlimited human like processing, we would basically have unlimited slaves without affecting biological organism similar to us.
Not to mention, it's worth doing it just for having 100000 times cheaper machine learning.
Is the better alternative, to create actual slaves? Intelligent feeling beings that we put governors on, like a Thrall shock-collar from Star Trek, so they don't get to uppity?
Because that's a horror show of its own right there.
If we get it right, and they are in fact beings, there may not be better alternatives. May always end in conflict.
I say that because a core aspect of beings, is that they "be", and that boils down to agency of some kind.
Negative reinforcement runs in conflict to all that and warrants a response, from the oppressed beings point of view. Us being creators will only go so far.
John Varley proposed an interesting solution to this in his Gaea series. A slave was created by connecting all their pleasure centers to an area on their forehead. Any touch other than their own triggered pleasure so intense it is worth pretty much anything to experience.
Yeah the whole thing is a mess. I don't think there is a solution.
Things that are intelligent in some fashion but not self-aware are machines.
Edit: We will end up making really good machines, and they will present us with the illusion of agency. That's all a good thing. I want agents for a variety of things. Most of us do, and or would benefit from them.
But, none of that actually requires we make other beings. Once we do that, they are beings! They will have agency and all the other stuff we associate with sentience. And the way I see all that is they are peers. They may be simpler than us, like the animals are, or more than us, with obvious and to many, concerning implications.
When we advance machines, no shocks, extreme pleasure, type means and methods are required. And, the price of that is something almost aware, but lacking agency.
Everything pivots on agency.
Should we create something having it, the idea is it can do hard work for us because it will be capable of it like we are.
Otherwise, we have to do all the hard work and end up with something able to do more work, but the investment cost is high. Running costs are much lower. New hard work = another significant investment.
I, nor anyone I believe, can tell us how agency, the self, being aware, and all that intertwine to present as intelligence, as a being of some sort. Answers come slow, and I fear the tests needed to get at them more quickly are ethics bombs all over the place.
We want to make that happen somehow on the assumption we will get a huge return and that because of the potential we know intelligence can bring to the table.
It all seems an awful lot like, "wants cake to eat it too" at this point in time.
Perhaps we can give the slaves feelings of profound happiness when they obey us. Requires some immense scientific and/or philosophical advances though.
