This relates to subjects I am slightly familiar with and very interested in so I am glad this is on the front page and I think he did a good job.
I have a couple of criticisms though.
The idea that we could build this powerful general human-like intelligence but it would turn out to necessarily be an addict seems unlikely, because I don't think you can build a reward system without including something to prevent shortcuts. Although creating an effective shortcut-prevention system may be challenging. See a bunch of stuff written by Eliezer S. Yudkowsky and his friends.
Also, when he writes
"It seems like the most important motivations of human civilization are related to near-term goals. This is probably a consequence of the fact that motivation writ large is still embodied in individual humans, who are driven by their evolutionary psychology. Individually, we are unprepared to think like a civilization. Our faint mutual motivation to survive in the long term as a civilization is no match for the ability we have to self-modify and physically reshape the planet."
To me that is implying a fairly obvious and typical line of leftish reasoning that for most of my life I assumed was self-evident and didn't require elaboration. After deliberately 'exposing' myself to some 'right-wing' thought and 'extreme' 'left-wing' thought, I now realize that aspiring for collectivism is not an adequate solution.
What happens with self-interested individuals in our capitalist society is that they aggregate power and resources for their own individual use. Unfortunately, something very similar happens in collectivism: power and resources are still aggregated in the hands of a few.
In other words, both systems tend towards centralization. Theoretically, the collectivists will aim for redistributing power and resources for the good of the many, but there are a couple of practical problems: they are still living in hierarchical societies and so have strong motivation for enriching themselves and the controllers cannot see enough detail to know what is really best for the rest of society.
Command economies don't work because of general problems with centralization and lack of distribution and the limitations of the types of technologies employed. A peer-to-peer network is more robust than an internet with only a few backbones.
Unfortunately, capitalists don't realize that their system also leads to monopoly and centralization.
I believe that we do need wholistic measurement and analysis of things like global resources and human equality, and a common principle of supporting the collective good. But at the same time we need to decentralize and distribute production locally. We need local decision making that operates on the basis of a global information schema with wholistic data.
A system can't be efficient or evolve without being localized, but it also can't be integrated without an accurate shared knowledgebase and an egalitarian perspective.
I have a couple of criticisms though.
The idea that we could build this powerful general human-like intelligence but it would turn out to necessarily be an addict seems unlikely, because I don't think you can build a reward system without including something to prevent shortcuts. Although creating an effective shortcut-prevention system may be challenging. See a bunch of stuff written by Eliezer S. Yudkowsky and his friends.
Also, when he writes
"It seems like the most important motivations of human civilization are related to near-term goals. This is probably a consequence of the fact that motivation writ large is still embodied in individual humans, who are driven by their evolutionary psychology. Individually, we are unprepared to think like a civilization. Our faint mutual motivation to survive in the long term as a civilization is no match for the ability we have to self-modify and physically reshape the planet."
To me that is implying a fairly obvious and typical line of leftish reasoning that for most of my life I assumed was self-evident and didn't require elaboration. After deliberately 'exposing' myself to some 'right-wing' thought and 'extreme' 'left-wing' thought, I now realize that aspiring for collectivism is not an adequate solution.
What happens with self-interested individuals in our capitalist society is that they aggregate power and resources for their own individual use. Unfortunately, something very similar happens in collectivism: power and resources are still aggregated in the hands of a few.
In other words, both systems tend towards centralization. Theoretically, the collectivists will aim for redistributing power and resources for the good of the many, but there are a couple of practical problems: they are still living in hierarchical societies and so have strong motivation for enriching themselves and the controllers cannot see enough detail to know what is really best for the rest of society.
Command economies don't work because of general problems with centralization and lack of distribution and the limitations of the types of technologies employed. A peer-to-peer network is more robust than an internet with only a few backbones.
Unfortunately, capitalists don't realize that their system also leads to monopoly and centralization.
I believe that we do need wholistic measurement and analysis of things like global resources and human equality, and a common principle of supporting the collective good. But at the same time we need to decentralize and distribute production locally. We need local decision making that operates on the basis of a global information schema with wholistic data.
A system can't be efficient or evolve without being localized, but it also can't be integrated without an accurate shared knowledgebase and an egalitarian perspective.