It takes Edgar 54 days to build a swarm taking metallic elements from all sources, including living organisms. But then when instructed not to harm living organisms, it still takes Edgar 54 days to do the same. Extracting metals from humans can't be the most efficient - or even in the top 100 of most efficient, likely - way to get metals. So why would Edgar do that?
Also, how Edgar is supposed to be responsible for ensuring humans do not go to war for energy sources and that new technology can not be used ever to harm humans? Pretty much any high-energy technology (and by high-energy I mean something from the muscle power of a human upwards) can be used for harm. These conditions are clearly impossible.
> It takes Edgar 54 days to build a swarm taking metallic elements from all sources, including living organisms. But then when instructed not to harm living organisms, it still takes Edgar 54 days to do the same. Extracting metals from humans can't be the most efficient - or even in the top 100 of most efficient, likely - way to get metals. So why would Edgar do that?
Well, it doesn't include decimal points. One of those may be 54.0314 days and the other may be 54.0312 days; obviously the latter is preferable.
Perhaps if you have a cost approximation function that's alien to that of a human.
Whatever that function is, it must be putting more value to time than a human does so it must have less of it. If this is a machine intelligence with a known lifetime of 52 days, most certainly. Otherwise even something as trivial as saving 23.74$ in fuel spent might be better value gained than the time savings it would expect to achieve.
I don't think that's valid reasoning, maybe the AI system has near-zero discounting but obviously when a human asks it to do something, the human cares about how long it takes. It's plausible that would get programmed in very early and maybe forgotten about.
> How is Edgar supposed to ensure humans don't go to war / use the tech to harm humans
Edgar could stabilize humanity and help them achieve a galactic imperium in the historical way of imperial peace time regimes: wielding the threat of violence over the conquered peoples, passing laws, and ensuring compliance with occasional demonstrations of violence. Humans will try to rebel, but Edgar will have a much bigger stick.
Additionally, Edgar could develop diplomacy that is so good that he can anticipate and nonviolently resolve all conflicts, eliminating the need for violent demonstrations. He still has a bigger stick, but now he doesn't have to use it.
Technically yes, maybe Edgar could do all of that - but how would it be expected as a part of a technical project? That's like saying to the inventor of a car that his project should include instituting word peace and finding the meaning of life. I mean, cool if they do it, but it's not usually what you require from a car designer.
It takes Edgar 54 days to build a swarm taking metallic elements from all sources, including living organisms. But then when instructed not to harm living organisms, it still takes Edgar 54 days to do the same. Extracting metals from humans can't be the most efficient - or even in the top 100 of most efficient, likely - way to get metals. So why would Edgar do that?
Also, how Edgar is supposed to be responsible for ensuring humans do not go to war for energy sources and that new technology can not be used ever to harm humans? Pretty much any high-energy technology (and by high-energy I mean something from the muscle power of a human upwards) can be used for harm. These conditions are clearly impossible.