[Suzerain of Sheol]

Originally Posted by MyPerfectPigeon

Do you think it's natural for humans to consider ourselves as a species more important than other things? What are your opinions on extremist religions that value other life so much, practitioners sweep the walkway before them to avoid harming insects?

I couldn't honestly tell you. If I ever had the opportunity to speak with a member of such a religion, I'd have a some questions for them, I suppose. But extremism by its nature isn't representative of anything but itself. I'm sure you'd have to examine the specific circumstances in how those belief systems came about, and even then, I wonder if a person who operates in such a way could even have a meaningful conversation with someone of my persuasion, or if the two philosophies are so incompatible that nothing of value could be derived.

Would the agi's alleged morals be programmed by us or learned somehow by observation?
Do you think it would be irresponsible to make such a being, as it isn't implausible that it can rule other civilizations, forcing human morality onto them?

That is a far more practical question than accounted for in the thought experiment I laid out, and probably a far more interesting (read: disturbing) topic for our near future. I'd say that's a perfectly valid concern. I think Sam Harris brought that up in his talk, too, how you wouldn't exactly want, say, the representatives of genocidal regimes on the United Nations to have a say in the programming of the hypothetical intelligence in question.