Quote:
Originally Posted by iLLuS10n-
if we are capable of creating AGI, I don't see any difference between creating less smart AI for our own purposes and creating less smart humans for our own purposes. Do you think this is moral, even if we treat them nicely?
Intentionally creating less-smart ones? Not sure why we'd do that. You're saying for "our own purposes", so I'll assume there are some purposes for which a dumber being would somehow be more useful than a smarter being.
You seem to be asking about two things at once. Is it moral to:
A) Create a dumber conscious being than ourselves.
B) Use a conscious being for our own purposes.
If A and B are moral then so is the combination, "A followed by B". If either are immoral then the combination is immoral.
I think A is "yes". Overall-intelligence is the amount of reality you're able to see. A dumber conscious being can't experience as much of reality as a smarter one. Reality is infinite and humans are infinitely dumb. Creating a being that sees a little less of reality, I don't see how that would be immoral in itself (ignoring the fact that we haven't even defined "moral").
For B, I can construct an example of where I think it would be moral. We program the AGI to desire to do the things it will be used to do. This way, it wouldn't have to be our slave (not against its will), it would "choose" to do exactly what we would have commanded it to do anyway. This is assuming the consciousness does not choose the desires, which I think is a true assumption.
So "yes" to the question in full.