I wouldn’t put one in charge of a water treatment plant or a set of traffic lights, but even if it stuffed up and killed people, the responsibility would still not be the AI’s. The responsibility would lie with those who built it, trained it, and prompted it, just as, if a bunch of people are fried by a nuclear missile, the responsibility lies with the person who pushed the button rather than with the missile itself. It’s a piece of software. It does what it’s told. It may be complex enough to display unexpected emergent behaviour, but so is Dwarf Fortress. Any destruction it accomplishes is just the real-world version of !!Fun!!, disgusting concept though that may be.
I wouldn’t put one in charge of a water treatment plant or a set of traffic lights, but even if it stuffed up and killed people, the responsibility would still not be the AI’s. The responsibility would lie with those who built it, trained it, and prompted it, just as, if a bunch of people are fried by a nuclear missile, the responsibility lies with the person who pushed the button rather than with the missile itself. It’s a piece of software. It does what it’s told. It may be complex enough to display unexpected emergent behaviour, but so is Dwarf Fortress. Any destruction it accomplishes is just the real-world version of !!Fun!!, disgusting concept though that may be.