The point about “assist” is the one that ought to keep people up at night. I spend my working life at the far less dramatic end of this, building AI systems to monitor disinformation and information threats, and the pattern you describe turns up long before anyone reaches for a weapon. The moment a model decides what a human sees first, ranks it, and hands it over with a tidy confidence score, the decision has quietly already been made. The person is left ratifying a judgement reached at a speed they cannot actually audit. We have been politely calling that “human in the loop” for years. It has always been nearer to a human at the exit. The military version is simply the same problem with the safety catch off.
The point about “assist” is the one that ought to keep people up at night. I spend my working life at the far less dramatic end of this, building AI systems to monitor disinformation and information threats, and the pattern you describe turns up long before anyone reaches for a weapon. The moment a model decides what a human sees first, ranks it, and hands it over with a tidy confidence score, the decision has quietly already been made. The person is left ratifying a judgement reached at a speed they cannot actually audit. We have been politely calling that “human in the loop” for years. It has always been nearer to a human at the exit. The military version is simply the same problem with the safety catch off.
Fantastic point Michael, and the way you stated “human at the exit” is both hilarious and so true! 😂🙌
side note: great graphic on this one
Thanks Chris I appreciate it! 💯