A rogue killer drone 'hunted down' a human target without being instructed to, UN report says

- A deadly drone "hunted down" a human target without being instructed to do so, according to a UN report.
- The incident took place during clashes in Libya last year, the Daily Star reported.
- Experts are sounding the alarm about the lack of regulation around using "killer robots."

...The Turkish-built KARGU-2, a deadly attack drone designed for asymmetric warfare and anti-terrorist operations, targeted one of Haftar's soldiers while he tried to retreat, according to the paper.

The drone, which can be directed to detonate on impact, was operating in a "highly effective" autonomous mode that required no human controller...

"The lethal autonomous weapons systems were programmed to attack targets without requiring data connectivity between the operator and the munition: in effect, a true 'fire, forget and find' capability," ...

This is likely the first time drones have attacked humans without instructions to do so, Zak Kellenborn, a national security consultant who specializes in unmanned systems and drones, confirmed in the report...

Human Rights Watch has called for an end to so-called "killer robots" and is campaigning for a "preemptive ban on the development, production, and use of fully autonomous weapons," according to a report by the charity...

Interesting case indeed. It is the first reported case of this type I have seen.

I don't find the idea of autonomous weapons, with artificial intelligences I suspect are subpar to your average bargain bin robot-hoover, a particularly appealing one. I don't think you need to be a technology you expert to see how these weapons will ultimately be linked to massive data gathering, an area where we already are in murky waters.

But if international agreements will lead anywhere is anyone's guess. Just getting international agreement on dumb anti-personnel mines has been challenging, with major movers like Russia, China and the US still to accept them. This even though such weapons inflict enormous civilian losses. With that in mind, I predict little success in stopping further developments in "smart" weapons.

Yup. But I can't see this doing anything but advancing in this regard and moving from Military to some forms of Policing.

As drone tech improves along with the ways to deliver a lethal (stopping) payload I can only imagine future scenarios such as a Waco encampment or the North Hollywood Bank Robbery Shootout being ones where police pull way back setting a 'contain' perimeter while sending in teams of lethal (stopping) drones with AI that target and incapacitate anyone with an identified weapon in hand within that zone.

Especially if the AI does get sophisticated enough that the chance of innocent bystander death is less than having cops volleying shots back and forth with them.

The US Army is already testing networked drone swarms so i think it is undeniable that we will one day have miniaturized such swarms that flood an area and deliver one shot (bullet or tranquilizer or other) with the lethal accuracy only AI can have, all the while allowing for many to be shot down on route before the target(s) are neutralized.

I would be shocked if we DON"T get there.

It's such an extreme case of international agreements being useless that they become dangerous.

Trouble is the 'advantages' for the non ethical are massive while there's no real gap between developing human controlled drones and developing AI controlled ones. There may be some small mileage in banning preprogrammed drones for a while but we simply have to face the fact that AI is here and it's going to define everything.

The road to autonomous drones that actually use weapons for use by law enforcement is a lot, lot steeper than for military purposes.

But on the other end of the scale, for terrorism and crime, drones will likely see a lot of use.

I was going to agree simply but then i thought of how police have been increasingly militarized with almost no push back. I think we might just see them used in the field with no public debate or consultation. They will just be there.

And as the drones advance and are tested and proven by military that tech will become available for criminals too. You could see criminals using AI drones to go after pursuing cops before cops are empowered with their own drones. One AI drone with a small ordnance could back off advancing police cars or even police helicopters because of the threat of proximity detonation.

I very much doubt autonomous weapons will see much use in law enforcement anytime soon, there are simply too many legal hurdles. There are some legal frameworks that is related, in data gathering and recogniton, but the step up to deciding force is still enormous legally. We might see robots with a human acting as a panic switch.

As for military application, military forces already use military hardware that don’t give a toss who triggers them in the form of anti-personell mines, which cause enormous civilian losses. A smart robot with questionable artificial intelligence isn’t going to face much legal challenge.

I'd rather be policed by a robot as opposed to some racist psychopath.

Unless a racist psycho programs the robot

Probably need to. Have to maintain consistency!

I get that it is a joke, and I also get that what you describe is far beyond a "mere" autonomous weapon system.

But intelligent systems that do autonomous decisions based on large amounts and streams of data are fully capable of being racist. It doesn't become fair just because decisions are made by a computer, because these decision hinge on quality datasets and neural nets might enforce tendencies that already exist in data. We've seen this already in face recognition solutions deployed by law enforcement. In fact, they might make such things worse, because we have a tendency to view computer decisions and coldly logical and done without prejudice. Thus prejudiced autonomous decisions could gain legitimacy.

If we assume a autonomous weapon (and ignore the legal hurdles behind such systems for law enforcement), the deployment of such a weapon can certainly be the result of prejudice.

That link is ****ing wild.
There is obviously a massive flaw in the American code. Like so bad that it probably needs to be started from scratch again and really should not even be allowed to be sold without a disclaimer.

Very interesting how that can happen and almost proof of systemic racism at the subconscious level.

Wonder how European facial recognition fares.

Yes, it is wild. Technology also tends to be foundational. Meaning that if we don't fix this now, future technological advancement might rest on already flawed implementations.

As for Europe, there are talks of restrictions on the EU level, but this tends to be vested more in right to privacy and lack of oversight. In regards to a debate on systemic prejudice, it might crop up in some white papers. Other than that, we see the usual trend; European countries are very content with their prejudices and would rather not make a fuss about it.

Do you see that as an absolute statement, meaning no 'autonomous creep'?

Law enforcement currently use human controlled drones for surveillance and other uses. AI could creep in to tracking software such that the drone could map an area or follow a suspect with its AI engine and not require the operator to keep it over the target.

Do you foresee big debate/controversy over such AI creep or do you think that one day it will not be present and the next it just will used without consultation?

I get that is a bridge far from arming a drone and allowing it to effect an arrests so I am not trying to make that connection, but I do think that type of creep, in practice and with citizens getting used to that over time could open the door to the other 'enforcement' type aspects.

My understanding based on what i have read prior on this topic is that the facial recognition inaccuracy problems with POC in US based firms and AI applications as opposed to the Chinese ones are all based on the lack of diversity in the data set.

The main way facial recognition AI is trained is thru gamification, in the same way Chess engines are trained. AI engines are put to the task versus other AI engines to 'match faces while seeking to hit a score success threshold (70% accurate, etc) and the ones that get closest, then identify all the data points they used for their success and that is blended with other AI engines who may have achieved a success hurdle but with different date points and they then blend those into a new or advancing AI and run the game again. And again. And again.

The issue American firms have is that the main data set is compromised largely of white males and white males have the highest percentage of unique data points. Meaning the game and success score is far easier to achieve. So your AI engine might match white faces with 95% accuracy and POC much lower but the companies blend that to say we have achieved X% success over all and thus this tech is ready to use.


A fix for the US systems is to simply buy massive data sets of POC from CHina and elsewhere if they can get it Heck i would think Facebook worldwide picture access could be a huge chunk of that and FB model is built on selling that type of data.

AI and neutral nets are already used in law enforcement, so the creep is there. But there is a big difference between a reversible outcome that passes by a human and allowing AI to decide non-reversible outcomes. This regardless if the former isn't actually checked, the point is more about responsibility than reality.

I'm no legal expert, but my layman mind see some legal hurdles for law enforcement use which can be exemplified.

Let's assume a weaponized and autonomous drone kills someone in the field under law enforcement operation. When this happens, which of the following is the legal equivalent of the drone:
1. A human's signature ("I ordered the killing"").
2. A human's gun ("I killed")
3. A human's manual of procedures ("These are our parameters for acceptable lethal force")
4. A law enforcement officer ("the drone decided to kill")

Furthermore, if the killing was wrongful, but in accordance with drone programming, would it be accidental or intentional?

-----

I can imagine a future where there is a big legal difference between an autonomous drone which is simply turned on vs. a drone operated by keeping it turned on ("dead man's switch"), the latter would (to my layman's mind) make some of the legal hurdles simpler.

Indeed.

Society is being forced to deal with some lesser version of many of those questions with AI vehicles and the question of 'when they kill'? I think that if there is a 'way around' many of those questions it will be based on what will be seen as a pragmatic decision.

What I mean by that is the best answer to why AI should be allowed on the roads will be 'it gets in 90% (C%) less accidents/deaths as compared to human operators.

I think if AI Enforcement data (maybe not first from the US but from somewhere like China or another country that might eagerly adopt it) shows exponentially less death of Officials, Bystanders and Offenders, that would be the tipping point even if all the questions you ask above remain difficult to answer.

It will be an interesting journey.

lol ya ****ing right. the drone did exactly what it was supposed to do. and exactly what the people in charge wanted it to do.

What could wrong with putting everything in the hands of a computer and taking the people out of the loop?

Just unplug the damn thing!