6 ms·

Palantir demos AI to fight wars but says it will be ethical

32 points by konart 3y ago

scrum-treats 3y ago
[flagged]
dtagames 3y ago
This is absolutely unreal. They want it to execute battle ops. As in, kill people. Using chat prompts. What could go wrong?
wackycat 3y ago
This headline (and Palantir and our country) ignores the large swath of the population who doesn't find war ethical in the first place.
- belter 3y ago
  What happens when: You are walking a beautiful forest with a beautiful Ukranian girl, and somebody drops a war on you, nobody so much asked for it...That until the last minute they did not believe was going to happen?
- VagueMag 3y ago
  Oh I'd say Palantir and the U.S. military understand quite well how many people, especially our own soldiers, find what we're doing unethical, which is why they're so keen to replace human soldiers with amoral robots.
TechBro8615 3y ago
Killer demo...
throwawayjs 3y ago
That was fast. Wondering what the pace of work is like at Palantir to have shipped this so quickly.
- seattle_spring 3y ago
  When I interviewed with them in 2018 they plainly stated that most teams expect 60+ hours. One of the interviewers said she works 10 hours a day, 7 days a week. She did, however, insist that it was a personal choice to do so. I did not end up working there.
  - throwawayjs 3y ago
    Oh wow. That’s a no from me dawg.
- Our_Benefactors 3y ago
  Anecdotally, I spent a Thanksgiving with a family friend, besides an hour for the meal he worked all day on a project due the following day.
danShumway 3y ago
Ignoring the obvious ethical questions, there's also no real assurance that this won't be vulnerable to prompt injection. Is it going to do image analysis? Is it going to work with 3rd-party text? Can it do web searches? And they want this crap to be able to automatically execute workflows without human intervention. > What AIP does not do is walk through how it plans to deal with the various pernicious problems of LLMs and what the consequences might be in a military context. AIP does not appear to offer solutions to those problems beyond “frameworks” and “guardrails” it promises will make the use of military AI “ethical” and “legal.” Utterly, utterly ridiculous. But it's OK because they said it's "ethical." The computing industry has gone completely off the rails with LLMs; we spent a month or two debating hypothetical sci-fi questions until everyone got bored of them, and now nobody is thinking about ethics or security at all for any of their products. It would be embarrassing if it wasn't so horrifying. What we've learned from this is that "how does the AI escape the box?" is a fun thought experiment, but "is this technology inherently vulnerable to extremely basic malicious input attacks?" is a boring question that's not fun to debate and might have implications for the stuff we're actually building, so we're just not going to have that conversation. Or if we do have that conversation, we're definitely not going to delay anything we're building until we have an answer to that question. We're just going to say it's an ongoing issue that will be fixed sometime in the future, probably. It's not reasonable to ask us to solve that problem before we let an LLM control a military drone. /s But every single journalist or writer who covers this stuff should be pointing out that so far, the industry has basically zero idea of how to actually secure an LLM in production. And that's not even getting into "what happens when your Chat AI hallucinates an answer to a question that's wrong and then orders a drone strike off of that hallucination?" Figuring out how to build an LLM agent that can't be hacked is the baseline, and the industry hasn't even passed that point yet.
- nullsense 3y ago
  Agreed. This is just ludicrous. People just keep saying "but think of the benefits!" and then you repeatedly start seeing things like this and you have to ask yourself... when it's all said and done are you going to like the world you wind up with?
- mannerheim 3y ago
akomtu 3y ago
Some generals believe that an army of machines is simply a more efficient army at their disposal. But what AI really enables is excluding those pesky generals, and all other humans, from the chain of command. AI is that One Ring to rule them all.
deleted 3y ago
[deleted]
partlysean 3y ago
Makes me think of the Star Trek episode “A Taste of Armageddon” where a computer simulates the outcome of a war and its causalities, and then those civilizations actually go to murder that number of people in real life.

32 points by konart 3y ago

[flagged]

This is absolutely unreal. They want it to execute battle ops. As in, kill people. Using chat prompts. What could go wrong?

wackycat 3y ago

This headline (and Palantir and our country) ignores the large swath of the population who doesn't find war ethical in the first place.

belter 3y ago
What happens when: You are walking a beautiful forest with a beautiful Ukranian girl, and somebody drops a war on you, nobody so much asked for it...That until the last minute they did not believe was going to happen?
VagueMag 3y ago
Oh I'd say Palantir and the U.S. military understand quite well how many people, especially our own soldiers, find what we're doing unethical, which is why they're so keen to replace human soldiers with amoral robots.

TechBro8615 3y ago

Killer demo...

throwawayjs 3y ago

That was fast. Wondering what the pace of work is like at Palantir to have shipped this so quickly.

seattle_spring 3y ago
When I interviewed with them in 2018 they plainly stated that most teams expect 60+ hours. One of the interviewers said she works 10 hours a day, 7 days a week. She did, however, insist that it was a personal choice to do so. I did not end up working there.
- throwawayjs 3y ago
  Oh wow. That’s a no from me dawg.
Our_Benefactors 3y ago
Anecdotally, I spent a Thanksgiving with a family friend, besides an hour for the meal he worked all day on a project due the following day.

danShumway 3y ago

Ignoring the obvious ethical questions, there's also no real assurance that this won't be vulnerable to prompt injection. Is it going to do image analysis? Is it going to work with 3rd-party text? Can it do web searches? And they want this crap to be able to automatically execute workflows without human intervention. > What AIP does not do is walk through how it plans to deal with the various pernicious problems of LLMs and what the consequences might be in a military context. AIP does not appear to offer solutions to those problems beyond “frameworks” and “guardrails” it promises will make the use of military AI “ethical” and “legal.” Utterly, utterly ridiculous. But it's OK because they said it's "ethical." The computing industry has gone completely off the rails with LLMs; we spent a month or two debating hypothetical sci-fi questions until everyone got bored of them, and now nobody is thinking about ethics or security at all for any of their products. It would be embarrassing if it wasn't so horrifying. What we've learned from this is that "how does the AI escape the box?" is a fun thought experiment, but "is this technology inherently vulnerable to extremely basic malicious input attacks?" is a boring question that's not fun to debate and might have implications for the stuff we're actually building, so we're just not going to have that conversation. Or if we do have that conversation, we're definitely not going to delay anything we're building until we have an answer to that question. We're just going to say it's an ongoing issue that will be fixed sometime in the future, probably. It's not reasonable to ask us to solve that problem before we let an LLM control a military drone. /s But every single journalist or writer who covers this stuff should be pointing out that so far, the industry has basically zero idea of how to actually secure an LLM in production. And that's not even getting into "what happens when your Chat AI hallucinates an answer to a question that's wrong and then orders a drone strike off of that hallucination?" Figuring out how to build an LLM agent that can't be hacked is the baseline, and the industry hasn't even passed that point yet.

nullsense 3y ago
Agreed. This is just ludicrous. People just keep saying "but think of the benefits!" and then you repeatedly start seeing things like this and you have to ask yourself... when it's all said and done are you going to like the world you wind up with?
mannerheim 3y ago

akomtu 3y ago

Some generals believe that an army of machines is simply a more efficient army at their disposal. But what AI really enables is excluding those pesky generals, and all other humans, from the chain of command. AI is that One Ring to rule them all.

deleted 3y ago

[deleted]

partlysean 3y ago

Makes me think of the Star Trek episode “A Taste of Armageddon” where a computer simulates the outcome of a war and its causalities, and then those civilizations actually go to murder that number of people in real life.