4 ms·

I disagree with that analogy. Palantir isn’t getting them more data. It’s helping them use it more effectively. The organizations already have all the data reco

I disagree with that analogy. Palantir isn’t getting them more data. It’s helping them use it more effectively. The organizations already have all the data recorded. If you don’t like the data that organizations record, I’m right there with you, but demonizing the tools that allow them to access data more effectively is an absurd stance to take. Their system also does things like add an immutable audit log, or enforce requirements on how data can be accessed. Working with law enforcement, before Palantir there was nothing preventing an officer from looking up their ex husband or wife, or celebrities, or anyone at random. With it, there’s an audit trail of every search, and most of them need a case number as justification. Even then, certain searches would automatically get flagged for review. We can go back to giving people direct and unaudited SQL access if you’d like, but I’d prefer some more accountability, even if the downside is organizations being able to use data _they already have_ more effectively.

conanbatt 6y ago
A rationalization at best. Facilitating an immoral act is active participation.
- taywrobel 6y ago
  Do you buy things from amazon? They treat their warehouse employees terribly, so I hope not, lest you facilitate an immoral act. Do you ever use uber? They work to keep their drivers legally declared as contractors, despite treating them employees in all other regards. That’s pretty immoral, so I hope you don’t encourage it by supplying them with business.
  - conanbatt 6y ago
    I don't think that warehouse employee situation is immoral so I'm good on that. A neither on the UBER case. I'm safe!
catalogia 6y ago
The analogy seems apt; weapons dealers don't give warlords armies (if they did, they'd be called mercenaries.) They give the armies of warlords the tools they need to be more effective. The violent acts, killing people or holding PII, are still performed by the warlord's organization, but does that excuse the people selling the tools used?
- taywrobel 6y ago
  I guess we’ll just have to agree to disagree. I just don’t think that complaining about a tech company when what you actually dislike is what governments or other organizations are doing is going to impact anything. But Palantir has been the “tech boogieman” since before I even left in 2014, so it probably always will be. It is admittedly easier to blame a smaller tech company as a scapegoat than try to address the larger governmental or organizational actors that you actually have grievances against. Just know it won’t do anything. The people who complain online aren’t their target market and never will be. And the paranoia lends way more credence to their dressed up search engine than it really deserves on technical merit alone. So in a way the people complaining about it are making the issue they’re so upset about worse.
  - catalogia 6y ago
    > It is admittedly easier to blame a smaller tech company as a scapegoat than try to address the larger governmental or organizational actors that you actually have grievances against. Or you can do both. You might as well suppose that people are critical of the mercenary company Blackwater because they want to avoid criticizing America foreign policy and military exploits. I don't think that's accurate at all, it's been my experience that people who are critical of one are very often critical of the other as well. Similarly, it's been my experience that people critical of Palintar are also critical of the organizations and governments who use the services of Palintar.
  - int_19h 6y ago
    The reason why Palantir is the "tech boogieman" is because it willingly and enthusiastically sells its tech to governments specifically for surveillance reasons, and even consults them as to how best integrate it all. And then we look at the politics of its owner, and it's explicitly anti-democratic - so it's not a coincidence. And yes, of course, complaining about Palantir online isn't going to change the government policy. But if working for Palantir means that no other self-respecting software engineer will want to shake your hand, then fewer people will want to work there, and their surveillance tech will be lower quality and have more holes in it.
deleted 6y ago
[deleted]
jvehent 6y ago
So governments already have all the potassium nitrate, Palantir is just helping them make bullets?
- taywrobel 6y ago
  Sure, if you think querying a database is the same as shooting someone. If so, please don’t tell me you’re a DBA.
  - untog 6y ago
    The example given was hyperbolic but I do think it’s true that a database can be “weaponised” once it’s got the power of state violence behind it.
  - zxcmx 6y ago
    I think this is position is a bit naive. Like saying "but child porn is just bits, like any other type of file". A crucial step in shooting someone (or a drone strike) is figuring out who to target and where they are. The fact that this is an important use case for Palantir makes it not hyperbole to talk about "weaponising information".
    - taywrobel 6y ago
      > I think this is position is a bit naive. Like saying "but child porn is just bits, like any other type of file". Everything has a domain it operates in, so I believe that statement has nuance. In particular when you look at how general or specific the domain is. From the perspective of something uploaded to S3? It sure is like any other file. From the perspective of a website dedicated to the dissemination of child porn? It clearly is more than "just bits" in that context. How about from the perspective of a search engine, where that site may be indexed? Welcome to the grey area that all of these debates are rooted in. Technically it's just indexed strings or ngrams, so it is like any other type of file. But there's an argument that a search engine should "know more" than just the raw data, and should be able to understand that context somehow. Palantir is in this grey area. The software isn't built for spying. It's built for managing and understanding vast amounts of data. Can this be used for spying? Yes. It can also be used for double blind clinical trials. Or maintaining insurance systems. Or coordinating disaster relief efforts. It operates technologically in a very general domain, but their flexible user-defined ontology makes it very powerful at operating in more specific domains. So it's a lot less cut and dry than the dissenters make it seem, IMHO.
dave5104 6y ago
I'm curious if "accountability" actually means anything in practice, or if it's just some of the Palantir corporate kool-aid. I don't think everyone associates government agencies with pristine records of holding themselves accountable.
- taywrobel 6y ago
  I’ve only worked with a handful of LEOs in my time there, but the ones I did work with were genuinely concerned about it. If you’ve never worked with a government agency before, let me tell you, Hanlon’s Razor is in full effect. Never attribute to malice that which is adequately explained by stupidity. The higher ups know that free reign for their officers to look up anything is a recipe for legal nightmares. The individual officers are often just too dumb to realize that they shouldn’t search for something, and do so out of curiosity. Having blocks in the way like requiring a case number be entered to run a search, and knowing that that search and results are forever associated to that case, cut down a lot on abuse. Are all forms of abuse the same, or so easily mitigated? No, but I’d like for some system to audit that and at least try to enforce it.
  - patrickaljord 6y ago
    Not sure why you're being downvoted. All your comments have been very well and politely argued (whether people agree with them or not) and this is the first time I got a pick at what palantir is actually doing. Thanks for the info.
    - taywrobel 6y ago
      Meh, it’s fine, downvotes don’t bother me. Especially when you take the position that goes against common opinion, it’s to be expected. The irony to me is that when I was at Palantir they talked extremely openly about all their technology. A ton of it was open sourced, and in depth tech talks were posted to YouTube, but almost no one watched them. I think people just like writing them off as shady or sketchy because it’s easier than needing to reason about the clear ethical nuance that exists in the government contractor space.
pmiller2 6y ago
I'd rather we not help our own government "more effectively" spy on American citizens, thanks.
- taywrobel 6y ago
  Oof, you’re going to have a hard time using pretty much any technology then. Especially if the EARN IT act passes.
  - pmiller2 6y ago
    Well, in the context of this post, that is irrelevant at best, or whataboutism at worst. I notice you don't offer a defense of Palantir here. Why is that?
- taywrobel 6y ago
  > Well, in the context of this post, that is irrelevant at best, or whataboutism at worst. I notice you don't offer a defense of Palantir here. Why is that? HN won't let me reply to that, so I'll do it here. I didn't feel the need to offer a "defense" of Palantir there since I already outlined it above and don't feel like there's value in repeating myself. But you asked, here you go - The government is going to use the data regardless, so long as they have it. The only effective recourse is to make them collecting or storing the data illegal, but good luck with that, seeing as even when it is illegal they do it anyways (see: Snowden). Do you think is Palantir weren't there they'd just say "Ah well, I guess that's that" and be done with it? Nope; they'd go to Lockheed, Raytheon, IBM, or another long-time contractor to build a replacement almost immediately. The technology is genuinely nothing special. It federates searches across disparate database, and stitches results together. But for me, having worked at Palantir, I know what mechanisms are provided to attempt to mitigate abuse. So as long as that data does exist and is going to be used, I'd prefer to have it be as well controlled and audited as possible. And sure, you can argue that the government may forego auditing, or be entirely corrupt, but that doesn't seem to me to be a good reason to not use tools that at least provide that capability.
  - vertex-four 6y ago
GekkePrutser 6y ago
I don't agree entirely. Machine Learning/AI adds an entirely new perspective on big data. It can see patterns in data that would be meaningless without it, correlations that can't be found in other ways. Some data is OK for a government to have if they can just look it up in exceptional cases, but not if they act on every single bit of data collected. In essence, using ML is creating new data (or at least: Information from the raw data). This information can be very revealing and not at all in line with the scope the data was initially collected under, which is one of the principles of GDPR. You can't really view data as static once it's collected. Combining with other existing data and new methods of data analysis make it a totally different picture in terms of privacy.

conanbatt 6y ago

A rationalization at best. Facilitating an immoral act is active participation.

taywrobel 6y ago
Do you buy things from amazon? They treat their warehouse employees terribly, so I hope not, lest you facilitate an immoral act. Do you ever use uber? They work to keep their drivers legally declared as contractors, despite treating them employees in all other regards. That’s pretty immoral, so I hope you don’t encourage it by supplying them with business.
- conanbatt 6y ago
  I don't think that warehouse employee situation is immoral so I'm good on that. A neither on the UBER case. I'm safe!

catalogia 6y ago

The analogy seems apt; weapons dealers don't give warlords armies (if they did, they'd be called mercenaries.) They give the armies of warlords the tools they need to be more effective. The violent acts, killing people or holding PII, are still performed by the warlord's organization, but does that excuse the people selling the tools used?

taywrobel 6y ago
I guess we’ll just have to agree to disagree. I just don’t think that complaining about a tech company when what you actually dislike is what governments or other organizations are doing is going to impact anything. But Palantir has been the “tech boogieman” since before I even left in 2014, so it probably always will be. It is admittedly easier to blame a smaller tech company as a scapegoat than try to address the larger governmental or organizational actors that you actually have grievances against. Just know it won’t do anything. The people who complain online aren’t their target market and never will be. And the paranoia lends way more credence to their dressed up search engine than it really deserves on technical merit alone. So in a way the people complaining about it are making the issue they’re so upset about worse.
- catalogia 6y ago
  > It is admittedly easier to blame a smaller tech company as a scapegoat than try to address the larger governmental or organizational actors that you actually have grievances against. Or you can do both. You might as well suppose that people are critical of the mercenary company Blackwater because they want to avoid criticizing America foreign policy and military exploits. I don't think that's accurate at all, it's been my experience that people who are critical of one are very often critical of the other as well. Similarly, it's been my experience that people critical of Palintar are also critical of the organizations and governments who use the services of Palintar.
- int_19h 6y ago
  The reason why Palantir is the "tech boogieman" is because it willingly and enthusiastically sells its tech to governments specifically for surveillance reasons, and even consults them as to how best integrate it all. And then we look at the politics of its owner, and it's explicitly anti-democratic - so it's not a coincidence. And yes, of course, complaining about Palantir online isn't going to change the government policy. But if working for Palantir means that no other self-respecting software engineer will want to shake your hand, then fewer people will want to work there, and their surveillance tech will be lower quality and have more holes in it.

deleted 6y ago

[deleted]

jvehent 6y ago

So governments already have all the potassium nitrate, Palantir is just helping them make bullets?

taywrobel 6y ago
Sure, if you think querying a database is the same as shooting someone. If so, please don’t tell me you’re a DBA.
- untog 6y ago
  The example given was hyperbolic but I do think it’s true that a database can be “weaponised” once it’s got the power of state violence behind it.
- zxcmx 6y ago
  I think this is position is a bit naive. Like saying "but child porn is just bits, like any other type of file". A crucial step in shooting someone (or a drone strike) is figuring out who to target and where they are. The fact that this is an important use case for Palantir makes it not hyperbole to talk about "weaponising information".
  - taywrobel 6y ago
    > I think this is position is a bit naive. Like saying "but child porn is just bits, like any other type of file". Everything has a domain it operates in, so I believe that statement has nuance. In particular when you look at how general or specific the domain is. From the perspective of something uploaded to S3? It sure is like any other file. From the perspective of a website dedicated to the dissemination of child porn? It clearly is more than "just bits" in that context. How about from the perspective of a search engine, where that site may be indexed? Welcome to the grey area that all of these debates are rooted in. Technically it's just indexed strings or ngrams, so it is like any other type of file. But there's an argument that a search engine should "know more" than just the raw data, and should be able to understand that context somehow. Palantir is in this grey area. The software isn't built for spying. It's built for managing and understanding vast amounts of data. Can this be used for spying? Yes. It can also be used for double blind clinical trials. Or maintaining insurance systems. Or coordinating disaster relief efforts. It operates technologically in a very general domain, but their flexible user-defined ontology makes it very powerful at operating in more specific domains. So it's a lot less cut and dry than the dissenters make it seem, IMHO.

dave5104 6y ago

I'm curious if "accountability" actually means anything in practice, or if it's just some of the Palantir corporate kool-aid. I don't think everyone associates government agencies with pristine records of holding themselves accountable.

taywrobel 6y ago
I’ve only worked with a handful of LEOs in my time there, but the ones I did work with were genuinely concerned about it. If you’ve never worked with a government agency before, let me tell you, Hanlon’s Razor is in full effect. Never attribute to malice that which is adequately explained by stupidity. The higher ups know that free reign for their officers to look up anything is a recipe for legal nightmares. The individual officers are often just too dumb to realize that they shouldn’t search for something, and do so out of curiosity. Having blocks in the way like requiring a case number be entered to run a search, and knowing that that search and results are forever associated to that case, cut down a lot on abuse. Are all forms of abuse the same, or so easily mitigated? No, but I’d like for some system to audit that and at least try to enforce it.
- patrickaljord 6y ago
  Not sure why you're being downvoted. All your comments have been very well and politely argued (whether people agree with them or not) and this is the first time I got a pick at what palantir is actually doing. Thanks for the info.
  - taywrobel 6y ago
    Meh, it’s fine, downvotes don’t bother me. Especially when you take the position that goes against common opinion, it’s to be expected. The irony to me is that when I was at Palantir they talked extremely openly about all their technology. A ton of it was open sourced, and in depth tech talks were posted to YouTube, but almost no one watched them. I think people just like writing them off as shady or sketchy because it’s easier than needing to reason about the clear ethical nuance that exists in the government contractor space.

pmiller2 6y ago

I'd rather we not help our own government "more effectively" spy on American citizens, thanks.

taywrobel 6y ago
Oof, you’re going to have a hard time using pretty much any technology then. Especially if the EARN IT act passes.
- pmiller2 6y ago
  Well, in the context of this post, that is irrelevant at best, or whataboutism at worst. I notice you don't offer a defense of Palantir here. Why is that?
taywrobel 6y ago
> Well, in the context of this post, that is irrelevant at best, or whataboutism at worst. I notice you don't offer a defense of Palantir here. Why is that? HN won't let me reply to that, so I'll do it here. I didn't feel the need to offer a "defense" of Palantir there since I already outlined it above and don't feel like there's value in repeating myself. But you asked, here you go - The government is going to use the data regardless, so long as they have it. The only effective recourse is to make them collecting or storing the data illegal, but good luck with that, seeing as even when it is illegal they do it anyways (see: Snowden). Do you think is Palantir weren't there they'd just say "Ah well, I guess that's that" and be done with it? Nope; they'd go to Lockheed, Raytheon, IBM, or another long-time contractor to build a replacement almost immediately. The technology is genuinely nothing special. It federates searches across disparate database, and stitches results together. But for me, having worked at Palantir, I know what mechanisms are provided to attempt to mitigate abuse. So as long as that data does exist and is going to be used, I'd prefer to have it be as well controlled and audited as possible. And sure, you can argue that the government may forego auditing, or be entirely corrupt, but that doesn't seem to me to be a good reason to not use tools that at least provide that capability.
- vertex-four 6y ago

GekkePrutser 6y ago

I don't agree entirely. Machine Learning/AI adds an entirely new perspective on big data. It can see patterns in data that would be meaningless without it, correlations that can't be found in other ways. Some data is OK for a government to have if they can just look it up in exceptional cases, but not if they act on every single bit of data collected. In essence, using ML is creating new data (or at least: Information from the raw data). This information can be very revealing and not at all in line with the scope the data was initially collected under, which is one of the principles of GDPR. You can't really view data as static once it's collected. Combining with other existing data and new methods of data analysis make it a totally different picture in terms of privacy.