16 ms·

Palantir is using War on Terror tools to track American citizens

523 points by pdog 8y ago

technologia 8y ago
Is this really surprising? Palantir's value comes from its data not that god awful interface it has.
- danso 8y ago
  Really? Because others have frequently claimed the opposite -- that it is a visualization company with the insight to focus on lucrative government contracts. https://news.ycombinator.com/item?id=14674460 https://news.ycombinator.com/item?id=14674460
  - technologia 8y ago
    I'm just speaking from my interactions with their platform during my time as a govie. I don't disagree with what you are saying, they definitely capitalized on gov contracts in this space; I should have been clearer with my statement. For groups that know their data well, they tend to push for custom features from Palantir in lieu of developers resulting in some crap shoot interfaces.
chinathrow 8y ago
I can't wait that EU citizen will hit them hard after May 25th.
- paulie_a 8y ago
  I think palantir is CIA related... They probably are not going to worry about pesky international regulations
  - sschueller 8y ago
    That's what sanctions are for. But I highly doubt the EU will ever sanction the US as they are so dependant on them.
    - noir_lord 8y ago
      EU has more people and an equivalent sized economy. The EU could stand up to the US if they felt it mattered. Other than defence partnerships the EU is no more dependent on the US than the US is on the EU. It's a two way street. Unless you think the US would be happy to walk a way from the largest single market in the world (by GDP).
      - briandear 8y ago
        Except the EU isn’t a country. Poland doesn’t have the same incentives that Greece might, as an example.
        noir_lord 8y ago
        Except trade deals and regulations are negotiated/implemented top down. So the EU has teeth if the US pushed it too far. It's hard to get the EU to agree on a lot of things but something sufficient to make the EU consider punishing the US would likely be important enough they would agree.
      - sschueller 8y ago
        The USD is currently 62.70%[1] of all currency reserves. If you lose the ability to trade in dollars you are screwed and the US knows this. [1] https://en.wikipedia.org/wiki/Reserve_currency https://en.wikipedia.org/wiki/Reserve_currency
  - 2close4comfort 8y ago
    Sanctions? I am not sure the EU has enough clout to pull that off against something the likes of Palantir, I am sure that they would have to know it is there before they could ask for removal and FIOA or other legal devices are not going to be enough pry that open under this administration.
- adventured 8y ago
  As the yearned-for fake privacy walls go up, the only entities that will still have the ability to abuse privacy, will be politically connected corporations and governments. Germany supposedly has some of the best privacy laws in the world. Their government freely abuses their people with domestic espionage. They'll continue to work with technology businesses to that end as it fits their purposes. There's a lot of outrage about Facebook. Meanwhile, absolutely nothing has changed, or is going to change, about the NSA spying. Nothing got fixed. More or less the Snowden revelations were entirely meaningless in terms of changing the status quo. The NSA is spying on EU and US citizens just as they were five years ago, just as they will be five years from now. Where's the outrage today about the NSA? Almost everyone gave up, that's why Facebook is the new target, the people couldn't do anything about the government spying. GDPR creates the haves and have nots when it comes to access to abuse privacy. That's all it will change. If you have the political access (which Palantir does and will), you're going to get the security keys to abuse privacy (the alternative is for EU governments to entirely shut down all domestic espionage, that will never happen).
beaconstudios 8y ago
Why does the media seem to be publishing so many "well duh" tier articles about various companies at the moment? "Big data company collects big data", "company known for monetising private data collects private data", "political consulting company profiles voters and tries to affect vote". No shit?
- Harvey-Specter 8y ago
  There are obviously people out there for whom this is new information.
- empath75 8y ago
  I know this might be a surprise to you, but not everyone knows everything that you do.
  - beaconstudios 8y ago
    I'm not exactly tapping my own research here. Palantir is a big data/spying company, what do you expect from them? Facebook asks you for lots of private data and it's clearly obvious that when you post data on Facebook, Facebook gets that data. So what's the shock that they hold this private data? The only surprising aspect of the current Facebook drama is that they apparently mine your phone for additional data but that's not the main focus of the outcry. Why are people surprised that Cambridge Analytica, a political consultancy, did political research and affected a voter outreach strategy? None of this requires any real background knowledge, and yet the outcry seems extremely disproportionate to the revelations at hand. It seems like the Snowden reports all over again, where the world was up in arms that an intelligence agency was collecting intelligence.
    - xg15 8y ago
      > Palantir is a big data/spying company, what do you expect from them? How can expect anything of them when I don't even know that company exists, let alone what it does? > Why are people surprised that Cambridge Analytica, a political consultancy, did political research and affected a voter outreach strategy? None of this requires any real background knowledge, and yet the outcry seems extremely disproportionate to the revelations at hand. It seems like the Snowden reports all over again, where the world was up in arms that an intelligence agency was collecting intelligence. This is simplifying the discussion a lot. The general assumption about both intelligence and research was probably that they stick to rules and are concerned with their own fields. I'd guess the average joe would (before snowden at least) think that an intelligence agency is concerned with foreign diplomats and military strategists and a political consultancy does opinion polls. It's something different if said agencies collect intimate data of random citizens without their consent. At least in public mind, that was something the stasi or similar organisations did - but certainly not their own side.
eptcyka 8y ago
Whilst the article may not have any surprising information for the regular hacker news reader, it may come as a surprise that a non-insurance, non-government business holds more private data about an individual than Facebook does, especially since most individuals will not recall ever interacting with said company. I think it is beneficial if more people become aware of this.
- jonnybgood 8y ago
  > it may come as a surprise that a non-insurance, non-government business holds more private data about an individual than Facebook does Where in the article is that stated?
  - drcongo 8y ago
    In the title?
    - jonnybgood 8y ago
      It’s clickbait. It doesn’t state it or give a citation for it anywhere in the actual article.
      - lgl 8y ago
        I don't think it's directly stated in the article. But with a little common sense we can probably safely assume that a company that is designed specifically to analyze huge amounts of data for law enforcement agencies, governments and 3 letter agencies will probably have a lot more data than Facebook. In fact, we can probably also assume that people's Facebook data has already been inputted in Palantir's system as just another data source. Sure that's a lot of assumptions and probabilities, but on the other hand: Peter Thiel.
      - drcongo 8y ago
        Yeah, but you edited your post to specify "in the article" after I'd posted. At the point of my reply, it was correct.
jondiggsit 8y ago
I’ve always thought - I don’t care if the NSA is spying on US citizens because their job is national security. The problem is when those tools trickle down. All of a sudden you have a local cop with too much time on their hands spying on you. But if you’re not doing anything wrong, you shouldn’t have anything to hide, right? Isn’t that how the saying goes?
- EGreg 8y ago
  Counterpoint: http://magarshak.com/blog/?p=169 http://magarshak.com/blog/?p=169
- dTal 8y ago
  Must be a nice way to live, having full faith that your government will always do the right thing. Relaxing.
  - nerdponx 8y ago
    It's more a matter of focusing the discussion. Once you bring government into it, it makes the conversation a lot more complicated. This is not a complicated issue.
    - pfortuny 8y ago
      But it is the Government that complicates the thing: USSR, China, Fascist regimes: they persecute OPINIONS not deeds.
  - mtgx 8y ago
    We literally have thousands of years of proof that governments always abuse their power when they think someone is their "enemy". So it's strange that people default to "trusting their government." I blame this on America's blue-team/red-team political thinking, which forces one to stick to a party no matter what. The standard for "enemy" could also be as low as they choose it to be, which is also weird that people think the government wouldn't go after them because they are "innocent", as if these people get to decide who is innocent and who isn't. No, it's the government that does that. Courts may have the final say in the end (unless you're indefinitely detained for multiple years first), but until then the government can do a lot of damage to your life if it comes after you. Even putting you through years of lawsuits, stress and money spent on lawyers would be bad enough. You may win in the end, but it will be a Pyrrhic victory. And that's if you don't take their plea deal first when they scare you with multiple bogus charges that you think shouldn't even apply to you but you're too scared of the 50 years in prison sentence they're threatening you with.
dmitriid 8y ago
> As shown in the privacy breaches at Facebook and Cambridge Analytica, the pressure to monetize data at tech companies is ceaseless Says Bloomberg, and includes Facebook code on their pages
ckastner 8y ago
> Aided by as many as 120 “forward-deployed engineers” This term popping up again made me curious. To my surprise, that's not just some overloaded nickname (what's cooler than styling yourself with a military or martial arts term, right?), that's what they are actually called: https://www.palantir.com/careers/teams/forward-deployed-software-engineers https://www.palantir.com/careers/teams/forward-deployed-soft...
- OldFatCactus 8y ago
  they fear the words "consultant" or "contractor" I think
  - mcguire 8y ago
    Forward deployment had a much hipper, militaristic sound.
  - ng12 8y ago
    Yup. And FDE is just a consultant, an FDSE is a software engineer attached to a specific client. A regular SE works only on generic products.
- ataturk 8y ago
  Because they wish to appeal to the spirit of adventure these people have. They're washed up mercenaries of the state who desperately want to feel relevant one more time. I wish people would stop helping the war machine. Stay the hell home for once.
- jakelazaroff 8y ago
  What's the difference between a normal software engineer and a "forward-deployed" software engineer, though? It sounds to me like that's just their military-flavored version of "rockstar developer" or "code ninja".
  - beaconstudios 8y ago
    my understanding is forward deployed engineers set up shop in Palantir's clients' offices and implement the solution using Palantir's tech. Basically doing rollout and front-of-house tech support.
amelius 8y ago
Why isn't there a privacy watchdog that constantly monitors these kind of companies? I also expect this watchdog to test for data leakage from one site/company to another (e.g. by taking on the role of customer). They should do this on a frequent basis. Without this, companies seem to be free to do whatever they want.
- justherefortart 8y ago
  Hahahaha This is how the fucking government gets around the Bill of Rights, primarily the 4th. When it's outsourced, you have no rights. Parallel construction needs to be the next addition to the 4th or it's own amendment.
common_ 8y ago
Pop Quiz: Which two companies had their first office at 101 University Avenue in Palo Alto? The answer says quite a lot, I think.
- drcongo 8y ago
  I've found Piazza's Fine Foods Inc and PhysioTouch Inc. Not sure what I'm supposed to take away from this answer though.
  - gberger 8y ago
    Facebook
- deleted 8y ago
  [deleted]
ataturk 8y ago
My biggest issue with former military people is that their skills don't easily translate into any other fields and that their mindset is usually heavily canted towards statism, which is a big problem when you combine the kinds of surveillance technology we have with needing to turn a profit. It makes anything East Germany did look like they were underachievers.
diskandar27 8y ago
this is kind of scary and sad when powerful technology is being misused ...probably it is better to live in China ??? ..the government openly monitoring its citizen instead of spying on them
- pmlnr 8y ago
  Definitely not, although that specific aspect - you _know_ you're being watched - makes life different, in some way, simpler. The rules are obvious, however, if they change, you might find yourself in a much worse situation, than in the West - for now.
  - gnode 8y ago
    > you _know_ you're being watched - makes life different, in some way, simpler. Arguably, what happens when people are aware of surveillance is one of the worst parts of it. Following the Snowden revelations, there were noticeable chilling effects: https://www.washingtonpost.com/news/wonk/wp/2016/04/27/new-study-snowdens-disclosures-about-nsa-spying-had-a-scary-effect-on-free-speech/?utm_term=.824c5af75415 https://www.washingtonpost.com/news/wonk/wp/2016/04/27/new-s...
    - deleted 8y ago
      [deleted]
- kirubakaran 8y ago
  Reminds me of a friend who said he never wears a condom, because condoms can break.
opportune 8y ago
I'm much less concerned with Palantir being used internally by corporations to detect information leaks (although in practice this also means it will be used to quash whistleblowers), since that's at least a viable use for their technology. And it's good that Palantir is pivoting towards less invasive applications of their technology such as supply chain tracking. But the article touches on the biggest issue with Palantir, which is that they seem to be completely fine with allowing law enforcement agencies to use their software for parallel construction, which is in my opinion a pretty egregious violation of the 4th amendment. Although it may be constitutional for police departments to purchase data from e.g. Facebook or some data broker, even local police departments are known to employ dragnet-style surveillance through things like Stingray devices. Of course we have a number of three-letter agencies, also Palantir customers, who don't even need to pretend that they're following the 4th amendment. Since Palantir is essentially enabling the construction of all the tools required for a complete surveillance state, and actively using their tools for that purpose for financial gain, I think that makes them one of the least ethical companies in tech. I certainly wouldn't be able to live with myself if I knew I was writing software with the explicit purpose of being able to track down rebels in Yemen, catch illegal immigrants, and violate the constitutional rights of millions of Americans.
- jonnybgood 8y ago
  Can you expand on exactly how Palantir is used for parellel construction?
  - taneq 8y ago
    They dig up data telling the police where to look. The police then look in just the right place and "legitimately find" the evidence that they would never have stumbled on and couldn't have legally known was there.
  - opportune 8y ago
    Law enforcement uses dragnet warrantless surveillance to gather data (such as who is calling whom, who is in physical proximity with whom). That data is loaded into Palantir's software. Law enforcement uses that data to acquire admissible evidence of the perpetrators' crimes. This is the crucial step, because although the original evidence couldn't be admissible in court due to its collection without a warrant, it can be used to acquire more evidence. If the warrantless surveillance includes information such as the time and place of where you plan to commit a crime, law enforcement can stop you for a random check on the way there, giving them actual evidence for prosecution. Now that evidence can be used in court, even though the only way law enforcement were able to acquire it was through unconstitutional practices. The thing about Palantir is that it needs big data to work. Police departments don't just have random chunks of data lying around containing large numbers of connections between large numbers of people - they might have something like that for something like suspected criminal gangs, but that wouldn't be enough data to make Palantir's software worthwhile. So acquiring the data necessary to use Palantir's software requires mass data collection. Some of their mass data collection techniques may not require a warrant, such as using license plate captures, but some does, such as who you're calling. So does Palantir perform warrantless surveillance themselves? To my knowledge, no. But when they sell their software to law enforcement agencies, they must know that the only way their customer could get bang for their buck is through mass surveillance. In my opinion, the person willingly and knowingly selling tools for oppressive purposes holds a lot of blame for the actual oppression that occurs as a result.
klunger 8y ago
Culture flows from the top. Peter Thiel apparently views women's suffrage and poor people as "tough for libertarians [like him]". Can we talk about, well, how problematic it is that a company that has information on just about everyone (and all the power this yields) is lead by someone like this? > As Thiel’s wealth has grown, he’s gotten more strident. In a 2009 essay for the Cato Institute, he railed against taxes, government, women, poor people, and society’s acquiescence to the inevitability of death. (Thiel doesn’t accept death as inexorable.) He wrote that he’d reached some radical conclusions: “Most importantly, I no longer believe that freedom and democracy are compatible.” The 1920s was the last time one could feel “genuinely optimistic” about American democracy, he said; since then, “the vast increase in welfare beneficiaries and the extension of the franchise to women—two constituencies that are notoriously tough for libertarians—have rendered the notion of ‘capitalist democracy’ into an oxymoron.”
- opportune 8y ago
  "I no longer believe that freedom and democracy are compatible." So it seems like Thiel's been a fascist for quite a while. And he's working on surveillance technology with law enforcement agencies... You're absolutely right that Thiel is probably the worst person, for society, to be running a business like this.I can't believe YCombinator partnered with him after he said things like this, and managed an entire two years with him until they dumped him over Trump. Look, I'm not saying that libertarianism is fascism. But what kind of libertarianism is non-democratic? It would either be authoritarian capitalism (fascism), or anarcho-capitalism.
  - exelius 8y ago
    Values will always be secondary to money in this world.
    - chillingeffect 8y ago
      1. That's a gloomy self-fulfilling prophecy. Try thinking more constructively. 2. Money is a value, so your assertion is a paradox.
occamrazor 8y ago
And they sell GPDR compliance solutions: https://www.palantir.com/solutions/gdpr/ https://www.palantir.com/solutions/gdpr/ Oh, the irony!
- prepend 8y ago
  Is it though? It’s pretty hard for companies with lots of diverse data on their users to comply. Finding it all and deleting it is pretty hard. The hope I had was that companies wouldn’t track it at all. I fear that only companies that afford stuff like Palantir will collect it, because they can comply and show evidence of complying.
- djabatt 8y ago
  There is a lot of irony in capitalism
prepend 8y ago
This isn’t Palantir, this is the companies who buy Palantir software. The data from JPMorganChase isn’t visibile to Palantir as an organization, but to the bank. This headline is weird and as accurate as “Microsoft knows everything about you” because the bank uses SQL Server. Or “the Python developers” because python and pandas is used to link a bunch of data together. Since this is Bloomberg, I would expect them to know the difference between software and data services. Google gets your data when you use them (except for enterprise), most other products don’t. Cynically, this seems like oppo PR against Thiel. How could you prove this suspicion?
- jawns 8y ago
  Is it really the case that data does not cross boundaries among Palantir clients? And that the algorithms used by any given client are only informed by the data that client provides?
  - creaghpatr 8y ago
    Unless proven otherwise, then that is hearsay.
    - prepend 8y ago
      Good point, but this is the risk for every cloud hosted software. So every AWS and azure product has this suspicion that’s impossible or extremely difficult to remove. If you’re really paranoid, then all non-OSS software. So again, why is Palantir more worrisome than Salesforce?
  - mcguire 8y ago
    If so, then Palantir is missing a big revenue source: integration with public data that each client would otherwise have to collect on its own.
  - ryan-c 8y ago
    When I worked there (mid 2009 to early 2014), almost all customers had dedicated installations (as in a rack of servers) with just their data on-site. Even the hosted instances had siloed data. There are features for peering, but AFAIK that has to be explicitly set up between installations.
ideophobia 8y ago
I used to be intelligence analyst that utilized Palantir on a daily basis. Ask me anything.
- chillingeffect 8y ago
  What kind of search filtering options are available for fishing expeditions?
  - ideophobia 8y ago
    You can search on or around nearly anything. That's one of their selling points to customers. The search function is multi-faceted, in that 1) it can index the data of any files you upload to Palantir (making everything text searchable), 2) Analysts can indicate specific attributes of an entity just by highlighting text in a file and marking it as a DOB or SSN or whatever. 3) The # of attributes is nearly limitless. One of the primary screens in Palantir is a simple search page designed to work (and look) like Google, to search upon your entire database however you'd like. You can enhance the search function via the various attributes built into your installation or using advanced search tools. So I guess a "fishing expedition" in the traditional sense is very easy, as it's limited by the data you have, not the system itself.
    - zrobotics 8y ago
      Are there options to restrict search space (i.e. limit data sources that could potentially contaminate an investigation)? And if so, is that a common practice, or is it more typical to search through all avaliable data for anything potentially useful and weed out later?
      - ideophobia 8y ago
        I don't specifically remember an option to weed out searching, but I'm reasonably certain there were advanced options that included "does not equal" or "does not contain" type boolean searching. From my experience though, it wasn't common practice to weed out from the start, but rather to start with a general search and slowly narrow it down to the targeted data. A lot of data sources weren't automatically imported, they were more like extended queries. So if I wanted employment records, for example, I had to initiate that as a custom search to query that external system, and pull back the data for the specific individual I was investigating, it was not available in the database by default. But once it was imported and tagged, it was available in any future searches for the remainder of the data retention period.
EGreg 8y ago
Let’s suppose you have an actual solution, like a free open source platform that is used by millions of people and took you 7 years and half a million dollars to build, because you saw the problems coming back in 2011? A platform with 5 million users in 110 countries? It seems the press of today won’t really care about solutions, but only highlighting a scandal. Am I wrong? What would one do to get them to write a story about actual solutions like solid.mit.edu or ours: https://m.youtube.com/watch?v=pZ1O_gmPneI https://m.youtube.com/watch?v=pZ1O_gmPneI Press kit? News conference? Some kinda weird publicity stunt? Serious question.
binarymax 8y ago
This isn't surprising. In fact it was expected. This company was born out of the Total Information Awareness mandate by GWB shortly after 9/11. Total Information Awareness means monitoring of everyone and everything. They got contracts and funding and grew to the original vision set in motion 15 years ago.
2close4comfort 8y ago
Wouldn't be interesting if all the Facebook data being collected is really to feed info to Palantir...with a layer of deniability and they get data on billions.
dangerlibrary 8y ago
> The U.S. Department of Health and Human Services uses Palantir to detect Medicare fraud. Having worked in this field, I can say that this is hilarious overkill. The vast majority (in dollar terms) of Medicare/Medicaid fraud is drug companies misreporting their Average Market Price (AMP), the critical variable in how much they get paid. In 2012, Glaxo settled(!) a small bundle of cases, some of which I worked on, for three billion dollars [0]. You don't need police-state surveillance of individuals, you need visibility into company transaction records. [0] https://www.justice.gov/opa/pr/glaxosmithkline-plead-guilty-and-pay-3-billion-resolve-fraud-allegations-and-failure-report https://www.justice.gov/opa/pr/glaxosmithkline-plead-guilty-...
- refurb 8y ago
  Medicare fraud goes well beyond price reporting by manufacturers. Physicians can bill for more expensive procedures, bill phantom patients, etc. That's a huge part of the multi billion dollar fraud that Medicare has. I assume being able to identify physicians will outlier billing patterns would be helpful in figuring out where to look closer.
  - dangerlibrary 8y ago
    I mean, if you want to get into it, you're both right and wrong. The second largest cases are typically against hospital chains that encourage (read: basically force) their coding teams to use particular billing codes. It's easy enough to model this from claims data (and we did). When a chain buys a new hospital, suddenly patients that present with the same diagnosis codes are all getting extensive exams that ostensibly require 45+ minutes of a physician's time and cost 4-10x more. Again - the large operators (hospital chains) absolutely dwarf individuals (doctors). DOJ barely has the resources to prosecute the largest offenders, and have faced repeated budget cuts. Wasting time on individual doctors or private practices (beyond maybe making examples of a few egregious actors) would be an inefficient allocation of their resources.
    - ng12 8y ago
      Do you know the details of Palantir's investigation? How do you know they're not doing exactly what you just described? Palantir is just a data analysis company, they're not the NSA.
everdev 8y ago
This was a huge debate when the Patriot act was passed, that US citizens would get caught up in surveillance. It seems common in history that during war time, a country aggressively pursues it's own population for spies or sympathizers for the enemy. It send also common that the country goes overboard so as not to let anyone through the cracks and a lot of innocent people get swept up in the process.
ajeet_dhaliwal 8y ago
My main takeaway is that the corporations don't have the funds to waste on this and don't see the ROI (given how many large companies listed in the article stopped using Palantir) whereas the government does. Peter Thiel being a libertarian is clearly aware of the scale of government wastage and decides to go in for the win.
hapnin 8y ago
No surprise. Thiel is a grade-A asshole who appears to be amoral.
thestephen 8y ago
> The LAPD uses Palantir’s Gotham product for Operation Laser, a program to identify and deter people likely to commit crimes. Information from rap sheets, parole reports, police interviews, and other sources is fed into the system to generate a list of people the department defines as chronic offenders [...] The list is distributed to patrolmen, with orders to monitor and stop the pre-crime suspects as often as possible, using excuses such as jaywalking or fix-it tickets. This is pretty much everything I was afraid of when it comes to this field - big data "prediction" being used to predispose people into crime, and by extension, algorithmic racism.
- _salmon 8y ago
  Minority Report. Literally.
- zo1 8y ago
  For all we know, keeping a closer eye on previous criminals deters them from repeating their crimes. This strikes me as a very benign way of preventing a crime from happening in the first place. Rather than letting it happen, creating a victim and perpetrator, and then having to punish the latter yet again. At the very least, this should give them the impression that policing is much higher than it actually is. Sounds like a good deterrent to me.
  - archerface 8y ago
    There is a lot of problems with this strategy when introduced into the real world; the chaotic vortex that it is. The data can be wrong. This could be easily fixed if law enforcement agencies had to actually verify that the data was correct, but in practice this seems to rarely occur. When people are told by a group that they are a thing, repeatedly, they eventually accept that they are that thing. If the cops keep harassing you for being a gangbanger when you have never been one your whole, you just happen to be associated with a few individuals who are, you eventually just move towards that group. Why not? The powers that be already think you are one and they treat you like one, which causes other people to treat you like one, so why not just be one? You also need to consider the idea of enforcing crime that hasn't happened yet. That defeats the whole point of law enforcement. It is meant as a reactionary force, not an offensive force. Policing someone who has the potential for crime assumes that they will do it again, which makes it so your essentially treating them as guilty for something they haven't done yet. From my perspective, that seems to defeat the purpose of our legal system. Anyway, I think I will end my rant here...
clavalle 8y ago
I think I heard the maxim once "If it can be abused, it will be abused." We need to start getting ahead of worst-case scenarios and to hell with people that think that approach is paranoid.
sevensor 8y ago
After September 11, thre was a big push for "Visual Analytics," which is basically rebranded statistics plus visual data mining tools. There's a report called Illuminating the Path that lays out the research agenda. I read it in grad school when my advisor was trying to tag everything under the sun as Visual Analytics in an attempt to get access to the Visual Analytics money pot. (We didn't.) It's creepy to see it come back around, fully realized by Palantir.
lerie82 8y ago
From someone who builds software like this all the time, I am tired of seeing these "privacy" issues people are crying about. People want to get wifi off a FREE wifi connection and then bitch about their data being leaked, even though they ticked the box that said the wifi owner could do whatever he wanted with the data collected over his own network. People have to learn to be responsible for their own data, use their own services and secure their own technologies and stop blaming everyone else for data they out online or ALLOW to be put online but storing it on a device that has access to the internet, we're talking mobile here. I see people bitching about Facebook when it comes to privacy, but Facebook isn't about privacy, its exactly the opposite, its "social media" where you "share" things, not hide them. Palantir has nice solutions that not only gather data but uses sophisticated techniques to tie the data together. If you think your constitutional rights have been violated then you need to speak up, and not on yc, and not on some nameless forum, not on your blog and not on social media. You need to take your ass down and file a lawsuit, otherwise, just sit there quietly while the big boys secure your country.
return1 8y ago
i find the peter thiel conspiracy-theoretic spiderweb in the article ridiculous (i believe you could make a similar one for any well known vc). I wonder however why he invests in these companies. i mean centralizing intelligence seems a very anti-libertarian thing to do.
amelius 8y ago
How can we build a case against them, and sue them? EFF?
avs733 8y ago
Ignoring the enormous argument over whether what I am about to say is legal in the US/America's particularly constructed law... Is it time to consider whether companies who's primary business is the collection, summation, synthesis, and re purposing of information should be defacto illegal? Its not that having the government do this is meaningfully better, but I'm convinced that is a better overall path. The public seem more afraid and more willing to engage in a more serious discussion over ethics. At the very least, it would enable the public to have more oversight over how such data is collected, stored, embargoed (or not), and used. The level of abstraction and fuzzing that is allowed by allowing a non-governmental entity like Palantir or Facebook to have (whether as a collector or aggregator) the data and governments be only a user seems too difficult to regulate and even have a transparent discussion about because claims of national security and secrecy are nested within claims of trade secret and intellectual property in ways that create a moat around effective oversight. So seriously...if we made this business model illegal, what do we lose? What exceptions would need to exist?
- ng12 8y ago
  > Is it time to consider whether companies who's primary business is the collection, summation, synthesis, and re purposing of information should be defacto illegal? You mean IBM/SAP/Oracle?
  - avs733 8y ago
    I don't think it is unreasonable within what I said to differentiate the industry that supports data collection and archiving as a broad process (e.g., those you describe) with those that seek and intent specifically to collect and operationalize data about people without fully informed consent.
    - ng12 8y ago
      Palantir does not do data collection in that sense. They're much closer to IBM than you realize.
      - 8y ago
otakucode 8y ago
Does anyone else remember a time when paranoia was considered a mental illness rather than a virtue? Or am I imagining that?
flyingcircus3 8y ago
When a company is named after the crystal ball an evil wizard uses to spy on his enemies in The Lord of The Rings, and that company's founder was the first large investor in Facebook, and that same founder is able to fund company ending lawsuits against media outlets, headlines like this are akin to "water is wet"
shmerl 8y ago
So it's a double play on the name now. I.e. it's in the hands of Sauron?
snowwrestler 8y ago
> Cavicchia was in charge of forensic investigations at the bank. Through Palantir, he gained administrative access to a full range of corporate security databases that had previously required separate authorizations and a specific business justification to use. He had unprecedented access to everything, all at once, all the time, on one analytic platform. He was a one-man National Security Agency, surrounded by the Palantir engineers, each one costing the bank as much as $3,000 a day. I've long thought that the "secret sauce" of Palantir is just that signing a Palantir contract gives executives a reason and excuse to pull together data that had never before been indexed against each other. I mean, if you take any old dumb BI platform and give it access to data that people have not previously connected, you're going to get new insights. Someone once described Palantir as a data mapping consultancy that marketed itself as a software platform.
anigbrowl 8y ago
Sort of off-topic, but I have a need for a graph/relationship oriented database like this (on a relatively tiny scale) and am having a hard time choosing between the many alternatives. Open source is a must, but otherwise I'm primarily interested in ease of use and visualization compatibility - I want to focus on my problem domain, rather than writing a lot of code or hacking for its own sake. Right now looking at Neo4j and GraphDB, would welcome other suggestions. Thanks in advance!
mdekkers 8y ago
"the vast increase in welfare beneficiaries and the extension of the franchise to women [...] have rendered the notion of ‘capitalist democracy’ into an oxymoron." That guy is a nice piece of work.
billhendricksjr 8y ago
Given Thiel’s purported views on privacy, the hypocrisy is overwhelming.

523 points by pdog 8y ago

technologia 8y ago

Is this really surprising? Palantir's value comes from its data not that god awful interface it has.

danso 8y ago
Really? Because others have frequently claimed the opposite -- that it is a visualization company with the insight to focus on lucrative government contracts. https://news.ycombinator.com/item?id=14674460 https://news.ycombinator.com/item?id=14674460
- technologia 8y ago
  I'm just speaking from my interactions with their platform during my time as a govie. I don't disagree with what you are saying, they definitely capitalized on gov contracts in this space; I should have been clearer with my statement. For groups that know their data well, they tend to push for custom features from Palantir in lieu of developers resulting in some crap shoot interfaces.

chinathrow 8y ago

I can't wait that EU citizen will hit them hard after May 25th.

paulie_a 8y ago
I think palantir is CIA related... They probably are not going to worry about pesky international regulations
- sschueller 8y ago
  That's what sanctions are for. But I highly doubt the EU will ever sanction the US as they are so dependant on them.
  - noir_lord 8y ago
    EU has more people and an equivalent sized economy. The EU could stand up to the US if they felt it mattered. Other than defence partnerships the EU is no more dependent on the US than the US is on the EU. It's a two way street. Unless you think the US would be happy to walk a way from the largest single market in the world (by GDP).
    - briandear 8y ago
      Except the EU isn’t a country. Poland doesn’t have the same incentives that Greece might, as an example.
      - noir_lord 8y ago
        Except trade deals and regulations are negotiated/implemented top down. So the EU has teeth if the US pushed it too far. It's hard to get the EU to agree on a lot of things but something sufficient to make the EU consider punishing the US would likely be important enough they would agree.
    - sschueller 8y ago
      The USD is currently 62.70%[1] of all currency reserves. If you lose the ability to trade in dollars you are screwed and the US knows this. [1] https://en.wikipedia.org/wiki/Reserve_currency https://en.wikipedia.org/wiki/Reserve_currency
- 2close4comfort 8y ago
  Sanctions? I am not sure the EU has enough clout to pull that off against something the likes of Palantir, I am sure that they would have to know it is there before they could ask for removal and FIOA or other legal devices are not going to be enough pry that open under this administration.
adventured 8y ago
As the yearned-for fake privacy walls go up, the only entities that will still have the ability to abuse privacy, will be politically connected corporations and governments. Germany supposedly has some of the best privacy laws in the world. Their government freely abuses their people with domestic espionage. They'll continue to work with technology businesses to that end as it fits their purposes. There's a lot of outrage about Facebook. Meanwhile, absolutely nothing has changed, or is going to change, about the NSA spying. Nothing got fixed. More or less the Snowden revelations were entirely meaningless in terms of changing the status quo. The NSA is spying on EU and US citizens just as they were five years ago, just as they will be five years from now. Where's the outrage today about the NSA? Almost everyone gave up, that's why Facebook is the new target, the people couldn't do anything about the government spying. GDPR creates the haves and have nots when it comes to access to abuse privacy. That's all it will change. If you have the political access (which Palantir does and will), you're going to get the security keys to abuse privacy (the alternative is for EU governments to entirely shut down all domestic espionage, that will never happen).

beaconstudios 8y ago

Why does the media seem to be publishing so many "well duh" tier articles about various companies at the moment? "Big data company collects big data", "company known for monetising private data collects private data", "political consulting company profiles voters and tries to affect vote". No shit?

Harvey-Specter 8y ago
There are obviously people out there for whom this is new information.
empath75 8y ago
I know this might be a surprise to you, but not everyone knows everything that you do.
- beaconstudios 8y ago
  I'm not exactly tapping my own research here. Palantir is a big data/spying company, what do you expect from them? Facebook asks you for lots of private data and it's clearly obvious that when you post data on Facebook, Facebook gets that data. So what's the shock that they hold this private data? The only surprising aspect of the current Facebook drama is that they apparently mine your phone for additional data but that's not the main focus of the outcry. Why are people surprised that Cambridge Analytica, a political consultancy, did political research and affected a voter outreach strategy? None of this requires any real background knowledge, and yet the outcry seems extremely disproportionate to the revelations at hand. It seems like the Snowden reports all over again, where the world was up in arms that an intelligence agency was collecting intelligence.
  - xg15 8y ago
    > Palantir is a big data/spying company, what do you expect from them? How can expect anything of them when I don't even know that company exists, let alone what it does? > Why are people surprised that Cambridge Analytica, a political consultancy, did political research and affected a voter outreach strategy? None of this requires any real background knowledge, and yet the outcry seems extremely disproportionate to the revelations at hand. It seems like the Snowden reports all over again, where the world was up in arms that an intelligence agency was collecting intelligence. This is simplifying the discussion a lot. The general assumption about both intelligence and research was probably that they stick to rules and are concerned with their own fields. I'd guess the average joe would (before snowden at least) think that an intelligence agency is concerned with foreign diplomats and military strategists and a political consultancy does opinion polls. It's something different if said agencies collect intimate data of random citizens without their consent. At least in public mind, that was something the stasi or similar organisations did - but certainly not their own side.

eptcyka 8y ago

Whilst the article may not have any surprising information for the regular hacker news reader, it may come as a surprise that a non-insurance, non-government business holds more private data about an individual than Facebook does, especially since most individuals will not recall ever interacting with said company. I think it is beneficial if more people become aware of this.

jonnybgood 8y ago
> it may come as a surprise that a non-insurance, non-government business holds more private data about an individual than Facebook does Where in the article is that stated?
- drcongo 8y ago
  In the title?
  - jonnybgood 8y ago
    It’s clickbait. It doesn’t state it or give a citation for it anywhere in the actual article.
    - lgl 8y ago
      I don't think it's directly stated in the article. But with a little common sense we can probably safely assume that a company that is designed specifically to analyze huge amounts of data for law enforcement agencies, governments and 3 letter agencies will probably have a lot more data than Facebook. In fact, we can probably also assume that people's Facebook data has already been inputted in Palantir's system as just another data source. Sure that's a lot of assumptions and probabilities, but on the other hand: Peter Thiel.
    - drcongo 8y ago
      Yeah, but you edited your post to specify "in the article" after I'd posted. At the point of my reply, it was correct.

jondiggsit 8y ago

I’ve always thought - I don’t care if the NSA is spying on US citizens because their job is national security. The problem is when those tools trickle down. All of a sudden you have a local cop with too much time on their hands spying on you. But if you’re not doing anything wrong, you shouldn’t have anything to hide, right? Isn’t that how the saying goes?

EGreg 8y ago
Counterpoint: http://magarshak.com/blog/?p=169 http://magarshak.com/blog/?p=169
dTal 8y ago
Must be a nice way to live, having full faith that your government will always do the right thing. Relaxing.
- nerdponx 8y ago
  It's more a matter of focusing the discussion. Once you bring government into it, it makes the conversation a lot more complicated. This is not a complicated issue.
  - pfortuny 8y ago
    But it is the Government that complicates the thing: USSR, China, Fascist regimes: they persecute OPINIONS not deeds.
- mtgx 8y ago
  We literally have thousands of years of proof that governments always abuse their power when they think someone is their "enemy". So it's strange that people default to "trusting their government." I blame this on America's blue-team/red-team political thinking, which forces one to stick to a party no matter what. The standard for "enemy" could also be as low as they choose it to be, which is also weird that people think the government wouldn't go after them because they are "innocent", as if these people get to decide who is innocent and who isn't. No, it's the government that does that. Courts may have the final say in the end (unless you're indefinitely detained for multiple years first), but until then the government can do a lot of damage to your life if it comes after you. Even putting you through years of lawsuits, stress and money spent on lawyers would be bad enough. You may win in the end, but it will be a Pyrrhic victory. And that's if you don't take their plea deal first when they scare you with multiple bogus charges that you think shouldn't even apply to you but you're too scared of the 50 years in prison sentence they're threatening you with.

dmitriid 8y ago

> As shown in the privacy breaches at Facebook and Cambridge Analytica, the pressure to monetize data at tech companies is ceaseless Says Bloomberg, and includes Facebook code on their pages

ckastner 8y ago

> Aided by as many as 120 “forward-deployed engineers” This term popping up again made me curious. To my surprise, that's not just some overloaded nickname (what's cooler than styling yourself with a military or martial arts term, right?), that's what they are actually called: https://www.palantir.com/careers/teams/forward-deployed-software-engineers https://www.palantir.com/careers/teams/forward-deployed-soft...

OldFatCactus 8y ago
they fear the words "consultant" or "contractor" I think
- mcguire 8y ago
  Forward deployment had a much hipper, militaristic sound.
- ng12 8y ago
  Yup. And FDE is just a consultant, an FDSE is a software engineer attached to a specific client. A regular SE works only on generic products.
ataturk 8y ago
Because they wish to appeal to the spirit of adventure these people have. They're washed up mercenaries of the state who desperately want to feel relevant one more time. I wish people would stop helping the war machine. Stay the hell home for once.
jakelazaroff 8y ago
What's the difference between a normal software engineer and a "forward-deployed" software engineer, though? It sounds to me like that's just their military-flavored version of "rockstar developer" or "code ninja".
- beaconstudios 8y ago
  my understanding is forward deployed engineers set up shop in Palantir's clients' offices and implement the solution using Palantir's tech. Basically doing rollout and front-of-house tech support.

amelius 8y ago

Why isn't there a privacy watchdog that constantly monitors these kind of companies? I also expect this watchdog to test for data leakage from one site/company to another (e.g. by taking on the role of customer). They should do this on a frequent basis. Without this, companies seem to be free to do whatever they want.

justherefortart 8y ago
Hahahaha This is how the fucking government gets around the Bill of Rights, primarily the 4th. When it's outsourced, you have no rights. Parallel construction needs to be the next addition to the 4th or it's own amendment.

common_ 8y ago

Pop Quiz: Which two companies had their first office at 101 University Avenue in Palo Alto? The answer says quite a lot, I think.

drcongo 8y ago
I've found Piazza's Fine Foods Inc and PhysioTouch Inc. Not sure what I'm supposed to take away from this answer though.
- gberger 8y ago
  Facebook
deleted 8y ago
[deleted]

ataturk 8y ago

My biggest issue with former military people is that their skills don't easily translate into any other fields and that their mindset is usually heavily canted towards statism, which is a big problem when you combine the kinds of surveillance technology we have with needing to turn a profit. It makes anything East Germany did look like they were underachievers.

diskandar27 8y ago

this is kind of scary and sad when powerful technology is being misused ...probably it is better to live in China ??? ..the government openly monitoring its citizen instead of spying on them

pmlnr 8y ago
Definitely not, although that specific aspect - you _know_ you're being watched - makes life different, in some way, simpler. The rules are obvious, however, if they change, you might find yourself in a much worse situation, than in the West - for now.
- gnode 8y ago
  > you _know_ you're being watched - makes life different, in some way, simpler. Arguably, what happens when people are aware of surveillance is one of the worst parts of it. Following the Snowden revelations, there were noticeable chilling effects: https://www.washingtonpost.com/news/wonk/wp/2016/04/27/new-study-snowdens-disclosures-about-nsa-spying-had-a-scary-effect-on-free-speech/?utm_term=.824c5af75415 https://www.washingtonpost.com/news/wonk/wp/2016/04/27/new-s...
  - deleted 8y ago
    [deleted]
kirubakaran 8y ago
Reminds me of a friend who said he never wears a condom, because condoms can break.

opportune 8y ago

I'm much less concerned with Palantir being used internally by corporations to detect information leaks (although in practice this also means it will be used to quash whistleblowers), since that's at least a viable use for their technology. And it's good that Palantir is pivoting towards less invasive applications of their technology such as supply chain tracking. But the article touches on the biggest issue with Palantir, which is that they seem to be completely fine with allowing law enforcement agencies to use their software for parallel construction, which is in my opinion a pretty egregious violation of the 4th amendment. Although it may be constitutional for police departments to purchase data from e.g. Facebook or some data broker, even local police departments are known to employ dragnet-style surveillance through things like Stingray devices. Of course we have a number of three-letter agencies, also Palantir customers, who don't even need to pretend that they're following the 4th amendment. Since Palantir is essentially enabling the construction of all the tools required for a complete surveillance state, and actively using their tools for that purpose for financial gain, I think that makes them one of the least ethical companies in tech. I certainly wouldn't be able to live with myself if I knew I was writing software with the explicit purpose of being able to track down rebels in Yemen, catch illegal immigrants, and violate the constitutional rights of millions of Americans.

jonnybgood 8y ago
Can you expand on exactly how Palantir is used for parellel construction?
- taneq 8y ago
  They dig up data telling the police where to look. The police then look in just the right place and "legitimately find" the evidence that they would never have stumbled on and couldn't have legally known was there.
- opportune 8y ago
  Law enforcement uses dragnet warrantless surveillance to gather data (such as who is calling whom, who is in physical proximity with whom). That data is loaded into Palantir's software. Law enforcement uses that data to acquire admissible evidence of the perpetrators' crimes. This is the crucial step, because although the original evidence couldn't be admissible in court due to its collection without a warrant, it can be used to acquire more evidence. If the warrantless surveillance includes information such as the time and place of where you plan to commit a crime, law enforcement can stop you for a random check on the way there, giving them actual evidence for prosecution. Now that evidence can be used in court, even though the only way law enforcement were able to acquire it was through unconstitutional practices. The thing about Palantir is that it needs big data to work. Police departments don't just have random chunks of data lying around containing large numbers of connections between large numbers of people - they might have something like that for something like suspected criminal gangs, but that wouldn't be enough data to make Palantir's software worthwhile. So acquiring the data necessary to use Palantir's software requires mass data collection. Some of their mass data collection techniques may not require a warrant, such as using license plate captures, but some does, such as who you're calling. So does Palantir perform warrantless surveillance themselves? To my knowledge, no. But when they sell their software to law enforcement agencies, they must know that the only way their customer could get bang for their buck is through mass surveillance. In my opinion, the person willingly and knowingly selling tools for oppressive purposes holds a lot of blame for the actual oppression that occurs as a result.

klunger 8y ago

Culture flows from the top. Peter Thiel apparently views women's suffrage and poor people as "tough for libertarians [like him]". Can we talk about, well, how problematic it is that a company that has information on just about everyone (and all the power this yields) is lead by someone like this? > As Thiel’s wealth has grown, he’s gotten more strident. In a 2009 essay for the Cato Institute, he railed against taxes, government, women, poor people, and society’s acquiescence to the inevitability of death. (Thiel doesn’t accept death as inexorable.) He wrote that he’d reached some radical conclusions: “Most importantly, I no longer believe that freedom and democracy are compatible.” The 1920s was the last time one could feel “genuinely optimistic” about American democracy, he said; since then, “the vast increase in welfare beneficiaries and the extension of the franchise to women—two constituencies that are notoriously tough for libertarians—have rendered the notion of ‘capitalist democracy’ into an oxymoron.”

opportune 8y ago
"I no longer believe that freedom and democracy are compatible." So it seems like Thiel's been a fascist for quite a while. And he's working on surveillance technology with law enforcement agencies... You're absolutely right that Thiel is probably the worst person, for society, to be running a business like this.I can't believe YCombinator partnered with him after he said things like this, and managed an entire two years with him until they dumped him over Trump. Look, I'm not saying that libertarianism is fascism. But what kind of libertarianism is non-democratic? It would either be authoritarian capitalism (fascism), or anarcho-capitalism.
- exelius 8y ago
  Values will always be secondary to money in this world.
  - chillingeffect 8y ago
    1. That's a gloomy self-fulfilling prophecy. Try thinking more constructively. 2. Money is a value, so your assertion is a paradox.

occamrazor 8y ago

And they sell GPDR compliance solutions: https://www.palantir.com/solutions/gdpr/ https://www.palantir.com/solutions/gdpr/ Oh, the irony!

prepend 8y ago
Is it though? It’s pretty hard for companies with lots of diverse data on their users to comply. Finding it all and deleting it is pretty hard. The hope I had was that companies wouldn’t track it at all. I fear that only companies that afford stuff like Palantir will collect it, because they can comply and show evidence of complying.
djabatt 8y ago
There is a lot of irony in capitalism

prepend 8y ago

This isn’t Palantir, this is the companies who buy Palantir software. The data from JPMorganChase isn’t visibile to Palantir as an organization, but to the bank. This headline is weird and as accurate as “Microsoft knows everything about you” because the bank uses SQL Server. Or “the Python developers” because python and pandas is used to link a bunch of data together. Since this is Bloomberg, I would expect them to know the difference between software and data services. Google gets your data when you use them (except for enterprise), most other products don’t. Cynically, this seems like oppo PR against Thiel. How could you prove this suspicion?

jawns 8y ago
Is it really the case that data does not cross boundaries among Palantir clients? And that the algorithms used by any given client are only informed by the data that client provides?
- creaghpatr 8y ago
  Unless proven otherwise, then that is hearsay.
  - prepend 8y ago
    Good point, but this is the risk for every cloud hosted software. So every AWS and azure product has this suspicion that’s impossible or extremely difficult to remove. If you’re really paranoid, then all non-OSS software. So again, why is Palantir more worrisome than Salesforce?
- mcguire 8y ago
  If so, then Palantir is missing a big revenue source: integration with public data that each client would otherwise have to collect on its own.
- ryan-c 8y ago
  When I worked there (mid 2009 to early 2014), almost all customers had dedicated installations (as in a rack of servers) with just their data on-site. Even the hosted instances had siloed data. There are features for peering, but AFAIK that has to be explicitly set up between installations.

ideophobia 8y ago

I used to be intelligence analyst that utilized Palantir on a daily basis. Ask me anything.

chillingeffect 8y ago
What kind of search filtering options are available for fishing expeditions?
- ideophobia 8y ago
  You can search on or around nearly anything. That's one of their selling points to customers. The search function is multi-faceted, in that 1) it can index the data of any files you upload to Palantir (making everything text searchable), 2) Analysts can indicate specific attributes of an entity just by highlighting text in a file and marking it as a DOB or SSN or whatever. 3) The # of attributes is nearly limitless. One of the primary screens in Palantir is a simple search page designed to work (and look) like Google, to search upon your entire database however you'd like. You can enhance the search function via the various attributes built into your installation or using advanced search tools. So I guess a "fishing expedition" in the traditional sense is very easy, as it's limited by the data you have, not the system itself.
  - zrobotics 8y ago
    Are there options to restrict search space (i.e. limit data sources that could potentially contaminate an investigation)? And if so, is that a common practice, or is it more typical to search through all avaliable data for anything potentially useful and weed out later?
    - ideophobia 8y ago
      I don't specifically remember an option to weed out searching, but I'm reasonably certain there were advanced options that included "does not equal" or "does not contain" type boolean searching. From my experience though, it wasn't common practice to weed out from the start, but rather to start with a general search and slowly narrow it down to the targeted data. A lot of data sources weren't automatically imported, they were more like extended queries. So if I wanted employment records, for example, I had to initiate that as a custom search to query that external system, and pull back the data for the specific individual I was investigating, it was not available in the database by default. But once it was imported and tagged, it was available in any future searches for the remainder of the data retention period.

EGreg 8y ago

Let’s suppose you have an actual solution, like a free open source platform that is used by millions of people and took you 7 years and half a million dollars to build, because you saw the problems coming back in 2011? A platform with 5 million users in 110 countries? It seems the press of today won’t really care about solutions, but only highlighting a scandal. Am I wrong? What would one do to get them to write a story about actual solutions like solid.mit.edu or ours: https://m.youtube.com/watch?v=pZ1O_gmPneI https://m.youtube.com/watch?v=pZ1O_gmPneI Press kit? News conference? Some kinda weird publicity stunt? Serious question.

binarymax 8y ago

This isn't surprising. In fact it was expected. This company was born out of the Total Information Awareness mandate by GWB shortly after 9/11. Total Information Awareness means monitoring of everyone and everything. They got contracts and funding and grew to the original vision set in motion 15 years ago.

2close4comfort 8y ago

Wouldn't be interesting if all the Facebook data being collected is really to feed info to Palantir...with a layer of deniability and they get data on billions.

dangerlibrary 8y ago

> The U.S. Department of Health and Human Services uses Palantir to detect Medicare fraud. Having worked in this field, I can say that this is hilarious overkill. The vast majority (in dollar terms) of Medicare/Medicaid fraud is drug companies misreporting their Average Market Price (AMP), the critical variable in how much they get paid. In 2012, Glaxo settled(!) a small bundle of cases, some of which I worked on, for three billion dollars [0]. You don't need police-state surveillance of individuals, you need visibility into company transaction records. [0] https://www.justice.gov/opa/pr/glaxosmithkline-plead-guilty-and-pay-3-billion-resolve-fraud-allegations-and-failure-report https://www.justice.gov/opa/pr/glaxosmithkline-plead-guilty-...

refurb 8y ago
Medicare fraud goes well beyond price reporting by manufacturers. Physicians can bill for more expensive procedures, bill phantom patients, etc. That's a huge part of the multi billion dollar fraud that Medicare has. I assume being able to identify physicians will outlier billing patterns would be helpful in figuring out where to look closer.
- dangerlibrary 8y ago
  I mean, if you want to get into it, you're both right and wrong. The second largest cases are typically against hospital chains that encourage (read: basically force) their coding teams to use particular billing codes. It's easy enough to model this from claims data (and we did). When a chain buys a new hospital, suddenly patients that present with the same diagnosis codes are all getting extensive exams that ostensibly require 45+ minutes of a physician's time and cost 4-10x more. Again - the large operators (hospital chains) absolutely dwarf individuals (doctors). DOJ barely has the resources to prosecute the largest offenders, and have faced repeated budget cuts. Wasting time on individual doctors or private practices (beyond maybe making examples of a few egregious actors) would be an inefficient allocation of their resources.
  - ng12 8y ago
    Do you know the details of Palantir's investigation? How do you know they're not doing exactly what you just described? Palantir is just a data analysis company, they're not the NSA.

everdev 8y ago

This was a huge debate when the Patriot act was passed, that US citizens would get caught up in surveillance. It seems common in history that during war time, a country aggressively pursues it's own population for spies or sympathizers for the enemy. It send also common that the country goes overboard so as not to let anyone through the cracks and a lot of innocent people get swept up in the process.

ajeet_dhaliwal 8y ago

My main takeaway is that the corporations don't have the funds to waste on this and don't see the ROI (given how many large companies listed in the article stopped using Palantir) whereas the government does. Peter Thiel being a libertarian is clearly aware of the scale of government wastage and decides to go in for the win.

hapnin 8y ago

No surprise. Thiel is a grade-A asshole who appears to be amoral.

thestephen 8y ago

> The LAPD uses Palantir’s Gotham product for Operation Laser, a program to identify and deter people likely to commit crimes. Information from rap sheets, parole reports, police interviews, and other sources is fed into the system to generate a list of people the department defines as chronic offenders [...] The list is distributed to patrolmen, with orders to monitor and stop the pre-crime suspects as often as possible, using excuses such as jaywalking or fix-it tickets. This is pretty much everything I was afraid of when it comes to this field - big data "prediction" being used to predispose people into crime, and by extension, algorithmic racism.

_salmon 8y ago
Minority Report. Literally.
zo1 8y ago
For all we know, keeping a closer eye on previous criminals deters them from repeating their crimes. This strikes me as a very benign way of preventing a crime from happening in the first place. Rather than letting it happen, creating a victim and perpetrator, and then having to punish the latter yet again. At the very least, this should give them the impression that policing is much higher than it actually is. Sounds like a good deterrent to me.
- archerface 8y ago
  There is a lot of problems with this strategy when introduced into the real world; the chaotic vortex that it is. The data can be wrong. This could be easily fixed if law enforcement agencies had to actually verify that the data was correct, but in practice this seems to rarely occur. When people are told by a group that they are a thing, repeatedly, they eventually accept that they are that thing. If the cops keep harassing you for being a gangbanger when you have never been one your whole, you just happen to be associated with a few individuals who are, you eventually just move towards that group. Why not? The powers that be already think you are one and they treat you like one, which causes other people to treat you like one, so why not just be one? You also need to consider the idea of enforcing crime that hasn't happened yet. That defeats the whole point of law enforcement. It is meant as a reactionary force, not an offensive force. Policing someone who has the potential for crime assumes that they will do it again, which makes it so your essentially treating them as guilty for something they haven't done yet. From my perspective, that seems to defeat the purpose of our legal system. Anyway, I think I will end my rant here...

clavalle 8y ago

I think I heard the maxim once "If it can be abused, it will be abused." We need to start getting ahead of worst-case scenarios and to hell with people that think that approach is paranoid.

sevensor 8y ago

After September 11, thre was a big push for "Visual Analytics," which is basically rebranded statistics plus visual data mining tools. There's a report called Illuminating the Path that lays out the research agenda. I read it in grad school when my advisor was trying to tag everything under the sun as Visual Analytics in an attempt to get access to the Visual Analytics money pot. (We didn't.) It's creepy to see it come back around, fully realized by Palantir.

lerie82 8y ago

From someone who builds software like this all the time, I am tired of seeing these "privacy" issues people are crying about. People want to get wifi off a FREE wifi connection and then bitch about their data being leaked, even though they ticked the box that said the wifi owner could do whatever he wanted with the data collected over his own network. People have to learn to be responsible for their own data, use their own services and secure their own technologies and stop blaming everyone else for data they out online or ALLOW to be put online but storing it on a device that has access to the internet, we're talking mobile here. I see people bitching about Facebook when it comes to privacy, but Facebook isn't about privacy, its exactly the opposite, its "social media" where you "share" things, not hide them. Palantir has nice solutions that not only gather data but uses sophisticated techniques to tie the data together. If you think your constitutional rights have been violated then you need to speak up, and not on yc, and not on some nameless forum, not on your blog and not on social media. You need to take your ass down and file a lawsuit, otherwise, just sit there quietly while the big boys secure your country.

return1 8y ago

i find the peter thiel conspiracy-theoretic spiderweb in the article ridiculous (i believe you could make a similar one for any well known vc). I wonder however why he invests in these companies. i mean centralizing intelligence seems a very anti-libertarian thing to do.

amelius 8y ago

How can we build a case against them, and sue them? EFF?

avs733 8y ago

Ignoring the enormous argument over whether what I am about to say is legal in the US/America's particularly constructed law... Is it time to consider whether companies who's primary business is the collection, summation, synthesis, and re purposing of information should be defacto illegal? Its not that having the government do this is meaningfully better, but I'm convinced that is a better overall path. The public seem more afraid and more willing to engage in a more serious discussion over ethics. At the very least, it would enable the public to have more oversight over how such data is collected, stored, embargoed (or not), and used. The level of abstraction and fuzzing that is allowed by allowing a non-governmental entity like Palantir or Facebook to have (whether as a collector or aggregator) the data and governments be only a user seems too difficult to regulate and even have a transparent discussion about because claims of national security and secrecy are nested within claims of trade secret and intellectual property in ways that create a moat around effective oversight. So seriously...if we made this business model illegal, what do we lose? What exceptions would need to exist?

ng12 8y ago
> Is it time to consider whether companies who's primary business is the collection, summation, synthesis, and re purposing of information should be defacto illegal? You mean IBM/SAP/Oracle?
- avs733 8y ago
  I don't think it is unreasonable within what I said to differentiate the industry that supports data collection and archiving as a broad process (e.g., those you describe) with those that seek and intent specifically to collect and operationalize data about people without fully informed consent.
  - ng12 8y ago
    Palantir does not do data collection in that sense. They're much closer to IBM than you realize.
    - 8y ago

otakucode 8y ago

Does anyone else remember a time when paranoia was considered a mental illness rather than a virtue? Or am I imagining that?

flyingcircus3 8y ago

When a company is named after the crystal ball an evil wizard uses to spy on his enemies in The Lord of The Rings, and that company's founder was the first large investor in Facebook, and that same founder is able to fund company ending lawsuits against media outlets, headlines like this are akin to "water is wet"

shmerl 8y ago

So it's a double play on the name now. I.e. it's in the hands of Sauron?

snowwrestler 8y ago

> Cavicchia was in charge of forensic investigations at the bank. Through Palantir, he gained administrative access to a full range of corporate security databases that had previously required separate authorizations and a specific business justification to use. He had unprecedented access to everything, all at once, all the time, on one analytic platform. He was a one-man National Security Agency, surrounded by the Palantir engineers, each one costing the bank as much as $3,000 a day. I've long thought that the "secret sauce" of Palantir is just that signing a Palantir contract gives executives a reason and excuse to pull together data that had never before been indexed against each other. I mean, if you take any old dumb BI platform and give it access to data that people have not previously connected, you're going to get new insights. Someone once described Palantir as a data mapping consultancy that marketed itself as a software platform.

anigbrowl 8y ago

Sort of off-topic, but I have a need for a graph/relationship oriented database like this (on a relatively tiny scale) and am having a hard time choosing between the many alternatives. Open source is a must, but otherwise I'm primarily interested in ease of use and visualization compatibility - I want to focus on my problem domain, rather than writing a lot of code or hacking for its own sake. Right now looking at Neo4j and GraphDB, would welcome other suggestions. Thanks in advance!

mdekkers 8y ago

"the vast increase in welfare beneficiaries and the extension of the franchise to women [...] have rendered the notion of ‘capitalist democracy’ into an oxymoron." That guy is a nice piece of work.

billhendricksjr 8y ago

Given Thiel’s purported views on privacy, the hypocrisy is overwhelming.