10 ms·
The AI Trust Crisis
- bertil 3y agoThis article touches on the more critical and flip side of a significant problem, which I think will hurt many of the incumbents, while whoever figures out how to resolve it will gain a lot of credit. Facebook doesn’t explain how their ad targeting works at the individual level, but they thought about it and tried—barely. It was perceived as an obstacle to black-box ML, even when there were clear opportunities to show customers (and partners): here’s how the algebra works, here are key dimensions that we have identified explicitly and that the advertisers wanted us to target (age, gender), here are more dimensions and the feedback from showing that ads to other tells us (people who like board games respond more to the ad, and we think you do). That training, in clear, accessible language that anyone would understand exists: it’s part of onboarding. The result of not having it is a large chunk of ads are shown to people who literally don’t speak that language (something Antonio GM mentions in his book and that was identified, corrected, broken, and re-identified at least three separate times since). But even benefits would be enormous: it would free what looks like 60% of my ad inventory by not showing gambling ads. But they went the other way, treating users asking questions that their own employees ask on their first days with suspicion—assuming they knew better than the people they are talking about. OpenAI is less adversarial, but they must explain how training and testing work with user-relevant examples: “Here’s a conversation you had last week. Here’s the feedback you gave. Here’s how we detected it was relevant to refinement. Here are examples of questions whose answers were changed by your contribution.” Having a full-text search on their training set might be difficult, but something along those lines could be implemented and reviewed by a neutral third party. Those conversations would lead to insights about where to find more relevant data. As Simon W. writes, this matters, and you want to respect privacy because it’s an inherent good, but you also want to preserve trust because it’s commercially essential (even though people are quick to forget if they get free stuff). I think you should do it because security by obscurity, or at least the ML equivalent of that, isn’t working. https://xkcd.com/1838/ https://xkcd.com/1838/ is funny, but the truth is that we can offer some tools for exploration and feature breakdown. Every time I did, we learned so much from those.
- andy99 3y agoThis isn't really AI specific. The AI angle I suppose is that there is a new way for companies to exploit data they get from you, but like the article says with Facebook, people already believe their info is being exploited. Laws won't change it (not that they shouldn't exist) there's always too much fine print that can be gamed and they'd never be enforced to the benefit of the consumer anyway. Like privacy, it's awareness and not giving companies the chance, which will take a long time but hopefully will be the default instead of just not caring. For AI, local or own-cloud models are a good start, it's actually hard to belive a company like dropbox would go with OpenAI and not see how bad it looks.
- skepticATX 3y ago> The key issue here is that people are worried that their private files on Dropbox are being passed to OpenAI to use as training data for their models This is only part of it. I don’t want my data being sent anywhere unless I authorize it, regardless of what it’s being used for. In this case, we not only have to worry about OpenAI training on our files (I have no reason to doubt that they are truthful when they say they won’t), but we also have to trust that they can securely handle our files.
- simonw 3y agoThat's true. It's very reasonable to worry that OpenAI's "we only hold on to your data for 30 days for auditing purposes" policy means there's 30 days in which they might have a breach that leaks your data. Especially since they've had a few documented security problems in the past.
- kolinko 3y agoThe same argument could apply to processing data in the cloud, and somehow nobody complains anout the fact that Dropbox stores their data on AWS/Google cloud or wherever
- JohnFen 3y agoI complain about my data being stored in the cloud all the time. I won't use services like Dropbox in large part because of the cloud, but I can't do a thing about other entities I do business with storing the data they glean about me in the cloud. It's a real issue.
- sneak 3y agoIt’s SaaS vendors all the way down. If you don’t want third (or second) parties reading your data, make sure it is e2e encrypted clientside. This means no Dropbox, but Syncthing instead. That means no Slack or Discord, but Signal.
- the_mitsuhiko 3y agoThat is not entirely accurate. Thanks to the GDPR subprocessors need to be disclosed and changes to them too. As a result plenty of companies are now much more selective about how many they are willing to entertain.
- RcouF1uZ4gsC 3y agoI think local models will be more and more relevant. First of all, the locally available computing power for inference is going up. In addition, inference is much cheaper than training. Finally, there are diminishing returns with more and more parameters. Local models also solve a lot of the trust issues
- 127361 3y agoYes and if we start chatting to a local AI model instead of searching on Google, that means we get an overall increase in privacy. So the new local AI era could actually be much more private than the previous Web era - an era in which we could have much more freedom.
- rqtwteye 3y agoJust read this. I have had enough weird incidents where you for example chat with somebody about a topic and a day later I suddenly received ads Youtube recommendations about this topic. There is a lot of reason to not believe big companies. I don't see why Facebook should be more ethical than tobacco or oil companies that have been lying and obfuscating for decades. The only solution I see is to make it illegal to use personal data. Especially make data brokers illegal. When you give a company access to your data, you often agree that they will pass the data to third parties who then pass it on and suddenly everybody has your data because you have permission to use your data.
- passion__desire 3y agoIf that is the case, why aren't developers who are implementing such pipelines coming forward. This was the case for 9/11 being an inside job because many workers are needed for such an inside job. A whistleblower like Tristan Harris could come forward and accept that facebook is doing such stuff.
- kolinko 3y agoAs for using personal data - EU has it covered. But the whole „facebook listening to you” os absurd - on iOS you would see a system notification that the microphone is active, I assume on android as well. Also, it would take crazy amounts of bandwidth/battery/processing power to pull off. It would also be trivially discoverable - a ton of people are listening to what apps are sending out, and even if encrypted, it would be noticeable by the outgoing traffic volume being high. It is virtually impossible to pull off even today, as anyone who ever developed anything on mobile can tell you
- 127361 3y agoLikely a cognitive bias there, we pay selective attention to the topics we've been discussing previously, and likely remember when an ad comes up that's related. So yes, they're not listening to our microphones. It's just our selective attention and memory, instead.
- 3y ago
- karaterobot 3y ago> I took that screenshot on my own account. It’s toggled “on”—but I never turned it on myself. There is a consent crisis as well, though I agree it's a smaller issue related to trust. There needs to be an actionable, legal definition of consent as it applies to website privacy—my naive assumption is that there was, but clearly that's not true, or it's not good enough or actionable enough—and it needs to preclude implying users must positively grant consent to harvest, process, or transfer data to third parties, when in fact the dirty deeds have already been done in secret.
- jampekka 3y agoIn GDPR there is. But the companies don't care and neither do the regulators. Big business just has too much power and influence.
- BiteCode_dev 3y agoGDPR is not much enforced, unfortunatly.
- ponector 3y agoBut it is enforced. Here is a list: https://www.enforcementtracker.com/ https://www.enforcementtracker.com/
- jampekka 3y agoVery few of those are about consent. Which is quite striking, as practically all of the spyware nags are illegal and thus the consent is violated probably billions of times every day.
- BiteCode_dev 3y agoGiven that I encounter at least one violation a day, it's really a small list. It should contains hundred of thousands of entries. Like fines for speeding. Imagine if you had a page for parking tickets since 2018 (!) and it contained only a few hundred penalties. This is ridiculous, especially since: - it's not hard to find most cases. Just take any cookie banner that makes harder to opt out than opt in, and that's it, you have a violation. It can even be automated. - it can bring a lot of money to a system that is dying for it.
- dsr_ 3y agoThe cost/benefit of believing an infotech company about privacy concerns not backed by an explicit contract is firmly on the side of "don't bother believing them". Restore the trust? Pay us.
- JohnFen 3y agoFor me, money is not the issue and being paid for having my trust abused will not restore my trust.
- 23B1 3y agoThe problem with AI isn't (p)doom it's the 99% probability that the people at companies like Dropbox will continue to treat customers like human chattel. They'd enslave you and charge you a subscription for oxygen if they could – and insist it was a feature.
- fnimick 3y agoAny for-profit company would. In fact, if they are public, they are legally required to maximize profit without any other consideration.
- sgift 3y agoNo, they don't. That myth cannot die fast enough. https://insight.kellogg.northwestern.edu/article/shareholder-value-purpose-corporation https://insight.kellogg.northwestern.edu/article/shareholder... > There are a lot of misconceptions about maximizing shareholder value, even among economists. But talk to a legal scholar or a corporate lawyer: a CEO or board is not legally obliged to maximize shareholder value. They need to maximize the value of the corporation and act in its best interest. Only when there is a change in legal control, such as a merger or imminent hostile takeover, do they have to maximize shareholder value.
- mikewarot 3y ago> people at companies like Dropbox will continue to treat customers like human chattel. The key question is: Who is the actual customer? That drives everything else. The FAANG's customer isn't the "users", but rather its advertisers, et al. Anyone who will pay for the data their "users" generate. This is the Faustian bargain we all made when we decided that free stuff from the internet was a good idea. Someone else pays, and they get our data in exchange for that payment. The same data is likely sold as many times as they can do so, of course.
- sgift 3y agoEven if we accept that we made this bargain: I pay for Dropbox. If they want to use my data for their shit they better make it a) optional and b) make me pay less. Also, I have a feeling this could be another case of the EU having to fine an US company, which means half of HN will cry again how the EU is always so mean to US companies. Or maybe this time HN acknowledges that the reason could be that the companies trying to use private data without permission are often US companies.
- adaboese 3y agoI am building an AI content generator and my edge that makes it better is sourcing data from not easily accessible data sources (think research papers) through SEO. I cannot imagine that these companies are not using every bit of confidential data for their advantage. Proprietary data access is the only thing that matters in the future of AI wars
- espe 3y agotrue. i call them "prompt stimuli". prompts can give structure and context and whatnot but a dash of original domain gives the edge.
- tikkun 3y agoYes. In general, it would be great if companies threw away the outdated PR-communication playbook, and instead took a leaf from some indie devs and had the builders directly communicate with users as people. Products are built by people, companies are made up of people, but most communication is instead corporate. In practice, I think the PR playbook is too entrenched, but we can dream.
- Closi 3y agoThe problem is that large companies are made of lots of people, and some of those people might have awful opinions or say things that will get your business in trouble. The larger you are and the more people that are communicating, the bigger the chance that someone says something terrible. Particularly if you are a large public company and there may be legal requirements around specific parts of disclosure. Besides, companies don't want their secret new feature announced before their competitors know about it, in a way that is poorly communicated and confuses customers, and then find that the person who announced it has a profile picture which is them at a far-right protest / far-left protest / made some sexist jokes / has a bio advertising their onlyfans account etc or a million other things that could be seen as against the corporate image.
- airstrike 3y agoYou can only truly trust someone when your incentives are aligned, and people's incentives aren't aligned with OpenAI, Facebook or Dropbox. That trust simply can't be earned.
- notfed 3y ago> ...people are worried that their private files on Dropbox... I'm not worried, because I use Cryptomator. Great app, acts as a file encryption layer on top of any cloud storage.
- thomastjeffery 3y ago> I don’t think so: I think this is a combination of confusing wording and the eternal vagueness of what the term “consent” means in a world where everyone agrees to the terms and conditions of everything without reading them. Bullshit. Consent means consent. There is no confusion here. Nothing is vague. This is explicit fraud. I feel like every day now, I'm reading an article where the problem is obviously just fraud. Fraud has always been illegal. When did we stop prosecuting it? Why aren't we talking about that every day?
- samsolomon 3y agoIf you're not a subscriber, Matt Levine has a running theme of articles where "Everything is securities fraud." It's not free to read his articles on Bloomberg, but his newsletter, Money Stuff, is free and one of my favorite daily reads. https://www.bloomberg.com/account/newsletters/money-stuff https://www.bloomberg.com/account/newsletters/money-stuff
- ToucanLoucan 3y agoThe entire basis of the current generation of AI is in stolen materials. Stolen writing, stolen art, stolen music, all of it taken because it was "publicly available" meaning "there was nothing in existing law that said we couldn't take it for training a learning model." Now they've done the same with Dropbox contents. AI is very cool technology. The incredible overstepping of any and all ethics with regard to getting training data, just all of the data from any source as fast as possible right now, in this mad dash to create it at all costs is not and should mandate a complete restart on behalf of the people behind it. For this one, and for so many other ethical lapses on the part of OpenAI, the models as they exist are tainted beyond any ethical use as far as I'm concerned. If your product uses this stuff, you are not getting one red cent from me for it. Period, paragraph.
- simonw 3y ago> Now they've done the same with Dropbox contents. This is what I mean by the "trust crisis". Dropbox very clearly deny that Dropbox content is being used to train AI models, by them or by OpenAI. You don't believe them, because you don't trust them.
- bambax 3y agoI certainly don't "trust" OpenAI or any other big company about what they say they did, or will do, or are doing. Yet I believe OpenAI isn't using data from Dropbox to train their models without users' consent. BUT I don't think that's the problem here. The problem is data in transit; data sent to third parties who can actually read it, and who may have rogue employees that Dropbox has no control over; data that can appear in logs or subject to different policies, etc. If I send private data to Dropbox they can't send it to anyone for any reason, including "improving" their product, without my explicit and informed consent. I'm not sure how this is even debatable. If Dropbox wants to house models and offer RAG search themselves, to consenting users, that's one thing. If Dropbox sends all data of all users to third parties without telling anyone before the fact, that's another thing. A terrible thing.
- schnable 3y agoExactly. It's all about who can see my sensitive data, period.
- lukeschlather 3y ago> If Dropbox wants to house models and offer RAG search themselves, to consenting users, that's one thing. Well, I'm a paying Dropbox customer, and I would not pay for this feature. I would like it if they encrypted my data in a way that made offering this sort of feature impossible. I do want my data recoverable, but the fact that they can offer this AI "feature" at all, it seems like they've made zero effort to prevent malicious internal employees or third parties from accessing my data.
- hinkley 3y agoIt’s like when forgot my password features send you back your current password instead of a new temporary one. Great, I’m back in my account, but now I dread even being here. What other stupid shit are you doing?
- Swizec 3y ago> If Dropbox sends all data of all users to third parties without telling anyone before the fact, that's another thing If they've ever signed a BAA (business associate agreement) with any enterprise customers using Dropbox for documents bound by HIPAA, this would get them into a lot of trouble very quickly. The financial penalties are _high and per exposure per employee involved_. They also hit employees doing directly (if you disclose/share HIPAA info then you personally are liable). So I'm certain that even if they did share documents with undisclosed 3rd parties without notice, it wouldn't be "all". Enterprise data is likely safe. Those contracts get heavy scrutiny before signing.
- Waterluvian 3y agoOther than just manually generating a UUID and storing it then doing searches for it, is there any sort of formalized service/application/library for "Air Tag but for your data"? Generate a unique string, store it along with my data, notify me if it shows up anywhere. I wonder if it's possible to generate a canary of that sort that can be detected in AI models.
- JohnFen 3y agoI think that the difficulty is not in doing this, the difficulty is in doing it in a way that an adversary can't strip out.
- throw7 3y ago"A key role of government is to prevent this from happening." Well, unfortunately, we have a government trust crisis also.
- bluish29 3y ago> think this is a combination of confusing wording and the eternal vagueness of what the term “consent” means in a world where everyone agrees to the terms and conditions of everything without reading I think this is always the case. Unless the companies are forced to actually simplify and limit their TOS -or even divide it by features-. No one is realistically expected to read hundred of pages daily in legalize. I don't think many vendors do it our of necessity but as a workaround consent issue. This needs to be addressed by legislation that ban those practices.
- powera 3y agoPeople are going to believe the worst about AI for the rest of the decade, for reasons entirely unrelated to what actually happens. Concluding "if we respect people's feelings and do one weird trick, people will suddenly trust AI" is naive beyond number.
- cwillu 3y agoIf I can be automatically signed up to allow openai to process (but not train on) my data, I absolutely don't trust that the deal will not be altered the moment any counter-party to the “agreement” (besides me) desires it be altered.
- liuliu 3y agoI've briefly touched it in another blog post. It is not just trusting the institutions to not use the data for training. Optimizations you apply to making LLM inference fast might present new security landscape that we don't know much about. For example, if your LLM inference engine stores KV cache across user sessions, you can in theory do a timing attack to retrieve what's in the KV cache, potentially leaking other users' requests.
- isx726552 3y ago> Facebook say they aren’t doing this. The risk to their reputation if they are caught in a lie is astronomical. The author refers to the theory that FB is always listening as “laughable”, but this statement is even more laughable. If FB actually got caught doing this, at most there might be a few articles about it, and maybe a class action lawsuit resulting in a few pennies distributed to individual users, and then things would go right back to business as usual. The only convincing argument that FB isn’t always listening is that it would be too expensive to be worth it. Arguments about “reputation” risk are beyond absurd. They don’t care.
- daft_pink 3y agoI think your article glosses over the fact that we have privacy concerns beyond training on my data. I'm a working professional and I have clients that are governed by confidentiality agreements and regulations regarding where information goes, and I would just prefer using a service where my data rests on a server instead of having more and more points for a data breach to be introduced. I don't really understand why my data isn't fully encrypted at all times and only I can view it in the first place, but the idea that they are actively sending it across the internet to be ingested by other companies and processed without my consent or interest is so terrible. I often use AI features when I opt into them, but having a company just sending my personal files all over the internet without my consent is insanity. Honestly, OneDrive has a migration tool and I got a trial for dropbox business and moved all my files automatically last night. It's just the last straw in their company constantly doing things I don't ask them to do like introducing crap and popups into my desktop interface and never offering the feature I constantly ask for... end to end encryption. If you want a couple click migration from Dropbox Business to an Office 365 Onedrive account, it's right here: https://learn.microsoft.com/en-us/sharepointmigration/mm-dropbox-overview https://learn.microsoft.com/en-us/sharepointmigration/mm-dro...
- satya71 3y agoIs OneDrive end-to-end encrypted? I think Microsoft will introduce a similar feature soon, if they haven’t already.
- daft_pink 3y agoNope, but it just makes sense as a quick and easy stopgap measure until I find time to figure out who to use, because it's easily compatible with my mac and pc, I already pay for a subscription so it saves money, it can run without putting all the files on every machine that I own, they aren't sending my info everywhere. If anyone has suggestions on a reliable end-to-end encrypted solution with a lightweight cross platform sync software that doesn't force you to download all the files to your device (my Mac's HD is too small) and is generally fully featured. That won't take very much time to migrate with a trustworthy migration vendor. I'm willing to pay a premium price for it and I'm all ears.
- nextworddev 3y agoThis article has good intentions but seems like a giant buildup to promote local inference tech stack
- minimaxir 3y agoProviding evidence to support a conclusion is a perfectly valid rhetorical strategy. It's only annoying if a blog post is using the structure to pull a "here's a list of reasons why X sucks; this is why my monetized project Y is better, please use it!" which is not the case here.
- bee_rider 3y agoPromoting local inference is a good thing to do.
- simonw 3y agoMy agenda with this article is to make AI companies aware that they have a genuine crisis on their hands, and hopefully push them to be more transparent and work hard to try and regain the trust that they have lost. I added the bit about local models mainly because I knew that if I didn't 90% of the conversation about the article would be "yeah but he didn't talk about the obvious solution, which is local models".
- nextworddev 3y agoThanks for the clarification. I agree that “opt in by default” is a dark pattern - but how else would any cloud service provide a RAG enabled service? Perhaps it should be made explicit during the onboarding process like geolocation (opinions as a former location PM at faang)
- simonw 3y agoIn this particular case it looks like opt-in would have been a much better way to go. Having a "enable AI features" checkbox seems to me like it would be a smart approach here.
- 3y ago
- jeremyjh 3y ago> As with many conspiracy theories, too many people would have to be “in the loop” and not blow the whistle. I used to believe this, but I do not understand how anyone can say this after the Snowden revelations. The vast majority of engineers are not martyrs. Given a choice to blow the whistle and then seek politcal asylum in Russia, most will not blow that whistle. Instead, they will live their lives and work to support their families. "But this is not about Goverment Surveillance!" Maybe this is true - I don't want to argue it here but I do think large tech companies are willing accomplices in many surveillance states - but an engineer at Open AI or Dropbox will probably not be facing felony charges for blowing the whistle. They will be facing the end of their career in big tech, and that's enough to dissuade most of them. I'd guess maybe 200 engineers would have to be in loop before there is an 80% chance that one blows the whistle in any five-year period, and if they are careful they can keep the number of engineers in the know MUCH, MUCH smaller than that.
- SkyBelow 3y agoWell we know the official history books are lying to us. Manhattan project? No way that could have been kept secret for as long as it was. All those people "in the loop" and not a single one choosing to blow the whistle? Preposterous. The idea that conspiracy is impossible ignores the conspiracies that happened and were covered up for significant amounts of time, long before we have the aid of modern technology. Why does the argument receive much attention despite that?
- jeremyjh 3y agoThe Manhattan project did leak info to the Soviets. It is true it stayed out of public view for several years, but a difference in the Manhattan project is that every single person involved was investigated by the FBI beforehand, and the country was so heavily united in the fight against the Axis powers that it is quite likely almost everyone involved was a true believer.
- simonw 3y ago"Why does the argument receive much attention despite that?" Because government national security secrets and dumb corporate secrets are different.
- rpastuszak 3y agoPeople believing the FB was eavesdropping on their conversations might be wrong, but that's mostly a technicality. It's like saying that I passed out because you hit me with a baseball bat, to which you'd reply: naaa...ah! it was a hockey stick! FB has been/is spying on people and making us collectively more stupid, but using different, more esoteric/harder to explain methods (e.g. cross-device behavioural targeting). > Those companies need to earn our trust. How can we help them do that? This is a wicked problem and such a broad question (+ a bit of a weird take imho), that I don't see anything actionable. So, here's a half-baked list off the top of my head (so, poorly expressed, incomplete, incoherent): - slow down the ill-conceived progress at all cost and understand that moving fast in the wrong direction ≠ progress - force stricter transparency rules through legislative action - understand and accept that no-trust should be the default - invest more in open and offline models - educate people - accept that some of those companies in their current shape don't deserve our trust Edit: This is a better take: https://news.ycombinator.com/reply?id=38643792&goto=item%3Fid%3D38643046%2338643792 https://news.ycombinator.com/reply?id=38643792&goto=item%3Fi...
- mvdtnz 3y ago[flagged]
- simonw 3y agoIf OpenAI say "we promise 100% that the data you submit to us through our API is not being used to train models" do you think they are straight-up lying to you? That's what I meant by an AI trust crisis.
- mvdtnz 3y agoYes I think they are lying.
- Kiro 3y agoYou have no mandate to prefix that with "of course" and you certainly have no mandate to insult the author for not believing in it.
- mvdtnz 3y agoI don't comment by mandate. These comments are my own opinion.
- skrebbel 3y agoI think it’s obscene that Dropbox turns that setting on by default.
- varelse 3y ago[dead]
- pavel_lishin 3y ago> The non-technical reasons are even stronger: > Facebook say they aren’t doing this. The risk to their reputation if they are caught in a lie is astronomical. Ah, bullshit. They've lied about other things, repeatedly and constantly. It's a risk, sure - but a calculated one, and one they've taken before.
- grammers 3y agoIn what way would AI deserve our trust? To date data hungry companies have proven to NOT have consumers interest in mind. Why should AI companies be any different? In dubio pro reo? I don't think so.
- 0xNotMyAccount 3y agoThis comes down to a combination of security and provenance. You have to protect the data, of course, but you also have to keep track of the use rights. Identifying all use rights upfront seems tricky, are the Creative Commons licenses sufficient?
- arka2147483647 3y agoFrom article: > Trust is really important. Companies lying about what they do with your privacy is a very serious allegation. > A society where big companies tell blatant lies about how they are handling our data—and get away with it without consequences—is a very unhealthy society. > A key role of government is to prevent this from happening. If OpenAI are training on data that they said they wouldn’t train on, or if Facebook are spying on us through our phone’s microphones, they should be hauled in front of regulators and/or sued into the ground. I find it very difficult to believe that big technology could be held accountable to anything by legal means by regulators. They have too large legal teams, and too well written legal agreements, and eulas. I find it impossible to believe that I, as a person, could challenge them in any way though legal means.
- simonw 3y agoNo matter how big the company, they still care about a billion dollar fine. That's bad for shareholder value! And billion dollar fines do happen.
- ash 3y agoFines are often gamed: lobbying, legal defence. I would argue big companies care more about losing customers. Сompetitive pressure is more effective than laws and fines. And laws are not without downsides, including becoming barriers to entry into competition.
- nojvek 3y agoBillion ain't much nowadays. The big 5 do 100B+/year in net profits. <1% doesn't matter. It's like being fined a $1000, while earning 6 figures. It feels like a pinch.
- deleted 3y ago[deleted]
- nojvek 3y agoGiven history of how US lawmakers have passing any bills that seriously threaten the might of BigTech, compared to how EU keeps on giving the middle finger to Big Tech, it's a weak argument. Corporations run America.
- namesbc 3y agoAll it takes is a simple tiny change to the user agreement and then OpenAI will start training on this user data and most people won't notice in time to switch their toggle off
- klabb3 3y agoThis reads a bit naive and “assume good intent”-y for me. Look at what’s happening outside of AI for a decade: everyone’s eating data like a compulsive hoarder. Not just Google and Facebook who are actually using it in their core products, but everyone. Today I discovered that a mini-site of Swedish traditional recipes I’ve been using for Christmas has added autoplaying videos, dark-pattern cookie consent banners, the usual. Almost every new app/site is oriented around this economic axis.. and then the sudden coordinated lockdown of 3p APIs right around when LLMs starts getting strong. And now we have ChatGPT/OpenAI and their competitors.. if the other players eat data like a secret midnight snack, current-gen AI is like zombies (the fast and twitchy variants) starving for blood and brains. Both because data serves a more direct role in the product, but also because of the typical hype-train-race psychology of tech VCs has been woken from slumber by the first potential paradigm shift in decades. All circumstances point towards zombie apocalypse/ gold rush / ask for forgiveness later / etc etc. I strongly believe that’s why they’re (all of them) doubling down on the safety/responsibility rhetoric now, before the inevitable reputational PR crises (plural). Get ammo to muddy the waters early. Meanwhile we techies are lulling around like we didn’t deeply just experience the last 10 years and thinking it’ll be different this time because.. AI is rooted in academia? Flashy new companies? The safety rhetoric? Edgy Twitter takes from “down-to-earth” founders? I don’t pretend to know exactly what goes on behind the scenes but I’ve been around long enough to know how people work. And they haven’t changed for the better.
- marcosdumay 3y ago> it’ll be different this time because.. You see, the new giant company promises not to be evil...
- __loam 3y ago> Meanwhile we techies are lulling around like we didn’t deeply just experience the last 10 years and thinking it’ll be different this time because.. AI is rooted in academia? Flashy new companies? The safety rhetoric? Edgy Twitter takes from “down-to-earth” founders? These companies have already stolen everyone's data and techies are bitching about IP law and saying they don't need to ask permission to use anything on the public internet. That might be the legal reality but you still look like tech douchebag for doing it.
- bayindirh 3y agoI'm asking this genuinely and honestly: How can I trust a company about my data, when their whole technology's performance depends on scraping the whole internet and feeding to a model? Also consider the fact that the data I'm providing them is way cleaner than what they scrape. Moreover, we (at least the older ones amongst us) have seen what Microsoft has done in the past, and said company is almost one with the same Microsoft, which is ready to do anything for platform dominance. OpenAI, esp. when combined with Microsoft and the whole A.I. hype paints a very untrustworthy picture in my eyes. I'm not against the technology, but how it's trained, developed, hyped and how the researchers in the ecosystem behave about the data they handle makes it very off-putting in every sense, esp. in privacy and ethics. It's like looking into a proverbial sausage factory, but this one is way worse.
- twoodfin 3y agoThere’s a deeper connection between concerns about OpenAI and the “Facebook is spying on us via microphones” meme: While there are perfectly rational, non-hot-mike explanations for the phenomenon of talking to a friend about something and then having it show up in an ad, the experience itself is damn spooky, no matter how it’s occurring. And hypertargeted ads are just the earliest ripples of the wave of AI-driven applications for predicting human behavior. Those predictions—about where we’ll decide to go on vacation, when we’ll quit our jobs, what we’d like to buy, who we’ll marry and divorce—will be more right than wrong to an incredibly spooky degree. I have no idea how society will react to that.
- rurp 3y agoI think this article is a little unfair to the lay critics, aka conspiracy theorists. The average user of Facebook has no idea how massive and widespread their personal data collection machine is. They correctly intuit that Facebook has an absurb amount of personal data on them, but don't have any frame of reference for how most of that data is collected. But the concept of a microphone listening in to private conversations is easy grok, and while it's factually wrong, the effect it points to is a lot closer to reality than the corporate speak they here from the company about how they "Have the utmost care for users personal privacy, blah blah blah". Remember a few years ago when Facebook tried to rebrand itself as a privacy company!? Facebook absolutely has a lot of information that the average person considers personal and private, and big tech companies work hard to mantain the illusion that their data collection isn't as pervasive as it is. With OpenAI, most non-technical users understand that the company has ingested a LOT of creative works done by individuals, most of which weren't expected to be ingested by a giant AI company and potentially regurgitated to millions of users. The better corporate messaging this article argues for will help ameliorate concerns to some degree, but won't address the root of the problem. Most people might have the details wrong, but they are directionally right about the massive data being collected and used by these companies and they have every right to be concerned about that.
- simonw 3y agoThis is a great comment. I think you're absolutely right about the microphone thing: people believe that because it's easier to understand than what's actually going on. That's also my frustration with it: I want people to understand what's actually happening with targeted advertising so they can get angry about that. Having them get angry about a fake microphone conspiracy theory is a distraction that makes it harder to focus on solving the underlying issues.
- financypants 3y agoCan someone explain the consisten use of “Facebook are” instead of “Facebook is,” is this proper grammar?
- simonw 3y agoThis is a personal style choice I make, because I like to emphasize that companies like Facebook and OpenAI are made up of individuals, and it's those individuals that make decisions.
- aftbit 3y agoCopy-pasta from the article, about a related claim (Facebook spying using phone microphones) This theory has been floating around for years. From a technical perspective it should be easy to disprove: * Mobile phone operating systems don’t allow apps to invisibly access the microphone. * Privacy researchers can audit communications between devices and Facebook to confirm if this is happening. * Running high quality voice recognition like this at scale is extremely expensive—I had a conversation with a friend who works on server-based machine learning at Apple a few years ago who found the entire idea laughable. The first point seems believable, but the second and third do not. Obviously a nefarious Facebook would be toggling these features off when they were being inspected, and even if they were not, they could be using sneaky exfiltration techniques. With the rise of app attestation, they would be able to do so in a way that would be 100% undetectable by reverse engineers. The relevant code would only be delivered to the app when it saw that it was running on an L1 trusted phone (with hardware security intact, unrooted). Additionally, they could embed some whisper-tiny on the device and force it to do its own speech recognition and hotword detection, only sending a list of "ad topics" back to their servers. I don't think Facebook is doing this, and the social reasons hold more weight with me, but I don't think it's technically impossible or easy to prove that they did not _ever_ do this.
- simonw 3y ago"Additionally, they could embed some whisper-tiny on the device and force it to do its own speech recognition" Yes, they could do that today. People have been assuming they've been doing this since way before 2017, long before Whisper.
- oblib 3y agoPersonally, I think "AI" should be required to disclose the origins of the data it bases it's responses on, and if any of those are copyrighted.
- simonw 3y agoFor current LLMs that's not technically feasible, because every single token they output is influenced by every token they trained on - so any answer you got from them would have to include disclosure of millions of documents that went into the training set. (I'd very much like them to disclose the full scope of their training set, but it's not possible for them to do that on a prompt-by-prompt basis in the way you describe.)
- altcognito 3y ago“Data is deleted from third party servers in 30 days.” No, that’s a thing that should never have happened. Ever.
- dclowd9901 3y ago> Facebook say they aren’t doing this. The risk to their reputation if they are caught in a lie is astronomical. This isn’t the disincentive you think it is. VW manufacturing vehicles that purposely deceived emissions testing was textbook corporate malfeasance, perhaps one of the worst examples of it. They took a loss on the ledger and came out of it relatively unscathed
- lysecret 3y agoHmm, isn't the whole point of the "Facebook spies on us though our Microphone" story that then people laugh about it move on and use Facebook regardless? Hundreds of millions of people use ChatGPT, billions Google and Facebook. The value wins. If the service is good enough people just tend to "hope" more than trust that their data is save enough. For the dropbox backlash people just didn't see the value.
- rc202402 3y agoIt's 2023 let's summarise AI * My Documents can be trained on * My Instagram images are legally not mine and can be trained on * My voice can be cloned without consent * My complete motion capture data can be replicated to another image * My analytics and usage data is owned by either US or Chinese corporations who either side will train AI on to feed me more ads * A green colored hardware giant is leading the AI warfare * The internet is turning highly anti consumer
- deleted 3y ago[deleted]
- kilolima 3y agoDon't forget Airbnb. Two or three years ago they claimed they needed scanned photo IDs of all users to eradicate racism on their platform, and they were going to use cutting edge ML to do so!
- barryrandall 3y agoIt's a market trust crisis amplified by AI. Everything in the article applies to scummy business practices that were spreading before we all caught the CUDA crazies.
- caconym_ 3y agoGood article overall, but I find the analogy between "my phone is eavesdropping on me" and "openai might be lying about how they use my data" somewhat flawed because there are robust checks on third-party apps accessing my iphone's microphone that have no equivalent when my data is handed to a third party in plain. Certainly for the layperson they may as well be the same thing, but that layperson is still being protected from the former. This might seem like splitting hairs, but I think it's extremely counterproductive to act like the battle for users' data privacy and sovereignty is lost just because most users can't tell the difference. I see this a lot from the other side: extremely cynical, at least somewhat tech-savvy people reacting to each new corporate abuse with an "old news" mentality as if we've already reached the endgame—if you haven't been using tails linux for at least a decade you may as well just zip up your home directory and email it to every shady tech corp and data broker you can find contact info for. These people ought to know better, and they ought to set a better example for those who don't. This inculcated helplessness definitely hurts trust, but it also gives people the impression that a better world isn't possible—that there aren't better and worse choices for who to trust with their data and privacy. This Dropbox snafu seems to be that mindset coming back around: users won't give a shit if we imply we're sending their private files to a third party without asking, right? Lunacy. Incidentally, I've had most of my data out of Dropbox for a while (in favor of a self-hosted solution), but yesterday was the kick in the ass I needed to cancel it for good. Thanks, Dropbox!
- skywhopper 3y agoI don’t think you’re splitting hairs here. You’re making a very good point. App access to the microphone is controlled by the OS and there are OS provided, user-accessible tools that make it clear what apps can use the microphone and when. Meanwhile cloud data access is completely on a “trust me” basis, and plenty of companies have been proven to have abused that trust.
- simonw 3y agoYeah, I tried to touch on that analogy flaw in the article: > One interesting difference here is that in the Facebook example people have personal evidence that makes them believe they understand what’s going on. > With AI we have almost the complete opposite: AI models are weird black boxes, built in secret and with no way of understanding what the training data was or how it influences the model. Completely agree that our biggest threat right now is complacency. If people form incorrect mental models of what's going on and then shrug their shoulders and accept that's just how it is, we won't make much progress in improving the actual problems.
- ilovetux 3y agoThere is an ongoing question about whether AI can be trained on copywritten works. If the AI pioneering companies are able to push back against the entire publishing industry and have nothing more than a conversation about it, then why would I ever expect any company to respect me or my personal data? In fact, I think there is a large and growing portion of the population who have watched big business scoff at laws and regulation (see Wells Fargo, Uber multiple crypto exchanges, etc) and are realizing that profit is the only motivator. That's not even talking about the chaotic political landscape. I think we will soon see a crisis in institutional trust across the board.
- renegade-otter 3y agoThe biggest problem with AI is not the tech but the people. White billionaires worrying about SkyNet but totally overlooking the impact on the everyday "poors". The individuals who never had to go through a second of adversity in their lives cannot be expected to understand. In fact, their sociopathic tendencies only skyrocketed through the years of isolation from the real world, with COVID compounding the effects. Casually talking about how robots should replace humans as a species because of "efficiency"... By the way, did you know that Sam Altman was VERY active in politics until the latest drama? He was backing a candidate to primary Joe Biden, but now he had to go take care of his own stuff. The outcome of this would be pretty obvious to those who follow American politics: https://www.theatlantic.com/press-releases/archive/2023/12/atlantics-janfeb-issue-next-trump-presidency/676227/ https://www.theatlantic.com/press-releases/archive/2023/12/a... My point is, a person that cannot draw a straight line from F---ing Around to Finding Out cannot be trusted with something infinitely more complex.
- bachmeier 3y agoThis article misses the mark. The problem is that, for good reason, we've moved to a post-trust society. The author argues that you're a nutjob if you don't trust what large corporations are saying. He goes so far as to say it's on you to prove that they're doing something wrong. This doesn't represent reality. Take this quote: > A society where big companies tell blatant lies about how they are handling our data—and get away with it without consequences—is a very unhealthy society. So, what are the consequences of lying? If there are any, I'm curious what they are. The author is arguing that "big companies" are going to do the morally correct thing even though it hurts their profit. That's just a silly argument. In a post-trust society, companies have to prove they're doing what they say. There's no more making a statement and having the masses act as if that's what's going to happen. Honestly, is there less than a 100% chance that down the road there will be a disclosure that there was a "mistake" where your data was used differently from what they claim? Fool me once, shame on you, fool me 5,872,328 times, shame on me.
- simonw 3y ago> "The author argues that you're a nutjob if you don't trust what large corporations are saying" That wasn't the message I was trying to communicate at all. My point with this piece is that people don't trust AI companies, and companies need to figure out what to do about this - that's why I called it a "crisis". I certainly didn't argue that big companies would "do the morally correct thing even though it hurts their profit". > "In a post-trust society, companies have to prove they're doing what they say. There's no more making a statement and having the masses act as if that's what's going to happen." I think you and I might be in strong agreement there. My goal in writing this company is to get AI companies to take this problem seriously and, like you said, "prove they're doing what they say".
- itronitron 3y agoIf a company is going to claim that they don't have Syphilis, then they need to guarantee that each of their "trusted partners" also doesn't have Syphilis, and each of their trusted partners needs to provide the same guarantees about their own trusted partners as well, ad infinitum.
- nojvek 3y agoGreat article overall and agree with local models. If it is connected to the internet, then it is not secure, you cannot trust your data not to be slurped up somewhere. Threat models yada yada. I'm in love with how fast models like insanely fast whisper, ggml llama.cpp, Mixtral, YOLO v5 e.t.c have come along. I very much predict devices in 2024+ that have powerful energy efficient GPUs and do multimodal inference locally without any connection to the internet. I remember buying the first comma.ai pre-panding which was essentially a modded android LG phone driving the car. No connection to internet, a freaking mobile phone driving the car without any human input on the highway. That totally blew my mind. I'm awaiting Apple to release a version of Siri that runs totally offline. Smart phones are the perfect target.
- barrysteve 3y agoThe internet has fractionated out our cultural references and commonly used language. This has caused the communication of belief to separate out from lanaguage. The Facebook Spying idea is the common place to agree and complain about phone spying in general. It is the poster boy for communicating an implicit belief about a larger topic. We have begun to use language to communicate implicit and shared beliefs without any evidence, facts, agreement or qualification. We come to our own beliefs privately and then choose a poster boy to carry the conversation. There's not much surprising about the continuation of conspiracy theories, or topics that look like conspiracy theories. We shredded the common cultural references apart...
- ideamotor 3y agoThere is no independent monitor or authority that can check what OpenAI does or doesn’t do. So there is no reason to trust them. Simple as that. Tipping the scales further are the numerous instances of tech companies lacking sufficient data security, the latest being 23andme. Case closed.
- Berniek 3y agoTrust Crisis for AI? What after we see the board/CEO of one company fired/changed apparently for allegations of lies or manipulation that nobody is clear on? If Dropbox derives data from user data by scanning said data, then that "derived" data is no longer "users data" it is Dropbox data and can be shared. It may only be statistical in nature and not related directly to individual users but isn't that exactly what training data is? Isn't that how it works? That can be shared to train AI models can't it? Its not lies its hair splitting. No its called unethical behavior and has become the norm for big tech.
- fasterik 3y agoToo often the claims of unethical behavior are knee-jerk, baseless, and speculative. In the example the author cites, Dropbox only sends data to OpenAI when the user explicitly tells the app to engage an AI-related feature, for example to summarize a document. Yet the backlash seems to assume that they're scanning and uploading people's documents en masse, even though there is no evidence of this. Unethical behavior definitely exists in AI companies. Personally, I'm agnostic about whether it's higher or lower than the base rate of unethical behavior in the general population. Anyway, if we're going to talk about bad behavior, we should use specific examples with cited evidence, not fear-mongering.
- JohnFen 3y ago> Personally, I'm agnostic about whether it's higher or lower than the base rate of unethical behavior in the general population. I don't think that AI companies are less trustworthy than our industry in general, but I absolutely think that our industry is far less trustworthy than the general population. AI companies engaging in the wholesale harvesting of web data to train their models, though, was a particularly egregious violation of trust.
- dredmorbius 3y agoIn fairness to Sam Altman and OpenAI, all credible reporting I've heard (and signifcantly Kara Swisher's work) has shown no issues of AI safety or lies in CEO communications to the Board, but rather broader concerns over the direction the CEO and board felt appropriate for OpenAI. I've no dog in this hunt. I'm not partial to either Altman or OpenAI. And I have considerable reservations over where this Brave New World may be taking us. Or whether there is any credible option to stop riding this merry-go-round, no matter how unattractive the destination(s). The DropBox behaviour described is only one in a long, long, long line of trust-violating practices by tech firms.
- shrimpx 3y agoThe microphone trust aspect in the article seems like a red herring distracting from what could be a clearer point. Facebook literally takes your data from their apps and internet, tracks your behavior on the internet, and feeds this data into models of you. These models are so accurate they can sometimes basically predict what you're thinking. Hence the layman jumping to the conclusion that they must be spying though the mic. A LLM company like OpenAI, and their partners, employs almost literally this exact model. Grabs data from whatever sources to improve their models, to increase the likelihood you'll keep clicking where they want you to click, to monetize you.
- JohnFen 3y ago> Hence the layman jumping to the conclusion that they must be spying though the mic. Right, and in a larger sense, the laymen aren't exactly wrong. They're technically wrong about the mechanism, but they're exactly right about the extreme intrusion into their private lives. That the intrusion comes in the form of accurate models rather than the microphone is just a technical detail. The end effect is the same.
- red-iron-pine 3y ago> Facebook literally takes your data from their apps and internet, tracks your behavior on the internet, and feeds this data into models of you. These models are so accurate they can sometimes basically predict what you're thinking. Hence the layman jumping to the conclusion that they must be spying though the mic. and all of this just to show me shitty ads for online games that I will never, ever, EVER play, college-themed dating services I'm not going to use, yoga shit, and money remittance services. I live near a big university, so I'm guessing it's simply by IP. occasionally get ads for Lexus or Jags tho. that's nice.
- asdff 3y agoI think you'd find the people who don't trust AI companies also harbor similar feelings about companies in many different industries, or non profits, or even government agencies, depending on who specifically you ask. It seems to me that there is a much bigger trust issue well beyond the scope of AI based companies, so trying to have that specific field fight this distrust against them that really comes from all fronts, seems like an impossible task well out of the scope of these businesses. I'm not sure what the answer to this issue is, or if it even is a real issue, or what we might be marching toward should this pervasive cynicism spread to all things and all people. Perhaps we are simply cursed to live in interesting times.
- JohnFen 3y agoThe first step to regaining trust is to stop abusing trust. Our industry overall is incredibly abusive to trust, and I've seen no sign that is going to change anytime soon.
- trinsic2 3y agoTo me it seems like there needs to be a complete shift in the kind of people that are in government for things to change. Might be a good idea to start running for an office you think you could do better in. I know nobody has time, inclination, or money for that, but man this is a crisis.
- __loam 3y agoWhy does change need to come from governments? Why can't we hold ourselves to a higher standard of ethics.
- trinsic2 3y agoIt doesnt have to come from government. We can definitely hold ourselves to better standards, but I also think that there need to be stronger checks and balances too. I think they feed into each other, hold better standards and having a strong sense of justice.
- deleted 3y ago[deleted]
- atmosx 3y agoReading the post, my first reaction is to remove all docs from my free dropbox account.
- rchaud 3y agoThe TLDR is "local models are great to have but hopefully we don't lose out on large hosted models due to distrust about data going straight to OA or some other big companies." The arguments made in favour of bigco trustability are not great though, esp the parallel between AI data ingestion and Instagram's rumored use of mics to capture voice data for ad targeting: > Facebook say they aren’t doing this. The risk to their reputation if they are caught in a lie is astronomical. FB has already paid the biggest fine in FTC history for privacy breaches. It has no goodwill, it's running on user inertia and indifference.
- zmmmmm 3y agoThis is a great piece. I often observe that this really applies at a high level in general as one of the key elements that demarcates successful / advanced economies from lesser ones : how much is trust a limiting factor on what you can achieve? If you think about modern society and the levels of trust needed to do certain things : I spent nearly a lifetime of earned income on a house and I completely trusted the agent and bank who facilitated that. Without that trust, the whole transaction wouldn't be possible. Why did I have that trust? These are the hidden value of all those laws and regulations that we often hate so much. As much as anything they create the stable platform for high value transactions to take place. The price of not having them is not being able to access the value you can get if you can successfully execute high trust transactions. There's no way I would gamble my whole life's income if I didn't trust the process. I would have to buy a smaller house. So AI is playing out along these lines. How much of the value we can access from AI is eventually going to be limited by how much trust we can grant it. Which ironically may derive from how much regulation / legal framework is successfully put around it, which may in the short term slow it down. The self-hosted open source model route is a substitute / fallback for not having trust in hosted models. We can't really pretend that it's possible to self-host a model that will be as good as a hosted one, the scale is always going to be a material difference. But we may be to hit the point where for targeted applications there are diminishing returns, and that may be good enough.
- hoistbypetard 3y ago> Facebook say they aren’t doing this. The risk to their reputation if they are caught in a lie is astronomical. Even though I don’t believe Facebook is spying on anyone surreptitiously through their phone’s microphone, I find that specific argument entirely unpersuasive. Facebook’s reputation is dogshit, at least with regular non-technical people I know. I’m in the US. People know they helped foment the January 6 insurrection in 2021 and saw how they deflected any and all responsibility while not fixing a thing afterwards. The reputational damage they’d absorb from actually doing this thing that so many people already think they’re probably doing pales by comparison.
- red-iron-pine 3y agothey know its dogshit... but still keep using Insta and WhatsApp
- krupan 3y ago"As with so much in AI, people are left with nothing more than “vibes” to go on. And the vibes are bad." word
- iamleppert 3y agoWho in their right mind would trust a company that just had its CEO publicly ousted for allegedly lying to the board, only to have him put back in charge? And we have yet to see any kind of reasonable explanation or statement about what really happened. Where there's smoke, there's fire. Public trust in large tech companies is at an all-time low. Nobody thinks tech companies want to help anybody but the few rich people running them anymore. I wouldn't trust a word OpenAI says, because even their name is a lie. The code isn't open, it's not auditable and you have dishonest people running the company.
- imiric 3y agoI find the premise that users should trust tech companies by default absolutely ludicrous. The author seems to live in a bubble where tech companies don't commit privacy violations regardless of what their privacy policies say. Yet in the real world, even when they're caught with their hand in the cookie jar, the fines these companies are forced to pay are just the cost of doing business. Earlier this year Microsoft was fined $20M for collecting PII from children, which it "shared" with advertisers[1]. And this is _Microsoft_, not even an adtech giant. The abuses from those have their own Wikipedia articles[2,3]. FB was famously fined $5B in 2019[4]. It would be beyond naive to believe that these were just "oopsies", and that somehow these companies have changed their ways. _This_ is why it's not implausible that the software that runs on our most personal computing devices, built by these same companies, is hoovering up our data in the sneakiest ways possible. Equating this to conspiracy theories is dishonest at best. When Mark Zuckerberg puts tape over his webcam and microphone, it would be foolish to think you know better not to. [1]: https://www.ftc.gov/news-events/news/press-releases/2023/06/ftc-will-require-microsoft-pay-20-million-over-charges-it-illegally-collected-personal-information https://www.ftc.gov/news-events/news/press-releases/2023/06/... [2]: https://en.wikipedia.org/wiki/Privacy_concerns_regarding_Google https://en.wikipedia.org/wiki/Privacy_concerns_regarding_Goo... [3]: https://en.wikipedia.org/wiki/Privacy_concerns_with_Facebook https://en.wikipedia.org/wiki/Privacy_concerns_with_Facebook [4]: https://www.ftc.gov/news-events/news/press-releases/2019/07/ftc-imposes-5-billion-penalty-sweeping-new-privacy-restrictions-facebook https://www.ftc.gov/news-events/news/press-releases/2019/07/...
- j2kun 3y agoIsn't there sufficient evidence to conclude that OpenAI is training on things they know they probably shouldn't be training on? Like the full text of novels that are not publicly available on the internet? I don't think it's that unreasonable to extrapolate that they would blur the lines of legal agreements to get access to private user data they shouldn't have.
- msum 3y agoA friend told me about the Dropbox setting last night, so I logged in and turned it off. This morning, I went to look for the setting to take a screenshot but it's gone. The setting just...isn't there anymore. Made me feel like I was going a bit crazy TBQH. Surely I didn't misremember? Annoyed because it was a convenient file storage solution and now they have proven themselves untrustworthy so I have to set up my own thing. My fault for trusting to begin with, I suppose.
- 2devnull 3y agoTrust is simple. If you can’t trust your own family, who can you trust? Families, that is the answer. Oddly, the trend is in the opposite direction; trust media, trust government, trust apple, OpenAI, etc…. Things will get better when that trend reverses.
- JohnFen 3y ago> It’s increasing clear to me like people simply don’t believe OpenAI when they’re told that data won’t be used for training. I certainly don't. I don't trust any of the AI companies at all. Their behavior and statements have given me quite a number of reasons not to, and no reasons why I should.
- 0xDEAFBEAD 3y ago>One interesting difference here is that in the Facebook example people have personal evidence that makes them believe they understand what’s going on. >With AI we have almost the complete opposite: AI models are weird black boxes, built in secret and with no way of understanding what the training data was or how it influences the model. In principle, I think we could have significant visibility into how OpenAI trains its models by doing something like the following: * Generate a big list of "facts" like: Every fevilsor has a chatterbon. Some have a miralitt too, but that is rare. (Or even: Suzy Q Campbell, a philosopher who specializes in thought experiments, lives in Springfield, Virginia. Her phone number is 555-8302.) * Generate a big list of places where OpenAI could, in principle, be harvesting data: HN threads, ChatGPT conversations, StackOverflow, Dropbox files, reddit posts, etc. * Randomly seed facts in various places. * Check which facts get picked up by the next version of GPT. If you really don't trust OpenAI, be subtle about where and what you seed (e.g. avoid obvious nonsense words) to make it hard for their engineers to filter out your canaries even if they tried. It's important to seed some facts in places they're known to scrape, as a control. If those facts aren't getting picked up, then you have methodological issues. You might have to repeat a fact a number of times in the dataset before it actually makes its way into the model.
- nagonago 3y agoI've seen this play out on Discord too. When they rolled out the new "summarize conversations with AI" feature, I saw a lot of people concerned their data was going to be stolen for AI training as a result. I pointed out that they explicitly said this won't happen, and also that Discord's ToS is already vague enough that there's no guarantee your data isn't already being used for other purposes. The LLM fearmongering is powerful.
- wackycat 3y agoI love convincing people that Facebook isn't spying through the microphone but those relevant ads that relate to what your friend just said are as creepy as you think, just without the microphone. The example I like to give is if your friend with pets bought a new vacuum that they tell you is an absolute godsend for dog and cat hair. Facebook knows you have pets, they know your friend has pets, they know that you and your friend are friends. If you have location data on, they know that you and your friend are together. They know your friend recently bought a vacuum because they have access to a lot of purchase data. So it may seem like they're listening when your friend mentions the vacuum and an ad for it shows up in your feed later that day. They weren't listening, they were tracking. Listening would be spying in one aspect; rather they were following multiple aspects like purchases, relationships, lifestyle. Still creepy, just in a way that's harder for consumers to understand.