14 ms·
The data that powers AI is disappearing fast
- masherm 2y agoWhat if everyone resorts to training on synthetic data ...
- lindafassett927 2y ago[dead]
- blackeyeblitzar 2y agoWe should make the data on large platforms like YouTube and social media in general accessible to all companies for AI use (with the actual creator’s positive consent).
- jsheard 2y agoHow many creators do you think would actually go out of their way to consent to their work being used for training? Maybe if they get paid, but otherwise forget about it.
- janice1999 2y agoThere's actually an interesting perverse incentive here. A group of politically motivated actors (perhaps funded by a government or private interests) could create content with the sole purpose of opting in to all the AI data harvesting feeds. That way they could bias the output to their cause (assuming the AI company doing the training does take active measures to counter it).
- jsheard 2y agoI would assume that's happening regardless, platforms like Reddit are already selling their data firehoses to AI companies, so anyone who manages to slip propaganda bots under Reddits radar will end up having their talking points fed into future AI training sets.
- im3w1l 2y agoPayment is out of the question - too much of a hassle. What is likely is websites either requiring you allow ai training in exchange for hosting your content, or giving some minor perk/incentive for it. Consider this comment itself. How much do you think it is worth for an AI company? Maybe 0.00001 dollars? How would you handle the logistics of money that little money?
- oldkinglog 2y agoThis comment neatly explains why the LLM bubble will burst as soon as prosecutors remember that the DMCA doesn't have a carve-out for AI.
- BadBadJellyBean 2y agoI think all is about consent. And I don't even think that people would be so upset if the whole AI traing wasn't abot profit. But they way it is, companies are training their models on other people's work and try to make money with the models.
- gregGisme 2y ago[dead]
- add-sub-mul-div 2y agoImagine if we'd been appropriately skeptical of the way social media might worsen society in exchange for some short term fun and convenience, rather than blindly conflate invention and novelty with progress. We can forgive ourselves for that naivete, but having seen that what excuse do we have now?
- deleted 2y ago[deleted]
- GaggiX 2y ago>Those restrictions are set up through the Robots Exclusion Protocol Well so it's not really disappearing at all.
- buildbot 2y agoWell, unless you exclude common crawl and block all robots…it’s still going to end up in a dataset someday. Or deleted and gone forever!
- janice1999 2y ago> We’re seeing a rapid decline in consent to use data across the web That is such a weird and misleading way to put it. There was no consent in the first place. Take YouTube for example. Google did not consent to the videos it hosts being used by OpenAI. The uploader certainly did not ever consent to their face, voice and content being used to train models either.
- advael 2y agoI think this is an important thing to point out. This is the first really compelling test of the legal theory that the overbroad content-ownership boilerplate licenses big user-generated-content hosts use could actually bind people to allow arbitrary use of their content, likeness, etc. Before the uses of it were actually quite egregious, but kind of abstract, things like "build a profile on you to advertise products" has incredibly deep and creepy implications but they're for whatever reason not as unifying an objection as this
- add-sub-mul-div 2y agoRight, I think a better way to put it is that there was complacency and neutrality (inattention) on the topic before. But now that people see that their work is going to be used to flood the culture with slop, the new default stance is to deny consent.
- linkjuice4all 2y agoI agree that these content creators were not contacted individually and asked for specific consent to have their materials ingested for training purposes. That being said - the general crux of the third-party doctrine (at least in the United States) is that information told to another party has no expectation of privacy. That seems to apply to a vast amount of 'user generated content' on the internet. Someone decided to utilize someone else's megaphone to post into the commons and now expects to retain some ownership. It's hard to have it both ways "just because someone else noticed." Unfortunately this also extends to general artistic styles. If you, as an artist, continues to create in a consistent style and someone else notices you don't have a lot of recourse when some entity is able to recreate a similar style. On a technical side this seems similar to your server responding to any incoming request with a 200 status and some content (LinkedIn v HiQ). In this case a user-agent asked (was it a person? scraper? AI training process?) and you gave it to "them". I guess we end up in the position where if you don't want to train AI then don't post your content publicly.
- rectang 2y agoThe NYT had another article a couple of days ago about Getty leveraging the images it owns to go into AI. https://www.nytimes.com/2024/07/19/technology/generative-ai-getty-shutterstock.html https://www.nytimes.com/2024/07/19/technology/generative-ai-...
- buildbot 2y agoGetty often makes false claims and generally is a super shitty organization, for example (there are many): https://www.latimes.com/business/hiltzik/la-fi-hiltzik-getty-copyright-20160729-snap-story.html https://www.latimes.com/business/hiltzik/la-fi-hiltzik-getty...
- rectang 2y agoGetty will have to indemnify users against infringement claims in the event that the images it generates turn out to be based on unlicensed materials and judged as derivative by a court. So will all other AI companies eventually, it's just that Getty has _some_ content while other companies rely much more on unlicensed content.
- buildbot 2y agoTo me, what getty is doing is actual theft. How do you “accidentally” issue copyright claims using images you don’t own? How did getty start claiming this image as their own? In contrast, training an AI model you download a piece of info once and then don’t resell that (exact) piece of info. Nobody is claiming they own the image when they don’t.
- rcxdude 2y agoIt's fairly easy to do accidentally in the case of a large corporation like this: different parts of it don't check with other parts. In this case one part added some public domain photos to their site, which is scummy but legal, and then another part assumed that those photos were like the many others they have exclusive rights to. In another case mentioned in that article it's a third party which lied to them. Accidental doesn't mean they aren't reckless with their threats, however, which large-scale copyright license collection systems almost by necessity are, and that causes a lot of harm (and they are rightfully losing cases over it).
- thatxliner 2y agoI would be fine if you use my data to train your AI models if you let me use your models for free. If you can’t do that, you can’t have my data.
- thatxliner 2y agoI would be fine if you used my data to train your AI models as long as I’m able to use your models for free in return. If not, you can’t have my data.
- jsheard 2y ago> Those restrictions are set up through the Robots Exclusion Protocol If anyone would like to join in, there's an actively maintained robots.txt here: https://github.com/ai-robots-txt/ai.robots.txt https://github.com/ai-robots-txt/ai.robots.txt Yes, I know this isn't legally binding and scrapers can ignore it if they want to.
- thriftwy 2y agoI look forward for AI trained entirely on Wikipedia and classical literature with no twitter and no contemporary art in sight. It would be sublime. Let's face it, the creators of century XXI way overestimate the importance of their stuff. It's mostly deleterious to the culture.
- to11mtm 2y ago> trained entirely on Wikipedia does the CC used by wikipedia allow training? > and no contemporary art in sight will note per above, they may be able to scrape other CC if they -can- scrape wikipedia. > It's mostly deleterious to the culture. Yep and based on how I already sometimes catch people treating LLM hallucinations as fact, it's likely gonna get worse anyway.
- regularfry 2y ago> does the CC used by wikipedia allow training? This is backwards. Absent something that prohibits it, training is allowed. The question should be this: what do you believe prohibits it in this case?
- jprete 2y agoThe author of a work has copyright automatically, so what would prohibit it by default is copyright law. In the specific case of Wikipedia I would guess it's allowed by the license, but that's not generally true.
- regularfry 2y agoTraining isn't copying (any more than a browser or CDN cache is) or distribution. Copyright is out of scope. Is there anything else that would prohibit training?
- jprete 2y agoAll of those things are copying. Browser caches probably fall under fair use. CDNs are contracted by the distributor so literally licensed.
- elAhmo 2y agoNYT assumes that LLMs = AI, which is far from truth. This is just a recent hype which relies on getting insane amounts of data to train, but we had and will have AI models that do not rely on training using data without consent.
- kredd 2y agoTechnicalities make things harder to understand for general masses. “The Data That Powers LLMs” means little to nothing to an average user. My 70 year old parents use ChatGPT, and they have no idea what LLM is, but call it AI.
- wkat4242 2y agoIt's not just LLMs but all generative models that rely on extreme amounts of training data. Text but also images, video, speech, music. And it makes sense that LLMs were the ones to trigger the hype. AI in general has been making steady progress but most usecases are really hard to explain to layman. Give them a chatbot and they naturally understand what's happening. The only thing I'm seeing is that people are using them wrong. They're using them like all-seeing oracles, which is exacerbated by the confidence with which LLMs provide their answers, and the innate human idea that "it's a computer so it must be right". But knowledge isn't really where LLMs shine, at least not without search engine integrations. It's rather generation, summarisation, translation (with context sensitivity), rewriting styles etc.
- jll29 2y ago"disapparing" = people getting aware that their data has value, and setting their robots.txt permissions acordingly?
- skeledrew 2y agoLike people getting up in arms when they notice someone found a way to create valuable medicine from the litter from some trees in their yard that they always ignored people taking shade under. Now they want to block that novel use or get a cut of the profits, when they never saw any value before beyond use as shade. Senseless IMO.
- tensor 2y agoNo, disappearing = people thinking their data has more value than it has and setting their robots.txt so that it's no longer even indexed for search because they've banned any use of AI. Then, because it's actually not that valuable, no one notices and that data eventually ceases to exist when the creator eventually abandons it or dies.
- wkat4242 2y agoIn the near future I see less real, quality human data that will go behind paywalls, but also much more AI generated data feeding into the next generation of AI. Because more and more people are using it to publish stuff online. Which then gets scooped up by AI training crawlers. And if it made sense to train an LLM on its own output, it would be done already :) I doubt they can continue the progress at the same speed they have been at so far. Because the game is set to become more difficult.
- khana 2y ago[dead]
- to11mtm 2y agoIt honestly makes me hesitant to publish my special personal code on gists or private repos vs at least 3-2 backup the really good stuff... TBH I would not feel any sadness if LLM models plateaued, greatly decellerated development, or even -regressed- as they start to ingest all the garbage already being spewed by LLMs.
- hedora 2y agoI think the worst possible outcome is a licensing regime that means that Disney or Paramount or Elsevier or whoever all get to have a monopoly on training large models within their niche. My guess is that any successful calls for regulation will have this outcome, which means that individuals won't be able to legally use AI-based tools except when creating works for hire, etc. Currently, I think most of the training use cases can be covered by the existing "you can't copyright a fact" carve out in the law. That's probably better for society and creators than my licensing regime scenario. Anyway, I'm rooting for "no regulation" for now. The whole industry is still being screwed over by market distortions created by the DMCA, and this could easily be 10x worse.
- JohnFen 2y ago> Anyway, I'm rooting for "no regulation" for now. Not me. I want appropriate regulation. Without any regulation, these companies will just continue their abusive practices without restraint. Will regulation encourage a concentration in a couple of large companies? Maybe. It depends on the nature of the regulation. Even if it does, though, I'd take that over the current status quo.
- DevX101 2y agoIt's already happening. Mega content aggregators will either build their own LLMs, or license for $$$$. Twitter's doing both and reddit did the latter.
- janoc 2y agoIt is good that you are rooting for the poor industry because they are "being screwed". Sad that you didn't consider the content creators, people who's faces, voices, writings, art, personal information, etc. are being used without consent and without any compensation as the ones "being screwed" here.
- Terr_ 2y agoIt's kind of like the history of the web with regular human consumers and various attempts to allow microtransactions, which receded in favor of ads and tracking. If the pattern holds, the "cost" of training non-paywalled data will be attempts to hack/influence the model, as opposed to hacking humans.
- doctorpangloss 2y ago“Tell HN: Stop Reading The New York Times”
- greatpostman 2y agoAi negativity clickbait
- neonate 2y agohttps://archive.ph/EEQQB https://archive.ph/EEQQB
- lionelholt 2y agoThe decision to block bots is not always about protecting intellectual property. A practical consideration I haven't seen mentioned is that some of these AI bots are stupidly aggressive with their requests, even ignoring robots.txt. I had to activate Cloudflare WAF and block a variety of bots to prevent my web app servers from crashing. At least they're reasonable enough to identify themselves!
- PaulDavisThe1st 2y agoyeah, we had a bunch of them crawling out git repositories in a very aggressive way, repeating the crawl within a few days, etc. etc. 403 to the lot of them, regardless of the bot's purpose.
- aitchnyu 2y agoWhats the deal with (Amazon) bots crawling private Gitlabs aggressively?
- j16sdiz 2y agoThose aggressive bot.... Crawls each and every date link on my synology dsm hosted calendar without throttling.
- worstspotgain 2y agoOutside of tech, AI isn't making any friends. It's an unexpected threat on the horizon for the bearers of accumulated privilege, such as monopolies and nation-state autocrats. They hear echoes of the computer and internet revolutions, which were key in upending many prior "garden patches" of power. Their cronies have been stirring up resistance among workers on the front lines of disruption, such as some content creators. The truth is that no one knows where the dust will settle. All covered wagons are venturing west of the Rockies at the same time. Content creators are not really facing a worse outlook than everyone else. They are, however, in a better position to hinder the advance. In principle, an AI learning from a scientific textbook is no different than a human student doing the same. Neither will violate copyright law when they're done learning, except perhaps accidentally - paraphrasing and facts are not violations. Unfortunately, legal and ethical principles can differ from legal reality. We're left hoping that some altruistic legal minds will open up a Northwest Passage for us, like Thomas Penfield Jackson in US v. Microsoft. The worst possible outcome is that we end up with an all-powerful AI cartel which negotiates massive deals with IP conglomerates, locking out competition and open and free alternatives.
- autoexec 2y ago> In principle, an AI learning from a scientific textbook is no different than a human student doing the same. Neither will violate copyright law when they're done learning If AI were only learning from scientific textbooks very few people would have problem with it. When AI learns from creative works like novels, it runs into the same problems we have as humans. We can read a book, but the minute we start distributing copies of it from memory we're in legal trouble.
- atlasunshrugged 2y agoWhy wouldn't nation state autocrats be interested in AI? Seems like a great tool for facial recognition at scale and eventually maybe even automated policing via drones and the like, thus sparing them the costs (and danger) of having to maintain a major police/military force to protect their regimes
- worstspotgain 2y ago
- skybrian 2y ago> Some companies believe they can scale the data wall by using synthetic data — that is, data that is itself generated by A.I. systems — to train their models. But many researchers doubt that today’s A.I. systems are capable of generating enough high-quality synthetic data to replace the human-created data they’re losing. I'm thinking synthetic datasets are how it's going to go. Of course you can't get information from nothing, but they might not need nearly as much seed data to generate lots of examples that are specifically designed to train reasoning skills. Maybe it will take a while to get over the hump, but they'll be motivated to make it work.
- Out_of_Characte 2y agoGarbage in Garbage out is real and shouldn't be underestimated. Its about as useful as trying to understand quantum physics from the perspective of a flat-earther
- WalterSear 2y agoSynthetic and generated data is also real and shouldn't be underestimated. LLM training can be very counterintuitive at times.
- Out_of_Characte 2y agoSynthetic and generated data might not be garbage. Consider random chess positions. Those are generated,yet there's an engine out there that has objectively a higher chance of winning from that position. There was also a paper on persistent computer programs after running random data iteratively over millions of cycles. This doesn't even have a 'goal'. which by itself is very interesting because of how similar it feels to how organisms develop by the act of persisting after a few billion years. That is to say I largely agree with the premise that you can train and improve alot by absense of data. But if your original dataset is garbage, then training over and over might get limited by a hidden nash equilibrium that the algoritm can't escape due to lack of new information. Obligatory Anton petrov: https://www.youtube.com/watch?v=L_IWVZPmc-E https://www.youtube.com/watch?v=L_IWVZPmc-E Paper: https://arxiv.org/pdf/2406.19108 https://arxiv.org/pdf/2406.19108
- amelius 2y agoLet's not fool ourselves and think that these big AI companies care about licenses. It's easier to ask for forgiveness than consent, the cost of doing business, etcetera, etcetera.
- skeledrew 2y agoOn one hand this really sucks, particularly for newcomers starting from scratch, as now only the larger companies that already scraped the web can carry on in this vein, finding ways to improve the architecture to better utilize the data they already have. On the other hand, I see this being a forcing function to move away from generative AI sooner, which I've always considered to be a dead end for AGI, to future architectures that train on data similar to how humans learn, ie raw audio, video, etc streams from the environment.
- ChrisMarshallNY 2y ago> now only the larger companies that already scraped the web can carry on in this vein That will have a finite lifetime. We've already seen issues with just the couple of years' difference that the first LLMs have, and it will definitely get worse, as time goes on. They will, however, be able to improve what they do with the data, and some of them will probably get enough funding to buy the data. Don't be shocked to find that the reason so many organizations are restricting access to their data, has nothing, whatsoever, to do with things like privacy and copyright. They just see a lot of money sloshing around, and want to hold out for their share. The pipes will reopen, but the free lunch is probably done. One big issue will probably be, that the free data will be extremely low-quality (possibly poisonous) data, so that might be very bad for folks on the lower tiers.
- skeledrew 2y agoWithout that "free lunch", the incumbents will still likely have the advantage in perpetuity. There will be fewer competitors as the cost will be a non-starter. Unless backed by some deep-pocketed entity that doesn't mind the possibility of losing a lot. The tech is already proven, and there's not enough value in the incremental improvements that'd make a new entrant sustainable. So even if the pipes were reopened, behind walls there won't be much demand.
- 6510 2y agoThe only interesting part is that it will become possible to remove a book without leaving an empty spot on the shelve. Eager to create "safety" the wikipedia-ish discussion where no two truths can exist simultaneously will happen in private and the official narrative of everything however preposterous will be the only one. We should also create LLM driven content moderation so that wrong think can be silenced immediately. The masses of brainwashed slaves will no doubt learn to like it.
- deleted 2y ago[deleted]
- bastien2 2y ago.
- wslh 2y agoAlthough LLMs are different, politically speaking Google et al have been harvesting our data and controlling our focus for decades now. I find strange that the topic was not discussed enough in the past. Seems like "The Groundhog Day" movie.
- globalnode 2y agoThere will be a new law which states that copyrights don't have effect if the data is being used to train machine learning models (except in the EU perhaps? heh).
- 1024core 2y ago> The data that powers AI is disappearing fast .... says the article behind a paywall.
- adultSwim 2y agoThe data is already incorporated into existing models. Those models will generate derivative data used to train the next models. They only needed to rip it all off once, after that it's actually preferable to not let anyone do it again.
- cozzyd 2y agoIs it easy to recognize the AI content stealers by user agent? Can we just feed them garbage when detected?
- Havoc 2y ago> consent I mean the entire industry is based on doing stuff without seeking consent. All the major players seem to have used at some point at least the books dataset so def no regard for consent or copyright. Bigger issue is mechanical limits like Twitter and Reddit - walled gardens of info. That could entrench existing players, whether via money (pay me for data), ethics (oh now suddenly consent matters) or just timing (more stuff mechanically restricted)
- rldjbpin 2y agowhile "data is the new oil" might be so late 2000s and early 10s, it is finally coming to fruition in public discourse after people realized that one can dump them into a model and get something you can work with. having worked in this topic, creating synthetic data is not always easy, and you still need real data to get the best results. if you look beyond basic internet media, there are tons of fields where "real data" does not exist in quantities that could help create effective models. these might be good case studies on how to proceed further in small endeavours.
- lofaszvanitt 2y agoYes, this is the key point: "Changing the license on the data doesn’t retroactively revoke that permission, and the primary impact is on later-arriving actors, who are typically either smaller start-ups or researchers." And also this: "Mr. Longpre said that one of the big takeaways from the study is that we need new tools to give website owners more precise ways to control the use of their data." This will happen once everything is looted/not needed anymore.
- 1vuio0pswjnm7 2y agoWorks where archive.ph is blocked: https://web.archive.org/web/20240720013944if_/https://archive.ph/EEQQB https://web.archive.org/web/20240720013944if_/https://archiv... This is how I read NYT now: tnftp -4o"|yy093" https://www.nytimes.com/2024/07/19/technology/ai-data-restrictions.html > 1.htm links 1.htm Using HTTP/1.1 pipelining, i.e., yy025 and tcpclient instead of tnftp, I retrieve many articles in a single HTTP request. yy093 outputs a single HTML page with all the fulltext articles, the way I like it. 100% Javascript-free. The web keeps getting better for text-only. More JSON, less HTML.
- pas 2y agowhat's yy025 and yy093? (google only finds random Chinese(?) pages.) and ... woah, may I ask where your username came from (does it mean something)? edit: okay, it seems those are scripts written by you, right? https://news.ycombinator.com/item?id=41035155 https://news.ycombinator.com/item?id=41035155 okay, WTF is this https://news.ycombinator.com/item?id=40894851 https://news.ycombinator.com/item?id=40894851 ? :D
- 63stack 2y agoTempleOS guy borne again