15 ms·
Analyzing the websites that are used to train AI chatbots
- 1vuio0pswjnm7 3y ago"Also high on the list: b-ok.org No. 190, a notorious market for pirated e-books that has since been seized by the U.S. Justice Department. At least 27 other sites identified by the U.S. government as markets for piracy and counterfeits were present in the data set." There is copying for personal or academic use and then there is copying for commercial purposes. What if the later is done without permission from the copyright owner. The incessant promotion of so-called "AI" sure seems commercial in nature. "Tech" companies rarely spam us so incessantly without some desperate scheme to make money, for example data collection about computer users, persistent surveillance and annoying, inefficient online advertising.
- cynicalsecurity 3y ago> 1 And, of course, put me in a team that uses Blub, and I’ll pick up Blub in a heartbeat. Except php. I tried to give php an honest chance recently (“It has changed”, they said, “It is much better with modern practices”) but it was painful all the way through, even when I tried to do everything right. With this level of the author's expertise, you can throw this article right in the garbage bin.
- dougb5 3y agoAs the article acknowledges, the training data that goes into the chatbots that people actually use in 2023 -- ChatGPT (both the GPT-3.5 and GPT-4 flavors) -- are probably very different from c4, which was released 4 years ago. GPT-4's training data is probably a similarly big chunk of the public Web plus a bunch of licensed content. But we can't know for sure because OpenAI has chosen not to disclose any information about the people that they take content from to build their service.
- mdrzn 3y agohttps://archive.ph/kNEyR https://archive.ph/kNEyR
- Entinel 3y agoI am shocked that Discord is not on this list. I would bet my mortgage that Discord would sell their chats as a dataset to train on if the price was right.
- EGreg 3y agoI often get downvoted for suggesting this, but why not simply host all your chats, videoconferencing etc. on your own website? Make it a gated community and only post little teasers to the likes of Instagram and YouTube and TikTok and Twitter and Facebook. Let them come to YOUR site to participate and abide by YOUR terms. Why make a Facebook Group or a Discord server? Why do so many of us, including the biggest influencers in the world, spend so much money only to post all of our content on the Big Tech’s sites and infrastructure? You give all your social capital to them too, and they turn around and sell it back to you (access to your own followers) and can deplatform you. They encourage you to pay them to attract eyeballs away from other videos, and vice versa, in a zero-sum game on their own platform. They now increasingly even control our conversations and our democracy. Why not have these conversations on our own forums, powered by our own software, the way we do on HN? We gave away all our social capital and our content, to these platforms. And then they turn around and train big data on it internally, and scrape it for their AIs. People have complained for years yet do nothing about it: https://readwrite.com/mark-cuban-facebooks-sponsored-posts-are-driving-away-brands/ https://readwrite.com/mark-cuban-facebooks-sponsored-posts-a... I ask again: why not simply host it ourselves?
- gomox 3y agoWhen ChatGPT first came out, I was surprised at the depth of information it had about region-specific market sizing in our (relatively niche) industry. Turns out the model had ingested a pay-to-read article that cost upwards of $2000 [0] and was quoting the figures in it and referencing it directly (i.e. attributing the info to the article in question). I knew about the paper but had never purchased it. I was actually surprised I could access the data through ChatGPT. A few days later the same information was gone. I assumed someone might have decided to keep it on the low regarding having ingested all of these sources of restricted access content. I now get a generic blurb about the same question. The opacity of it all made me a little worried. What similar models exist, trained on actually private information, for more nefarious purposes? [0] https://www.marketresearch.com/Infiniti-Research-Limited-v2680/ITSM-Latin-America-30367235/ https://www.marketresearch.com/Infiniti-Research-Limited-v26...
- jameshart 3y agoThe IT Services Market is not 'relatively niche'.
- gomox 3y agoImagine how your comment would come off if your guess was wrong. That's how it comes off.
- astrange 3y ago> A few days later the same information was gone. I assumed someone might have decided to keep it on the low regarding having ingested all of these sources of restricted access content. I now get a generic blurb about the same question. This didn't happen. There is no way to remove information from the model and retraining it costs millions of dollars.
- moneywoes 3y agoHow did they patch all the illegal stuff ChatGPT said at release?
- RugnirViking 3y agowhoa, they claimed deviantart is a text-to-image generator being sued by artists? is that really a thing? Last I heard Deviantart was really against it
- Nicksil 3y agohttps://news.ycombinator.com/item?id=33567202 https://news.ycombinator.com/item?id=33567202
- bufo 3y agoThis article is so misleading. First there’s nothing secret about all this. Also these chat bots don’t just “sound” smart, they are providing value to millions of users.
- dang 3y agoOK, but the article itself is (or at least may be?) more interesting than the misleading title. Let's focus on that instead. We've edited the title now. From https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html: "Please don't pick the most provocative thing in an article or post to complain about in the thread. Find something interesting to respond to instead."
- Turing_Machine 3y ago> First there’s nothing secret about all this. Right. It's going to come as a shock to exactly no one that (e.g.) Wikipedia was used as a source. Then there's the fear-mongering about using public databases (e.g., voter registration lists) in "unforeseen ways". While that may be true, they don't seem to be concerned about non-AI actors using these databases in "unforeseen ways". Pretty much pure clickbait.
- newaccount74 3y agoWhen asking ChatGPT about programming questions, it feels pretty similar to asking questions on stack overflow, except you get answers instantly. I assume that Q&A sites must be among the most important training data.
- ChatGTP 3y agoAlso gists and GitHub issues, they would’ve had near perfect data sets of everything they had permission to touch at GitHub. I saw something on Twitter, where someone was offered a job at OpenAI to write code for common coding problems and provide very verbose reasoning and comments. Can’t find it now..
- tyronehed 3y ago[dead]
- htss2013 3y agoIt states that if an LLM is good at the LSAT, it must have trained on many practice LSAT tests. How do we know this? The test measures things like reading comprehension and logical reasoning. LLMs have reasoning ability with novel material, and there are plenty of ways to train logical reasoning outside of LSAT test prep materials.
- archon1410 3y ago> more than 72,000 instances of “swastika,” one of the banned terms from the list. Isn't swastika being a common sacred symbol in Eastern and specially Dharmic religions common knowledge? This, alongwith most of the religious websites being Christian, Islamic and so on is quite concerning.
- pier25 3y agoIt's disheartening to see AI companies profiting from free content I've created online over years and years. StackOverflow, forums, Github, even my personal crappy website is there.
- geoduck14 3y agoCan we mess with ChatGPT by posting wrong answers on SO?
- ck_one 3y agoFor niche topics you probably can.
- UncleEntity 3y agoLucky for you all these companies have figured out how to directly profit from your free labor by charging for using the data to train AIs. Users provide free content, company sells free content, AI replaces users.