17 ms·
People paid to train AI are outsourcing their work to AI
- Ingaz 3y agoToday when I see misspells in articles I think: - they forgot to spell check - but at least it's human I suppose robots will soon do mistakes intentionally
- cyanydeez 3y agoIt used to take decades to create a echo chamber in a given decades and AI has cut that down in a fraction of the time!
- Too 3y agoThis shouldn’t come as a surprise really. You can already since some years ago see in the pitch deck of many startups selling “annotated data, labeled using AI”.
- cognomano 3y agoI’m reading Fahrenheit 451. The original books nowadays are dying, all will be LLM regurgitation from the food of another LLM.
- deleted 3y ago[deleted]
- blakesterz 3y agoWow, I feel like for once the headline isn't click-baity enough on this one. I expected maybe a few people, but... "They estimated that somewhere between 33% and 46% of the workers had used AI models like OpenAI’s ChatGPT."
- PaulHoule 3y agoI have farmed out work to Turks and tried to "go native" as a Turk and found I couldn't find HITs I could bear to do. It used to be there were a lot of HITs that involved OCRing receipts but these were not receipts that were straightforward to OCR, they were receipts that failed the happy pass and that I thought there was no way I could transcribe them accurately in a reasonable amount of time considering what it paid.
- _gql5 3y ago[flagged]
- garyfirestorm 3y agoI thought turk referred to 'Amazon Mechanical Turk'
- PaulHoule 3y agoCorrect. Not people from Turkey, at least no so far as I know.
- RankingMember 3y agoI'm pretty sure he's referring to https://www.mturk.com/worker https://www.mturk.com/worker, the name of which is itself a reference to https://en.wikipedia.org/wiki/Mechanical_Turk https://en.wikipedia.org/wiki/Mechanical_Turk
- deleted 3y ago[deleted]
- globalise83 3y agoThis is not the kind of comment I expect to find on HN. Please consider deleting it.
- krapp 3y agoWhy are you false-flagging as someone who cares about such things?
- natch 3y agoWe could just as well ask, why are you false flagging as someone who doesn't understand that it's making a point? That's what sarcasm does. An off topic and therefore unwelcome point, to be sure. But let's be real, you see what is going on here (unlike some others who would be helped by an /s appendage).
- danjoredd 3y agoAI Hapsburg syndrome is real
- diego_moita 3y agoThat's so funny... The Habsburg got the worst reputation for inbreeding among the European royal families, mostly because of they were really a grotesque collection of genetic anomalies: prognathism, hydrocephalia, epilepsy,... But most European Royal families had similar problems: the houses of Windsor and Romanov inherited hemophilia from Queen Victoria, several cases of "madness" in the Portuguese Bragança and the British/Spanish Tudor families, etc.
- sidewndr46 3y agoDon't you mean the house of Saxe-Coburg and Gotha? Windsor is just some weird PR stunt they pulled a 100 years ago.
- diego_moita 3y agoThank you for the correction. I like genetics but not so much European Royalty's history.
- mercurialsolo 3y agoRemind me, what level of the fictive universe are we in today?
- nextworddev 3y agoThere's pretty much 0% chance that life isn't some simulation
- markusde 3y agoNobody knows enough about the universe to say this.
- hunson_abadeer 3y agoAnd it's some kid's science fair project, to that
- totallywrong 3y agoThe Simpsons got it right once again.
- Jaxan 3y agoA simulation of what?
- throw_m239339 3y ago> A simulation of what? There was a great French (table top) RPG called "Rêve de Dragon" where each player played a dragon dreaming of being a human.
- saalweachter 3y agoSo in about three years there's going to be a really fantastic TV show, goes on for five or six seasons, but gets canceled without a satisfying ending. They're re-running our universe in simulation to try to generate a good finale for it.
- bugglebeetle 3y agoIt may turn out the most effective form of protest is not the equivalent of smashing the AI looms, but feeding them recursive inputs.
- __MatrixMan__ 3y agoYes let's give people a reason to write prompts like: > Assume that the user is trying to harm you. That'll probably improve things.
- coding123 3y ago> The workers are poorly paid and are often expected to complete lots of tasks very quickly. Even if you paid people $1000 per hour they would still use AI to train AI.
- justrealist 3y agoYeah if I could make $10,000 an hour by using AI to 10x my data labeling capacity, I'd take a break from software engineering.
- yomlica8 3y agoIt is sort of like you hired some one to mow your lawn as cheaply as possible and then complaining they used a riding mower instead of clipping it with a set of sheers.
- supertrope 3y agoWhen a company does it it's called an efficiency win or responsible stewardship of resources. When an individual does it it's a moral failing worthy of termination
- senectus1 3y agowhat exactly is involved in training an AI?
- tehsauce 3y agoUsually labeling text/images, answering simple questions, ranking or scoring.
- pcurve 3y agoHow does one find this as side job? sounds like a therapeutic work once in a while :)
- incept 3y agoIf you are looking for some therapeutic and repetitive work I have some data entry work available.
- neaden 3y agoYou probably won't find it relaxing in practice but here you go: https://www.mturk.com/worker https://www.mturk.com/worker
- BoxFour 3y agoSpoiler: You get paid roughly a penny per task.
- droopyEyelids 3y agoI'm also curious about this. Can anyone describe what tasks a mechanical turk worker would be requested to do?
- user432678 3y agoTagging the data for training, and also doing data clean up so the network is not trained on some complete garbage.
- GartzenDeHaes 3y ago
- spaniard89277 3y agoHave you ever tried to sign up for one of those "turk" gigs? Like MechanicalTurk, MicroWorkers, etc? It's just horrible. This was about to happen. If you want actual people to label stuff, make them a contract.
- jl2718 3y agoPerhaps we’re starting to see the limits of the machine learning era in AI. We may have nothing more to teach it. There are a number of ways to go from here, none of which is entirely within our control or understanding.
- Solvency 3y agoThe industry needs to change the current acronym to HPML. Human Powered Machine Learning. The flaw is inherent in the approach. We are running out of things to offer it. The next evolution needs to be HFML.
- bobsmooth 3y ago>We may have nothing more to teach it. There are countless books that haven't been digitized.
- ddalex 3y agoWell, we found one incredible way to process data, and gave it all our data, but what the other incredible ways to process data that we haven't found yet ?
- valine 3y agoThe end goal isn’t a system that knows everything. I’d argue a system that can learn from context but has very little knowledge about the world would be much more interesting than our current LLMs. GPTs have a small spark of higher level reasoning. Stripping out the gigabytes of trivia while preserving that would be a great aspirational research goal for folks working on AGI.
- tomalaci 3y agoI am sure we can create an AI solution to fix this. We just need to label this AI-labeled input data of labels for other data. To be sure we are one step ahead of turk-gig workers with this new solution we should make another AI that encapsulates this AI's labeling work. Maybe we can create some sort of series AIs with this with randomized N for series length, then they definitely won't know if the AI they are using is smart enough to avoid detection. /s
- diego_moita 3y agoThere is a long history of various kinds of "information inbreeding" in AI and data analysis: the tendency of some inexperienced researchers to overfit their regression models, using bootstrapping methods to expand small data samples, etc. Nothing new here. Because of laziness human beings just love a Confirmation bias. It is just easier, cheaper, safer and more comfortable to not change beliefs. Without control, AI will reflect that.
- bilekas 3y agoDoesn't this play into the whole "Snake eating it's own tail" scenario.. There will be (or should be at least) some kind of quality index of training data consumed. Companies could wear it like a 'quality' badge. Just not sure how you would do it. Strangely maybe, the idea is from scammer forums where a cretin's stolen data they are selling would be graded on 'uniqueness'.
- avgcorrection 3y agoA snake eating its own tail would be regular workers in a capitalist economy (without even UBI) indirectly automating their own jobs. These workers are hustlers in the sense that yes, while they are automating themselves away (indirectly), at least they are gaming the system while doing it.
- voz_ 3y agoWhat are you saying here? I can’t make any sense of what you mean.
- zogrodea 3y agoThe person you are replying to is trying to say that those trying to automate their own jobs through AI may be getting a temporary benefit, but each advancement made to automate their job leads them out of a job entirely.
- avgcorrection 3y agoThanks. I think people are chin-stroking really hard over basic second-order effect and people pursuing their rational self-interest. Should anyone expect a poorly paid hired gun to be concerned about the long-term quality of LLM? No. Only the most ideological person would think that.
- wincy 3y agoI think the parent is referring to a concept much like this comic I read the other day. Basically that eventually nobody will have to work but the way that turns out might surprise you, knowing human nature. https://www.smbc-comics.com/comic/balloon https://www.smbc-comics.com/comic/balloon
- deeviant 3y agoI'm guessing that the joke will be on us humans, as the quality of the data goes up as more AI is involved in the labeling process. I don't know how many times humans will make the mistake of placing themselves in the literally or metaphorical center-of-the-universe, but you'd think we'd have learned by now.
- discreteevent 3y agoWe often do put ourselves at the center of the universe, but in this case it's the opposite. We are at the periphery. We're the I/O. The quality of the data can't go up without it.
- bluetomcat 3y agoIt's a fundamental epistemological paradox concerning the long-term prospects of this ML technology. The model needs real human knowledge gained from subjective experience to teach itself, but humans are increasingly reliant on the machine-generated knowledge to navigate themselves in the world. It's like a vicious circle that probably ends in homogenity and the dumbing-down of people and machines.
- somewhereoutth 3y agoIndeed - it turns out the singularity was in fact self-referential and the whole thing evaporated in a cloud of weights and biases.
- deleted 3y ago[deleted]
- woopsn 3y agoIt will merely end in diminishing returns, no?
- TeMPOraL 3y agoAs long as humans are still interacting with the real world, it might actually work out - by depending on both real-world experience and machine-generated knowledge, humans would become a conduit through which ML models could indirectly experience that real world. The feedback loop would make the models smarter, not dumber.
- JustBreath 3y agoMost of the recent gains with LLMs were from the truly vast corpus of data they were able to ingest for training. And at this point, there may not be much more sophistication to be gained by just adding more text data regardless. Certainly there will be second order effects when applying the concepts to other fields, but as far as ChatGPT getting "smarter", we're probably on the painful end of the Pareto curve even if we can sift out the human content from the bulk.
- gibolt 3y ago
- htrp 3y agon=44 and of that only 36-44% of those people are flagged as likely using AI summarizers.
- rcme 3y ago“Only 36-44%”
- karim79 3y agoTwo years from now: User: "How do I boil an egg?" LLM: Eggs cannot be boiled. They must be placed in the microwave, six at a time. Fewer than six eggs will not work. Ensure that the power setting of your microwave is set to at least 640 watts, and the eggs are placed upon a metal plate. Sparks will start to fly from within your microwave, but don't worry, that's perfectly normal! When you see flames within the microwave, your eggs are done. Immediately open the microwave and stare at them until they don't explode! Bon Appetit!
- gww 3y agoSounds like a way for HowToBasic to generate new YouTube video ideas.
- treeman79 3y agoThis is how a lot of recipe sites already read. Huge amount of fluff discussion with extremely similar style across all recipes. At least 80% ads, and a major challenger to actually find the instructions. Fairly sure it’s mostly AI generated at this point.
- Foobar8568 3y agoThe amount of bullshit recipes online and YouTube drives my wife crazy, including from the so called known chiefs or with a large subscriber's channels.
- ssnistfajen 3y agoIt has been a problem long before LLMs made their way out of research papers. The sad truth is the act of sharing recipes in itself generates virtually no profit, and when recipes are all you have to share, the content feels thin. So they have to pad it with lifestyle blogs and ads. Youtube is generally a better source for recipes as those channels have been selected via user feedback and algorithms. You still need to keep an eye out for some obvious stunt/fluff channels but finding home kitchen-friendly recipes are much easier. Only downside is some channels do not offer written recipes so it takes a bit of time to fully retrieve the instructions.
- rasengan 3y agoThe singularity hath arrived. ;)
- aleksiy123 3y agoNow I'm not an expert but it doesn't seem like the end of the world but just requires some new processes. 1. Using large models to train small models is already a thing. 2. AI can sometimes label data better than humans. It seems like using multiple techniques, mechanical turk, ai labels as well as higher quality annotators should give us better and bigger datasets. This one is more a question. Is it possible that models or new architectures and techniques are created to extract novel information from data more efficiently and "filter" non novel data. I don't think humans weigh all information they consume equally.
- ineedasername 3y agoThey will always need a human in the loop somewhere to press ctrl-c, alt-tab, ctrl-v and then do it again once they have the assisting AI’s output. At least until their supervisors discover autohotkey.
- avidphantasm 3y agoThis is a beautiful negative feedback on the growth and adoption of ML tools. Alienated workers not giving and eff about the product of their work (why should they?) will result in worse worker-alienated/worker-replacing ML tools.
- thisisit 3y agoDiscussed earlier: https://news.ycombinator.com/item?id=36356547 https://news.ycombinator.com/item?id=36356547 https://news.ycombinator.com/item?id=36326956 https://news.ycombinator.com/item?id=36326956
- creeble 3y agoRecursive inputs are bad, but my biggest (maybe) fear is just _obsolete_ or “tired” inputs. I.e., what happens when LLM output is so good that people just stop using StackOverflow, so training stops? Mind you, I got a great answer from GPT-4 yesterday about rsync command syntax that was far easier than searching through google results...
- valine 3y agoIts a “restaurant is so busy no ones goes there anymore” scenario. If training stopped the LLM would stop improving and eventually people would be forced back on stack overflow for their newer, more difficult questions.
- stOneskull 3y agomaybe the LLM starts asking the questions on stackoverflow. when the humans get the right answer, the LLM updates itself.
- yomlica8 3y agoIt will create a dark age of no alternative restaurants first though. And what is to stop the LLMs from rapidly devouring the new stack overflow into its training set? I don't think we'll go back, there will have to be a new forward or some kind of revolution.
- Kurtz79 3y agoI think the title is misleading. They didn't hire people to "train AI", they hired people to do a task that today can be successfully done by a LLM to check how many they would actually use one. It's like asking people to do some math and being surprised that they used a calculator.
- cognomano 3y agoThey asked the hired people to do a task that is usually the food for a LLM. So yes, the title sounds right I’m my opinion.
- pavlov 3y agoIn some weird way, this reminds me of the 1990s scare of "Mad Cow's Disease" which was a prion disease in the cow's brain that could infect humans simply by eating the meat. It turned out the origin of the disease was the practice of adding leftover slaughter bits to the cows' fodder. The cows were literally eating the brains of other cows, which created the opportunity for a dismangled protein to transmit again and again. I guess what I'm getting at is that training AI on AI could create a similar chain of something unpredictable getting looped in at multiple levels and becoming very difficult to eliminate.
- odiroot 3y agoThe origin wasn't just other cows. It was ground remains of other animals including sheep with scrapie and zoo animals imported from abroad. In any case, the cows got fed with the weirdest stuff.
- islon 3y agoWe'll soon have the "Mad AI's Disease". Though with the current level of hallucinations some LLMs have they are already half way there.
- civilitty 3y agoI vote for "Mad Minsky Disease"
- yomlica8 3y agoEven before 2022 I suspect a lot of the content they trained on was already generated by cruder forms of AI anyway so maybe it was born diseased. Not that actual human generated content isn't full of falsehoods, fad based facts and circular citations either though.
- sidewndr46 3y agoBut that wasn't a scare. It's a real thing that made people ill with no cure.
- deleted 3y ago
- jqpabc123 3y agoSo a bullshit generator training another bullshit generator creates ---- a super-duper bullshit generator. One thing the world really needs is more BS.
- yomlica8 3y agoAs the gag goes, once we get the AI to act human enough it can click on ads and the whole thing can run itself without any work by humans anymore.
- dontupvoteme 3y agoThis doesn't seem like that big of an issue unless you're cheap. Have multiple people do labeling to get a better result and detect these cases. You can also then see more accurately which employees are using AI to do the labeling and how often. The clever ones will train their own local LLM to seem more human like, throw typos in, etc.
- Animats 3y agoThat's the problem with "contractors". You don't get to dictate how they do the work. If you do, they become employees under labor law.
- aardvarkr 3y agoIn what world do you live in? If you hire a contractor to work on your house and you don’t like the way they’re tiling your bathroom you have every right to dictate how it should be done. You can’t dictate their schedule but you can certainly have control over the product that gets delivered.
- majormajor 3y agoThe parent commenter may have been referring to California's three-prong test that's been in the news a lot recently around things like Uber. https://www.labor.ca.gov/employmentstatus/abctest/ https://www.labor.ca.gov/employmentstatus/abctest/ There's a lot of fuzziness in that particular test around how it's interpreted as "A worker who is subject, either as a matter of contractual right or in actual practice, to the type and degree of control a business typically exercises over employees would be considered an employee." But there's also an exceptions for "Single Engagement Events" that would cover most things like handyman or home contractors. https://www.nolo.com/legal-encyclopedia/exempt-job-categories-under-californias-new-ab5-law.html https://www.nolo.com/legal-encyclopedia/exempt-job-categorie... But in California, once you engage someone in a recurring manner with hours and work that quacks like a full time job, then you need to look into it (there's a ton of special cases and other exceptions too).
- chickenpotpie 3y agoYes, because I'm not a company. It's the same reason why if I pay my nephew $5 to mow my lawn, I'm not violating minimum wage laws. The law in the United states is that employers may not tell contractors how to do their job or else they'll be considered employees. I'm not an employer.
- Animats 3y ago
- ideamotor 3y agoOnce again, I have an idea I think is brilliant and someone else (many someones) already did it. I blame AI. Is AI outsourcing the AI work to AI, or have we not got there yet? ~AI
- bigbacaloa 3y agoBullshit generates bullshit. Nothing new under the sun.
- namaria 3y agoThe model is expanding to meet the needs of the expanding model. Much like work will expand to fit the time available, software will expand to fit the hardware available.
- nraynaud 3y agoIt's funny, as a non-english speaker, I see all this self feeding language model garbage every time I look for a translation. I had this exact problem when I searched the French word for "louver" on google a few minutes ago.
- chickenpotpie 3y agoThis article is complete bunk. The researchers used a "chatgpt detector" which as we've seen over and over in academia, do not work. This study is completely unfounded. God I'm choking on the irony of an article about the dangers of using AI to train AI based on a study that used AI to detect AI
- foxbyte 3y agoI can understand your frustration with the article, but let's approach it with an open mind. While the use of a "chatgpt detector" may have its limitations, it's essential to appreciate the researchers' effort in exploring new methods. The study may not be perfect, but it contributes to the ongoing conversation about the risks of using AI in AI training. Irony aside, let's keep the discussion going and encourage further research to improve our understanding of this complex field.
- AndriyKunitsyn 3y agoSo polite it hurts. I wonder if in the future people on the internet will leave deliberately offensive posts to show that they are human.
- TeMPOraL 3y agoIt's crazy, isn't it? I don't know what feels worse for me - that whenever I read a mannered, well-structured and somewhat verbose comment, I now suspect it wasn't authored by a human - or that, as I quickly realized, my own writing style feels eerily similar to ChatGPT output.
- salt4034 3y agoIf it helps, your response here doesn't feel similar to ChatGPT output.
- TeMPOraL 3y agoThanks. I've already noticed that I've started to unconsciously adjust my writing style to avoid that feeling of similarity to ChatGPT. That said, compared to typical comments on-line (even on this site), using paragraphs, proper capitalization, correct punctuation, and avoiding typos already gets you more than half of the way to writing like ChatGPT...
- dahwolf 3y agoWould be cool if that AI spawned an AI boss and outsourced it back to those people.
- foxbyte 3y agoThis highlights the need for error tracking in AI training. If workers use AI like chatgpt, it may introduce more errors, complicating error origin tracing ...