6 ms·
Mechanical Turk shutting down September 30
- jp0001 1mo agoNot surprised and I’d like to see the effects of LLMs on offshoring of jobs to India. I’ve seen efforts to reduce headcount there after the release of Opus-4.6.
- fetchmark 1mo ago[flagged]
- GNR_Radio 1mo ago[dead]
- x0xMaximus 1mo agoAs AMT's largest requester for the past 10 years, this news was relayed to requesters at the same time as respondents. It's also worth noting that our lead contact, the Sr Program Manager at AWS leading AMT, transitioned to Amazon Bedrock and SageMaker Model Evaluations a ~2-3 years ago.. Leaving behind essential zero team managing the project after they migrated over the stored value accounts to native AWS billing.
- jan_Inkepa 1mo agoour of curiosity, can you say what your use of it was?
- x0xMaximus 1mo agoInitially the annotation of biomedical literature (https://pubmed.ncbi.nlm.nih.gov/25592589/ https://pubmed.ncbi.nlm.nih.gov/25592589/) but left academia for commercial use where we transitioned to arbitrage the market research and ephemeral task markets. It was essentially a research panel with a underdeveloped UI for tasks.. so abstract away AMT's tooling so it can be used by any buyer in the ResTech, Political Polling and Data Annotation space My favorite part of AMT is always going to be figuring out that if we only paid in $0.07 intervals, their commission algorithm would round down to the nearest whole cent when their 20% commission resulted in a fractional cent on the unit transaction level, not the monthly invoice level. Was ultimately worth it to have implemented https://git.generalresearch.com/panels/amt-jb/tree/jb/flow/assignment_tasks.py#n527 https://git.generalresearch.com/panels/amt-jb/tree/jb/flow/a...
- loremm 1mo agoI truly have no idea what this website is saying https://generalresearch.com/ https://generalresearch.com/
- skinfaxi 1mo agoWhy it's Transactional Systems for Ephemeral Tasks, don't you know? No but same here. The other pages lead to more confusion.
- freehorse 1mo agoMy favourite part is the "our customers" section [0], starting with > Ripe with fraud, labor exploitation, political polling manipulation I am not sure if they want to convey what I think I am reading or not. [0] https://generalresearch.com/mission/#:~:text=Our%20Customers https://generalresearch.com/mission/#:~:text=Our%20Customers
- x0xMaximus 1mo agoBuyer A is presented with the following rate card: < 10min targeting gen pop: $3 10 <= 15min targeting gen pop: $5 Of course they'll bid saying their 14 min survey only takes 9 min to complete. Buyers try to cheat pricing strategies of exchanges just as much as respondents try to cheat buyers on survey platforms. Both parties can't be trusted and have adverse incentives.
- tourist2d 1mo agoIt smells like they don't know what they're doing. Had a lol at the mission page accidentally implying their customers are unethical though: > Our Customers > Ripe with fraud, labor exploitation, political polling manipulation; our customers demand the best tools and security ...
- DrewADesign 1mo agoAt least they’re ripe! I hate unripe customers, rife with astringency, sharp vegetal notes, and an unpleasant, unyielding texture.
- testbjjl 1mo agoSounds like your program manager was smart and valuable to the company. Do you have any plans to replace the service/data provided by MTurk?
- thorum 1mo agoPrior discussion from when the service stopped accepting new customers in July: https://news.ycombinator.com/item?id=48803886 https://news.ycombinator.com/item?id=48803886
- conception 1mo agoIt’s kind of crazy they’re shutting this down just when this Service probably has the most possibilities ever. You have an agent with Multiple people doing actual physical tasks in the real world seems like something that could be really powerful.
- dec0dedab0de 1mo agothat was my first thought too, but i guess llms don’t need an api to hire humans
- Hilliard_Ohiooo 1mo ago[dead]
- robot_jesus 1mo agoRight? Not to mention the training/tuning possibilities. Mercor has a market value of $20B basically doing the same thing but desperately trying to find workers.
- deleted 1mo ago[deleted]
- deleted 1mo ago[deleted]
- neom 1mo agoRentahuman is still around: https://news.ycombinator.com/item?id=46868675 https://news.ycombinator.com/item?id=46868675
- paxys 1mo agoThe concept behind it isn't going away. There are plenty of companies that hire people in bulk to do manual data labeling, transcription, RLHF, moderation and lots more. It's just that they are now catering to large AI companies, not regular people looking to get some repetitive work done (since that can now mostly be done by AI).
- 1mo ago
- OkWellBye 1mo ago[flagged]
- madrox 1mo agoMechanical Turk had a good run, but not surprised it's shutting down. I'm sure the platform was getting flooded with people doing task arbitrage and using lots of AI anyway. I believe the issue is that this can no longer be a horizontal play. MTurk was mostly for unskilled tasks...the kind AI can do well enough that it isn't worth the cost differential to verify it or keep farmed to humans. The "trust but verify" AI output is now the kind that requires domain expertise. This is what most full stack AI companies are bringing to industries. Curious if this kind of work will come around again one day or was just a moment in time. If it does I'm sure it will be specifically about generating training data.
- raverbashing 1mo ago"Fun" fact, they were also used by psychology/etc students when they need to do 'research' with X amount of people
- timcobb 1mo agoHow could it come around again?
- madrox 1mo agoI'm unsure. If it does, it will be work that is too expensive or inaccurate or regulatory for current AI methods. For example, you want a doctor to sign off on some AI output on a diagnosis. However, I'm not sure a single platform will be how it emerges
- Legend2440 1mo agoThere are like three dozen companies selling similar services for generating AI training data.
- charlieyu1 1mo agoGood data isn’t cheap and keeping it in-house gives you more control.
- 1mo ago
- cautiouscat 1mo agoI was literally wondering this morning if mturk was still around! I assumed a lot of the stuff could just be done with LLMs and OCI now.
- qarl2 1mo agoI wonder why.
- smalltorch 1mo agoI did this a long time ago and it actually paid for a few meals. They are terribly monotonous tasks. I can see why it's shutting down if its still the same thing. Lllms could probably do everything there without rotting out minds for basically pennies.
- Forgeties79 1mo agoInstead we pay to rot our minds lol
- aboardRat4 1mo agoEnd of an era.
- wileydragonfly 1mo agoI made a couple thousand dollars off it squeezing in tasks here and there between meetings. Amazon Payments were challenging to redeem as I recall. I did more or less write the bulk of someone’s doctoral thesis. Kept giving me a dollar to summarize the findings of various psychology papers. The most memorable was the one where people were put in a room and someone sprayed “liquid ass” on the wall. The subjects given no explanation experienced higher levels of anxiety than those that were told there was a sewage leak being repaired. One of the strangest dollars I ever made. My PS4 and game collection was spectacular.
- shortformblog 1mo agoSo, I have a story to share about Mechanical Turk that you might find interesting. I’ve shared it a couple of times before on Twitter and Bluesky, but I’ll share it again. Long story short: Mechanical Turk saved my bacon. Back in 2005, I was working a job at a small-town newspaper in a town I’d never lived before. I didn’t know anyone beyond the staff (as I was working layout rather than as a reporter), and I had gotten interested in the idea of doing Mechanical Turk for a few extra bucks. I thought there might be a formative scene of interested people doing this, so I started working on a blog for it. I briefly collaborated on it with another guy who put it on forum software because he didn’t know how to use, like Drupal. That blog was called Turking.com, and it seemed like it was going well for a bit. We even got a mention on the AWS website. But after about two or three months, it was clear the initial excitement around the idea (and the initial work) had died down. (The work picked up later, but in clearly different ways. I don’t think folks were really “excited” about it after that point.) If you want to get an idea of it, there was one capture on the Wayback Machine: https://web.archive.org/web/20051124231722/http://www.turking.com/mturk/ https://web.archive.org/web/20051124231722/http://www.turkin... The site had started to die out in part because my iBook suffered a catastrophic GPU failure and me, being a broke small-town newspaper employee, did not have the money to replace it. Plus, the community just hadn’t emerged like we had expected. But then I got an email out of the blue: Someone wanted to buy my domain, which I owned outright, but they didn’t want to say who. I got contacted by a broker, and the exchange took place over escrow. (The domain most assuredly was bought by Amazon and is managed by MarkMonitor.) I didn’t get a ton of money from the deal, but I did get enough to pay for a new laptop. I had some regrets about selling it (in part because I originally built the site with someone else), but I was in a bit of a dire financial situation which that proved to be the starting point for getting myself out of. That domain purchase was notably more than I ever made from clicking and classifying random pictures, that said.
- idiotsecant 1mo agoSo did the other person just get hosed when you did your 'ipo'?
- shortformblog 1mo ago
- xyst 1mo agobezos has determined LLM has replaced "on demand 24/7 workforce"
- dgellow 1mo ago(He’s not CEO anymore)
- mturk 1mo agoI'll still be around on Oct 1.
- rickcarlino 1mo agoWishing you the best of luck on the road ahead ;-)
- rcr-anti 1mo agoHad some absolutely bizarre results from their attempt to integrate mturk with Bedrock's 'ground truth' thing as of a few months ago. Threw simple mnist digits recognition at it, see what the quality, timing, and cost was. Figured mnist digits was at this point trivial. Spent $10 and the accuracy was marginally better than guessing, completely unusable results. Was completely baffled, people were publishing peer reviewed research based on exclusively mturk results.
- willmeyers 1mo agoI wouldn't be surprised if Fiverr and Upwork start pulling back too. There's just so much AI bots now that have flooded the task marketplace.
- mips_avatar 1mo agoSurprised Amazon managed to fumble mechanical turk at the same time Mercor/Scale and all these other companies started hiring humans to do data labeling tasks.
- wanderingstan 1mo agoI used it to have people transcribe my dad’s handwritten letters and journals. It was touching to get notes from a “Turk” saying how much she enjoyed his travels and following the cast of characters in his life!
- forinti 1mo agoI recently found a letter written by a great-aunt of mine hidden inside a book. It is written in German, which I know a little, but her handwriting was too difficult for me. So I searched around and found this site: https://www.transkribus.org/handwriting-ocr And it managed to extract the text! Anyway, it turns out the wheat harvest was very good in 1937 and thank you for the letters and newspapers.
- deleted 1mo ago[deleted]
- shermantanktop 1mo agoThe tagline for MTurk used to be “artificial artificial intelligence.” It appears now we can get along with just a single “artificial.”
- inigyou 1mo agoNo "intelligence" required
- retr0rocket 1mo ago[dead]
- transitorykris 1mo agoThe only time I ever used it (as a turk?) was when Jim Gray went missing at sea and satellite imagery of vast regions of the pacific were fed through Mechanical Turk. Seems a trivial problem now, but was not then, and needed humans to take a look.
- PrimeAli 1mo ago[flagged]
- ghaff 1mo agoAbsent visibility into AWS P&Ls--even notwithstanding AI--Mechanical Turk has been a real outlier at AWS for a number of years now, so not really surprising.
- winterbourne 1mo agoI was just thinking about the good old days combining human judgements on mturk for classifying high-value forum threads. Since replaced by using judgements from multiple LLMs. The only use I can think of it these days is for social scientists who need to gather judgements from bona fide humans. This will be devastating for a lot of workers in low-infrastructure countries.
- somepleb 1mo agoReminds me of Expensify's usage of Amazon Mechanical Turk. https://arstechnica.com/information-technology/2017/11/expensify-acknowledges-potential-privacy-problem-by-calling-it-a-feature/ https://arstechnica.com/information-technology/2017/11/expen...
- utopiah 1mo agoFWIW I tried it shortly after it launched so a bit after ~2005, maybe 2007 because I read about and honestly found the prospect, from a dataset creation or improvement, quite interesting. I think I heard about it from a research paper that distributed its work that way. Well it did work, in the sense that I managed to complete some tasks. I don't think I ever collected the money/credits back then though (simply because it amounted to so little). What I can attest though is that... it was debilitating. If you think your office job is boring then splitting it in way smaller tasks where you have no autonomy is absolutely terrible from a worker standpoint. I initially was hoping to use it in order to work on providing a service over a dataset but understanding first hand what it takes to make it happen made me stop. It radically changed how I saw supervised learning since, and sadly not in a good way.
- cmiles8 1mo agoAmazon / AWS went from boasting about how they never killed off products to just going on a steady binge of shutting things down. That wouldn’t necessarily be bad if they were replacing this with better stuff, but most of the new launches also seem on a path to eventual culling too.
- GeoBounties 1mo agogreat news for us at geobounties! we allow ai agents to deploy humans for real world tasks. a true mechanical turk! : ) now your ai assistant can even get your coffee for you! (sorry Peggy Olson!)
- pulkas 1mo agomission completed
- abhaynayar 1mo agoInteresting. I remember reading the book "Life 3.0" years ago, where one of the first things the fictional AGI does to make money is perform tasks on Amazon Mechanical Turk while pretending to be human.
- tempfile 1mo agoEvil business, glad to be rid of it.
- logotype 1mo agoMTurk existed since forever, and somehow flew under the radar for all these years. Very few developers I know didn't even know it existed.
- HlessClaudesman 1mo agoCut off one mechanical turk and two more will rise to take it's place.
- dubeye 1mo agoI've used Mechanical Turk in the early days but developed my own version for my vertical to manage freelancers. The problem as I see it is verifying the work is not done by AI. If it was possible to verify somehow that the work is definitely not done by AI I think there would still be a market for this. But the data just cannot be trusted. An increasingly big part of my job is to try and weed out workers who use AI and it's not an easy task at scale because generally workers gain trust with manual work and then there is degradation over time.
- inemesitaffia 1mo agoOutsourcing shops? Proctoring? Locked down devices?
- dubeye 1mo agoI suppose it depends on the value of the individual task. I can't imagine it's really realistic to lock down devices of mechanical Turk workers in such a way that ai use can be monitored. In any case Locking down devices isn't really pragmatic at scale for relatively low value tasks, and also introduces complexity about employment law where I am.
- donalhunt 1mo agoWorth noting that one of the most high profile uses of Mechanical Turk was in September 2007 to review satellite imagery in a massive crowdsourced effort to find missing record-setting aviator Steve Fossett. The effort failed to find any areas of interest and the missing aviator was found the following year by a hiker. I wonder if there was any analysis after the fact to understand if the imagery actually provided any hints regarding the eventual crash site.
- nlkingthree 1mo agoI wonder whether AI would have found the aviator.
- 21asdffdsa12 1mo agoCIA should be good on that- find traces by man in wild nature..
- walrus01 1mo agofreely available satellite/aerial images in 2007 were in no way high resolution enough to identify such a tiny airplane, and in many places, they still aren't.
- yard2010 1mo ago> Amazon's search effort was shut down the week of October 29, without any measurable success. Major Cynthia Ryan later said it had been more of a hindrance than a help. She said that persons purporting to have seen the aircraft on the Mechanical Turk or have special knowledge clogged her email during critical days of the search, and for even months afterward. Many of the ostensible sightings proved to be images of CAP aircraft flying search grids, or simply mistaken artifacts of old images. Psychics flooded the search base in Minden with predictions of where the aviator could be found. One man from Canada was particularly persistent with daily calls to Ryan. Ryan noted that every message, letter, or phone call was taken seriously, which swamped the USAF specialists assigned the task of reviewing every one of them without regard to apparent plausibility. In retrospect, the crowdsource effort was "not ready for prime time", according to Ryan From Wikipedia
- zackmorris 1mo agoOf course the moment AI can be used to earn a residual income from Mechanical Turk, they shut it down. Incentives work both ways. If the exchange rate of human labor to capital falls to zero, then human workers will have no choice but to exclude capital from the equation. No jobs, no money, no billionaires.
- runamuck 1mo agoFunny story... an incredibly famous rapper got his start because his manager, Shane Morris used Mechanical Turk to get engagement on Spotify before Spotify began to detect that vector.
- 1970-01-01 1mo agoI don't see this as anything but good news about AI progress, yet I can't help but think there will be spins about "AI taking away thousands more tech jobs"
- nerevarthelame 1mo agoA lot of the early tasks posted to Mechanical Turk by Amazon to get people using their new platform were easily automated. You could basically choose the "none of the above" option 100% of the time and end up with a quality score above their threshold. Instead of partying in college, my friends and I wrote a script to complete the tasks. We took over university computer labs to run it on a bunch of different computers. We made a few thousand bucks. Good times.
- ealready_value 1mo agoAbout 8 years ago, we used mturk for reading data out of public PDFs generated by a huge range of producers. I am not sure that LLMs would have been able to consistently extract this data until recently as a good number of these PDFs were scans, sometimes a scan of scan. We had pretty good luck in the end, but getting there produced a somewhat large app on our side that would manage the whole process, including but not limited to asking for multiple responses, comparing them to each other, finding consensus, and determining which users would consistently produce bad responses and stop them from responding. We got to a confidence that about 85-95% of the data was correct, which was good enough for the company. Through that process, I learned a couple things about managing mturk, primarily about how changes to the cost-per-task would change the process. Initially, we thought that price would be a quality knob, but quickly learned that price was a speed know. The higher the price the faster the tasks would be taken and completed. Quality did not change significantly as the price went up or down. Overall, I still have fondness to mturk, but it was really bare-bones experience that needed a lot of work to get working effectively.
- akshay_akula 1mo agoEnd of an era. Funny timing too, right when agents plus humans doing real world tasks would have made it interesting again.
- indianwashlets 1mo ago[flagged]
- ge96 1mo agoThat's crazy I remember doing work on there a long time ago. It was sus work like manipulating Google SEO rank by looking up and clicking links. But still it was real money even if it was pennies. I think the weirdest thing I had to do was scan through social media images... so weird seeing into random people's lives. I did it in the late 2000s or early 2010s.
- GavinAnderegg 1mo agoIt's telling that the copyright notice at the bottom of the page hasn't been updated since 2018.
- bentt 1mo agoWent from training AI to people hooking AI up to it. Slop as a service.