3 ms·

Projects like these highlight that as ML becomes more and more complex, a gap grows larger and larger which must be filled by manual or semi-manual labor. If it

Projects like these highlight that as ML becomes more and more complex, a gap grows larger and larger which must be filled by manual or semi-manual labor. If it isn't a team of volunteers combing the rendered results for errors, it's users who have succumbed to those errors and leave feedback in the hopes the system improves. That gets me wondering - is the future of AI really just a semi-autonomous twilight zone where cheap / free labor augments an already faulty system? If not, what possible application is there for an expensive and closed automated system which works 100% requiring no human input, when other options are cheaper and leave clear directions for improvement?

mikepurvis 7y ago
Surely there's a spectrum here? Conventional, manually-made maps are full of errors too, and there's no army of low-paid workers combing over those— nope, they're published as they are, and it's up to end users to report problems. The better the map maker is (whether it's an ML black box or a team of human experts), the fewer errors there are, and either way, one expects an upward quality trend over time.
tschwimmer 7y ago
Yes and no. There is a huge explosion of 'Ops Plus' startups that take an existing manual process and then build some basic tooling around it (with or without any substantial ML component). There are mild to moderate efficiency gains coming from this, but a lot of their valuation is coming from a bet that in the future they'll be able to fully automate the system and reap efficiency gains. In practice, almost nobody is even thinking about building a fully automated process for every case. The reason is simple: automating the first 60% of work takes x effort, automating the next 30% takes 10x, and automating the next 9% takes 100x and operating the final 1% is essentially impossible. So if you came to the table with the goal of 100% automation right out the gate you'd spent 10 years developing something with little to show. I think full automation of some systems is possible, but is actually blocked by generational norms. By and large the systems that "Ops Plus" startups are attempting to automate were designed by people who are not digital natives. They're not illiterate, but things like instant messaging, async communication and and structured data are not natural primitives for them. I'm not saying everyone in the Fortnite generation is a master data modeller, but I think that when they join the workforce they'll set up systems that are much more feasible to automate.
- nicolaskruchten 7y ago
  I’m interested in learning more about this “Ops Plus” term but it doesn’t seem very google friendly... Is this a term of art in some circles?
  - tschwimmer 7y ago
    I actually just made it up. Companies that come to mind are Scale.ai (doing data labelling stuff for ML that historically would have been done by outsourcing companies, etc), Flexport (freight forwarding, traditionally done via spreadsheets and emails), Checkr (background checks), Atrium (legal services), Oscar/Clover (health insurance), Cadre (real estate investing). There are tons and tons of them in the recruiting (really sourcing) space too: I'd say Triplebyte is an Ops Plus company, as is Sourceress and a couple more.
- iguy 7y ago
  What you describe is pretty much exactly parallel to the automation of physical work. Having a steam engine gets you a long way, then the first few robots a bit further... there's no particular reason to want a factory with zero employees (except the watchman and his dog...) unless you want it to work on Mars or something.
tomrod 7y ago
> If it isn't a team of volunteers combing the rendered results for errors, it's users who have succumbed to those errors and leave feedback in the hopes the system improves. This is the idea behind GANs, as it stands.

mikepurvis 7y ago

Surely there's a spectrum here? Conventional, manually-made maps are full of errors too, and there's no army of low-paid workers combing over those— nope, they're published as they are, and it's up to end users to report problems. The better the map maker is (whether it's an ML black box or a team of human experts), the fewer errors there are, and either way, one expects an upward quality trend over time.

tschwimmer 7y ago

Yes and no. There is a huge explosion of 'Ops Plus' startups that take an existing manual process and then build some basic tooling around it (with or without any substantial ML component). There are mild to moderate efficiency gains coming from this, but a lot of their valuation is coming from a bet that in the future they'll be able to fully automate the system and reap efficiency gains. In practice, almost nobody is even thinking about building a fully automated process for every case. The reason is simple: automating the first 60% of work takes x effort, automating the next 30% takes 10x, and automating the next 9% takes 100x and operating the final 1% is essentially impossible. So if you came to the table with the goal of 100% automation right out the gate you'd spent 10 years developing something with little to show. I think full automation of some systems is possible, but is actually blocked by generational norms. By and large the systems that "Ops Plus" startups are attempting to automate were designed by people who are not digital natives. They're not illiterate, but things like instant messaging, async communication and and structured data are not natural primitives for them. I'm not saying everyone in the Fortnite generation is a master data modeller, but I think that when they join the workforce they'll set up systems that are much more feasible to automate.

nicolaskruchten 7y ago
I’m interested in learning more about this “Ops Plus” term but it doesn’t seem very google friendly... Is this a term of art in some circles?
- tschwimmer 7y ago
  I actually just made it up. Companies that come to mind are Scale.ai (doing data labelling stuff for ML that historically would have been done by outsourcing companies, etc), Flexport (freight forwarding, traditionally done via spreadsheets and emails), Checkr (background checks), Atrium (legal services), Oscar/Clover (health insurance), Cadre (real estate investing). There are tons and tons of them in the recruiting (really sourcing) space too: I'd say Triplebyte is an Ops Plus company, as is Sourceress and a couple more.
iguy 7y ago
What you describe is pretty much exactly parallel to the automation of physical work. Having a steam engine gets you a long way, then the first few robots a bit further... there's no particular reason to want a factory with zero employees (except the watchman and his dog...) unless you want it to work on Mars or something.

tomrod 7y ago

> If it isn't a team of volunteers combing the rendered results for errors, it's users who have succumbed to those errors and leave feedback in the hopes the system improves. This is the idea behind GANs, as it stands.