6 ms·
How Clara Labs (YC S14) Is Using Humans to Build AI
- jgalt212 10y agoVery similar to x.ai, although x.ai seems to be currently constrained to one problem domain (meeting scheduling). However, I'd bet they're looking to move be beyond just this. https://x.ai/ https://x.ai/
- giardini 10y agoNavigating to https://claralabs.com/ https://claralabs.com/ freezes my Linux system - a reboot is necessary. Is this a SPAM blog post?
- jasonlaska 10y agoWe've posted a follow up on the humans behind Clara here: https://news.ycombinator.com/item?id=12074657 https://news.ycombinator.com/item?id=12074657
- PameVls 10y agoThis is SO interesting! I am surprised by how companies are trying to solve AI problems... Machines normally take you 80-90% there, and using humans for that 10-20% left is an amazing idea. It's also a great monetization model for many companies.!!! Also, Clara is awesome :)
- jasonlaska 10y agoWe think it's a good idea too ;) Machines and humans truly have different talents: machines are great at memory, keeping track of state, distributing information while people are great at understanding subtle nuance in natural language.
- visarga 10y agoAnd in another 3-5 years, AI will start understanding subtle nuances too (probably) judging by the number of papers in computer language understanding.
- jasonlaska 10y agoEven modern methods today (deep recurrent networks, etc.) can do pretty well with these kinds of tasks (a very large ontology for instance) if you have enough annotations of the nuance!
- YeGoblynQueenne 10y ago>> Even modern methods today (deep recurrent networks, etc.) can do pretty well with these kinds of tasks (a very large ontology for instance) if you have enough annotations of the nuance! Where do you get that from? NLP, with neural networks or not, stays as safely away from meaning as is humanely possible while working in an area very closely connected to it. Also, ontologies? Very few people are interested in that nowadays, though that includes the team that made Watson. The push is instead to do away with all that and rely on statistical approximation.
- YeGoblynQueenne 10y agoUnfortunately "machines" (AI software, really) are really bad at nuance, subtle or blatant. The number of papers in the field is a very good example of why: until you've read (enough of) them you have no idea what the state of the field is. Edit: By this I mean that your assessment about the 3-5 years to "subtle nuance" is extremely, unrealistically optimistic. We're nowhere near AI understanding language. Try 300 to 500 years and you might be closer.
- pmiller2 10y agoNot only that, but there are a lot of domains where you don't need to be terribly accurate in the first place, because existing methods are so terrible. A great example I encountered recently is identifying people who may be interested in switching jobs soon. How much do you think recruiters and companies would pay to increase their pool of interested candidates 5-10%?
- kandalf 10y agoTo me this seems like the right approach - either you start with some massive amount of data that's not quite adapted to the problem (think Google), you start with full automation and have to basically write the decision tree yourself, or you generate the appropriate labeled training data like this. However, it seems like there are some scale issues if you start upmarket like Clara Labs has been. I wonder if there's benefit in having a cheaper more mass-market version as well that can be used to generate larger amounts of data and test algorithms better?
- jasonlaska 10y agoIt's certainly possible. One advantage of our setup is that rather than getting ok -to- noisy labels from customers, our CRAs understand the end-goal of the application and generate pretty great data. We are also able to incentivize them to produce fewer errors.
- intenex 10y agoWould love to integrate the ML and context aspects of Clara into our existing assistant team. Curious if that's something that's doable, or if it's just a package deal of their trained operators + AI. But yeah, basically awesome and everything we'd need. Being able to go even more in-house and specialize our personal assistant and equip them with augmented capabilities here would be even more awesome.
- maxhm10 10y agoThis is AWESOME. Human+Machine is always going to be better and stronger than machine alone. Keep it up! I wonder how this can impact the amount of people that can participate in crowdsourced projects.
- kespindler 10y agoReminds me of the writeup a few days ago about Google's machine emotional intelligence for email suggested replys - except Clara is doing it (and more) with a team the fraction of the size. The business solution here is very clever. As an engineer, my instinct is to fully automate any system. Clara is a great example where the most effective answer is getting the 90% solution, then going the last mile with humans to bring it up to 99% (making that up) accuracy, with most of the same savings. Imagine similar solutions for customer support (especially over website chat bars. Some systems like this already exist.)
- jasonlaska 10y agoHi! I'm the author of the post and happy to answer any specific questions here.
- kayhi 10y agoHow much human tagging time does it take to process 1000 meeting requests or more general how does the annotation side work? Do you have people overlap items since there can be a fatigue issue as you mentioned in the post?
- jasonlaska 10y agoClara has a 1-hr SLA for the processing of an incoming message. While I cannot give numbers on the speed of annotators (or volume), I can say that our platform is designed to enable quick and accurate work via incentive mechanisms. We avoid fatigue in part by making it easy for CRAs to navigate and work with data. Wrt to overlapping annotator schemes, these are known to be effective. We'll be writing more about how our human backend works in future posts.
- kayhi 10y agoThanks, in the future post would also be interested in the trade offs between building internal system vs. using a 3rd party (maybe you do?) like mechanical turk and the incentive structure.
- jasonlaska 10y agoOur follow up on the humans behind is here: https://news.ycombinator.com/item?id=12074657 https://news.ycombinator.com/item?id=12074657
- jasonlaska 10y agoThis is noted, thanks!
- yeukhon 10y agoSide note: your website's Javascript is broken. "i" is undefined.
- dreeves 10y agoRelated: http://messymatters.com/ai-plus-ui/ http://messymatters.com/ai-plus-ui/ About how AI often needs a good UI to handle the last few percent of cases.
- dreeves 10y agoRelated: http://messymatters.com/ai-plus-ui/ http://messymatters.com/ai-plus-ui/ About how AI often needs a good UI to handle the last few percent of cases.
- sakabaro 10y agoThat seems very cool and modern. I don't want to sound like nit-picking, but choosing Clara as a company name seems a bit misogynist.
- Hydraulix989 10y agoContemporary ML is really (and by that I mean human-level) good for some things: classifying images, guiding a car along prominently visible lane lines in perfect conditions, predicting timeseries that have latent patterns. It's NOT good at the general "bot" problem (or should I say "true AI" problem), and some "features" of human assistants like EQ, generalized task completion, and complex planning are decades away from getting solved. I would even argue that a mechanized/automated Clara would HAVE to pass the Turing test in order to work.
- jasonlaska 10y agoYour last comment on the seemingly circular relationship between Clara and a "turing-test-passing bot" is especially salient. This reference (found in the second footnote in the post) http://www.ijcai.org/Proceedings/95-1/Papers/125.pdf http://www.ijcai.org/Proceedings/95-1/Papers/125.pdf has a lot of interesting perspectives on the role of machine learning vs. human intelligence.
- Hydraulix989 10y agoI am going to read that actually, thanks. On the note of bots and your denouncement of the Turing Test as a metric, I do remember an article on here a couple of days ago hinting that one of the biggest frustrations of customer service calls is dealing with scripted responses, bots if you will, instead of real humans. I do wonder whether alleviating this user frustration will amount to passing some "similar" but possibly weaker test.
- jasonlaska 10y ago"The customer service test" -- I like it.
- harshnisar 10y agoRelevant: https://techcrunch.com/2016/06/07/crowdflower-series-d/ https://techcrunch.com/2016/06/07/crowdflower-series-d/