4 ms·
Being able to do many things at a level below what trained humans can isn't what any reasonable person would call superhuman performance. If machine translation
by simen 8y ago
Being able to do many things at a level below what trained humans can isn't what any reasonable person would call superhuman performance. If machine translation could perform at the level of human translators at even one pair of languages (like English-Mandarin), that would be impressive. That would be the standard people apply. But they very cleary can't.
- AndrewKemendo 8y agoIt's a problem with who you're comparing against. Generally people think superhuman = better than the best humans. I understand this and it's an obvious choice, but it assumes that humans are measured on a objective scale of quality for a task, which is rarely the case. Being on the front line of deploying ML systems, I think it's the wrong way to measure it. I think Superhuman should be considered relative to the competence level of average person who has the average amount of training on the task. This is because from the "business decision" level, if I am evaluating between hiring a human with a few months or a year of training and a tensorflow docker container that is reliably good/bad, then I am going to pick the container every time. That's what is relevant today - and the container will get better.
- simen 8y ago> Generally people think superhuman = better than the best humans. Because that's what it means. Also, I shudder to think of the prose coming out of a company that decides to "hire" an AI instead of a translator...
- AndrewKemendo 8y agoWell not explicitly or in any measurable terms [1]. The term 'Superhuman' lacks technical depth in the sense of measurement. So for the purposes of measuring systems we build vs human capability, it's a pretty terrible measure. [1] http://www.dictionary.com/browse/superhuman http://www.dictionary.com/browse/superhuman