4 ms·

> We like to say that Watson > is our marketing department. Did you forget a /s there? For me the marketing around Watson, with the unrealistic and not scient

> We like to say that Watson > is our marketing department. Did you forget a /s there? For me the marketing around Watson, with the unrealistic and not scientifically backed assumptions about the state of AI they put out to the general public, epitomize what's wrong with how the world sees our field. More on topic: the underlying problem with the original post here seems to be one of selection bias in the training data. Somewhat of a remnant of human decision making that, likely unintentionally, ended up in creating biased decisions in the end. While the system you linked seems to potentially create nicely decomposable decisions, is there anything that inherently prevents "bias" in what it learns? The approach seems to face many of the same problems modern ML systems in the Linked Data / Semantic Web space would given unrestricted learning from the web.

YeGoblynQueenne 8y ago
>> Did you forget a /s there? For me the marketing around Watson, with the unrealistic and not scientifically backed assumptions about the state of AI they put out to the general public, epitomize what's wrong with how the world sees our field. IBM's marketing Watson as a medical application is one thing. The system itself is quite another. And the system itself remains the most advanced NLP system created so far. Note that I say "system". Most NLP research consists of testing the performance of various algorithms against very specific benchmarks, but very little work goes towards creating a unified system that can integrate multiple NLP abilities. Watson on the other hand is exactly such a unified system. It integrates statistical machine learning, symbolic reasoning (frames, fer chrissake, frames! In this day and age!), pattern-matching (with Prolog) and so on and so forth. Far as I can tell there isn't anything like it anywhere - but of course, I don't know much about what Google, Facebook et al are doing internally.
- tastroder 8y ago
  Sure, I specifically meant the marketing though. I kind of get why they do it but I'm not sold on the idea that it's helpful to present AI as an infallible silver bullet. In terms of systems I'm not sure why IBM would stand out, Microsoft, Amazon and Google all offer relatively coherent pipelines for the engineering side of NLP and all of those are backed up by accomplished research teams. I'm sure you can find everything from total horror stories to beautiful testimonials for all of these platforms.
  - YeGoblynQueenne 8y ago
    Unfortunately it's difficult to tell what Google etc are doing with their AI systems, but I get the feeling that Google and Facebook in particular have a culture of focusing on end-to-end statistical machine learning to the exclusion of everything else. I base this intuition on the way google translate is trained end-to-end and without any attempt to integrate grammatical or other background knowledge (which causes it to make mistakes that a rule-based system would easily avoid, e.g. getting the gender of nouns wrong when translating between two languages that have gendered nouns via English, that doesn't, etc). I think these two companies in particular would be very unwilling to design and build a system like Watson, integrating symbolic techniques alongside statistical ones. They probably recruit so much for statistical machine learning skills that they don't have the know-how to do it anyway. The funny thing is that, like many large corporations, they probably have ad-hoc expert systems except they don't call them that. Pretty much any sufficiently large and complex system that encodes "business rules" is essentially an expert system. But, because "expert systems failed" companies and engineers will not use the knowledge that came out of expert system research to make their systems better. That AI winder really got us good.
_bxg1 8y ago
The joke may not have translated the way I wrote it; IBM makes promises about Watson's intelligence that it can't live up to, and we've had clients in the past who've come to us because we can do (some of) what Watson advertised but couldn't deliver. The main way our system would combat bias is by shedding light on it. No human-built system could be completely impartial, but ours will say "I decided X because of A, B, C, and D", and if people decide that C is biased then that piece of knowledge can be adjusted accordingly.
- tastroder 8y ago
  That makes more sense than what I read into it, apologies for that (and glad to hear that since I've had similar discussions). Thanks for clarifying.

YeGoblynQueenne 8y ago

>> Did you forget a /s there? For me the marketing around Watson, with the unrealistic and not scientifically backed assumptions about the state of AI they put out to the general public, epitomize what's wrong with how the world sees our field. IBM's marketing Watson as a medical application is one thing. The system itself is quite another. And the system itself remains the most advanced NLP system created so far. Note that I say "system". Most NLP research consists of testing the performance of various algorithms against very specific benchmarks, but very little work goes towards creating a unified system that can integrate multiple NLP abilities. Watson on the other hand is exactly such a unified system. It integrates statistical machine learning, symbolic reasoning (frames, fer chrissake, frames! In this day and age!), pattern-matching (with Prolog) and so on and so forth. Far as I can tell there isn't anything like it anywhere - but of course, I don't know much about what Google, Facebook et al are doing internally.

tastroder 8y ago
Sure, I specifically meant the marketing though. I kind of get why they do it but I'm not sold on the idea that it's helpful to present AI as an infallible silver bullet. In terms of systems I'm not sure why IBM would stand out, Microsoft, Amazon and Google all offer relatively coherent pipelines for the engineering side of NLP and all of those are backed up by accomplished research teams. I'm sure you can find everything from total horror stories to beautiful testimonials for all of these platforms.
- YeGoblynQueenne 8y ago
  Unfortunately it's difficult to tell what Google etc are doing with their AI systems, but I get the feeling that Google and Facebook in particular have a culture of focusing on end-to-end statistical machine learning to the exclusion of everything else. I base this intuition on the way google translate is trained end-to-end and without any attempt to integrate grammatical or other background knowledge (which causes it to make mistakes that a rule-based system would easily avoid, e.g. getting the gender of nouns wrong when translating between two languages that have gendered nouns via English, that doesn't, etc). I think these two companies in particular would be very unwilling to design and build a system like Watson, integrating symbolic techniques alongside statistical ones. They probably recruit so much for statistical machine learning skills that they don't have the know-how to do it anyway. The funny thing is that, like many large corporations, they probably have ad-hoc expert systems except they don't call them that. Pretty much any sufficiently large and complex system that encodes "business rules" is essentially an expert system. But, because "expert systems failed" companies and engineers will not use the knowledge that came out of expert system research to make their systems better. That AI winder really got us good.

_bxg1 8y ago

The joke may not have translated the way I wrote it; IBM makes promises about Watson's intelligence that it can't live up to, and we've had clients in the past who've come to us because we can do (some of) what Watson advertised but couldn't deliver. The main way our system would combat bias is by shedding light on it. No human-built system could be completely impartial, but ours will say "I decided X because of A, B, C, and D", and if people decide that C is biased then that piece of knowledge can be adjusted accordingly.

tastroder 8y ago
That makes more sense than what I read into it, apologies for that (and glad to hear that since I've had similar discussions). Thanks for clarifying.