6 ms·

The required knowledge is around entity recognition, with "Obama" referring to the 44th POTUS, and not somebody else who happens to have the same surname (and t

The required knowledge is around entity recognition, with "Obama" referring to the 44th POTUS, and not somebody else who happens to have the same surname (and there are multiple of them actually, at least 4 given his family).

Gigachad 4y ago
Fair, although I’d think it acceptable if the response to your prompt was “which one? I found 4”
- squeaky-clean 4y ago
  It would have way more than 4 people in the search results though. GP said there's at least 4 because there's him, his wife and his kids. Even knowing Obama is a person is a knowledge-based leap. (To us humans) it's obvious the question means Barack Obama because he's the most notable subject for that name. But how do you prevent your AI from responding that the "Obama JS library is 5 years old" https://github.com/rgbkrk/obama https://github.com/rgbkrk/obama
  - pixl97 4y ago
    If it's based on frequency of training data, the ex-president will have far more hits in the training corpus. Now, on less talked about topics it doesn't sound any different than what happens with people Q "How old is Tim?" A "Which Tim are you talking about, you didn't give me crap to work with?"
    - xeromal 4y ago
      The answer is obviously 62 because that's how old Tim Apple is.
      - mensetmanusman 4y ago
        Ha, it might give an age of 7 years or so when the moniker was invented.
- lordnacho 4y ago
  Isn't it a sort of cultural knowledge to understand what is meant? Ie when we say Obama we mean one specific guy who is very important but if I'm talking about Mrs Watanabe it's a generic Japanese person?
elcomet 4y ago
This can clearly be guessed from a search as well. Popularity can be well defined, and in the case of Obama, there is clearly one much more popular than the others.
- PartiallyTyped 4y ago
  The model still needs to infer from the sentence the entity to look it up. It is also the case that this is a relatively simple example as 'Obama' refers to a single class of entities and there is not a lot of ambiguity around resolution of class, only resolution of specific entity. Take this sentence: > When was KitKat released? I could refer to the sweet, or the Android OS. Vastly different classes, and the model here needs to "decide" to ask for more information to disambiguate the class, and if the class is the sweet, then it needs to disambiguate the taste particular flavour possibly, and even ask the geographic location.
  - thfuran 4y ago
    Yes, but the amount of knowledge necessary to decide how to make those sorts of decisions is far smaller than the amount of knowledge necessary to answer all such questions.
  - elcomet 4y ago
    And that's perfectly fine. Humans have exactly the same problem. They will get this wrong, and you will reply "no, I'm talking about the android version". Language is ambiguous so we cannot expect machines to get it right all the time.
    - PartiallyTyped 4y ago
      I do agree with you that it is fine, what I was getting at was that there needs to be a way to measure uncertainty in a manner that is robust to unbalanced distributions or context drifting.
      - elcomet 4y ago

Gigachad 4y ago

Fair, although I’d think it acceptable if the response to your prompt was “which one? I found 4”

squeaky-clean 4y ago
It would have way more than 4 people in the search results though. GP said there's at least 4 because there's him, his wife and his kids. Even knowing Obama is a person is a knowledge-based leap. (To us humans) it's obvious the question means Barack Obama because he's the most notable subject for that name. But how do you prevent your AI from responding that the "Obama JS library is 5 years old" https://github.com/rgbkrk/obama https://github.com/rgbkrk/obama
- pixl97 4y ago
  If it's based on frequency of training data, the ex-president will have far more hits in the training corpus. Now, on less talked about topics it doesn't sound any different than what happens with people Q "How old is Tim?" A "Which Tim are you talking about, you didn't give me crap to work with?"
  - xeromal 4y ago
    The answer is obviously 62 because that's how old Tim Apple is.
    - mensetmanusman 4y ago
      Ha, it might give an age of 7 years or so when the moniker was invented.
lordnacho 4y ago
Isn't it a sort of cultural knowledge to understand what is meant? Ie when we say Obama we mean one specific guy who is very important but if I'm talking about Mrs Watanabe it's a generic Japanese person?

elcomet 4y ago

This can clearly be guessed from a search as well. Popularity can be well defined, and in the case of Obama, there is clearly one much more popular than the others.

PartiallyTyped 4y ago
The model still needs to infer from the sentence the entity to look it up. It is also the case that this is a relatively simple example as 'Obama' refers to a single class of entities and there is not a lot of ambiguity around resolution of class, only resolution of specific entity. Take this sentence: > When was KitKat released? I could refer to the sweet, or the Android OS. Vastly different classes, and the model here needs to "decide" to ask for more information to disambiguate the class, and if the class is the sweet, then it needs to disambiguate the taste particular flavour possibly, and even ask the geographic location.
- thfuran 4y ago
  Yes, but the amount of knowledge necessary to decide how to make those sorts of decisions is far smaller than the amount of knowledge necessary to answer all such questions.
- elcomet 4y ago
  And that's perfectly fine. Humans have exactly the same problem. They will get this wrong, and you will reply "no, I'm talking about the android version". Language is ambiguous so we cannot expect machines to get it right all the time.
  - PartiallyTyped 4y ago
    I do agree with you that it is fine, what I was getting at was that there needs to be a way to measure uncertainty in a manner that is robust to unbalanced distributions or context drifting.
    - elcomet 4y ago