8 ms·

DO we have to expect _that_ level of understanding from the agent, though? If my wife said that to me, I may have a good chance of queuing up the song she has i

by pmx 3y ago

DO we have to expect _that_ level of understanding from the agent, though? If my wife said that to me, I may have a good chance of queuing up the song she has in mind, but anyone else? No chance. I don't expect tools like this to be able to understand cryptic requests and always come to the right answer. I'm happy if I can request a song or an action, or anything else in the same way i might ask another human who doesn't know me intimately.

swexbe 3y ago
If not, how is this more useful than something like Siri?
- famouswaffles 3y ago
  Natural language understanding. Siri doesn't get context at all. You can twist unstructed data or requests however you like and the LLM will deal with it just fine. "Play my favorite" is just a knowledge problem. If GPT fails there, it's because it doesn't know your favorite, not because it can't parse the request or understand what you need it to do. You have to speak certain ways to Siri to get it to do things. Unless specifically hard-coded, Siri will never receive "damn I'm finding it hard to read" as input as decide to turn on the lights. GPT will.
  - troupo 3y ago
    "Siri doesn't get context at all." and yet immediately "GPT fails there, it's because it doesn't know your favorite" "Knowing your favorite" is the context. > Unless specifically hard-coded, Siri will never receive "damn I'm finding it hard to read" as input as decide to turn on the lights. GPT will. Of course it won't. You have to very specifically fine tune it to understand what light conditions are, where you are in the house, and what it is you need to turn on.
    - circuit10 3y ago
      They mean context in the sentence or that or that be inferred from “common sense” and without any specific knowledge, I think
      - troupo 3y ago
        I hate Siri as much as anyone, but Chat GPT has no context in the "common sense" either. The sibling comment literally says "I had to provide a long-ish sentence as a context/programming instructions before it could do anything". https://news.ycombinator.com/item?id=37464563 https://news.ycombinator.com/item?id=37464563
        circuit10 3y ago
        You expect it to know things like that the user wants it to act as a home assistant without being told? That’s not common sense, that’s mind reading
        troupo 3y ago
        No, I expect people to stop pretending that LLMs somehow know context unlike the stupid Siri. There's considerably more in that prompt besides just "you need to act like a home assistant"
        omniglottal 3y ago
        There was insufficient context. Imagine I tell you "turn on that light, where I'm pointing". You'd do no better. No one here is under the conviction magical prescience is involved. This tooling provides the mechanism for an initial API call to be tied to the event described, in natural language, as "look where I'm pointing". The first response (to ask for clarification) is precisely what a human agent would do to get context to clarify the coarse-grained request. The second guess, assuming you disabled the (explicit) allowance for clarifying questions, is also a magnificent recognition of implicit, common-sense context. Seems it's even more effective than you at following the true context for this tools appropriate placement.
    - regularfry 3y ago
      > You have to very specifically fine tune it to understand what light conditions are, where you are in the house, and what it is you need to turn on. Where you are in the house and what needs to turn on, at least, is an API query job, not a fine-tuning job. As far as whether it can understand the relevance of lighting to the situation, I just asked ChatGPT 3.5 the question 'Acting as an AI home assistant, if you hear me say "I'm finding it hard to read", what actions would you take?' and 'Adjust the lighting' was the second option it gave back (after 'ask for clarification'). I think we're there, honestly, we just don't have the different parts connected yet.
      - troupo 3y ago
        > Where you are in the house and what needs to turn on, at least, is an API query job, not a fine-tuning job. And that API magically comes form where? > I just asked ChatGPT 3.5 the question 'Acting as an AI home assistant, if you hear me say "I'm finding it hard to read", what actions would you take?' So, basically: - you had to pre-program Chat GPT to act as a home assistant - you had to provide it with specific context and specific phrasing for it - it still failed, asked for clarification, and only then responded And now you have to this song and dance every time you want to coax GPT into doing what you need (and that's what RestGPT does).
        regularfry 3y ago
        > And that API magically comes form where? HomeAssistant, or any number of other providers. Do you think this part is somehow difficult? > you had to pre-program Chat GPT to act as a home assistant That is what we call "a prompt". It is a well-known technique. I am surprised that this should look strange to you. > you had to provide it with specific context and specific phrasing for it That is what we call "a prompt". It is a well-known technique. I am surprised that this should look strange to you. > it still failed, asked for clarification, and only then responded You have misunderstood. In its list of actions to take, the first and only response it gave, the first thing it said it would do in context is ask for clarification as to why I was finding it hard to read. That seems entirely reasonable to me. Does it not to you? > And now you have to this song and dance every time you want to coax GPT into doing what you need (and that's what RestGPT does). So what? It's not something the person sat in the dark ever has to care about.
    - famouswaffles 3y ago
      >You have to very specifically fine tune it to understand what light conditions are, where you are in the house, and what it is you need to turn on. No, you don't.
  - jakderrida 3y ago
    Human: "damn I'm finding it hard to read" GPT w/ memory: "Because you still have dyslexia."
    - ftkftk 3y ago
      Thanks for the chuckle.
eternityforest 3y ago
Why would we want this at all if it doesn't know you that well? Current voice assistants without AI can already handle songs and actions like that. Seems like it's largely solved.

by pmx 3y ago

swexbe 3y ago

If not, how is this more useful than something like Siri?

famouswaffles 3y ago
Natural language understanding. Siri doesn't get context at all. You can twist unstructed data or requests however you like and the LLM will deal with it just fine. "Play my favorite" is just a knowledge problem. If GPT fails there, it's because it doesn't know your favorite, not because it can't parse the request or understand what you need it to do. You have to speak certain ways to Siri to get it to do things. Unless specifically hard-coded, Siri will never receive "damn I'm finding it hard to read" as input as decide to turn on the lights. GPT will.
- troupo 3y ago
  "Siri doesn't get context at all." and yet immediately "GPT fails there, it's because it doesn't know your favorite" "Knowing your favorite" is the context. > Unless specifically hard-coded, Siri will never receive "damn I'm finding it hard to read" as input as decide to turn on the lights. GPT will. Of course it won't. You have to very specifically fine tune it to understand what light conditions are, where you are in the house, and what it is you need to turn on.
  - circuit10 3y ago
    They mean context in the sentence or that or that be inferred from “common sense” and without any specific knowledge, I think
    - troupo 3y ago
      I hate Siri as much as anyone, but Chat GPT has no context in the "common sense" either. The sibling comment literally says "I had to provide a long-ish sentence as a context/programming instructions before it could do anything". https://news.ycombinator.com/item?id=37464563 https://news.ycombinator.com/item?id=37464563
      - circuit10 3y ago
        You expect it to know things like that the user wants it to act as a home assistant without being told? That’s not common sense, that’s mind reading
        troupo 3y ago
        No, I expect people to stop pretending that LLMs somehow know context unlike the stupid Siri. There's considerably more in that prompt besides just "you need to act like a home assistant"
        omniglottal 3y ago
        There was insufficient context. Imagine I tell you "turn on that light, where I'm pointing". You'd do no better. No one here is under the conviction magical prescience is involved. This tooling provides the mechanism for an initial API call to be tied to the event described, in natural language, as "look where I'm pointing". The first response (to ask for clarification) is precisely what a human agent would do to get context to clarify the coarse-grained request. The second guess, assuming you disabled the (explicit) allowance for clarifying questions, is also a magnificent recognition of implicit, common-sense context. Seems it's even more effective than you at following the true context for this tools appropriate placement.
  - regularfry 3y ago
    > You have to very specifically fine tune it to understand what light conditions are, where you are in the house, and what it is you need to turn on. Where you are in the house and what needs to turn on, at least, is an API query job, not a fine-tuning job. As far as whether it can understand the relevance of lighting to the situation, I just asked ChatGPT 3.5 the question 'Acting as an AI home assistant, if you hear me say "I'm finding it hard to read", what actions would you take?' and 'Adjust the lighting' was the second option it gave back (after 'ask for clarification'). I think we're there, honestly, we just don't have the different parts connected yet.
    - troupo 3y ago
      > Where you are in the house and what needs to turn on, at least, is an API query job, not a fine-tuning job. And that API magically comes form where? > I just asked ChatGPT 3.5 the question 'Acting as an AI home assistant, if you hear me say "I'm finding it hard to read", what actions would you take?' So, basically: - you had to pre-program Chat GPT to act as a home assistant - you had to provide it with specific context and specific phrasing for it - it still failed, asked for clarification, and only then responded And now you have to this song and dance every time you want to coax GPT into doing what you need (and that's what RestGPT does).
      - regularfry 3y ago
        > And that API magically comes form where? HomeAssistant, or any number of other providers. Do you think this part is somehow difficult? > you had to pre-program Chat GPT to act as a home assistant That is what we call "a prompt". It is a well-known technique. I am surprised that this should look strange to you. > you had to provide it with specific context and specific phrasing for it That is what we call "a prompt". It is a well-known technique. I am surprised that this should look strange to you. > it still failed, asked for clarification, and only then responded You have misunderstood. In its list of actions to take, the first and only response it gave, the first thing it said it would do in context is ask for clarification as to why I was finding it hard to read. That seems entirely reasonable to me. Does it not to you? > And now you have to this song and dance every time you want to coax GPT into doing what you need (and that's what RestGPT does). So what? It's not something the person sat in the dark ever has to care about.
  - famouswaffles 3y ago
    >You have to very specifically fine tune it to understand what light conditions are, where you are in the house, and what it is you need to turn on. No, you don't.
- jakderrida 3y ago
  Human: "damn I'm finding it hard to read" GPT w/ memory: "Because you still have dyslexia."
  - ftkftk 3y ago
    Thanks for the chuckle.

eternityforest 3y ago

Why would we want this at all if it doesn't know you that well? Current voice assistants without AI can already handle songs and actions like that. Seems like it's largely solved.