3 ms·

You need filter assisted decoding to make that possible in all cases (as in, working for both models all the time provably) I'm surprised it worked in chatGPT

You need filter assisted decoding to make that possible in all cases (as in, working for both models all the time provably) I'm surprised it worked in chatGPT this time, but it won't work all the time due to lack of lexical, semantic, or phonetic info in the tokenizer. https://paperswithcode.com/paper/most-language-models-can-be-poets-too-an-ai https://paperswithcode.com/paper/most-language-models-can-be...

cubefox 4y ago
It works in GPT-4 (but not GPT-3.5) even without a special tokenizer: Some problems simply solve themselves with increasing scale. But Bard uses a smaller LaMDA model to reduce inference cost. Quote [1] from Sundar Pichai in February: > We’re releasing it initially with our lightweight model version of LaMDA. This much smaller model requires significantly less computing power, enabling us to scale to more users, allowing for more feedback. [1] https://blog.google/technology/ai/bard-google-ai-search-updates/amp/ https://blog.google/technology/ai/bard-google-ai-search-upda...
- Der_Einzige 4y ago
  I don't think it works reliably over 100 invokations of the prompt. The paper tries to get provable situations where 0 ignored constraints happen/are possible. I'm impressed that it is working more often than not, but we need provable guarantees.
  - zeven7 4y ago
    If you ask 100 humans the same thing, you'll get some mistakes too. I don't think the point is whether it makes mistakes. The Chat GPT response shown above is clearly more interesting and impressive / displays something more seemingly like intelligence than the Bard response provided. At some point scaling these things up enough becomes indistinguishable from intelligence, and at that point maybe it is.
    - KajMagnus 4y ago
      > If you ask 100 humans the same thing, you'll get some mistakes too Yes, still, it's pretty easy to find a somewhat good judgement human who never does that mistake. If you tell him/her: "This is of utmost importance: ..." — But as of now, these text gen machine learning programs don't totally understand that?

cubefox 4y ago

It works in GPT-4 (but not GPT-3.5) even without a special tokenizer: Some problems simply solve themselves with increasing scale. But Bard uses a smaller LaMDA model to reduce inference cost. Quote [1] from Sundar Pichai in February: > We’re releasing it initially with our lightweight model version of LaMDA. This much smaller model requires significantly less computing power, enabling us to scale to more users, allowing for more feedback. [1] https://blog.google/technology/ai/bard-google-ai-search-updates/amp/ https://blog.google/technology/ai/bard-google-ai-search-upda...

Der_Einzige 4y ago
I don't think it works reliably over 100 invokations of the prompt. The paper tries to get provable situations where 0 ignored constraints happen/are possible. I'm impressed that it is working more often than not, but we need provable guarantees.
- zeven7 4y ago
  If you ask 100 humans the same thing, you'll get some mistakes too. I don't think the point is whether it makes mistakes. The Chat GPT response shown above is clearly more interesting and impressive / displays something more seemingly like intelligence than the Bard response provided. At some point scaling these things up enough becomes indistinguishable from intelligence, and at that point maybe it is.
  - KajMagnus 4y ago
    > If you ask 100 humans the same thing, you'll get some mistakes too Yes, still, it's pretty easy to find a somewhat good judgement human who never does that mistake. If you tell him/her: "This is of utmost importance: ..." — But as of now, these text gen machine learning programs don't totally understand that?