3 ms·

"Fake it until you make it" - very human behavior indeed. It's unfair to hold the AI to a bar we don't apply to ourselves.

brianpan 4y ago
I expect a calculator to multiply 2 numbers for me very quickly and correctly. Every time. I don't hold any human to that bar and that's perfectly ok.
- Spivak 4y ago
  Yeah, and you can have GPT do that quickly and accurately. You just have to give it access to a calculator just like you would for a human. Tell it not to compute anything and explain how to format the output that can be ingested by a calculator or interpreter. People have crazy unrealistic expectations of what the raw model can do but undersell what it’s possible to build with a machine that grasps language better than basically every human.
  - withinboredom 4y ago
    I don’t need a calculator to tell me 50 * 12 is 600. But you’re saying an AI does?
    - Spivak 4y ago
      For simple calculations no, for more complicated calculations yes. Trying to determine in advance if a Llm can do the math "in its head" is a task unto itself that pretty much boils down to sampling multiple responses and seeing if it produces different answers. So it's often just easier (and cheaper) to make it use a calculator for everything and not have to worry.
      - withinboredom 4y ago
        It tried to convince me the square root of two was negative one the other week… It was so sure of it and couldn’t be convinced otherwise. I doubt they’ll be able to convince it to use a calculator at all. It’s like arguing with a six year old child to put a coat on in the middle of winter…
        Spivak 4y ago
        Take a look at https://github.com/williamcotton/empirical-philosophy/blob/main/articles/from-prompt-alchemy-to-prompt-engineering-an-introduction-to-analytic-agumentation.md https://github.com/williamcotton/empirical-philosophy/blob/m... https://langchain.readthedocs.io/en/latest/ https://langchain.readthedocs.io/en/latest/ They can be taught!

brianpan 4y ago

I expect a calculator to multiply 2 numbers for me very quickly and correctly. Every time. I don't hold any human to that bar and that's perfectly ok.

Spivak 4y ago
Yeah, and you can have GPT do that quickly and accurately. You just have to give it access to a calculator just like you would for a human. Tell it not to compute anything and explain how to format the output that can be ingested by a calculator or interpreter. People have crazy unrealistic expectations of what the raw model can do but undersell what it’s possible to build with a machine that grasps language better than basically every human.
- withinboredom 4y ago
  I don’t need a calculator to tell me 50 * 12 is 600. But you’re saying an AI does?
  - Spivak 4y ago
    For simple calculations no, for more complicated calculations yes. Trying to determine in advance if a Llm can do the math "in its head" is a task unto itself that pretty much boils down to sampling multiple responses and seeing if it produces different answers. So it's often just easier (and cheaper) to make it use a calculator for everything and not have to worry.
    - withinboredom 4y ago
      It tried to convince me the square root of two was negative one the other week… It was so sure of it and couldn’t be convinced otherwise. I doubt they’ll be able to convince it to use a calculator at all. It’s like arguing with a six year old child to put a coat on in the middle of winter…
      - Spivak 4y ago
        Take a look at https://github.com/williamcotton/empirical-philosophy/blob/main/articles/from-prompt-alchemy-to-prompt-engineering-an-introduction-to-analytic-agumentation.md https://github.com/williamcotton/empirical-philosophy/blob/m... https://langchain.readthedocs.io/en/latest/ https://langchain.readthedocs.io/en/latest/ They can be taught!