3 ms·

Once it can read, has a broad vocabulary and can reason enough to synthesize information from what is given then you don’t need to train it any further. We’re a

Once it can read, has a broad vocabulary and can reason enough to synthesize information from what is given then you don’t need to train it any further. We’re at that point now. Everything going forward is just engineering, even just finding ways to increase the context length will allow these models to work with any data available. OpenAI is very publicly working on that and so is Anthropic. You can also apply some finesse and combine the model with external tools like search and databases or custom built APIs, practically everyone and their dog is experimenting with this approach. So even if no better models are made, which seems unlikely to be the case, we’ll be utilizing the current generation in all kinds of ways from here on out.

JimtheCoder 3y ago
"...and can reason enough..." Are we actually at this point now? I am less confident than you are...
- throwuwu 3y ago
  I use it every day to help solve problems, that’s enough for me to say it has some level of reasoning ability. Not claiming its perfect but it’s better than some people I know.
mvdtnz 3y ago
> Once it can read, has a broad vocabulary and can reason enough to synthesize information from what is given then you don’t need to train it any further And what is it going to read? How will it distinguish anything it reads today from AI generated content? Don't you see you've just set up the exact same circumstance that the article talks about?
- throwuwu 3y ago
  Do you know the difference between training and inference? If not, this conversation isn’t going anywhere.
  - mvdtnz 3y ago
    Yes.
nerdbert 3y ago
> and can reason enough to synthesize information from what is given That's a qualitative leap from what it does now, and there's no clear basis for believing the current trajectory of these systems can make that leap.
- throwuwu 3y ago
  How? It can use what’s in its context window. I can copy paste documentation or code it’s never seen before and ask it to operate on it and get useful answers. This is what practically all the new AI tools are built on.
  - nerdbert 3y ago
    It's not reasoning, it's predicting likely text, which it can do because the bits of input you provide overlap with enough bits in its training corpus, because most programming tasks are repetitions of similar patterns. The outcome approximates reasoning in some cases but the process is completely different. It is not building up from principles, it is sifting down from examples.

JimtheCoder 3y ago

"...and can reason enough..." Are we actually at this point now? I am less confident than you are...

throwuwu 3y ago
I use it every day to help solve problems, that’s enough for me to say it has some level of reasoning ability. Not claiming its perfect but it’s better than some people I know.

mvdtnz 3y ago

> Once it can read, has a broad vocabulary and can reason enough to synthesize information from what is given then you don’t need to train it any further And what is it going to read? How will it distinguish anything it reads today from AI generated content? Don't you see you've just set up the exact same circumstance that the article talks about?

throwuwu 3y ago
Do you know the difference between training and inference? If not, this conversation isn’t going anywhere.
- mvdtnz 3y ago
  Yes.

nerdbert 3y ago

> and can reason enough to synthesize information from what is given That's a qualitative leap from what it does now, and there's no clear basis for believing the current trajectory of these systems can make that leap.

throwuwu 3y ago
How? It can use what’s in its context window. I can copy paste documentation or code it’s never seen before and ask it to operate on it and get useful answers. This is what practically all the new AI tools are built on.
- nerdbert 3y ago
  It's not reasoning, it's predicting likely text, which it can do because the bits of input you provide overlap with enough bits in its training corpus, because most programming tasks are repetitions of similar patterns. The outcome approximates reasoning in some cases but the process is completely different. It is not building up from principles, it is sifting down from examples.