4 ms·

I've seen ChatGPT draw logical conclusions from (most-likely) previously unseen axioms. I've seen it invent ideas that (most-likely) were not in his training se

I've seen ChatGPT draw logical conclusions from (most-likely) previously unseen axioms. I've seen it invent ideas that (most-likely) were not in his training set. I've seen it fail too, but when it succeeds I can not substantially distinguish it from the novelty or logical reasoning of a human. On niche subjects, it fails often. But you can get it to correct itself by making it think logically by spelling out axioms, which seems to fit your definition of novelty. All the arguments I've read about it not being intelligent seem to stem from some form of essentialism. Right now it's dumb intelligent but I think giving it an adjustable recursive depth, similarly to what BulgarianIdiot suggests, will go a long way to improving it given its base capabilities.

BulgarianIdiot 4y ago
Indeed. And telling it to spell out the steps in math solution is essentially (ab)using the chat log as the memory for an iteration loop. This demonstrates then when you instruct it "break down the problem and solve it iteratively" it comes out with superior solutions. More accurate, more novel. Ergo, then if it was allowed to iterate in this way internally, it'd produce better answers by default. This thread sums up the same idea: https://twitter.com/ItIsFinch/status/1634730897520951296 https://twitter.com/ItIsFinch/status/1634730897520951296
- ad404b8a372f2b9 4y ago
  Yeah, I think it's going to be a promising area of research. It's known as Chain-Of-Thought (CoT) in the literature and there are a few papers already out: https://openreview.net/forum?id=_VjQlMeSB_J https://openreview.net/forum?id=_VjQlMeSB_J https://openreview.net/forum?id=5NTt8GFjUHkr https://openreview.net/forum?id=5NTt8GFjUHkr https://openreview.net/forum?id=qFVVBzXxR2V https://openreview.net/forum?id=qFVVBzXxR2V I'm also looking forward to explicit long-term memory mechanisms that don't rely on the main architecture's weights nor on CoT.
- fock 4y ago
  There's no memory or iteration log. You are just priming the (more or less deterministic even...) predictor with some history. You are doing a lot of work yourself, setting a factually correct context (and this has severe limits...). As for "we just need a way to include iterations": isn't this what attention is supposed to do (kind of dynamically updating weights). The usual way to really update weights are variations on gradient descent. Can you link a paper outlining how we integrate your proposal into the current framework (which took around 20years to mature)? Otherwise your statements are Sci-Fi (at the flying cars-level).
  - BulgarianIdiot 4y ago
    Oh so there's no memory, just recallable history. Oh so it's not learning by example, it's just primed by a sample. Oh so there's no iteration of thought, just a loop of prediction. OK. Also, the papers you asked about: https://news.ycombinator.com/item?id=35115563 https://news.ycombinator.com/item?id=35115563 BTW, people are already implementing recursive/iterative queries on the ChatGPT API, and getting promising results.
    - LawTalkingGuy 4y ago
      The langchain project is an example of the iterative queries approach. It comes with constructs for working memory, factual lookup / calculation agents, etc. https://github.com/hwchase17/langchain https://github.com/hwchase17/langchain The general (non-technical) guideline is that the LLMs can "answer" anything you just gave them the answer for. So you give it a problem, ask it how to solve it, tell it to use that method and explain the data it needs, give it that data, and then show it everything at once: "With this data you requested and summarized, use this technique to answer this question".
BulgarianIdiot 4y ago
Are you on other social media BTW? Thx.
- ad404b8a372f2b9 4y ago
  Afraid not, only here. You can at me at scrap.titanium@fastmail.com though.
WoodenChair 4y ago
> I've seen it invent ideas that (most-likely) were not in his training set. How can you possibly know what's in its training set? Its training set is so large that any "guess" about what is or is not in it is a fool's errand. Given that we know how the technology works, there's no reason to claim emergent behavior when Occam's razor would say the items are likely in the training set.
- ad404b8a372f2b9 4y ago
  It's not a fools errand, you can't know definitively but you can have a level of confidence. It's quite easy to invent a sentence that's never been uttered before, it's also fairly easy to hit the limits of the internet's knowledge in pretty much any complex discipline that's not software engineering. And it's apparent when you speak to ChatGPT, he will hallucinate a lot to compensate for niche topics it doesn't know much about. Occam's razor wouldn't point to ChatGPT having omniscient knowledge of all things ever thought of and that will ever be invented.
- Sunhold 4y ago
  This kind of stuff clearly shows that GPT can generalize concepts: https://twitter.com/emollick/status/1626084142239649792 https://twitter.com/emollick/status/1626084142239649792
  - vkou 4y ago
    And this kind of stuff shows that it clearly cannot: https://acoup.blog/2023/02/17/collections-on-chatgpt/ https://acoup.blog/2023/02/17/collections-on-chatgpt/ > If you know nothing about either book, this answer almost sounds useful (it isn’t). Now this is a trivial research task; simply typing ‘the limits of empire review’ into Google and then clicking on the very first non-paywalled result (this review of the book by David Potter from 1990) and reading the first paragraph makes almost immediately clear the correct answer is that Isaac’s book is an intentional and explicit rebuttal of Luttwak’s book, or as Potter puts it, “Ben Isaac’s The Limits of Empire offers a new and formidable challenge to Luttwack.” > A human being who understands the words and what they mean could immediately answer the question, but ChatGPT which doesn’t, cannot: it can only BS around the answer by describing both books and then lamely saying they “intersect in some ways.” > The information ChatGPT needed was clearly in its training materials (or it wouldn’t have a description of either book to make a lossy copy of), but it lacks the capacity to understand that information as information (rather than as a statistically correlated sequence of words). Consequently it cannot draw the right conclusion and so talks around the question in a convincing, but erronous way.

BulgarianIdiot 4y ago

Indeed. And telling it to spell out the steps in math solution is essentially (ab)using the chat log as the memory for an iteration loop. This demonstrates then when you instruct it "break down the problem and solve it iteratively" it comes out with superior solutions. More accurate, more novel. Ergo, then if it was allowed to iterate in this way internally, it'd produce better answers by default. This thread sums up the same idea: https://twitter.com/ItIsFinch/status/1634730897520951296 https://twitter.com/ItIsFinch/status/1634730897520951296

ad404b8a372f2b9 4y ago
Yeah, I think it's going to be a promising area of research. It's known as Chain-Of-Thought (CoT) in the literature and there are a few papers already out: https://openreview.net/forum?id=_VjQlMeSB_J https://openreview.net/forum?id=_VjQlMeSB_J https://openreview.net/forum?id=5NTt8GFjUHkr https://openreview.net/forum?id=5NTt8GFjUHkr https://openreview.net/forum?id=qFVVBzXxR2V https://openreview.net/forum?id=qFVVBzXxR2V I'm also looking forward to explicit long-term memory mechanisms that don't rely on the main architecture's weights nor on CoT.
fock 4y ago
There's no memory or iteration log. You are just priming the (more or less deterministic even...) predictor with some history. You are doing a lot of work yourself, setting a factually correct context (and this has severe limits...). As for "we just need a way to include iterations": isn't this what attention is supposed to do (kind of dynamically updating weights). The usual way to really update weights are variations on gradient descent. Can you link a paper outlining how we integrate your proposal into the current framework (which took around 20years to mature)? Otherwise your statements are Sci-Fi (at the flying cars-level).
- BulgarianIdiot 4y ago
  Oh so there's no memory, just recallable history. Oh so it's not learning by example, it's just primed by a sample. Oh so there's no iteration of thought, just a loop of prediction. OK. Also, the papers you asked about: https://news.ycombinator.com/item?id=35115563 https://news.ycombinator.com/item?id=35115563 BTW, people are already implementing recursive/iterative queries on the ChatGPT API, and getting promising results.
  - LawTalkingGuy 4y ago
    The langchain project is an example of the iterative queries approach. It comes with constructs for working memory, factual lookup / calculation agents, etc. https://github.com/hwchase17/langchain https://github.com/hwchase17/langchain The general (non-technical) guideline is that the LLMs can "answer" anything you just gave them the answer for. So you give it a problem, ask it how to solve it, tell it to use that method and explain the data it needs, give it that data, and then show it everything at once: "With this data you requested and summarized, use this technique to answer this question".

BulgarianIdiot 4y ago

Are you on other social media BTW? Thx.

ad404b8a372f2b9 4y ago
Afraid not, only here. You can at me at scrap.titanium@fastmail.com though.

WoodenChair 4y ago

> I've seen it invent ideas that (most-likely) were not in his training set. How can you possibly know what's in its training set? Its training set is so large that any "guess" about what is or is not in it is a fool's errand. Given that we know how the technology works, there's no reason to claim emergent behavior when Occam's razor would say the items are likely in the training set.

ad404b8a372f2b9 4y ago
It's not a fools errand, you can't know definitively but you can have a level of confidence. It's quite easy to invent a sentence that's never been uttered before, it's also fairly easy to hit the limits of the internet's knowledge in pretty much any complex discipline that's not software engineering. And it's apparent when you speak to ChatGPT, he will hallucinate a lot to compensate for niche topics it doesn't know much about. Occam's razor wouldn't point to ChatGPT having omniscient knowledge of all things ever thought of and that will ever be invented.
Sunhold 4y ago
This kind of stuff clearly shows that GPT can generalize concepts: https://twitter.com/emollick/status/1626084142239649792 https://twitter.com/emollick/status/1626084142239649792
- vkou 4y ago
  And this kind of stuff shows that it clearly cannot: https://acoup.blog/2023/02/17/collections-on-chatgpt/ https://acoup.blog/2023/02/17/collections-on-chatgpt/ > If you know nothing about either book, this answer almost sounds useful (it isn’t). Now this is a trivial research task; simply typing ‘the limits of empire review’ into Google and then clicking on the very first non-paywalled result (this review of the book by David Potter from 1990) and reading the first paragraph makes almost immediately clear the correct answer is that Isaac’s book is an intentional and explicit rebuttal of Luttwak’s book, or as Potter puts it, “Ben Isaac’s The Limits of Empire offers a new and formidable challenge to Luttwack.” > A human being who understands the words and what they mean could immediately answer the question, but ChatGPT which doesn’t, cannot: it can only BS around the answer by describing both books and then lamely saying they “intersect in some ways.” > The information ChatGPT needed was clearly in its training materials (or it wouldn’t have a description of either book to make a lossy copy of), but it lacks the capacity to understand that information as information (rather than as a statistically correlated sequence of words). Consequently it cannot draw the right conclusion and so talks around the question in a convincing, but erronous way.