5 ms·

It used to be that classical software was useful for verification, but not so good at generating ideas. With LLMs we have the opposite: machines are now good a

It used to be that classical software was useful for verification, but not so good at generating ideas. With LLMs we have the opposite: machines are now good at idea generation and exploration, but their ideas need to be verified by a human (or classical logic-based software).

jeron 4y ago
So all we need to do is have LLMs generate classical logic-based software to qualify its own ideas
- TaylorAlexander 4y ago
  I’m quite confident they’re using GitHub copilot or similar inside of OpenAI while working on improving GPT-4. So yeah, that’s happening.
  - zingar 4y ago
    But isn’t copilot itself a fuzzy llm? Many of its solutions don’t seem formally verified.
    - leereeves 4y ago
      Not entirely, but it will already be doing pretty well when it reliably outputs code that compiles and passes the test cases it generates.
- chaxor 4y ago
  There are plenty of different ways this is happening. 1) already happened with Galactica - the model included python scripts as part of it's "working memory" as part of a <work> token, which could be used in a container 2) via copilot, and copilot 365, as someone stated elsewhwre 3) GPT-> Wolfram alpha as a plugin, which will be out very soon Galactica is the only one that is already (technically) available
dclowd9901 4y ago
But, by virtue of how they work, are they any good at generating novel ideas?
- eximius 4y ago
  That raises an interesting point. On the one hand, novelty is in the eye of the beholder, if you will. What has been done before might still be new to many. On the other, randomness is built in and language has a remarkable capacity for uniqueness (see statistics about novel queries on Google). So it's possible and likely for LLMs to generate novel sentences. Whether those are valuable is a harder thing to think about.
  - soco 4y ago
    Given that even an army of monkey typists can generate a Shakespeare, why wouldn't an army of llms? Whether we call that "generating" or "creating" might be only philosophical.
    - wizofaus 4y ago
      An army of monkeys can only generate Shakespeare if you have enough manpower to sit around monitoring the masses of output, which makes it effectively not possible in any meaningful sense.
      - yoshyosh 4y ago
        Some combination of users interactions with the endless output might create a decent enough opportunity. For example, Spotify discover combined with auto generated music (of a certain level) maybe is a possibility for this in that genre?
      - melagonster 4y ago
        in now condition, some company rely on copy artists' style to make sure it good enough, and forbidden copyright.
    - 4y ago

jeron 4y ago

So all we need to do is have LLMs generate classical logic-based software to qualify its own ideas

TaylorAlexander 4y ago
I’m quite confident they’re using GitHub copilot or similar inside of OpenAI while working on improving GPT-4. So yeah, that’s happening.
- zingar 4y ago
  But isn’t copilot itself a fuzzy llm? Many of its solutions don’t seem formally verified.
  - leereeves 4y ago
    Not entirely, but it will already be doing pretty well when it reliably outputs code that compiles and passes the test cases it generates.
chaxor 4y ago
There are plenty of different ways this is happening. 1) already happened with Galactica - the model included python scripts as part of it's "working memory" as part of a <work> token, which could be used in a container 2) via copilot, and copilot 365, as someone stated elsewhwre 3) GPT-> Wolfram alpha as a plugin, which will be out very soon Galactica is the only one that is already (technically) available

dclowd9901 4y ago

But, by virtue of how they work, are they any good at generating novel ideas?

eximius 4y ago
That raises an interesting point. On the one hand, novelty is in the eye of the beholder, if you will. What has been done before might still be new to many. On the other, randomness is built in and language has a remarkable capacity for uniqueness (see statistics about novel queries on Google). So it's possible and likely for LLMs to generate novel sentences. Whether those are valuable is a harder thing to think about.
- soco 4y ago
  Given that even an army of monkey typists can generate a Shakespeare, why wouldn't an army of llms? Whether we call that "generating" or "creating" might be only philosophical.
  - wizofaus 4y ago
    An army of monkeys can only generate Shakespeare if you have enough manpower to sit around monitoring the masses of output, which makes it effectively not possible in any meaningful sense.
    - yoshyosh 4y ago
      Some combination of users interactions with the endless output might create a decent enough opportunity. For example, Spotify discover combined with auto generated music (of a certain level) maybe is a possibility for this in that genre?
    - melagonster 4y ago
      in now condition, some company rely on copy artists' style to make sure it good enough, and forbidden copyright.
  - 4y ago