6 ms·

Growing among who? The more I learn about and use LLMs the more convinced I am we're in a local maxima and the only way they're going to improve is by getting s

Growing among who? The more I learn about and use LLMs the more convinced I am we're in a local maxima and the only way they're going to improve is by getting smaller and cheaper to run. They're still terrible at logical reasoning. We're going to get some super cool and some super dystopian stuff out of them but LLMs are never going to go into a recursive loop of self-improvement and become machine gods.

ben_w 3y ago
> They're still terrible at logical reasoning. Are they even trying to be good at that? Serious question; using LLMs as a logical processor are as wasteful and as well-suited as using the Great Pyramid of Giza as an AirBnB. I've not tried this, but I suspect the best way is more like asking the LLM to write a COQ script for the scenario, instead of trying to get it to solve the logic directly.
- staunton 3y ago
  Indeed, AI reinforcement-learning to deal with formal verification is what I'm looking forward to the most. Unfortunately it seems a very niche endeavour at the moment.
- fsckboy 3y ago
  > using the Great Pyramid of Giza as an AirBnB , were you allowed to do it, would be an extremely profitable venture. Taj Mahal too, and yes, I know it's a mausoleum.
  - ben_w 3y ago
    I can see the reviews in my head already: 1 star: No WiFi, no windows, no hot water 1 star: dusty 1 star: aliens didn't abduct me :( 5 stars: lots of storage room for my luggage 4 stars: service good, but had weird dream about a furry weighing my soul against a feather 1 star: aliens did abduct me :( 2 stars: nice views, but smells of camel
TeMPOraL 3y ago
> The more I learn about and use LLMs the more convinced I am we're in a local maxima Not sure why would you believe that. Inside view: qualitative improvements LLMs made at scale took everyone by surprise; I don't think anyone understands them enough to make a convincing argument that LLMs have exhausted their potential. Outside view: what local maximum? Wake me up when someone else makes a LLM comparable in performance to GPT-4. Right now, there is no local maximum. There's one model far ahead of the rest, and that model is actually below it's peak performance - side effect of OpenAI lobotomizing it with aggressive RLHF. The only thing remotely suggesting we shouldn't expect further improvements is... OpenAI saying they kinda want to try some other things, and (pinky swear!) aren't training GPT-4's successor. > and the only way they're going to improve is by getting smaller and cheaper to run. Meaning they'll be easier to chain. The next big leap could in fact be a bunch of compressed, power-efficient LLMs talking to each other. Possibly even managing their own deployment. > They're still terrible at logical reasoning. So is your unconscious / system 1 / gut feel. LLMs are less like one's whole mind, and much more like one's "inner voice". Logical skills aren't automatic, they're algorithmic. Who knows what is the limit of a design in which LLM as "system 1" operates a much larger, symbolic, algorithmic suite of "system 2" software? We're barely scratching the surface here.
behnamoh 3y ago
My thoughts exactly. It's hard to see signal among all the noise surrounding LLMs, Even if they say they're gonna hurt you, they have no idea about what it means to hurt, what is "you", and how they're going to achieve that goal. They just spit out things that resemble people have said online. There's no harm from a language model that's literally a "language" model.
- visarga 3y ago
  A language model can do many things based on language instructions, some harmless, some harmful. They are both instructable and teachable. Depending on the prompt, they are not just harmless LLMs.
- forgetfreeman 3y ago
  You appear to be ignoring a few thousand years of recorded history around what happens when a demagogue gets a megaphone. Human-powered astroturf campaigns were all it took to get randoms convinced lizard people are an existential threat and then -act- on that belief.
  - nullsense 3y ago
    I think I'm just going to build and open source some really next gen astroturf software that learns continuously as it debates people online in order to get better at changing people's minds. I'll make sure to include documentation in Russian, Chinese and Corporate American English. What would a good name be? TurfChain? I'm serious. People don't believe this risk is real. They keep hiding it behind some nameless, faceless 'bad actor', so let's just make it real. I don't need to use it. I'll just release it as a research project.
    - forgetfreeman 3y ago
      It's not like there isn't a market waiting impatiently for the product...
      - 3y ago
berniedurfee 3y ago
I’m agreeing with this viewpoint the more I use LLMs. They’re text generators that can generate compelling content because they’re so good at generating text. I don’t think AGI will arise from a text generator.
stuckkeys 3y ago
I was looking at the A100 80gb cards. 14k a pop. We gonna see another GPU shortage when these models become less resource dependent. CRYPTO era
ux-app 3y ago
>They're still terrible at logical reasoning. 2 years ago a machine that understands natural language and is capable of any arbitrary, free-form logic or problem solving was pure science fiction. I'm baffled by this kind of dismissal tbh. >but LLMs are never going to go into a recursive loop of self-improvement never is a long time.
- deleted 3y ago
  [deleted]
- leonidasv 3y ago
  Two years ago we already had GPT-2, that was capable of some problem solving and logic following. It was archaic, sure, it produced a lot of gibberish, yes, but if you followed OpenAI releases closely, you wouldn't think that something like GPT3.5 was "pure science fiction", it would just look as the inevitable evolution of GPT-2 in a couple of years given the right conditions.
  - ux-app 3y ago
    that's pedantic. switch 2 years to 5 years and the point still stands.
    - edgyquant 3y ago
      No it isn’t. Even before transformers people were doing cool things with LSTMs and RNNs before that. People following this space haven’t really been surprised by any of these advancements. It’s a straight forward path imo
  - canjobear 3y ago
    In hindsight it’s an obvious evolution, but in practice vanishingly few people saw it coming.
    - leonidasv 3y ago
      Few people saw it coming in just two years, sure. But most people following this space were already expecting a big evolution like the one we saw in 5-ish years. For example, take this thread: https://news.ycombinator.com/item?id=21717022 https://news.ycombinator.com/item?id=21717022 It's a text RPG game built on top of GPT-2 that could follow arbitrary instructions. It was a full project with custom training for something that you can get with a single prompt on ChatGPT nowadays, but it clearly showcased what LLMs were capable of and things we take for granted now. It was clear, back then, that at some point ChatGPT would happen.

ben_w 3y ago

> They're still terrible at logical reasoning. Are they even trying to be good at that? Serious question; using LLMs as a logical processor are as wasteful and as well-suited as using the Great Pyramid of Giza as an AirBnB. I've not tried this, but I suspect the best way is more like asking the LLM to write a COQ script for the scenario, instead of trying to get it to solve the logic directly.

staunton 3y ago
Indeed, AI reinforcement-learning to deal with formal verification is what I'm looking forward to the most. Unfortunately it seems a very niche endeavour at the moment.
fsckboy 3y ago
> using the Great Pyramid of Giza as an AirBnB , were you allowed to do it, would be an extremely profitable venture. Taj Mahal too, and yes, I know it's a mausoleum.
- ben_w 3y ago
  I can see the reviews in my head already: 1 star: No WiFi, no windows, no hot water 1 star: dusty 1 star: aliens didn't abduct me :( 5 stars: lots of storage room for my luggage 4 stars: service good, but had weird dream about a furry weighing my soul against a feather 1 star: aliens did abduct me :( 2 stars: nice views, but smells of camel

TeMPOraL 3y ago

> The more I learn about and use LLMs the more convinced I am we're in a local maxima Not sure why would you believe that. Inside view: qualitative improvements LLMs made at scale took everyone by surprise; I don't think anyone understands them enough to make a convincing argument that LLMs have exhausted their potential. Outside view: what local maximum? Wake me up when someone else makes a LLM comparable in performance to GPT-4. Right now, there is no local maximum. There's one model far ahead of the rest, and that model is actually below it's peak performance - side effect of OpenAI lobotomizing it with aggressive RLHF. The only thing remotely suggesting we shouldn't expect further improvements is... OpenAI saying they kinda want to try some other things, and (pinky swear!) aren't training GPT-4's successor. > and the only way they're going to improve is by getting smaller and cheaper to run. Meaning they'll be easier to chain. The next big leap could in fact be a bunch of compressed, power-efficient LLMs talking to each other. Possibly even managing their own deployment. > They're still terrible at logical reasoning. So is your unconscious / system 1 / gut feel. LLMs are less like one's whole mind, and much more like one's "inner voice". Logical skills aren't automatic, they're algorithmic. Who knows what is the limit of a design in which LLM as "system 1" operates a much larger, symbolic, algorithmic suite of "system 2" software? We're barely scratching the surface here.

behnamoh 3y ago

My thoughts exactly. It's hard to see signal among all the noise surrounding LLMs, Even if they say they're gonna hurt you, they have no idea about what it means to hurt, what is "you", and how they're going to achieve that goal. They just spit out things that resemble people have said online. There's no harm from a language model that's literally a "language" model.

visarga 3y ago
A language model can do many things based on language instructions, some harmless, some harmful. They are both instructable and teachable. Depending on the prompt, they are not just harmless LLMs.
forgetfreeman 3y ago
You appear to be ignoring a few thousand years of recorded history around what happens when a demagogue gets a megaphone. Human-powered astroturf campaigns were all it took to get randoms convinced lizard people are an existential threat and then -act- on that belief.
- nullsense 3y ago
  I think I'm just going to build and open source some really next gen astroturf software that learns continuously as it debates people online in order to get better at changing people's minds. I'll make sure to include documentation in Russian, Chinese and Corporate American English. What would a good name be? TurfChain? I'm serious. People don't believe this risk is real. They keep hiding it behind some nameless, faceless 'bad actor', so let's just make it real. I don't need to use it. I'll just release it as a research project.
  - forgetfreeman 3y ago
    It's not like there isn't a market waiting impatiently for the product...
    - 3y ago

berniedurfee 3y ago

I’m agreeing with this viewpoint the more I use LLMs. They’re text generators that can generate compelling content because they’re so good at generating text. I don’t think AGI will arise from a text generator.

stuckkeys 3y ago

I was looking at the A100 80gb cards. 14k a pop. We gonna see another GPU shortage when these models become less resource dependent. CRYPTO era

ux-app 3y ago

>They're still terrible at logical reasoning. 2 years ago a machine that understands natural language and is capable of any arbitrary, free-form logic or problem solving was pure science fiction. I'm baffled by this kind of dismissal tbh. >but LLMs are never going to go into a recursive loop of self-improvement never is a long time.

deleted 3y ago
[deleted]
leonidasv 3y ago
Two years ago we already had GPT-2, that was capable of some problem solving and logic following. It was archaic, sure, it produced a lot of gibberish, yes, but if you followed OpenAI releases closely, you wouldn't think that something like GPT3.5 was "pure science fiction", it would just look as the inevitable evolution of GPT-2 in a couple of years given the right conditions.
- ux-app 3y ago
  that's pedantic. switch 2 years to 5 years and the point still stands.
  - edgyquant 3y ago
    No it isn’t. Even before transformers people were doing cool things with LSTMs and RNNs before that. People following this space haven’t really been surprised by any of these advancements. It’s a straight forward path imo
- canjobear 3y ago
  In hindsight it’s an obvious evolution, but in practice vanishingly few people saw it coming.
  - leonidasv 3y ago
    Few people saw it coming in just two years, sure. But most people following this space were already expecting a big evolution like the one we saw in 5-ish years. For example, take this thread: https://news.ycombinator.com/item?id=21717022 https://news.ycombinator.com/item?id=21717022 It's a text RPG game built on top of GPT-2 that could follow arbitrary instructions. It was a full project with custom training for something that you can get with a single prompt on ChatGPT nowadays, but it clearly showcased what LLMs were capable of and things we take for granted now. It was clear, back then, that at some point ChatGPT would happen.