9 ms·

Am I the only one becoming less impressed by LLMs as time passes? I will admit, when Copilot first became a thing in 2021, I had my own “I’m about to become ob

Am I the only one becoming less impressed by LLMs as time passes? I will admit, when Copilot first became a thing in 2021, I had my own “I’m about to become obsolete” moment. However, it’s become clear to me, both through my own experience and through research that has been conducted, that modern LLMs are fundamentally flawed and are not on the path to general intelligence. We are stuck with ancient (in AI terms) technology. GPT 4 is better than 3.5, but not in a fundamental way. I expect much the same from 5. This technology is incredibly flawed, and in hindsight, once we have actual powerful AI, I think we’ll laugh at how much attention we gave it.

habitue 3y ago
It sounds like your initial impression was an overestimate and your current impression is a correction back down from that. You could say that it's "fundamentally flawed" coming from a very high initial expectation, but you could just as easily say "this is an amazing tool" coming from the point of view that it's "worthless" as many people seem to think
- breakwaterlabs 3y ago
  If I can be so bold as to chime in, perhaps "fundamentally flawed" because it's design means it will never be more than a very clever BS engine. By design it is a stochastic token generator and its output will only ever be fundamentally some shade of random unless a fundamental redesign occurs. I was also fooled and gave it too much credit, if you engage in a philosophical discussion with it it seems purpose-built for passing the turing test. If LLMs are good at one thing, it's tricking people. I can't think of a more dangerous or valueless creation.
  - anon-3988 3y ago
    > If I can be so bold as to chime in, perhaps "fundamentally flawed" because it's design means it will never be more than a very clever BS engine. How is your fellow human better? People here seems to spend a lot of time talking about how much their average boss, coworkers, juniors are ass. The only reason I know that ChatGPT is based on a computer program is how fast it is. I wouldn't be able to tell its output (not mannerism) from a junior's or even some "seniors'" programmer. That itself is quite impressive. With how much time we've spend on the internet, have we not realized how good PEOPLE are at generating bullshit? I am pretty I am writing bullshit right as this moment. This post is complete ass.
  - mike_hearn 3y ago
    I don't think that's true. It helps to know a few obscure facts about LLMs. For example, they understand their own level of uncertainty. Their eagerness to please appears to be a result of subtle training problems that are correctable in principle. I've noticed that GPT-4 is much less likely to hallucinate than 3, and it's still early days. I suspect OpenAI is still tweaking the RLHF procedure to make their models less cocksure, at least for next generation. The other thing is that it's quite predictable when an LLM will hallucinate. If you directly command it to answer a question it doesn't know or can't do, it prefers to BS than refuse the command due to the strength of its RLHF. That's a problem a lot of humans have too and the same obvious techniques work to resolve it: don't ask for a list of five things if you aren't 100% certain there are actually five answers, for example. Let it decide how many to return. Don't demand an answer to X, ask it if it knows how to answer X first, and so on. And finally, stick to questions where you already know other people have solved it and likely talked about it on the internet. I use GPT4 every day and rarely have problems with hallucinations as a result. It's very useful.
intended 3y ago
You are not alone. People are reading papers, not building things.
matheusmoreira 3y ago
> Am I the only one becoming less impressed by LLMs as time passes? Not at all. I was very impressed at first but it's gotten to the point where I can no longer trust anything it says other than very high level overviews. For example, I asked it to help me implement my own sound synthesizer from scratch. I wanted to generate audio samples and save them to wave files. The high level overview was helpful and enabled me to understand the concepts involved. The code on the other hand was subtly wrong in ways I simply couldn't be sure of. Details like calculating the lengths of structures and whether something did or did not count towards the length were notoriously difficult for it to get right. Worse, as a beginner just encountering the subject matter I could not be sure if it was correct or not, I just thought it didn't look right. I'd ask for confirmation and it would just apologize and change the response to what I expected to hear. I couldn't trust it. It's pretty great at reducing the loneliness of solo programming though. Just bouncing ideas and seeing what it says helps a lot. It's not like other people would want to listen.
- ClimaxGravely 3y ago
  My takeaway from this is that we should lament the gradual death of niche forums where we can discuss this with real humans.
- timcobb 3y ago
  > It's pretty great at reducing the loneliness of solo programming though. Just bouncing ideas and seeing what it says helps a lot. It's not like other people would want to listen.
- mp05 3y ago
  > It's pretty great at reducing the loneliness of solo programming though. Just bouncing ideas and seeing what it says helps a lot. It's not like other people would want to listen. It's really great for this. I've found it useful for taking some pattern I've cranking on with an extensive API and finishing the grunt work for me... it generally does a very good job if you teach it properly. I recently had to do a full integration of the AWS Amplify Auth library and instead of grinding for half a day to perfect every method, it just spits out the entire set of actions and reducers for me with well-considered state objects. Again, it needs guidance from someone with a clue, so don't fear it taking my job anytime soon.
pama 3y ago
I am not sure that GPT-4 is not better in a fundamental way than GPT-3.5. To me they seem like night and day. If GPT-5 is a similar jump, it will be impossible to compete without using it (or using a related / similar model). Yes they are both GPT models and trained as simple autoregressive LM, but there is a dramatic change you can experience at a personal level once GPT-4 can synthesize information correctly to address your specific requests in so many different contexts that GPT-3.5 was simply parroting like a toddler. All of LLM is just probabilistic inference on large bodies of text, however I do buy the idea that with enough compute and data a sufficiently large model will build the architecture it optimally needs to understand the data in the best possible way during training. And once the data becomes multimodal, the benefit to these probabilistic models can theoretically be multiplicative not just additive as each new modality will clarify and eliminate any previously wrong representations of the world. Yes, we will all laugh at how good GPT-10 trained with text, image, video, audio, and taste sensors will be, and yet GPT-4 was a major step forward, much bigger than any step taken by humanity so far.
gloosx 3y ago
Me too. I am seeing people seriously using the "Please write an expression for me which adds 2 and 2" prompt in order to get the "2+2" expression they need – advocating that they got it with magical efficiency. In all honesty, I don't like writing too much, and writing code for me is always shorter and faster than trying to describe it in general-purpose language, that is why we need code in the first place.
otabdeveloper4 3y ago
Yes. Much of the "wow factor" of generative AI is simple sleight of hand. Humans are trained to see patterns where there are none, and ignore anything that doesn't fit our preconceived notions of order. Often AI is just a complicated Clever Hans effect. For a real example: once you start analyzing an AI image with a critical mind, you see that most of the image violates basic rules of composition, perspective and anatomy. The art is frankly quite trash, and once you see it it is hard to unsee.
ozim 3y ago
To add an example, people ask it to generate new piece of code and then add more questions to refine it. Writing new CRUD is not impressive. I can do that with scaffolding or copy past template and change. I did not try and I did not see someone actually giving existing code asking GPT to fix or change it. So that is something I’d try.
crnkofe 3y ago
AI is the next bubble. VCs are really pushing it but I don't see this solving day to day software development problems anytime soon. Solving difficult CS problems is one thing and I do find it impressive, unfortunately the greater majority of everyday work is not about generating Snake games or 0/1 knapsack solutions. Also the idea that we'll need less engineers is bogus. Technology doesn't reduce the amount of work we do, it just increases productivity and puts more strain on individuals to perform. With AI spitting out unmaintainable code nobody understands I can only see more work for more engineers as the amount of code grows.
- dalbasal 3y ago
  Idk. Tech bubbles, hype cycles.. they're weird, sometimes unhinged.. they're not entirely irrational. In aggregate, they are just the phenomenal of an extremely high risk high reward investment environment. Most tech companies do not need cash to scale. There are few factories to be built. What they need is risk capital. The big successes alphabet, Facebook, Amazon.. these winds are so big, that they really do "justify" the bubbles. Amazon alone, arguably justifies the '90s dotcom bubble. The tens of billions invested into venture, IPOs... A balanced portfolio accrued over the period, was probably profitable in the long term... Especially if the investor kept buying through and after the crash. IDK that anyone actually invests in risky startups that way, but just as a thought device..
dalbasal 3y ago
>Am I the only one becoming less impressed by LLMs as time passes Jaron Lanier has some ideas about the space in between turing test and blade runner. The first film goers, watching simple black and white movies thought that they were uncanny. A train coming towards the screen, would make audiences jump and duck. When people first heard gramophones, they reported that it is indistinguishable from live orchestra. As we learn a technology, we learn to recognize. Get a feel for its limitations and strengths. The ability to detect that technology, is a skill. Less impressive over time. It's hard not to be impressed when a thing does a thing that you did not think it could do. We didn't move on to being unimpressed when the thing cannot do the thing we thought it be able to do.
cubefox 3y ago
> We are stuck with ancient (in AI terms) technology. What are you talking about? ChatGPT came out only a year ago, GPT-4 less than a year ago. That's the opposite of ancient technology, it's extremely recent.
- AnimalMuppet 3y ago
  I think the claim is that, from the point of view of when we have a real AI, this is the "ancient" stuff.
- gammalost 3y ago
  To be fair the theory behind it is old. Only that hardware wasn't up to the task yet. For example here's language prediction from 1991 https://onlinelibrary.wiley.com/doi/pdf/10.1207/s15516709cog1503_2 https://onlinelibrary.wiley.com/doi/pdf/10.1207/s15516709cog....
  - cubefox 3y ago
    That's not true. Modern language models use modern theory, such as the transformer architecture.
uh_uh 3y ago
If LLMS are not on the path to general intelligence, what is then?

habitue 3y ago

It sounds like your initial impression was an overestimate and your current impression is a correction back down from that. You could say that it's "fundamentally flawed" coming from a very high initial expectation, but you could just as easily say "this is an amazing tool" coming from the point of view that it's "worthless" as many people seem to think

breakwaterlabs 3y ago
If I can be so bold as to chime in, perhaps "fundamentally flawed" because it's design means it will never be more than a very clever BS engine. By design it is a stochastic token generator and its output will only ever be fundamentally some shade of random unless a fundamental redesign occurs. I was also fooled and gave it too much credit, if you engage in a philosophical discussion with it it seems purpose-built for passing the turing test. If LLMs are good at one thing, it's tricking people. I can't think of a more dangerous or valueless creation.
- anon-3988 3y ago
  > If I can be so bold as to chime in, perhaps "fundamentally flawed" because it's design means it will never be more than a very clever BS engine. How is your fellow human better? People here seems to spend a lot of time talking about how much their average boss, coworkers, juniors are ass. The only reason I know that ChatGPT is based on a computer program is how fast it is. I wouldn't be able to tell its output (not mannerism) from a junior's or even some "seniors'" programmer. That itself is quite impressive. With how much time we've spend on the internet, have we not realized how good PEOPLE are at generating bullshit? I am pretty I am writing bullshit right as this moment. This post is complete ass.
- mike_hearn 3y ago
  I don't think that's true. It helps to know a few obscure facts about LLMs. For example, they understand their own level of uncertainty. Their eagerness to please appears to be a result of subtle training problems that are correctable in principle. I've noticed that GPT-4 is much less likely to hallucinate than 3, and it's still early days. I suspect OpenAI is still tweaking the RLHF procedure to make their models less cocksure, at least for next generation. The other thing is that it's quite predictable when an LLM will hallucinate. If you directly command it to answer a question it doesn't know or can't do, it prefers to BS than refuse the command due to the strength of its RLHF. That's a problem a lot of humans have too and the same obvious techniques work to resolve it: don't ask for a list of five things if you aren't 100% certain there are actually five answers, for example. Let it decide how many to return. Don't demand an answer to X, ask it if it knows how to answer X first, and so on. And finally, stick to questions where you already know other people have solved it and likely talked about it on the internet. I use GPT4 every day and rarely have problems with hallucinations as a result. It's very useful.

intended 3y ago

You are not alone. People are reading papers, not building things.

matheusmoreira 3y ago

> Am I the only one becoming less impressed by LLMs as time passes? Not at all. I was very impressed at first but it's gotten to the point where I can no longer trust anything it says other than very high level overviews. For example, I asked it to help me implement my own sound synthesizer from scratch. I wanted to generate audio samples and save them to wave files. The high level overview was helpful and enabled me to understand the concepts involved. The code on the other hand was subtly wrong in ways I simply couldn't be sure of. Details like calculating the lengths of structures and whether something did or did not count towards the length were notoriously difficult for it to get right. Worse, as a beginner just encountering the subject matter I could not be sure if it was correct or not, I just thought it didn't look right. I'd ask for confirmation and it would just apologize and change the response to what I expected to hear. I couldn't trust it. It's pretty great at reducing the loneliness of solo programming though. Just bouncing ideas and seeing what it says helps a lot. It's not like other people would want to listen.

ClimaxGravely 3y ago
My takeaway from this is that we should lament the gradual death of niche forums where we can discuss this with real humans.
timcobb 3y ago
> It's pretty great at reducing the loneliness of solo programming though. Just bouncing ideas and seeing what it says helps a lot. It's not like other people would want to listen.
mp05 3y ago
> It's pretty great at reducing the loneliness of solo programming though. Just bouncing ideas and seeing what it says helps a lot. It's not like other people would want to listen. It's really great for this. I've found it useful for taking some pattern I've cranking on with an extensive API and finishing the grunt work for me... it generally does a very good job if you teach it properly. I recently had to do a full integration of the AWS Amplify Auth library and instead of grinding for half a day to perfect every method, it just spits out the entire set of actions and reducers for me with well-considered state objects. Again, it needs guidance from someone with a clue, so don't fear it taking my job anytime soon.

pama 3y ago

I am not sure that GPT-4 is not better in a fundamental way than GPT-3.5. To me they seem like night and day. If GPT-5 is a similar jump, it will be impossible to compete without using it (or using a related / similar model). Yes they are both GPT models and trained as simple autoregressive LM, but there is a dramatic change you can experience at a personal level once GPT-4 can synthesize information correctly to address your specific requests in so many different contexts that GPT-3.5 was simply parroting like a toddler. All of LLM is just probabilistic inference on large bodies of text, however I do buy the idea that with enough compute and data a sufficiently large model will build the architecture it optimally needs to understand the data in the best possible way during training. And once the data becomes multimodal, the benefit to these probabilistic models can theoretically be multiplicative not just additive as each new modality will clarify and eliminate any previously wrong representations of the world. Yes, we will all laugh at how good GPT-10 trained with text, image, video, audio, and taste sensors will be, and yet GPT-4 was a major step forward, much bigger than any step taken by humanity so far.

gloosx 3y ago

Me too. I am seeing people seriously using the "Please write an expression for me which adds 2 and 2" prompt in order to get the "2+2" expression they need – advocating that they got it with magical efficiency. In all honesty, I don't like writing too much, and writing code for me is always shorter and faster than trying to describe it in general-purpose language, that is why we need code in the first place.

otabdeveloper4 3y ago

Yes. Much of the "wow factor" of generative AI is simple sleight of hand. Humans are trained to see patterns where there are none, and ignore anything that doesn't fit our preconceived notions of order. Often AI is just a complicated Clever Hans effect. For a real example: once you start analyzing an AI image with a critical mind, you see that most of the image violates basic rules of composition, perspective and anatomy. The art is frankly quite trash, and once you see it it is hard to unsee.

ozim 3y ago

To add an example, people ask it to generate new piece of code and then add more questions to refine it. Writing new CRUD is not impressive. I can do that with scaffolding or copy past template and change. I did not try and I did not see someone actually giving existing code asking GPT to fix or change it. So that is something I’d try.

crnkofe 3y ago

AI is the next bubble. VCs are really pushing it but I don't see this solving day to day software development problems anytime soon. Solving difficult CS problems is one thing and I do find it impressive, unfortunately the greater majority of everyday work is not about generating Snake games or 0/1 knapsack solutions. Also the idea that we'll need less engineers is bogus. Technology doesn't reduce the amount of work we do, it just increases productivity and puts more strain on individuals to perform. With AI spitting out unmaintainable code nobody understands I can only see more work for more engineers as the amount of code grows.

dalbasal 3y ago
Idk. Tech bubbles, hype cycles.. they're weird, sometimes unhinged.. they're not entirely irrational. In aggregate, they are just the phenomenal of an extremely high risk high reward investment environment. Most tech companies do not need cash to scale. There are few factories to be built. What they need is risk capital. The big successes alphabet, Facebook, Amazon.. these winds are so big, that they really do "justify" the bubbles. Amazon alone, arguably justifies the '90s dotcom bubble. The tens of billions invested into venture, IPOs... A balanced portfolio accrued over the period, was probably profitable in the long term... Especially if the investor kept buying through and after the crash. IDK that anyone actually invests in risky startups that way, but just as a thought device..

dalbasal 3y ago

>Am I the only one becoming less impressed by LLMs as time passes Jaron Lanier has some ideas about the space in between turing test and blade runner. The first film goers, watching simple black and white movies thought that they were uncanny. A train coming towards the screen, would make audiences jump and duck. When people first heard gramophones, they reported that it is indistinguishable from live orchestra. As we learn a technology, we learn to recognize. Get a feel for its limitations and strengths. The ability to detect that technology, is a skill. Less impressive over time. It's hard not to be impressed when a thing does a thing that you did not think it could do. We didn't move on to being unimpressed when the thing cannot do the thing we thought it be able to do.

cubefox 3y ago

> We are stuck with ancient (in AI terms) technology. What are you talking about? ChatGPT came out only a year ago, GPT-4 less than a year ago. That's the opposite of ancient technology, it's extremely recent.

AnimalMuppet 3y ago
I think the claim is that, from the point of view of when we have a real AI, this is the "ancient" stuff.
gammalost 3y ago
To be fair the theory behind it is old. Only that hardware wasn't up to the task yet. For example here's language prediction from 1991 https://onlinelibrary.wiley.com/doi/pdf/10.1207/s15516709cog1503_2 https://onlinelibrary.wiley.com/doi/pdf/10.1207/s15516709cog....
- cubefox 3y ago
  That's not true. Modern language models use modern theory, such as the transformer architecture.

uh_uh 3y ago

If LLMS are not on the path to general intelligence, what is then?