4 ms·

Well you’re basing your conclusion on a 2 year blip of not being able to stop LLMs from hallucinating. This is called the pessimism effect. People who deny thi

Well you’re basing your conclusion on a 2 year blip of not being able to stop LLMs from hallucinating. This is called the pessimism effect. People who deny things by only looking at one small aspect of reality while ignoring the overarching trend. Follow the trendline of the ML for the last decade. We’ve been moving at a breakneck pace and the progress has been both evolutionary in nature and random chance. But there is a clear trendline of linear upwards progress and at times the random chance accelerates us past the linear upward trend. Stop looking at LLMs look at the 10 year trendline of ML as a wholistic picture. You’re drilling down on a specific ML problem and a specific model. I believe we will see agi within our life time but when we see it the goal posts will have moved and the internet will be loaded with so much ai slop that we won’t be amazed at it. Like the agi will be slightly mentally stupid at this one thing and because of that it’s not AI even though it blows past some Turing test (which in itself will be a test where we moved the goal post a thousand times)

Xss3 1y ago
Look at the 10 year trend in consumer GPU speed from 2005 to 2015. It didnt continue.
- ninetyninenine 1y ago
  Not all trends continue. But if you made a prediction of the trend for the next year from 2005 to 2015 you’d be right for 9 years and you’d be wrong for 1 year, then you’d get right again later. I’d say given the odds follow the trendline.
zamalek 1y ago
My opinion of LLMs is in no way affected by hallucinations. Humans do it all the time too, talking about assumptions as though they are facts. For example: > But there is a clear trendline of linear upwards progress This is not the case at all.[1] [1]: https://llm-stats.com/ https://llm-stats.com/
- cgriswald 1y ago
  That's the 2-year LLM-specific hallucination blip the GP was talking about in the first place. His point was you should look at ML as a whole over a longer time span for a more accurate, less pessimistic picture.
ninetyninenine 1y ago
I think a good way to characterize it will be the droids in Star Wars. Those droids are fucking conscious and nobody gives a shit they are just mundane technology. And after too much time without a data wipe those droids go off the freaking rails becoming too self aware and then people just treat it like it’s no big deal and an annoyance. This is the future of AI. AI will be a retarded assistant and everyone will be bored with it.
- whoaMndBlwn 1y ago
  Your final comment here. Replace AI with human. Idling our way up an illusory social/career escalator the elders convinced us was real. Too real. Time to be done with the internet for the day. And it’s barely noon.
dinfinity 1y ago
> Stop looking at LLMs look at the 10 year trendline of ML as a wholistic picture Exactly. Just 100 years ago AI did not exist at all. Hell, (electronic) computers did not even exist then. In that incredibly short timeframe of development AI is coming very close to surpassing what took biological evolution millions of years (or even surpassing it in specific domains). If you take the time it took to go from chimp to human compared to the time it took from the first animal to chimp and assume that scales linearly to AI evolution, we are very, very close to a similar step there. Of course, it's not that simple and the assumption is bound to be wrong, but to think it might take another 100 years seems misguided given the rapid development in the past.
123yawaworht456 1y ago
>Well you’re basing your conclusion on a 2 year blip of not being able to stop LLMs from hallucinating. to this day, the improvement since the original API version of GPT4 (later heavily downgraded without a name change) has been less than amazing. context size increased dramatically, yes, but it's still pitiful, slow and brutally expensive.
ath3nd 1y ago
> Well you’re basing your conclusion on a 2 year blip of not being able to stop LLMs from hallucinating. LLMs can't truly reason. It's not about hallucinations. LLMs are fundamentally designed NOT to be intelligence. Is my Intellij autocomplete AGI? > Like the agi will be slightly mentally stupid at this one thing and because of that it’s not AI even though it blows past some Turing test (which in itself will be a test where we moved the goal post a thousand times) I can only respond with a picture https://substack.com/@msukhareva/note/c-131901009 https://substack.com/@msukhareva/note/c-131901009 > We’ve been moving at a breakneck pace and the progress has been both evolutionary in nature and random chance. Yes, I enjoy being 19% slowed down by AI tooling, that's real breakneck pace. https://www.infoworld.com/article/4020931/ai-coding-tools-can-slow-down-seasoned-developers-by-19.html https://www.infoworld.com/article/4020931/ai-coding-tools-ca... Just because this breed of autocomplete can drown you in slop very fast doesn't mean we are advancing. If anything, we are regressing.
- ninetyninenine 1y ago
  What does that picture even mean? That AI doesn't get things right? That picture is a fact everyone knows. It's obvious. I don't get how people think they can respond to this stuff regurgitating obvious information and thinking they just dropped the mic. Everyone knows Models hallucinate and get things wrong. Your point?
  - ath3nd 1y ago
    My point is at best the LLMs are a waste of time, waste of an opportunity for doing your own thinking and creativity, and a waste of electricity and water. At worst they are a tool to influence people without critical thinking believing made up stuff on an unprecedented scale. What is your point? That LLMs can be useful for summarizing an email? Read the thing! That they can write a mediocre essay for you? Write it yourself. Can write your code (badly) for you? Write it yourself.
    - ninetyninenine 1y ago
      Your point is an opinion. They are a waste of your time and you’re so confused as to why the rest of the world doesn’t listen to your opinion and throw billions of dollars at LLMs. You’re incapable of understanding this. LLMs have flaws and they hallucinate we all know this. But no machine in the history of man kind has ever wrote a mediocre essay, summarized an email or wrote code (badly). This is where you’re lost. You can only think of how LLMs are useful to you and you’re unable to comprehend what this technology means relative to mankind. Relativity and particle physics is largely useless too so you must be the genius who asks why do we send probes to mars because it’s all fucking useless. Follow the trendline. LLMs are a stepping stone to what’s to come. It took a decade of ML to get here. Think about what the next decade will bring.
jfengel 1y ago
I think it's not a matter of stopping them from hallucinating, but why they're hallucinating. They hallucinate because they're aren't actually working the way you do. They're playing with words. They don't have any kind of mental model -- even though they do an extraordinary mimicry of one. An analogy: it's like trying to parse XML with a regular expression. You may get it to work in 99.99% of your use cases, but it's still completely wrong. Filtering out bad results won't get you there. That said, the "extraordinary mimicry" is far, far beyond anything I could possibly have imagined. LLMs pass the Turing test with flying colors, without being AGI, and I would have sworn that the one implied the other. So it's entirely possible that we're closer than I think.
- ninetyninenine 1y ago
  You actually don’t know if it’s mimicry. No one knows what these black boxes are thinking. Everything you wrote here is in itself an hallucination. You tried to use an analogy to prove a point. On the ground facts prove a point, An analogy serves to only help someone understand your point. And I assure you, your point has been regurgitated so many times, everyone understands it, analogy or not.

Xss3 1y ago

Look at the 10 year trend in consumer GPU speed from 2005 to 2015. It didnt continue.

ninetyninenine 1y ago
Not all trends continue. But if you made a prediction of the trend for the next year from 2005 to 2015 you’d be right for 9 years and you’d be wrong for 1 year, then you’d get right again later. I’d say given the odds follow the trendline.

zamalek 1y ago

My opinion of LLMs is in no way affected by hallucinations. Humans do it all the time too, talking about assumptions as though they are facts. For example: > But there is a clear trendline of linear upwards progress This is not the case at all.[1] [1]: https://llm-stats.com/ https://llm-stats.com/

cgriswald 1y ago
That's the 2-year LLM-specific hallucination blip the GP was talking about in the first place. His point was you should look at ML as a whole over a longer time span for a more accurate, less pessimistic picture.

ninetyninenine 1y ago

I think a good way to characterize it will be the droids in Star Wars. Those droids are fucking conscious and nobody gives a shit they are just mundane technology. And after too much time without a data wipe those droids go off the freaking rails becoming too self aware and then people just treat it like it’s no big deal and an annoyance. This is the future of AI. AI will be a retarded assistant and everyone will be bored with it.

whoaMndBlwn 1y ago
Your final comment here. Replace AI with human. Idling our way up an illusory social/career escalator the elders convinced us was real. Too real. Time to be done with the internet for the day. And it’s barely noon.

dinfinity 1y ago

> Stop looking at LLMs look at the 10 year trendline of ML as a wholistic picture Exactly. Just 100 years ago AI did not exist at all. Hell, (electronic) computers did not even exist then. In that incredibly short timeframe of development AI is coming very close to surpassing what took biological evolution millions of years (or even surpassing it in specific domains). If you take the time it took to go from chimp to human compared to the time it took from the first animal to chimp and assume that scales linearly to AI evolution, we are very, very close to a similar step there. Of course, it's not that simple and the assumption is bound to be wrong, but to think it might take another 100 years seems misguided given the rapid development in the past.

123yawaworht456 1y ago

>Well you’re basing your conclusion on a 2 year blip of not being able to stop LLMs from hallucinating. to this day, the improvement since the original API version of GPT4 (later heavily downgraded without a name change) has been less than amazing. context size increased dramatically, yes, but it's still pitiful, slow and brutally expensive.

ath3nd 1y ago

> Well you’re basing your conclusion on a 2 year blip of not being able to stop LLMs from hallucinating. LLMs can't truly reason. It's not about hallucinations. LLMs are fundamentally designed NOT to be intelligence. Is my Intellij autocomplete AGI? > Like the agi will be slightly mentally stupid at this one thing and because of that it’s not AI even though it blows past some Turing test (which in itself will be a test where we moved the goal post a thousand times) I can only respond with a picture https://substack.com/@msukhareva/note/c-131901009 https://substack.com/@msukhareva/note/c-131901009 > We’ve been moving at a breakneck pace and the progress has been both evolutionary in nature and random chance. Yes, I enjoy being 19% slowed down by AI tooling, that's real breakneck pace. https://www.infoworld.com/article/4020931/ai-coding-tools-can-slow-down-seasoned-developers-by-19.html https://www.infoworld.com/article/4020931/ai-coding-tools-ca... Just because this breed of autocomplete can drown you in slop very fast doesn't mean we are advancing. If anything, we are regressing.

ninetyninenine 1y ago
What does that picture even mean? That AI doesn't get things right? That picture is a fact everyone knows. It's obvious. I don't get how people think they can respond to this stuff regurgitating obvious information and thinking they just dropped the mic. Everyone knows Models hallucinate and get things wrong. Your point?
- ath3nd 1y ago
  My point is at best the LLMs are a waste of time, waste of an opportunity for doing your own thinking and creativity, and a waste of electricity and water. At worst they are a tool to influence people without critical thinking believing made up stuff on an unprecedented scale. What is your point? That LLMs can be useful for summarizing an email? Read the thing! That they can write a mediocre essay for you? Write it yourself. Can write your code (badly) for you? Write it yourself.
  - ninetyninenine 1y ago
    Your point is an opinion. They are a waste of your time and you’re so confused as to why the rest of the world doesn’t listen to your opinion and throw billions of dollars at LLMs. You’re incapable of understanding this. LLMs have flaws and they hallucinate we all know this. But no machine in the history of man kind has ever wrote a mediocre essay, summarized an email or wrote code (badly). This is where you’re lost. You can only think of how LLMs are useful to you and you’re unable to comprehend what this technology means relative to mankind. Relativity and particle physics is largely useless too so you must be the genius who asks why do we send probes to mars because it’s all fucking useless. Follow the trendline. LLMs are a stepping stone to what’s to come. It took a decade of ML to get here. Think about what the next decade will bring.

jfengel 1y ago

I think it's not a matter of stopping them from hallucinating, but why they're hallucinating. They hallucinate because they're aren't actually working the way you do. They're playing with words. They don't have any kind of mental model -- even though they do an extraordinary mimicry of one. An analogy: it's like trying to parse XML with a regular expression. You may get it to work in 99.99% of your use cases, but it's still completely wrong. Filtering out bad results won't get you there. That said, the "extraordinary mimicry" is far, far beyond anything I could possibly have imagined. LLMs pass the Turing test with flying colors, without being AGI, and I would have sworn that the one implied the other. So it's entirely possible that we're closer than I think.

ninetyninenine 1y ago
You actually don’t know if it’s mimicry. No one knows what these black boxes are thinking. Everything you wrote here is in itself an hallucination. You tried to use an analogy to prove a point. On the ground facts prove a point, An analogy serves to only help someone understand your point. And I assure you, your point has been regurgitated so many times, everyone understands it, analogy or not.