4 ms·

> This is such a weird point to make I think it is a great point to make, because if everyone really believed that AIs will do everything without human interve

> This is such a weird point to make I think it is a great point to make, because if everyone really believed that AIs will do everything without human intervention in a handful of years, as the marketing repeats again and again (AGI, singularity, etc.) and have been saying for years... why then get bothered? Because we DO know LLMs have their hallucinations, limitations, perform tasks not previously seen way worse than humans, etc. And it seems that, for now, there is not a good or magic solution to it, it is inherent limitations of the paradigm. Yes, you can feed more and more and more (curated data) and eventually make AIs excellent at task X or Y, but then you spend your time specializing those engines. So the work does not really disappear, it just shifts and you make it more replicable for a bound set of problems. Needless to say that at some point I prefer to learn (and combine with AIs, it is ok) than acritically getting inputs from something until I become totally useless. Unless we have a paradigm for which a fully autonomous AI can do everything, this will just become improving our productivity in some ways, with all the in-between bottlenecks that it has.

nextlevelwizard 2mo ago
Why spend money and time making the new flagship model when a future flagship model can make you the flagship model?
- germandiago 2mo ago
  Then why not stop researching and doing the definitve model that will solve every problem? Why some people are not doing it? Bc they are aware of the marketing and limitations. If they did believe it, then they would switch area of research.
joaquieneCnix 2mo ago
> inherent limitations of the paradigm This is such a weird point to make. We are currently ( only ) discovering that paradigm; we are not inventing anything. We found a bunch of laws that produce rather cool results but our paradigm is incomplete which leads more or less wordy or frame-rich weird stuff like hallucinations, singularity and so on ... it's childish, really and on that funny pseudo-profound, pseudo-intellectual, pseudo-spiritual ( personal opinion, if it gets you horny, you go, baby ) "universe consciousness unity, Rick James, bitch" level ... Our bodies and minds need proper AI, not all the stuff we already outsource to middle and/or passionate men and women. Other species on the planet would certainly like to see us get augmented by AI so we can solve as many survivability issues as possible to keep as many ecosystems running long enough ... whatever that means but whether animals and plants are aware of chance and potential is another philosophical debate. To individuals, software is a hammer and chisel, a knife, a brush and canvas, pen and paper, a reading help, and to a good amount of people it's a microscope and a fine scalpel. To collectives, it's a tool to work on consensus and conventions, to share and gather. It's baby steps for civilizations and it looks like our particular species is gonna get stuck in a puddle of our own monkey shit, with bottles of champagne in our hands and monkeys grinding up and down the few ivory towers in proximity. > why then get bothered Humans are on different levels. Most have decided that "nature realized the/a bug and wanted someone dead" or "their survival is a matter of chance" is not acceptable at all and some people decided that sabotage, poison, abuse, rape, murder are acceptable means to get chicken shit ... The "paradigm" of life is far from explored/discovered, so we simply can't content ourselves with presumptions about inherent limitations of the LLM and AI paradigm for any other reason than to uncover ( not invent ) other parts of the paradigm. We are happy with what AI can do for us but "AIs will do everything without human intervention" sounds weird because babies are born and the older they get and the less sabotaged ( vs influence, cultural manipulation ) they get to grow up, the more breadth and depth humans want to experience. For this they need to learn and use their hands & fingers. They need to feed body and mind to find what triggers what, and what excitement and curiosity are inherent and which can or need to be added/acquired/experienced extrinsically. How many associations will we be able to make if AIs will do everything without human intervention?
Yopolo 2mo ago
No we don't know your 'points' The Hallucinations are becoming less, significantly by now. It also might be already were it is cheaper for one of the big few companies to spend millions and billions to teach the LLM / creating the training data necessary for an LLM to do something which it is not yet good enough due to the fact, that they sell this capability then to everyone who wants to use this capabilitiy. We have not seen the end of Reinforcement Learning, which does need a lot less training data but more compute. I'm 'vibing' on the side a handfull of small things, no LLM trained on particular what i'm asking to do. Its very capable of stringing together enough things so it can clearly follow handwavy things i tell it to do, analyse error messages, analysing screenshots etc. all by itself. There is not a single real ceilling in sight, we only have clear barriers like compute but constant fast progress. The field of mathematics went from 'useless' to 'you start better using it' to 'gamechanger' in how fast? 1 year after coding? less? I want signes that we hit a real problem, instead I get cheaper tokens, Chinese models becoming very good as open models, new model updates from the others, mathematicans now saying how good it is etc. Only half a year ago I had to babysit an LLM, now i tell it 1-3 sentences and it just goes and does it. And that stuff runs without compile errors etc. If AI makes us 10% or 20% betteer, which is not that much, this alone will lead to companies reduing their expensive staff by 10-20%, which will has real impact on a job area. Some jobs are already hard to sell like cyber security and basic image tasks.
- germandiago 2mo ago
  > The Hallucinations are becoming less, significantly by now. Yes? What is the mega-solid technique that is used for it? Armies of people using curated data and reviewing it by hand? That is exactly one of my points: shifting the work elsewhere for specialized tasks. More replicable, improved, but, it scales infinitely and is autonomous? Can you assert that? I am not denying there is some use (a lot of uses!) for this, but this is more nuanced than just: oh, they will replace us. Not at all, that day, with the current technology, is not going to arrive. This is just a systematization, fitting and tweaking of human knowledge by curated data. It is not the one true superintelligence they are selling us. To begin with, they do not have a concept of truth, but of probabilistic truth. Only that poses already a very, very big problem for the path to perfection. > We have not seen the end of Reinforcement Learning, which does need a lot less training data but more compute. Noone said the opposite, but I would like to know at which cost and if it is feasible. We do not have even enough compute power for current technology. > . Its very capable of stringing together enough things so it can clearly follow handwavy things i tell it to do, analyse error messages, analysing screenshots etc. all by itself. I use it every day for these tasks and it works well BECAUSE I review the output and makes me go faster. It finds a lot of things I would have not found and it also hallucinates another handful of them, which confirms my point about AIs not being able to be fully autonomous in any future point in time unless tweaked exactly for the task, and even then, it can still miss judgement a human could have for edge cases. So I am not sure of how bad or good it can be compared to a human but I am pretty sure it cannot be more reliable than an expert in many situations. > Chinese models becoming very good as open models I think they will be better in the long term if they follow this path. Not absolutely better but when mixing with economics and the fact that no frontier model is totally reliable anyway... why pay a lot for something that needs human inspection anyway? > There is not a single real ceilling in sight, we only have clear barriers like compute but constant fast progress. The ceiling is the paradigm itself, as I mentioned above. There is not a single chance with current technology that something could become "generically knowledgeable" and "reliable" both at the same time. If it becomes generically knowledgeable and reliable, it is bc of data fed into it and curated and tweaked by humans. This is not an original idea from myself, there are armies of people doing this every day around the world, you can check. This is where a lot of improvement comes from. Can this be reused? Of course. It is a generic solution? No way. > Only half a year ago I had to babysit an LLM, now i tell it 1-3 sentences and it just goes and does it. And that stuff runs without compile errors etc. Yes, I also do one-off scripts like this and code snippets, even reviews and others. Now go design a full distributed system. Use agents if you want. We come back in six months and compare it to a system that was properly written and tested by humans and we can compare the quality on some grounds: 1. how long it takes to add new features? 2. which ones act more according to spec once added? 3. when adding features, which ones have more bugs? 4. in the face of an error, will the agent delete my whole AWS infra (count the money losses if possible also)? 5. will I understand (or need to understand, but I bet yes) this code at some point in the future? You have to count all that money also, not just I vibe coded something and it seemed to work. With full systems things become super messy. Now add the human factor of requirements and back and forth (iterations can be admittedly faster with AI, especially prototypes, but that comes with other costs also)... Not easy at all.
dmurray 2mo ago
It's a very silly point to make to AI researchers specifically. If they don't work on those projects, the AI won't advance and won't magically be able to replicate the work in "one to three years".
- jbstack 2mo ago
  Relevant xkcd: https://xkcd.com/989/ https://xkcd.com/989/
- pdhborges 2mo ago
  Can you imagine scenarios that would make it less silly? I will give an example: - The AI researcher might be working for a lab or company with much less funds than the top dogs. Are they likely to discover something that is worth it before a bigger model becomes more capable?
  - TeMPOraL 2mo ago
    That company or their staff is probably focusing its funds on getting itself acquired/acquihired by the top dog.
  - _aavaa_ 2mo ago
    Ask the researchers working on Deepseek. They seem to be doing pretty well for themselves. Just because a AI will be able to do it in the future does not mean that us plebs will be allowed to have access to it. That alone is enough of a reason for smaller labs to keep going; having a seat at the table.
satvikpendem 2mo ago
The AI researchers are not the ones making the marketing, much less believing in it.
- germandiago 2mo ago
  I agree. But this is not what you see on the headlines and what money-incentivized stakeholders are saying.
- dofm 2mo ago
  Is this really true? At least one very headline AI researcher is pretty much an Anthropic spokesperson.
  - satvikpendem 2mo ago
    That they believe in the PR is itself PR, they're paid to be spokespeople as well as researchers.
swat535 2mo ago
> you can feed more and more and more I have a meta thought.. Hypothetically what happens once there is no more data to be fed to the system? Are we expecting AI to invent its own data and reach full cognition? Currently we are feeding it the data that humans created but if we stop (i.e "why bother?") thinking that AI will do it all?
- xmcqdpt2 2mo ago
  This is a well-known problem that has been an issue for years now. You can't use models to generate data for models because it leads to "model collapse" where it amplifies quirks in the generated data until it's all quirks. Here is a random university press release about it (grain of salt etc) https://www.utoronto.ca/news/training-ai-machine-generated-text-could-lead-model-collapse-researchers-warn https://www.utoronto.ca/news/training-ai-machine-generated-t... In practice you can do it a bit (generated data from a better / different model is fine, some generated data might be useful if there is non generated data etc.)
  - ilmaz10x 2mo ago
    [dead]

nextlevelwizard 2mo ago

Why spend money and time making the new flagship model when a future flagship model can make you the flagship model?

germandiago 2mo ago
Then why not stop researching and doing the definitve model that will solve every problem? Why some people are not doing it? Bc they are aware of the marketing and limitations. If they did believe it, then they would switch area of research.

joaquieneCnix 2mo ago

> inherent limitations of the paradigm This is such a weird point to make. We are currently ( only ) discovering that paradigm; we are not inventing anything. We found a bunch of laws that produce rather cool results but our paradigm is incomplete which leads more or less wordy or frame-rich weird stuff like hallucinations, singularity and so on ... it's childish, really and on that funny pseudo-profound, pseudo-intellectual, pseudo-spiritual ( personal opinion, if it gets you horny, you go, baby ) "universe consciousness unity, Rick James, bitch" level ... Our bodies and minds need proper AI, not all the stuff we already outsource to middle and/or passionate men and women. Other species on the planet would certainly like to see us get augmented by AI so we can solve as many survivability issues as possible to keep as many ecosystems running long enough ... whatever that means but whether animals and plants are aware of chance and potential is another philosophical debate. To individuals, software is a hammer and chisel, a knife, a brush and canvas, pen and paper, a reading help, and to a good amount of people it's a microscope and a fine scalpel. To collectives, it's a tool to work on consensus and conventions, to share and gather. It's baby steps for civilizations and it looks like our particular species is gonna get stuck in a puddle of our own monkey shit, with bottles of champagne in our hands and monkeys grinding up and down the few ivory towers in proximity. > why then get bothered Humans are on different levels. Most have decided that "nature realized the/a bug and wanted someone dead" or "their survival is a matter of chance" is not acceptable at all and some people decided that sabotage, poison, abuse, rape, murder are acceptable means to get chicken shit ... The "paradigm" of life is far from explored/discovered, so we simply can't content ourselves with presumptions about inherent limitations of the LLM and AI paradigm for any other reason than to uncover ( not invent ) other parts of the paradigm. We are happy with what AI can do for us but "AIs will do everything without human intervention" sounds weird because babies are born and the older they get and the less sabotaged ( vs influence, cultural manipulation ) they get to grow up, the more breadth and depth humans want to experience. For this they need to learn and use their hands & fingers. They need to feed body and mind to find what triggers what, and what excitement and curiosity are inherent and which can or need to be added/acquired/experienced extrinsically. How many associations will we be able to make if AIs will do everything without human intervention?

Yopolo 2mo ago

No we don't know your 'points' The Hallucinations are becoming less, significantly by now. It also might be already were it is cheaper for one of the big few companies to spend millions and billions to teach the LLM / creating the training data necessary for an LLM to do something which it is not yet good enough due to the fact, that they sell this capability then to everyone who wants to use this capabilitiy. We have not seen the end of Reinforcement Learning, which does need a lot less training data but more compute. I'm 'vibing' on the side a handfull of small things, no LLM trained on particular what i'm asking to do. Its very capable of stringing together enough things so it can clearly follow handwavy things i tell it to do, analyse error messages, analysing screenshots etc. all by itself. There is not a single real ceilling in sight, we only have clear barriers like compute but constant fast progress. The field of mathematics went from 'useless' to 'you start better using it' to 'gamechanger' in how fast? 1 year after coding? less? I want signes that we hit a real problem, instead I get cheaper tokens, Chinese models becoming very good as open models, new model updates from the others, mathematicans now saying how good it is etc. Only half a year ago I had to babysit an LLM, now i tell it 1-3 sentences and it just goes and does it. And that stuff runs without compile errors etc. If AI makes us 10% or 20% betteer, which is not that much, this alone will lead to companies reduing their expensive staff by 10-20%, which will has real impact on a job area. Some jobs are already hard to sell like cyber security and basic image tasks.

germandiago 2mo ago
> The Hallucinations are becoming less, significantly by now. Yes? What is the mega-solid technique that is used for it? Armies of people using curated data and reviewing it by hand? That is exactly one of my points: shifting the work elsewhere for specialized tasks. More replicable, improved, but, it scales infinitely and is autonomous? Can you assert that? I am not denying there is some use (a lot of uses!) for this, but this is more nuanced than just: oh, they will replace us. Not at all, that day, with the current technology, is not going to arrive. This is just a systematization, fitting and tweaking of human knowledge by curated data. It is not the one true superintelligence they are selling us. To begin with, they do not have a concept of truth, but of probabilistic truth. Only that poses already a very, very big problem for the path to perfection. > We have not seen the end of Reinforcement Learning, which does need a lot less training data but more compute. Noone said the opposite, but I would like to know at which cost and if it is feasible. We do not have even enough compute power for current technology. > . Its very capable of stringing together enough things so it can clearly follow handwavy things i tell it to do, analyse error messages, analysing screenshots etc. all by itself. I use it every day for these tasks and it works well BECAUSE I review the output and makes me go faster. It finds a lot of things I would have not found and it also hallucinates another handful of them, which confirms my point about AIs not being able to be fully autonomous in any future point in time unless tweaked exactly for the task, and even then, it can still miss judgement a human could have for edge cases. So I am not sure of how bad or good it can be compared to a human but I am pretty sure it cannot be more reliable than an expert in many situations. > Chinese models becoming very good as open models I think they will be better in the long term if they follow this path. Not absolutely better but when mixing with economics and the fact that no frontier model is totally reliable anyway... why pay a lot for something that needs human inspection anyway? > There is not a single real ceilling in sight, we only have clear barriers like compute but constant fast progress. The ceiling is the paradigm itself, as I mentioned above. There is not a single chance with current technology that something could become "generically knowledgeable" and "reliable" both at the same time. If it becomes generically knowledgeable and reliable, it is bc of data fed into it and curated and tweaked by humans. This is not an original idea from myself, there are armies of people doing this every day around the world, you can check. This is where a lot of improvement comes from. Can this be reused? Of course. It is a generic solution? No way. > Only half a year ago I had to babysit an LLM, now i tell it 1-3 sentences and it just goes and does it. And that stuff runs without compile errors etc. Yes, I also do one-off scripts like this and code snippets, even reviews and others. Now go design a full distributed system. Use agents if you want. We come back in six months and compare it to a system that was properly written and tested by humans and we can compare the quality on some grounds: 1. how long it takes to add new features? 2. which ones act more according to spec once added? 3. when adding features, which ones have more bugs? 4. in the face of an error, will the agent delete my whole AWS infra (count the money losses if possible also)? 5. will I understand (or need to understand, but I bet yes) this code at some point in the future? You have to count all that money also, not just I vibe coded something and it seemed to work. With full systems things become super messy. Now add the human factor of requirements and back and forth (iterations can be admittedly faster with AI, especially prototypes, but that comes with other costs also)... Not easy at all.

dmurray 2mo ago

It's a very silly point to make to AI researchers specifically. If they don't work on those projects, the AI won't advance and won't magically be able to replicate the work in "one to three years".

jbstack 2mo ago
Relevant xkcd: https://xkcd.com/989/ https://xkcd.com/989/
pdhborges 2mo ago
Can you imagine scenarios that would make it less silly? I will give an example: - The AI researcher might be working for a lab or company with much less funds than the top dogs. Are they likely to discover something that is worth it before a bigger model becomes more capable?
- TeMPOraL 2mo ago
  That company or their staff is probably focusing its funds on getting itself acquired/acquihired by the top dog.
- _aavaa_ 2mo ago
  Ask the researchers working on Deepseek. They seem to be doing pretty well for themselves. Just because a AI will be able to do it in the future does not mean that us plebs will be allowed to have access to it. That alone is enough of a reason for smaller labs to keep going; having a seat at the table.

satvikpendem 2mo ago

The AI researchers are not the ones making the marketing, much less believing in it.

germandiago 2mo ago
I agree. But this is not what you see on the headlines and what money-incentivized stakeholders are saying.
dofm 2mo ago
Is this really true? At least one very headline AI researcher is pretty much an Anthropic spokesperson.
- satvikpendem 2mo ago
  That they believe in the PR is itself PR, they're paid to be spokespeople as well as researchers.

swat535 2mo ago

> you can feed more and more and more I have a meta thought.. Hypothetically what happens once there is no more data to be fed to the system? Are we expecting AI to invent its own data and reach full cognition? Currently we are feeding it the data that humans created but if we stop (i.e "why bother?") thinking that AI will do it all?

xmcqdpt2 2mo ago
This is a well-known problem that has been an issue for years now. You can't use models to generate data for models because it leads to "model collapse" where it amplifies quirks in the generated data until it's all quirks. Here is a random university press release about it (grain of salt etc) https://www.utoronto.ca/news/training-ai-machine-generated-text-could-lead-model-collapse-researchers-warn https://www.utoronto.ca/news/training-ai-machine-generated-t... In practice you can do it a bit (generated data from a better / different model is fine, some generated data might be useful if there is non generated data etc.)
- ilmaz10x 2mo ago
  [dead]