4 ms·

How much better can they get chatgpt? It's super impressive, but it seems like this generative AI step, with accepting of mistakes, was a switch in mindset tha

How much better can they get chatgpt? It's super impressive, but it seems like this generative AI step, with accepting of mistakes, was a switch in mindset that made AI suddenly a lot more usefull But it seems like the last 10% that actually makes it take over human tasks almost fully might take another 5-30 years.

M4v3R 3y ago
Here's my semi-educated guess based on observing the rate of progress in this field over the last years: I think there is still improvement to be made in base models. The amount of progress between GPT-2 and GPT-4 in just 4 years is astonishing so I would be surprised if GPT-4 is the limit of what we can do with this technology. I expect at least one more major version that will significantly outperform GPT-4. After that I think the major focus will be making these models more performant, expanding the context window, making them less resource-hungry, adding multimodality and composing them into larger architectures using techniques similar to Chain of Thought - building fully fledged general-purpose agents. These agents will be able to outperform humans at most of the tasks we throw at them. If I were to guess I'd say this will happen by the end of this decade.
- avereveard 3y ago
  It seems to me we'll see large glue model, which will be able to read from model last layer and write directly into model first layer and are trained at composing them. Some is already happening, and it cannot be generalized, but the idea of being able to glue already trained medium sized model together and then do a retrain of the whole thing to improve performances seems already establishing itself in products and will allow much efficient expenditure of training resources.
ly3xqhl8g9 3y ago
So what if it will take another 30 years? Barely 71 years ago MIT demonstrated one of the first Numerical Control mill for manufacturing [1] that gave us the world as we have it today: from our chairs to our CPUs, nothing could be obtained so fast/cheap/good without automated milling, molding, and so on. It just seems completely unimaginable how by 2094 we won't have synthetic agents being not only great tutors, but also replacing every job that we have today or we will be able to imagine in the interim. The problem is then with our mental models, our economics, our metaphysics, not with the technology. [1] https://en.wikipedia.org/wiki/History_of_numerical_control#MIT's_machine https://en.wikipedia.org/wiki/History_of_numerical_control#M...
- layer8 3y ago
  We don’t know what the limits are, what the difficulties will be. You can’t just extrapolate from the past. We’ll have to wait and see where things go.
  - ly3xqhl8g9 3y ago
    As far as cognitive capacity and consciousness as self-monitoring agency we have N=1, ourselves. Are there limits above us? Probably, but we have no reason to believe we are anywhere near the pinnacle of cognitive or consciousness load. Sure we can extrapolate from the past, we do it daily when we build something, if we didn't we would get nothing done.
- palata 3y ago
  Well the world may have burned in 30 years (that doesn't seem unlikely at all). So I can perfectly imagine that by 2094 we don't have those synthetic agents. Maybe we'll just be focused on not starving.
  - ly3xqhl8g9 3y ago
    I am starving today, being starved by a system which puts me in artificial scarcity loops, me and who knows how many others. Starvation does not hinder the desire for resolution, if anything, it fosters it. But yes, the world might disappear tomorrow; it's just not a workable hypothesis. It's much more workable to think that the complexity of our tools will continue to increase, elevating themselves towards higher and higher abstraction layers.
    - palata 3y ago
      Well in 2023, I think it is reasonable to wonder what will happen to our tools and technology when oil becomes scarce (peak oil was 2008, oil is not unlimited). I was not saying that the Sun may explode. I was saying that we have good reasons to think that energy will become more and more expensive in the next few decades.
- 3y ago
FredPret 3y ago
The definition of what human tasks are is fluid. Every time a machine arrives that does what we used to do, human tasks are pushed up the chain of abstraction. This will only change if an AI arrives that is better at us at the only IQ test that counts - navigating the complexities of real life; and maybe not even then.

M4v3R 3y ago

Here's my semi-educated guess based on observing the rate of progress in this field over the last years: I think there is still improvement to be made in base models. The amount of progress between GPT-2 and GPT-4 in just 4 years is astonishing so I would be surprised if GPT-4 is the limit of what we can do with this technology. I expect at least one more major version that will significantly outperform GPT-4. After that I think the major focus will be making these models more performant, expanding the context window, making them less resource-hungry, adding multimodality and composing them into larger architectures using techniques similar to Chain of Thought - building fully fledged general-purpose agents. These agents will be able to outperform humans at most of the tasks we throw at them. If I were to guess I'd say this will happen by the end of this decade.

avereveard 3y ago
It seems to me we'll see large glue model, which will be able to read from model last layer and write directly into model first layer and are trained at composing them. Some is already happening, and it cannot be generalized, but the idea of being able to glue already trained medium sized model together and then do a retrain of the whole thing to improve performances seems already establishing itself in products and will allow much efficient expenditure of training resources.

ly3xqhl8g9 3y ago

So what if it will take another 30 years? Barely 71 years ago MIT demonstrated one of the first Numerical Control mill for manufacturing [1] that gave us the world as we have it today: from our chairs to our CPUs, nothing could be obtained so fast/cheap/good without automated milling, molding, and so on. It just seems completely unimaginable how by 2094 we won't have synthetic agents being not only great tutors, but also replacing every job that we have today or we will be able to imagine in the interim. The problem is then with our mental models, our economics, our metaphysics, not with the technology. [1] https://en.wikipedia.org/wiki/History_of_numerical_control#MIT's_machine https://en.wikipedia.org/wiki/History_of_numerical_control#M...

layer8 3y ago
We don’t know what the limits are, what the difficulties will be. You can’t just extrapolate from the past. We’ll have to wait and see where things go.
- ly3xqhl8g9 3y ago
  As far as cognitive capacity and consciousness as self-monitoring agency we have N=1, ourselves. Are there limits above us? Probably, but we have no reason to believe we are anywhere near the pinnacle of cognitive or consciousness load. Sure we can extrapolate from the past, we do it daily when we build something, if we didn't we would get nothing done.
palata 3y ago
Well the world may have burned in 30 years (that doesn't seem unlikely at all). So I can perfectly imagine that by 2094 we don't have those synthetic agents. Maybe we'll just be focused on not starving.
- ly3xqhl8g9 3y ago
  I am starving today, being starved by a system which puts me in artificial scarcity loops, me and who knows how many others. Starvation does not hinder the desire for resolution, if anything, it fosters it. But yes, the world might disappear tomorrow; it's just not a workable hypothesis. It's much more workable to think that the complexity of our tools will continue to increase, elevating themselves towards higher and higher abstraction layers.
  - palata 3y ago
    Well in 2023, I think it is reasonable to wonder what will happen to our tools and technology when oil becomes scarce (peak oil was 2008, oil is not unlimited). I was not saying that the Sun may explode. I was saying that we have good reasons to think that energy will become more and more expensive in the next few decades.
3y ago

FredPret 3y ago

The definition of what human tasks are is fluid. Every time a machine arrives that does what we used to do, human tasks are pushed up the chain of abstraction. This will only change if an AI arrives that is better at us at the only IQ test that counts - navigating the complexities of real life; and maybe not even then.