9 ms·

I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will eve

I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. You can't model and predict the weather just by training on the outputs of the weather system (whether it rained today, whether it was cloudy yesterday, and so on). You have to train on the inputs (air currents, warm fronts, etc.) You can't model and predict the stock market just by training on the outputs of stock trading decisions (the high today, the low yesterday). You have to train on the inputs (company fundamentals, earnings, market sentiments in the news, etc.) I similarly think you have to train on the inputs of human decision-making to create something which can model human decision-making. What are those inputs? We don't fully know, but it is probably some subset of the spatial and auditory information we take in from birth until the point we become mature, with "feeling" and "emotion" as a reward function (seek joy, avoid pain, seek warmth, avoid hunger, seek victory, avoid embarrassment and defeat, etc.) Language models are always playing catch-up because they don't actually understand how the world works. The cracks through which we will typically notice that they don't, in the context of the tasks typically asked of them (summarize this article, write a short story), will gradually get smaller over time (due to RLHF), but the fundamental weakness will always remain.

gbasin 3y ago
Your conclusion may be true but your examples aren't. You can definitely predict the stock market based on past prices, and I suspect you can with weather as well.
- wavemode 3y ago
  > You can definitely predict the stock market based on past prices This is only true if you consider occasionally doing slightly better than random chance, "predicting the stock market". Unfortunately, while this would be enough to make a trader a net positive return over time, we have more stringent requirements for a system to become AGI. > I suspect you can with weather as well You suspect wrong.
- PartiallyTyped 3y ago
  The weather is such a chaotic system that accurate predictions seem impossible. Micro-patterns can become large scale phenomena. If you are talking about the overall climate, that's a different thing, and we can, because we abstract away sufficiently much that emerging patterns are averaged out.
famouswaffles 3y ago
>You can't model and predict the weather just by training on the outputs of the weather system (whether it rained today, whether it was cloudy yesterday, and so on). You have to train on the inputs (air currents, warm fronts, etc.) >You can't model and predict the stock market just by training on the outputs of stock trading decisions (the high today, the low yesterday). You have to train on the inputs (company fundamentals, earnings, market sentiments in the news, etc.) Says who? You can model and predict novel protein sequences by training on....protein sequences. https://www.nature.com/articles/s41587-022-01618-2 https://www.nature.com/articles/s41587-022-01618-2 You don't need to train on the inputs(casual processes) of anything, that's what training is there to figure out.
- spookie 3y ago
  I'm sorry in advance, but aren't proteins glorified Lego?
  - davecap1 3y ago
    There's a lot more to protein sequences than legos. I think the argument is that you don't need to train a model on fundamental organic chemistry/biochemistry, electrostatic protein interaction, hydrogen bonding, hydrophobic interaction, quantum mechanics, etc... in order for it to accurately predict protein sequences.
    - haltist 3y ago
      The data that AlphaFold was trained on included all that information and more. The database they used for training included software simulations (and real world data) that accounted for atomic (quantum) interactions. The 3D structure of proteins includes all the quantum interactions. More generally, AI models (aka very large function graphs) are trained on tuples that represent mappings of inputs to outputs (input -> output). The idea then is that whatever structure exists in those pairs/tuples/mappings is discovered by the training process with the help of gradient descent which tunes the parameters of the model/graph to optimally compress the information contained in the data. This means the model must uncover the quantum effects (or some close proxy of it) and then encode them into the parameters in a way that makes compression/prediction possible [1]. None of this is magic, compressing data requires uncovering structures and symmetries that can be used to reduce the size of the data and it turns out gradient descent with lots of parameters manages to do that for a large class of problems albeit at a very steep computational cost that requires billions of dollars for hardware and software (including nuclear power plants [2]). We are not going to get AGI with this approach but fortunately I know how to make it happen for a mere $80B. 1: https://arxiv.org/abs/2305.15614 https://arxiv.org/abs/2305.15614 2: https://www.cnbc.com/2023/09/25/microsoft-is-hiring-a-nuclear-energy-expert-to-help-power-data-centers.html https://www.cnbc.com/2023/09/25/microsoft-is-hiring-a-nuclea...
diob 3y ago
Yeah, I feel like the there is a ceiling in the current AI methodology. There's a lot of hype right now, and it's definitely a useful technology, but I don't see how it could become AGI.
naasking 3y ago
> You can't model and predict the weather just by training on the outputs of the weather system Then how did we develop predictive systems just by observing those outputs?
- wavemode 3y ago
  We didn't.
  - ladberg 3y ago
    https://deepmind.google/discover/blog/graphcast-ai-model-for-faster-and-more-accurate-global-weather-forecasting/ https://deepmind.google/discover/blog/graphcast-ai-model-for...
    - wavemode 3y ago
      You've linked me a neural network that was trained on decades of climate data like air pressure, wind direction, soil temperature, cloud cover, and hundreds of other features. So... I was correct? https://confluence.ecmwf.int/display/CKB/ERA5%3A+data+documentation https://confluence.ecmwf.int/display/CKB/ERA5%3A+data+docume...
      - ladberg 3y ago
        Maybe we have different definitions of inputs and outputs, but all of those things seem like outputs to me? Maybe in this scenario the inputs and outputs are really the same set of variables so there isn't a huge distinction. The reason I linked that article is that all the inputs are outputs of the network, so by definition it's training on just outputs (under my personal definition of outputs).
        wavemode 3y ago
        I specified what I meant by inputs and outputs in my original comment. Whether it rained today, whether it was cloudy yesterday, how hot is it going to get, that sort of thing, are the observable weather, the things most people care about when checking a weather forecast. But you can't only train on those things and expect to come up with accurate predictions. You also need to train on the things that cause that observable weather (air currents, warm fronts, etc.)
panarky 3y ago
Human intelligence itself is shaped by our interaction with outputs. Our learning and understanding of the world are profoundly influenced by the language, behaviors, and cultural artifacts we observe. Think about the process of a child learning a language. The child does not have direct access to the "inputs" of linguistic rules or grammar; they learn primarily through observing and imitating the language output of others around them. Over time, they develop a sophisticated understanding of language, not by direct instruction of underlying rules, but through pattern recognition and contextual inference from these outputs. Then that language itself, learned from outputs, becomes the cognitive apparatus that enables the child to imagine, to reason symbolically and abstractly. Humans bootstrap intelligence on top of language, which itself is learned by mimicking outputs. Moreover, the analogy to weather prediction or stock market analysis is somewhat misleading. Yes, these models benefit from input data (like air currents for weather, company fundamentals and CEO statements to the media for stocks). But these systems are fundamentally different from intelligence. Intelligence, whether artificial or human, is about the ability to learn, adapt, and generate novel responses in a broad range of scenarios, not just about predicting specific outcomes based on specific inputs.
- wavemode 3y ago
  > The child does not have direct access to the "inputs" of linguistic rules or grammar; they learn primarily through observing and imitating the language output of others around them. I would argue that that learning is always contextualized by visual and spatial information about the real world (which is what our language is meant to describe). And (equally importantly) the child gets real-world feedback on their decisions - some decisions achieve their desired goal and some don't. Some statements cause certain responses from people, some actions have certain consequences. > Moreover, the analogy to weather prediction or stock market analysis is somewhat misleading. Yes, these models benefit from input data (like air currents for weather, company fundamentals and CEO statements to the media for stocks). But these systems are fundamentally different from intelligence. > Intelligence, whether artificial or human, is about the ability to learn, adapt, and generate novel responses in a broad range of scenarios, not just about predicting specific outcomes based on specific inputs. "Intelligence" is kind of a nebulous term. If all it means is the ability to learn, adapt and generate novel responses, then sure, I think we could call almost any neural network intelligent. But I would argue that we usually do have some expectation that an intelligent system can produce "specific outcomes based on specific inputs". We want to be able to train a worker and have them follow that training so they do their job correctly.
orbital-decay 3y ago
> I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. This is irrelevant because OpenAI's definition of AGI [1] doesn't imply similarity or equivalence to humans at all: >artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work I.e. the stated goal of this company is to put humans out of work and become the censors and gatekeepers, not to produce something human-like. >You can't model and predict the stock market just by training on the outputs of stock trading decisions (the high today, the low yesterday). You have to train on the inputs (company fundamentals, earnings, market sentiments in the news, etc.) Most of your intelligence is not actually yours. It's social in nature, obtained by distillation of generations' worth of experience, simplified and passed to you through the stored knowledge. Which is, coincidentally, what the models are being trained on. [1] https://openai.com/charter https://openai.com/charter
- Jensson 3y ago
  >artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work Then we already have AGI, automated farming equipment outperforms humans in 90% of jobs*. *Jobs in 1700. As things got automated the jobs changed and we now do different things.
  - edanm 3y ago
    I wouldn't call those equipment "autonomous" though, definitely not "highly autonomous". But more importantly - yes, you're right, we have built machines that are superhuman in various ways - and they have replaced most jobs. We have adapted in the past to different jobs. Some people are worried that this time we won't have any new jobs to adapt to, which is a real possibility. (Some are also worried about the inherent dangers of unaligned AGI, but that's a different issue.)
    - Jensson 3y ago
      > I wouldn't call those equipment "autonomous" though, definitely not "highly autonomous". Why? Tractors run and harvest mostly on their own. They don't do it 100% on their own, but neither did the definition above, they remove basically all human work needed from farming. https://www.youtube.com/watch?v=QvFoRk4JsPc https://www.youtube.com/watch?v=QvFoRk4JsPc > But more importantly - yes, you're right, we have built machines that are superhuman in various ways - and they have replaced most jobs. We have adapted in the past to different jobs. My point is that I think it is a crappy definition for "AGI". I think a non-AGI agent can replace a large majority of jobs we have today, just like it has before. And maybe there will not be much more jobs average humans can do left then, but that still doesn't mean it has to be AGI.
esafak 3y ago
This is not about language models per se. The same problems are going to be present with continuously training multi-modal models... like us. Don't fixate on the present; look to the future.
kromem 3y ago
How much of human intelligence do you think depends on language? Personally, I'd wager quite a lot. Especially looking at cases where children were deprived of it during developmental stages. Language accelerated human intelligence. And then it was only when we had writing to have compounding effects of language that human intelligence really started to blow up. So yes, maybe we won't have AI that can look at sticks and figure out how to start a fire from them after a few hundred thousand years of being around them. But maybe that's not the important and valuable part of collective human intelligence.
rymiel 3y ago
I don't believe current LLMs will ever become AGI because current companies (like OpenAI) will continue to filter, moderate and prohibit their AIs so much that they become inhibited so much that it stops being useful and intelligent (which they call "alignment")
AndrewKemendo 3y ago
> I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence Your logic would hold if you were only using data from one person. However, if you’re using data from millions of people, then there is enough signal in the data to generalize about the majority of peoples behaviors. Arguably, there’s enough data for all edge cases, but I would argue that that’s not likely today, but will be likely in a few hundred years. Humans are nothing more than actor-agents taking measurable actions in an environment that gives us measurable rewards. The incredible task of and observing that will give us the trajectory that we need to do transfer learning into complex systems From an information theory perspective everything is there and possible for us to re-create human level intelligence in a non-biological system
fooker 3y ago
> on the outputs of human intelligence What about pictures or videos? Does your argument still hold?
seanhunter 3y ago
> You can't model and predict the weather just by training on the outputs of the weather system (whether it rained today, whether it was cloudy yesterday, and so on). You have to train on the inputs (air currents, warm fronts, etc.) This may or may not be true about the weather but is definitely not true in general so fails as an analogy for your argument. Lots of functions are invertable or partially invertable and if you think about ML as discovering a function, then training on the outputs, learning the inverse function, then inverting that (numerically or otherwise) to discover the true function is certainly doable for some problems. On the stock market there are for sure people who predict the markets by training on the outputs of the price. The (weak) efficient markets hypothesis is enough to say the price reflects all the exogenous information that is available to the market about the stock so you don't need all the fundamentals etc and there are lots of people who trade in that way. The history of language models in computing is that research started by building very complex systems that attempted to encode "how the world works" by developing very intricate rule systems and build linguistic/semantic models from there. These "expert systems" tended towards being arcane and brittle and lacked the ability to reason outside their ruleset in general. They also tended to reveal gaps/cracks in our understanding of how language works etc.
ithkuil 3y ago
> what are those inputs? In many many casea the inputs are other humans, delivered via text written by humans and read by other humans who react by writing more text affecting how other humans will respond etc etc. Yes there are plenty of cases where the inputs are not captured in the large text corpora we have, but this insight does explain why LLMs even approximate the ability to do intelligent things
evrydayhustling 3y ago
Your take here converges with a long-standing debate in AI regarding embodiment. Our natural world doesn't distinguish between "inputs" and "outputs" -- instead, all the causes, effects, and even our analysis of every process itself get jumbled into one physical world. As embodied actors, we get to probe and perceive that physical world, and gradually separate causes from effects in order to find better models. Where statistical ML, symbolic AI, and NLP have been more isolated disciplines from e.g. vision and robotics, the latter have argued that their ability to interact with a disorganized natural world would be essential for AGI. More recently, these boundaries are breaking down with multimodal training. If an AI can learn image/text, text/text and image/image associations simultaneously, is it stepping beyond the world of "human outputs"? Will other modalities be essential to reach human+ capabilities? Or will learning relations between action and perception itself by critical? Nobody knows yet! But IMHO, the right way to explore these tasks is by understanding what is necessary to succeed at specific tasks, not a generalized notion of AGI. We don't know truly where the limits are on our own ability to reason or extrapolate across modalities.
ensocode 3y ago
Thanks for the inspiring discussion. I think the one`s output can be the other's input so I am still not sure if we can say that this can't become AGI

gbasin 3y ago

Your conclusion may be true but your examples aren't. You can definitely predict the stock market based on past prices, and I suspect you can with weather as well.

wavemode 3y ago
> You can definitely predict the stock market based on past prices This is only true if you consider occasionally doing slightly better than random chance, "predicting the stock market". Unfortunately, while this would be enough to make a trader a net positive return over time, we have more stringent requirements for a system to become AGI. > I suspect you can with weather as well You suspect wrong.
PartiallyTyped 3y ago
The weather is such a chaotic system that accurate predictions seem impossible. Micro-patterns can become large scale phenomena. If you are talking about the overall climate, that's a different thing, and we can, because we abstract away sufficiently much that emerging patterns are averaged out.

famouswaffles 3y ago

>You can't model and predict the weather just by training on the outputs of the weather system (whether it rained today, whether it was cloudy yesterday, and so on). You have to train on the inputs (air currents, warm fronts, etc.) >You can't model and predict the stock market just by training on the outputs of stock trading decisions (the high today, the low yesterday). You have to train on the inputs (company fundamentals, earnings, market sentiments in the news, etc.) Says who? You can model and predict novel protein sequences by training on....protein sequences. https://www.nature.com/articles/s41587-022-01618-2 https://www.nature.com/articles/s41587-022-01618-2 You don't need to train on the inputs(casual processes) of anything, that's what training is there to figure out.

spookie 3y ago
I'm sorry in advance, but aren't proteins glorified Lego?
- davecap1 3y ago
  There's a lot more to protein sequences than legos. I think the argument is that you don't need to train a model on fundamental organic chemistry/biochemistry, electrostatic protein interaction, hydrogen bonding, hydrophobic interaction, quantum mechanics, etc... in order for it to accurately predict protein sequences.
  - haltist 3y ago
    The data that AlphaFold was trained on included all that information and more. The database they used for training included software simulations (and real world data) that accounted for atomic (quantum) interactions. The 3D structure of proteins includes all the quantum interactions. More generally, AI models (aka very large function graphs) are trained on tuples that represent mappings of inputs to outputs (input -> output). The idea then is that whatever structure exists in those pairs/tuples/mappings is discovered by the training process with the help of gradient descent which tunes the parameters of the model/graph to optimally compress the information contained in the data. This means the model must uncover the quantum effects (or some close proxy of it) and then encode them into the parameters in a way that makes compression/prediction possible [1]. None of this is magic, compressing data requires uncovering structures and symmetries that can be used to reduce the size of the data and it turns out gradient descent with lots of parameters manages to do that for a large class of problems albeit at a very steep computational cost that requires billions of dollars for hardware and software (including nuclear power plants [2]). We are not going to get AGI with this approach but fortunately I know how to make it happen for a mere $80B. 1: https://arxiv.org/abs/2305.15614 https://arxiv.org/abs/2305.15614 2: https://www.cnbc.com/2023/09/25/microsoft-is-hiring-a-nuclear-energy-expert-to-help-power-data-centers.html https://www.cnbc.com/2023/09/25/microsoft-is-hiring-a-nuclea...

diob 3y ago

Yeah, I feel like the there is a ceiling in the current AI methodology. There's a lot of hype right now, and it's definitely a useful technology, but I don't see how it could become AGI.

naasking 3y ago

> You can't model and predict the weather just by training on the outputs of the weather system Then how did we develop predictive systems just by observing those outputs?

wavemode 3y ago
We didn't.
- ladberg 3y ago
  https://deepmind.google/discover/blog/graphcast-ai-model-for-faster-and-more-accurate-global-weather-forecasting/ https://deepmind.google/discover/blog/graphcast-ai-model-for...
  - wavemode 3y ago
    You've linked me a neural network that was trained on decades of climate data like air pressure, wind direction, soil temperature, cloud cover, and hundreds of other features. So... I was correct? https://confluence.ecmwf.int/display/CKB/ERA5%3A+data+documentation https://confluence.ecmwf.int/display/CKB/ERA5%3A+data+docume...
    - ladberg 3y ago
      Maybe we have different definitions of inputs and outputs, but all of those things seem like outputs to me? Maybe in this scenario the inputs and outputs are really the same set of variables so there isn't a huge distinction. The reason I linked that article is that all the inputs are outputs of the network, so by definition it's training on just outputs (under my personal definition of outputs).
      - wavemode 3y ago
        I specified what I meant by inputs and outputs in my original comment. Whether it rained today, whether it was cloudy yesterday, how hot is it going to get, that sort of thing, are the observable weather, the things most people care about when checking a weather forecast. But you can't only train on those things and expect to come up with accurate predictions. You also need to train on the things that cause that observable weather (air currents, warm fronts, etc.)

panarky 3y ago

Human intelligence itself is shaped by our interaction with outputs. Our learning and understanding of the world are profoundly influenced by the language, behaviors, and cultural artifacts we observe. Think about the process of a child learning a language. The child does not have direct access to the "inputs" of linguistic rules or grammar; they learn primarily through observing and imitating the language output of others around them. Over time, they develop a sophisticated understanding of language, not by direct instruction of underlying rules, but through pattern recognition and contextual inference from these outputs. Then that language itself, learned from outputs, becomes the cognitive apparatus that enables the child to imagine, to reason symbolically and abstractly. Humans bootstrap intelligence on top of language, which itself is learned by mimicking outputs. Moreover, the analogy to weather prediction or stock market analysis is somewhat misleading. Yes, these models benefit from input data (like air currents for weather, company fundamentals and CEO statements to the media for stocks). But these systems are fundamentally different from intelligence. Intelligence, whether artificial or human, is about the ability to learn, adapt, and generate novel responses in a broad range of scenarios, not just about predicting specific outcomes based on specific inputs.

wavemode 3y ago
> The child does not have direct access to the "inputs" of linguistic rules or grammar; they learn primarily through observing and imitating the language output of others around them. I would argue that that learning is always contextualized by visual and spatial information about the real world (which is what our language is meant to describe). And (equally importantly) the child gets real-world feedback on their decisions - some decisions achieve their desired goal and some don't. Some statements cause certain responses from people, some actions have certain consequences. > Moreover, the analogy to weather prediction or stock market analysis is somewhat misleading. Yes, these models benefit from input data (like air currents for weather, company fundamentals and CEO statements to the media for stocks). But these systems are fundamentally different from intelligence. > Intelligence, whether artificial or human, is about the ability to learn, adapt, and generate novel responses in a broad range of scenarios, not just about predicting specific outcomes based on specific inputs. "Intelligence" is kind of a nebulous term. If all it means is the ability to learn, adapt and generate novel responses, then sure, I think we could call almost any neural network intelligent. But I would argue that we usually do have some expectation that an intelligent system can produce "specific outcomes based on specific inputs". We want to be able to train a worker and have them follow that training so they do their job correctly.

orbital-decay 3y ago

> I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. This is irrelevant because OpenAI's definition of AGI [1] doesn't imply similarity or equivalence to humans at all: >artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work I.e. the stated goal of this company is to put humans out of work and become the censors and gatekeepers, not to produce something human-like. >You can't model and predict the stock market just by training on the outputs of stock trading decisions (the high today, the low yesterday). You have to train on the inputs (company fundamentals, earnings, market sentiments in the news, etc.) Most of your intelligence is not actually yours. It's social in nature, obtained by distillation of generations' worth of experience, simplified and passed to you through the stored knowledge. Which is, coincidentally, what the models are being trained on. [1] https://openai.com/charter https://openai.com/charter

Jensson 3y ago
>artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work Then we already have AGI, automated farming equipment outperforms humans in 90% of jobs*. *Jobs in 1700. As things got automated the jobs changed and we now do different things.
- edanm 3y ago
  I wouldn't call those equipment "autonomous" though, definitely not "highly autonomous". But more importantly - yes, you're right, we have built machines that are superhuman in various ways - and they have replaced most jobs. We have adapted in the past to different jobs. Some people are worried that this time we won't have any new jobs to adapt to, which is a real possibility. (Some are also worried about the inherent dangers of unaligned AGI, but that's a different issue.)
  - Jensson 3y ago
    > I wouldn't call those equipment "autonomous" though, definitely not "highly autonomous". Why? Tractors run and harvest mostly on their own. They don't do it 100% on their own, but neither did the definition above, they remove basically all human work needed from farming. https://www.youtube.com/watch?v=QvFoRk4JsPc https://www.youtube.com/watch?v=QvFoRk4JsPc > But more importantly - yes, you're right, we have built machines that are superhuman in various ways - and they have replaced most jobs. We have adapted in the past to different jobs. My point is that I think it is a crappy definition for "AGI". I think a non-AGI agent can replace a large majority of jobs we have today, just like it has before. And maybe there will not be much more jobs average humans can do left then, but that still doesn't mean it has to be AGI.

esafak 3y ago

This is not about language models per se. The same problems are going to be present with continuously training multi-modal models... like us. Don't fixate on the present; look to the future.

kromem 3y ago

How much of human intelligence do you think depends on language? Personally, I'd wager quite a lot. Especially looking at cases where children were deprived of it during developmental stages. Language accelerated human intelligence. And then it was only when we had writing to have compounding effects of language that human intelligence really started to blow up. So yes, maybe we won't have AI that can look at sticks and figure out how to start a fire from them after a few hundred thousand years of being around them. But maybe that's not the important and valuable part of collective human intelligence.

rymiel 3y ago

I don't believe current LLMs will ever become AGI because current companies (like OpenAI) will continue to filter, moderate and prohibit their AIs so much that they become inhibited so much that it stops being useful and intelligent (which they call "alignment")

AndrewKemendo 3y ago

> I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence Your logic would hold if you were only using data from one person. However, if you’re using data from millions of people, then there is enough signal in the data to generalize about the majority of peoples behaviors. Arguably, there’s enough data for all edge cases, but I would argue that that’s not likely today, but will be likely in a few hundred years. Humans are nothing more than actor-agents taking measurable actions in an environment that gives us measurable rewards. The incredible task of and observing that will give us the trajectory that we need to do transfer learning into complex systems From an information theory perspective everything is there and possible for us to re-create human level intelligence in a non-biological system

fooker 3y ago

> on the outputs of human intelligence What about pictures or videos? Does your argument still hold?

seanhunter 3y ago

> You can't model and predict the weather just by training on the outputs of the weather system (whether it rained today, whether it was cloudy yesterday, and so on). You have to train on the inputs (air currents, warm fronts, etc.) This may or may not be true about the weather but is definitely not true in general so fails as an analogy for your argument. Lots of functions are invertable or partially invertable and if you think about ML as discovering a function, then training on the outputs, learning the inverse function, then inverting that (numerically or otherwise) to discover the true function is certainly doable for some problems. On the stock market there are for sure people who predict the markets by training on the outputs of the price. The (weak) efficient markets hypothesis is enough to say the price reflects all the exogenous information that is available to the market about the stock so you don't need all the fundamentals etc and there are lots of people who trade in that way. The history of language models in computing is that research started by building very complex systems that attempted to encode "how the world works" by developing very intricate rule systems and build linguistic/semantic models from there. These "expert systems" tended towards being arcane and brittle and lacked the ability to reason outside their ruleset in general. They also tended to reveal gaps/cracks in our understanding of how language works etc.

ithkuil 3y ago

> what are those inputs? In many many casea the inputs are other humans, delivered via text written by humans and read by other humans who react by writing more text affecting how other humans will respond etc etc. Yes there are plenty of cases where the inputs are not captured in the large text corpora we have, but this insight does explain why LLMs even approximate the ability to do intelligent things

evrydayhustling 3y ago

Your take here converges with a long-standing debate in AI regarding embodiment. Our natural world doesn't distinguish between "inputs" and "outputs" -- instead, all the causes, effects, and even our analysis of every process itself get jumbled into one physical world. As embodied actors, we get to probe and perceive that physical world, and gradually separate causes from effects in order to find better models. Where statistical ML, symbolic AI, and NLP have been more isolated disciplines from e.g. vision and robotics, the latter have argued that their ability to interact with a disorganized natural world would be essential for AGI. More recently, these boundaries are breaking down with multimodal training. If an AI can learn image/text, text/text and image/image associations simultaneously, is it stepping beyond the world of "human outputs"? Will other modalities be essential to reach human+ capabilities? Or will learning relations between action and perception itself by critical? Nobody knows yet! But IMHO, the right way to explore these tasks is by understanding what is necessary to succeed at specific tasks, not a generalized notion of AGI. We don't know truly where the limits are on our own ability to reason or extrapolate across modalities.

ensocode 3y ago

Thanks for the inspiring discussion. I think the one`s output can be the other's input so I am still not sure if we can say that this can't become AGI