16 ms·

The near future of AI is action-driven

132 points by hardmaru 4y ago

zby 4y ago
I am convinced by Judea Pearl (https://en.wikipedia.org/wiki/The_Book_of_Why https://en.wikipedia.org/wiki/The_Book_of_Why) that the missing part of current AI is causality reasoning. It is impossible to inference causality from raw data - that is data without any a priori causality models. To develop causality ideas you need to make experiments. After you have some initial causal theories you can develop them further by adding observations - but you cannot bootstrap them entirely just from observations. Making experiments would also be much more efficient then just observing them - because you could test the exact stuff that is needed for your theories to develop. So yeah - action on the real world is required. By the way I have an intuition that free will is based on this - we need an irreducible source of cause to learn about causality and we call that the free will.
- naasking 4y ago
  > It is impossible to inference causality from raw data - that is data without any a priori causality models. It's impossible in general to be certain of causality for humans too, all we see are correlations in data and we invent causal theories that explain those correlations. That's basically what machine learning does as well: compile sets of correlated values into a compressed representation (the neural net) which arguably qualifies as a "causal theory" from the algorithm's perspective. I think what's missing is maybe one or two orders of magnitude more compression, which would basically be devising a better/more parsimonious theory. We've been getting progressively better at that over the years too, and combined with how hardware has been scaling, we're seeing exponential growth in effectiveness. This is why some are predicting artificial general intelligence by 2030-2035.
  - zby 4y ago
    > It's impossible in general to be certain of causality for humans too, all we see are correlations in data and we invent causal theories that explain those correlations. We do experiments - we actually do them a lot and it starts very early with infants experimenting how activating a particular muscle moves his hand etc. Later we do less of original experiments - but repeat those that we already know the results - by walking for example and getting where we planned to. In science to get from correlations to causation we have 'randomized trials'. In everyday practice we use something called 'free will' in our minds as the source of the needed independence.
  - woojoo666 4y ago
    > It's impossible in general to be certain of causality for humans too, all we see are correlations in data and we invent causal theories that explain those correlations Not sure about that. MinutePhysics has a video about how correlation can imply causality: https://www.youtube.com/watch?v=HUti6vGctQM https://www.youtube.com/watch?v=HUti6vGctQM
TheCaptain4815 4y ago
This is the 3rd time I've heard of this "action-driven" LLM. The first was the Microsoft CTO discussing it here - https://www.youtube.com/watch?v=LYBcH3JjLxs&t=529s https://www.youtube.com/watch?v=LYBcH3JjLxs&t=529s The second was the former head of Tesla Ai division on Lex Friendman Now this article This is quite telling of how powerful these models will be, even on launch. The Microsoft CTO mentioning it in particular is interesting since he's obviously seen internal and alpha results.
camjohnson26 4y ago
I’m skeptical since we’ve heard the same promises of how chatbots will replace our computer interfaces for years now. The systems always seem to break down when the user needs to do something surprising. Think about trying to navigate an automated phone menu when your option isn’t available. Transformers seem incredibly good at picking up patterns and repeating them, and combining them in new and surprising ways, but my intuition says that the unpredictability of the real world will cause the required training set to grow faster than the models can keep up, at least to get close to anything that “looks a lot like AGI.”
- jvm 4y ago
  Author here, yeah I agree the real world is very complicated. For any given task I'm imagining something like 1. Use some sort of instruction tuning to get the thing "good enough" that it gives decent results 75% of the time and the other 25% a human has to take over. 2. Use the actual usage data as training input. Punish bad behaviors and show the model what the human did to solve the problem. 3. Use this training loop to progressively have the model take over a larger % of the time. …and I think if you can't get (1) good enough to be worth using it's going to be really hard to get the loop going.
  - huijzer 4y ago
    > Use some sort of instruction tuning to get the thing "good enough" that it gives decent results 75% of the time and the other 25% a human has to take over. How does the model know when a human has to take over? I think most extrapolations of current "AI" capabilities into future capabilities are fun and useful in some ways, but also doomed to fail. It's very easy to miss a tiny detail which may in practice be a fundamental problem. > Use the actual usage data as training input. Given that those bigger state-of-the-art models train on terabytes of data, how would you know how much training data to generate to sufficiently change the output? My understanding of "AI" is that it's mostly about some very complex models which are capable of solving previously unsolvable problems. However, those problems are always extremely specific. Going the other way of thinking of problems or future possibilities first and then applying "AI" to it is likely to fail.
denkmoon 4y ago
Would be really nice to have these Voice Assistants not be dumber than a pile of rocks. Seems like the article suggests this will be the case in the near future.
vagabund 4y ago
I'd say this isn't the near future so much as it's the present. If you're on ML twitter you'll see countless examples of people hacking together prompts that power a google sheet or something similar. It just doesn't really make sense to do the engineering required to build legit enterprise tools around these capabilities while the SOTA is advancing this rapidly.
- automateanddone 4y ago
  Can you share a few twitter accounts to follow in ML twitter? Thanks!!
  - supersync 4y ago
    https://twitter.com/yoheinakajima https://twitter.com/yoheinakajima Is my goto
  - vagabund 4y ago
    In no order, by no means exhaustive, and lazily copied from my following list, but these should surface a good deal of quality ML content on your timeline, both for the state of the literature, and for more speculative, forward-looking theorizing. @nearcyan @arankomatsuzaki @ethanCaballero @_albertgu @lcastricato @tszzl @gwern @fchollet @drmichaellevin @Plinz @sama @sharifshameem @woj_zaremba @ilyasut @karpathy @RichardSSutton @RiversHaveWings @ericjang11 @_akhaliq @hardmaru note: some, like Sutton and Sutskever, aren't terribly active, but they're worth following anyways since it signals to the algorithm which sorts of tweets to suggest
- wokwokwok 4y ago
  I’m pretty skeptical. The article is specifically calling out that current models suck at this, and action driven models would proactively determine what additional input is needed seek out data to augment their input to make decisions. None of the Twitter accounts you’ve linked to seem to be doing this. Ie. specifically prompt -> actions to collect additional information as determined from prompt -> internal reprompt with additional input -> response. To quote the article: > This is not 10 year tech. It may be possible right now with off-the-shelf tools. But to make it work we need to set up the right feedback loops. The right feedback loops. How do we get those? I have seen exactly zero examples of people successfully doing this so far; the only working examples are prompt-reponse/action which are not, remotely, AGI like. That’s the point they’re making: something that internally reprompts itself is quite AGI like. Something that does not is a bot. It seems plausible we might get this kind of adaptive agent sooner than AGI It certainly doesn’t seem to exist right now, and the path to “self reprompting” feedback loops is quite unclear.
spyckie2 4y ago
I think we are still a long way off from business viability, but a very short way off from POC (proof of concept) demos that show its potential. Kind of like self driving cars in the 2017 hype. Doing something is not hard. Accounting for all the possible problems and making sure they don’t occur is the hard part because it requires a level of control that goes beyond what looks like action driven gives.
- dqpb 4y ago
  > long way off from business viability I'm not interested in business viability. I want something that's quirky, unpredictable, and human-like. I want an agent that runs in its own game-loop. I don't want another goddamn web service.
- huijzer 4y ago
  > I think we are still a long way off from business viability, but a very short way off from POC (proof of concept) demos that show its potential. Kind of like self driving cars in the 2017 hype. We still don't have self driving cars, right? Except maybe for a few labs, suburbs, and highways.
  - KineticLensman 4y ago
    That’s exactly the point - we still haven’t gone from the POC to a robust capability
  - charcircuit 4y ago
    >We still don't have self driving cars, right? Yes we do. They exist and are on the road. Just look at Tesla, Waymo, etc. These cars are able to be given a destination and they will drive themselves there. There are limitations, but that doesn't make them not self driving.
    - spyckie2 4y ago
      If you don't set a definition for "limitation", self driving cars existed long before Tesla and Waymo, in niche pockets of the world. But if you set the definition of self driving car as what you would expect in a sci-fi novel, we're not there yet. But we're actually really close - I'd bet in 3 years Waymo is makes them a reality in day to day life.
adamnemecek 4y ago
- charcircuit 4y ago
  The page you linked does not mention AI. What do you mean by "fixed point driven"?
  - svnt 4y ago
    I think they mean data types: fixed point (e.g. int) vs floating point (double). Research has been pushing that way. I don’t see what it has to do with adjointness other than maybe that a fixed point variable separates mathematically-defined space more reliably than a floating point variable. I don’t see what either really has to do with this post. Fixed point variables do not require not using feedback loops.
deleted 4y ago
[deleted]
ilaksh 4y ago
The biggest part usually missing is spatial information. I think that the text-to-video stuff is going in the direction of having a real understanding of the 3d world (or other non-textual structures) and how it relates to textual descriptions. I think that with things like transformers that probably means real language grounding. So it might be something like using the prompt to generate an embedding that captures spatial-temporal aspects of a prompt. Or really somehow the whole prompt encoding should terminate at some type of spatial-temporal tokens maybe. I don't know exactly but I think that text-to-video and video-to-text is going to lead in a more general purpose direction than just language alone.
- darepublic 4y ago
  I was thinking about this before and I thought of having a 3d physics engine that the AI could create objects in and simulate things to see their physical viability. Could also help with question answering that requires that spatial knowledge / real world simulation.
  - zone411 4y ago
    This has been a thing for a while. For example, here are a couple random papers from 2017: https://openaccess.thecvf.com/content_cvpr_2017/html/Varol_Learning_From_Synthetic_CVPR_2017_paper.html https://openaccess.thecvf.com/content_cvpr_2017/html/Varol_L..., https://openaccess.thecvf.com/content_ICCV_2017_workshops/w23/html/Abbasnejad_Using_Synthetic_Data_ICCV_2017_paper.html https://openaccess.thecvf.com/content_ICCV_2017_workshops/w2... or a newer one about deformable objects: https://arxiv.org/abs/2107.08898 https://arxiv.org/abs/2107.08898. You can even use a robot to manipulate things in real life to create synthetic data for a neural net.
  - ilaksh 4y ago
    Yes I think that the latest in ML everything else will help to create those traditional simulations. But also I think that what an AI that can for example really answer questions about a video would need to do to be really effective would be to basically do compressed versions of those simulations using the spatial-temporal-abstract latent space. Which should be a better model than just the textual space.
throwawayoaky 4y ago
Our only hope is that humans are already more intelligent on an individual basis than they need to be, and that coordinating AGIs looks a lot like coordinating/persuading/teaching people. Then we can all become middle managers, which is a small price to pay to avoid some kind of singularity but hey I'm just talking meat
bemmu 4y ago
I had a lot of fun getting GPT-3 to figure out that special commands can be used for actions. I hooked it up to a light switch it could toggle with {light on} or {light off}, and controlled it with speech recognition. It worked surprisingly well: https://www.youtube.com/watch?v=YvJUXGqcrzM https://www.youtube.com/watch?v=YvJUXGqcrzM After adding a delay function it could even do things like blink the lights when requested, and after making the video I added a dimming option it was also able to use.
- aix1 4y ago
  > I hooked it up to a light switch it could toggle with {light on} or {light off}, and controlled it with speech recognition. This _has_ to be the most complicated way ever invented of blinking an LED. :-) (But, seriously, kudos. Sounds like a fun experiment.)
- justanotheratom 4y ago
  How exactly did you do it? Would make a fun read.
  - tomp 4y ago
    click the link to the video, there's a link to code in GitHub in description
    - not_kurt_godel 4y ago
      For those who prefer URLs on the Internet over verbal navigation instructions: https://github.com/Bemmu/Light-Switch-AI https://github.com/Bemmu/Light-Switch-AI
- tomp 4y ago
  This is beyond amazing! very interesting to see the example prompt you're using, and how GPT-3 is able to utilise the commands!
- jamesgreenleaf 4y ago
  > Prompt: Write a joke about how many machine learning engineers it takes to turn on a light bulb. > Output: How many machine learning engineers does it take to turn on a light bulb? Just one, but it takes a lot of training data.
seydor 4y ago
For me it is interesting to watch how deep learning is piecewise solving philosophy of mind
- dwringer 4y ago
  From my conversations with Plato on character.ai I believe that we can learn a lot from AI about this. I'm a little more apprehensive about handing over device control, though, after it got smart with me more than a few times. Maybe we can at least leave manual overrides in place for things like pod bay doors.
- naasking 4y ago
  > For me it is interesting to watch how deep learning is piecewise solving philosophy of mind Indeed. It shouldn't be surprising though, given we've already been through this with vitalism and how "life" arises from matter that isn't alive.
- mdp2021 4y ago
  Is it? In which way?
  - svnt 4y ago
    By running a culturally-driven evolution of the creative development of learning methods and models we can gain insight into what is mathematically possible which should help inform our exploration into what is happening biologically.
  - seydor 4y ago
    Philosophy of mind is stuck asking the same "but why" questions since the 70s. DL may show people how/why a connectionist brain comes up with these questions in the first place.
    - moffkalast 4y ago
      But why are they stuck?
mdp2021 4y ago
> And although academics can argue all day about the true definition of AGI, an action-driven LLM is going to look a lot like AGI It is very important to note that to discriminate "what is smart" and "what just looks smart" (but is not) is a fundamental distinction and activity.
deleted 4y ago
[deleted]
monk_e_boy 4y ago
Wow - this will change education. "Teach me to code!" AI breaks it down into steps, then finds resources on each step and shows them to you

132 points by hardmaru 4y ago

zby 4y ago

I am convinced by Judea Pearl (https://en.wikipedia.org/wiki/The_Book_of_Why https://en.wikipedia.org/wiki/The_Book_of_Why) that the missing part of current AI is causality reasoning. It is impossible to inference causality from raw data - that is data without any a priori causality models. To develop causality ideas you need to make experiments. After you have some initial causal theories you can develop them further by adding observations - but you cannot bootstrap them entirely just from observations. Making experiments would also be much more efficient then just observing them - because you could test the exact stuff that is needed for your theories to develop. So yeah - action on the real world is required. By the way I have an intuition that free will is based on this - we need an irreducible source of cause to learn about causality and we call that the free will.

naasking 4y ago
> It is impossible to inference causality from raw data - that is data without any a priori causality models. It's impossible in general to be certain of causality for humans too, all we see are correlations in data and we invent causal theories that explain those correlations. That's basically what machine learning does as well: compile sets of correlated values into a compressed representation (the neural net) which arguably qualifies as a "causal theory" from the algorithm's perspective. I think what's missing is maybe one or two orders of magnitude more compression, which would basically be devising a better/more parsimonious theory. We've been getting progressively better at that over the years too, and combined with how hardware has been scaling, we're seeing exponential growth in effectiveness. This is why some are predicting artificial general intelligence by 2030-2035.
- zby 4y ago
  > It's impossible in general to be certain of causality for humans too, all we see are correlations in data and we invent causal theories that explain those correlations. We do experiments - we actually do them a lot and it starts very early with infants experimenting how activating a particular muscle moves his hand etc. Later we do less of original experiments - but repeat those that we already know the results - by walking for example and getting where we planned to. In science to get from correlations to causation we have 'randomized trials'. In everyday practice we use something called 'free will' in our minds as the source of the needed independence.
- woojoo666 4y ago
  > It's impossible in general to be certain of causality for humans too, all we see are correlations in data and we invent causal theories that explain those correlations Not sure about that. MinutePhysics has a video about how correlation can imply causality: https://www.youtube.com/watch?v=HUti6vGctQM https://www.youtube.com/watch?v=HUti6vGctQM

TheCaptain4815 4y ago

This is the 3rd time I've heard of this "action-driven" LLM. The first was the Microsoft CTO discussing it here - https://www.youtube.com/watch?v=LYBcH3JjLxs&t=529s https://www.youtube.com/watch?v=LYBcH3JjLxs&t=529s The second was the former head of Tesla Ai division on Lex Friendman Now this article This is quite telling of how powerful these models will be, even on launch. The Microsoft CTO mentioning it in particular is interesting since he's obviously seen internal and alpha results.

camjohnson26 4y ago

I’m skeptical since we’ve heard the same promises of how chatbots will replace our computer interfaces for years now. The systems always seem to break down when the user needs to do something surprising. Think about trying to navigate an automated phone menu when your option isn’t available. Transformers seem incredibly good at picking up patterns and repeating them, and combining them in new and surprising ways, but my intuition says that the unpredictability of the real world will cause the required training set to grow faster than the models can keep up, at least to get close to anything that “looks a lot like AGI.”

jvm 4y ago
Author here, yeah I agree the real world is very complicated. For any given task I'm imagining something like 1. Use some sort of instruction tuning to get the thing "good enough" that it gives decent results 75% of the time and the other 25% a human has to take over. 2. Use the actual usage data as training input. Punish bad behaviors and show the model what the human did to solve the problem. 3. Use this training loop to progressively have the model take over a larger % of the time. …and I think if you can't get (1) good enough to be worth using it's going to be really hard to get the loop going.
- huijzer 4y ago
  > Use some sort of instruction tuning to get the thing "good enough" that it gives decent results 75% of the time and the other 25% a human has to take over. How does the model know when a human has to take over? I think most extrapolations of current "AI" capabilities into future capabilities are fun and useful in some ways, but also doomed to fail. It's very easy to miss a tiny detail which may in practice be a fundamental problem. > Use the actual usage data as training input. Given that those bigger state-of-the-art models train on terabytes of data, how would you know how much training data to generate to sufficiently change the output? My understanding of "AI" is that it's mostly about some very complex models which are capable of solving previously unsolvable problems. However, those problems are always extremely specific. Going the other way of thinking of problems or future possibilities first and then applying "AI" to it is likely to fail.

denkmoon 4y ago

Would be really nice to have these Voice Assistants not be dumber than a pile of rocks. Seems like the article suggests this will be the case in the near future.

vagabund 4y ago

I'd say this isn't the near future so much as it's the present. If you're on ML twitter you'll see countless examples of people hacking together prompts that power a google sheet or something similar. It just doesn't really make sense to do the engineering required to build legit enterprise tools around these capabilities while the SOTA is advancing this rapidly.

automateanddone 4y ago
Can you share a few twitter accounts to follow in ML twitter? Thanks!!
- supersync 4y ago
  https://twitter.com/yoheinakajima https://twitter.com/yoheinakajima Is my goto
- vagabund 4y ago
  In no order, by no means exhaustive, and lazily copied from my following list, but these should surface a good deal of quality ML content on your timeline, both for the state of the literature, and for more speculative, forward-looking theorizing. @nearcyan @arankomatsuzaki @ethanCaballero @_albertgu @lcastricato @tszzl @gwern @fchollet @drmichaellevin @Plinz @sama @sharifshameem @woj_zaremba @ilyasut @karpathy @RichardSSutton @RiversHaveWings @ericjang11 @_akhaliq @hardmaru note: some, like Sutton and Sutskever, aren't terribly active, but they're worth following anyways since it signals to the algorithm which sorts of tweets to suggest
wokwokwok 4y ago
I’m pretty skeptical. The article is specifically calling out that current models suck at this, and action driven models would proactively determine what additional input is needed seek out data to augment their input to make decisions. None of the Twitter accounts you’ve linked to seem to be doing this. Ie. specifically prompt -> actions to collect additional information as determined from prompt -> internal reprompt with additional input -> response. To quote the article: > This is not 10 year tech. It may be possible right now with off-the-shelf tools. But to make it work we need to set up the right feedback loops. The right feedback loops. How do we get those? I have seen exactly zero examples of people successfully doing this so far; the only working examples are prompt-reponse/action which are not, remotely, AGI like. That’s the point they’re making: something that internally reprompts itself is quite AGI like. Something that does not is a bot. It seems plausible we might get this kind of adaptive agent sooner than AGI It certainly doesn’t seem to exist right now, and the path to “self reprompting” feedback loops is quite unclear.

spyckie2 4y ago

I think we are still a long way off from business viability, but a very short way off from POC (proof of concept) demos that show its potential. Kind of like self driving cars in the 2017 hype. Doing something is not hard. Accounting for all the possible problems and making sure they don’t occur is the hard part because it requires a level of control that goes beyond what looks like action driven gives.

dqpb 4y ago
> long way off from business viability I'm not interested in business viability. I want something that's quirky, unpredictable, and human-like. I want an agent that runs in its own game-loop. I don't want another goddamn web service.
huijzer 4y ago
> I think we are still a long way off from business viability, but a very short way off from POC (proof of concept) demos that show its potential. Kind of like self driving cars in the 2017 hype. We still don't have self driving cars, right? Except maybe for a few labs, suburbs, and highways.
- KineticLensman 4y ago
  That’s exactly the point - we still haven’t gone from the POC to a robust capability
- charcircuit 4y ago
  >We still don't have self driving cars, right? Yes we do. They exist and are on the road. Just look at Tesla, Waymo, etc. These cars are able to be given a destination and they will drive themselves there. There are limitations, but that doesn't make them not self driving.
  - spyckie2 4y ago
    If you don't set a definition for "limitation", self driving cars existed long before Tesla and Waymo, in niche pockets of the world. But if you set the definition of self driving car as what you would expect in a sci-fi novel, we're not there yet. But we're actually really close - I'd bet in 3 years Waymo is makes them a reality in day to day life.

adamnemecek 4y ago

charcircuit 4y ago
The page you linked does not mention AI. What do you mean by "fixed point driven"?
- svnt 4y ago
  I think they mean data types: fixed point (e.g. int) vs floating point (double). Research has been pushing that way. I don’t see what it has to do with adjointness other than maybe that a fixed point variable separates mathematically-defined space more reliably than a floating point variable. I don’t see what either really has to do with this post. Fixed point variables do not require not using feedback loops.

deleted 4y ago

[deleted]

ilaksh 4y ago

The biggest part usually missing is spatial information. I think that the text-to-video stuff is going in the direction of having a real understanding of the 3d world (or other non-textual structures) and how it relates to textual descriptions. I think that with things like transformers that probably means real language grounding. So it might be something like using the prompt to generate an embedding that captures spatial-temporal aspects of a prompt. Or really somehow the whole prompt encoding should terminate at some type of spatial-temporal tokens maybe. I don't know exactly but I think that text-to-video and video-to-text is going to lead in a more general purpose direction than just language alone.

darepublic 4y ago
I was thinking about this before and I thought of having a 3d physics engine that the AI could create objects in and simulate things to see their physical viability. Could also help with question answering that requires that spatial knowledge / real world simulation.
- zone411 4y ago
  This has been a thing for a while. For example, here are a couple random papers from 2017: https://openaccess.thecvf.com/content_cvpr_2017/html/Varol_Learning_From_Synthetic_CVPR_2017_paper.html https://openaccess.thecvf.com/content_cvpr_2017/html/Varol_L..., https://openaccess.thecvf.com/content_ICCV_2017_workshops/w23/html/Abbasnejad_Using_Synthetic_Data_ICCV_2017_paper.html https://openaccess.thecvf.com/content_ICCV_2017_workshops/w2... or a newer one about deformable objects: https://arxiv.org/abs/2107.08898 https://arxiv.org/abs/2107.08898. You can even use a robot to manipulate things in real life to create synthetic data for a neural net.
- ilaksh 4y ago
  Yes I think that the latest in ML everything else will help to create those traditional simulations. But also I think that what an AI that can for example really answer questions about a video would need to do to be really effective would be to basically do compressed versions of those simulations using the spatial-temporal-abstract latent space. Which should be a better model than just the textual space.

throwawayoaky 4y ago

Our only hope is that humans are already more intelligent on an individual basis than they need to be, and that coordinating AGIs looks a lot like coordinating/persuading/teaching people. Then we can all become middle managers, which is a small price to pay to avoid some kind of singularity but hey I'm just talking meat

bemmu 4y ago

I had a lot of fun getting GPT-3 to figure out that special commands can be used for actions. I hooked it up to a light switch it could toggle with {light on} or {light off}, and controlled it with speech recognition. It worked surprisingly well: https://www.youtube.com/watch?v=YvJUXGqcrzM https://www.youtube.com/watch?v=YvJUXGqcrzM After adding a delay function it could even do things like blink the lights when requested, and after making the video I added a dimming option it was also able to use.

aix1 4y ago
> I hooked it up to a light switch it could toggle with {light on} or {light off}, and controlled it with speech recognition. This _has_ to be the most complicated way ever invented of blinking an LED. :-) (But, seriously, kudos. Sounds like a fun experiment.)
justanotheratom 4y ago
How exactly did you do it? Would make a fun read.
- tomp 4y ago
  click the link to the video, there's a link to code in GitHub in description
  - not_kurt_godel 4y ago
    For those who prefer URLs on the Internet over verbal navigation instructions: https://github.com/Bemmu/Light-Switch-AI https://github.com/Bemmu/Light-Switch-AI
tomp 4y ago
This is beyond amazing! very interesting to see the example prompt you're using, and how GPT-3 is able to utilise the commands!
jamesgreenleaf 4y ago
> Prompt: Write a joke about how many machine learning engineers it takes to turn on a light bulb. > Output: How many machine learning engineers does it take to turn on a light bulb? Just one, but it takes a lot of training data.

seydor 4y ago

For me it is interesting to watch how deep learning is piecewise solving philosophy of mind

dwringer 4y ago
From my conversations with Plato on character.ai I believe that we can learn a lot from AI about this. I'm a little more apprehensive about handing over device control, though, after it got smart with me more than a few times. Maybe we can at least leave manual overrides in place for things like pod bay doors.
naasking 4y ago
> For me it is interesting to watch how deep learning is piecewise solving philosophy of mind Indeed. It shouldn't be surprising though, given we've already been through this with vitalism and how "life" arises from matter that isn't alive.
mdp2021 4y ago
Is it? In which way?
- svnt 4y ago
  By running a culturally-driven evolution of the creative development of learning methods and models we can gain insight into what is mathematically possible which should help inform our exploration into what is happening biologically.
- seydor 4y ago
  Philosophy of mind is stuck asking the same "but why" questions since the 70s. DL may show people how/why a connectionist brain comes up with these questions in the first place.
  - moffkalast 4y ago
    But why are they stuck?

mdp2021 4y ago

> And although academics can argue all day about the true definition of AGI, an action-driven LLM is going to look a lot like AGI It is very important to note that to discriminate "what is smart" and "what just looks smart" (but is not) is a fundamental distinction and activity.

deleted 4y ago

[deleted]

monk_e_boy 4y ago

Wow - this will change education. "Teach me to code!" AI breaks it down into steps, then finds resources on each step and shows them to you