3 ms·

Plenty of AI companies that are cons or extremely overvalued, but the technology is the real deal and delivering huge improvements over previous techniques in a

by xnx 2y ago

Plenty of AI companies that are cons or extremely overvalued, but the technology is the real deal and delivering huge improvements over previous techniques in all kinds of domains: language translation, weather prediction, code completion, self driving, etc.

advael 2y ago
I swear no matter how many times people say it people will still conflate all ML with LLMs. No, chatGPT is not driving advances in self-driving or weather prediction
- xnx 2y ago
  For better or worse, "LLM" or "generative ai" has become roughly synonymous with the current wave of ML. I know very little about ChatGPT, but Waymo is using an LLM: "Powered by Gemini, a multimodal large language model developed by Google, EMMA employs a unified, end-to-end trained model to generate future trajectories for autonomous vehicles directly from sensor data." (https://waymo.com/blog/2024/10/introducing-emma https://waymo.com/blog/2024/10/introducing-emma)
  - advael 2y ago
    Huh. I mean it makes sense to train end-to-end on all the interrelated tasks involved in driving but putting a whole-ass language model in the middle of that seems like a stunt. I wonder if it does better than like, any random transformer not trained on language first? Still, I hadn't heard that so I guess I was wrong about that one
    - magicalist 2y ago
      No, you were right, this appears to be just research on how applicable LLMs could be to the space. They talk about the improvements their LLM makes, especially in being multimodal vs training multiple independent models, but also the limitations that appear to prevent it from being useable as it is. Maybe some form if it will be used some day (it does seem like it would be useful to have semantic understanding of the world integrated into the system), but at least as of when this was published, it's not actually used.
  - porridgeraisin 2y ago
    Waymo uses reinforcement learning (what it was before LLMs) (TD3+BC according to one of their blogs) Emma is something they tried, but further down the article they explain why they don't use it as such yet.
    - xnx 2y ago
      Yep. It's an interesting experiment and really stretched my understanding of what an LLM is and can do.
- famouswaffles 2y ago
  Vision Language Models are absolutely being trialed for self-driving https://wayve.ai/thinking/lingo-2-driving-with-language/ https://wayve.ai/thinking/lingo-2-driving-with-language/
  - advael 2y ago
    Okay so because of the ambiguity of the other reply I'm just gonna say, I don't think we should be surprised that someone is trying to use LLMs to do basically anything. That's basically what prints funding money right now, so long as you're the kind of company or guy the VCs or whoever will believe in. The signal here is "does it do something to appreciably advance the state of the art over previous methods"?
    - famouswaffles 2y ago
      Seems to be the case to me, reading this and waymo's attempts. There's paper on EMMA here - https://arxiv.org/abs/2410.23262 https://arxiv.org/abs/2410.23262 And there are state of the art weather prediction transformers. https://arxiv.org/abs/2312.03876 https://arxiv.org/abs/2312.03876
      - advael 2y ago
        Yeah so like, this is a cool result, and it uses a transformer architecture. I actually do think that it's fair to say that transformers have proven widely useful, especially in tasks that look like sequence modeling. It's a step change akin to the now-pervasive use of convolutional neural networks that started in the 2010s, and is deeply significant of course. This is also really different from "this is an LLM" The reason I want to specifically harp on this is because a lot of people are selling this narrative where "AI is becoming superintelligent" or whatever by making an amorphous blob out of a bunch of separate advances that use machine learning techniques. This has been happening for a while, is a great thing for science, and it's clear that machine learning methods are here to stay in science. I'm a machine learning researcher. I've understood, celebrated, and tried to help with this as best I can manage over the last 9 years of my life. And it's been going on for a lot longer than the general public has been in this AI hype wave. The entire modern field of bioinformatics is arguably built on the backbone of machine learning, and has been since before I went to grad school. This is really different from "We fed everything into a language model and now it's superintelligent and is making scientific advances all by itself" or even "scientists just ask chatGPT shit and it figures it out for them". The breathless tech press really makes it sound like anything that happens in AI research, which increasingly includes the entire usage of ML toolkits in the sciences (Which is pervasive, and expectedly so! ML is an extension of statistics and statistics has been the basis of science for like a century) is just some amorphous force called "AI" that's suddenly gained this aggregate body of competency. Imagine if we anthropomorphized statistics that way. Or Math for that matter. This kind of narrative gives me the overall impression that this is not being talked about honestly, and it's clear that this is profitable to do. I don't have to use charged words like "con" or "fraud" to think this deceptive framing is not a great thing
camdenreslink 2y ago
LLMs can do weather prediction? There is no way that is true. Considering ChatGPT sometimes insists 2+2=5, I sincerely doubt it is solving PDEs used to model weather systems.
- famouswaffles 2y ago
  Transformers can do weather prediction yes. https://arxiv.org/abs/2312.03876 https://arxiv.org/abs/2312.03876
  - camdenreslink 2y ago
    This thread is about large language models and generative AI. AFAIK LLMs perform pretty badly at time series prediction. Maybe some specially trained weather prediction neural network could perform well, but that isn’t really what we’re talking about here.
    - famouswaffles 2y ago
      A Large Language Model is just a text predicting Transformer (and sometimes image and/or audio predicting if they're multimodal). Transformers are general sequence to sequence predictors. The only difference between this and GPT is the data it's been trained on. They're the same kind of neural network.
- lblume 2y ago
  Think base models. LLMs are extremely good at predicting the future when applied to human languages, as this is literally their only optimization goal. Why couldn't they also be good at predicting the future when applied to other complex forecasting tasks? Of course what can be mathematically calculated without inference is going to be. LLMs may however be able to interpret the results of these calculations better than humans or current stochastical evaluations.
scinadier 2y ago
Sounds like airlines need to start using ChatGPT to autopilot their airplanes.

by xnx 2y ago

advael 2y ago

I swear no matter how many times people say it people will still conflate all ML with LLMs. No, chatGPT is not driving advances in self-driving or weather prediction

xnx 2y ago
For better or worse, "LLM" or "generative ai" has become roughly synonymous with the current wave of ML. I know very little about ChatGPT, but Waymo is using an LLM: "Powered by Gemini, a multimodal large language model developed by Google, EMMA employs a unified, end-to-end trained model to generate future trajectories for autonomous vehicles directly from sensor data." (https://waymo.com/blog/2024/10/introducing-emma https://waymo.com/blog/2024/10/introducing-emma)
- advael 2y ago
  Huh. I mean it makes sense to train end-to-end on all the interrelated tasks involved in driving but putting a whole-ass language model in the middle of that seems like a stunt. I wonder if it does better than like, any random transformer not trained on language first? Still, I hadn't heard that so I guess I was wrong about that one
  - magicalist 2y ago
    No, you were right, this appears to be just research on how applicable LLMs could be to the space. They talk about the improvements their LLM makes, especially in being multimodal vs training multiple independent models, but also the limitations that appear to prevent it from being useable as it is. Maybe some form if it will be used some day (it does seem like it would be useful to have semantic understanding of the world integrated into the system), but at least as of when this was published, it's not actually used.
- porridgeraisin 2y ago
  Waymo uses reinforcement learning (what it was before LLMs) (TD3+BC according to one of their blogs) Emma is something they tried, but further down the article they explain why they don't use it as such yet.
  - xnx 2y ago
    Yep. It's an interesting experiment and really stretched my understanding of what an LLM is and can do.
famouswaffles 2y ago
Vision Language Models are absolutely being trialed for self-driving https://wayve.ai/thinking/lingo-2-driving-with-language/ https://wayve.ai/thinking/lingo-2-driving-with-language/
- advael 2y ago
  Okay so because of the ambiguity of the other reply I'm just gonna say, I don't think we should be surprised that someone is trying to use LLMs to do basically anything. That's basically what prints funding money right now, so long as you're the kind of company or guy the VCs or whoever will believe in. The signal here is "does it do something to appreciably advance the state of the art over previous methods"?
  - famouswaffles 2y ago
    Seems to be the case to me, reading this and waymo's attempts. There's paper on EMMA here - https://arxiv.org/abs/2410.23262 https://arxiv.org/abs/2410.23262 And there are state of the art weather prediction transformers. https://arxiv.org/abs/2312.03876 https://arxiv.org/abs/2312.03876
    - advael 2y ago
      Yeah so like, this is a cool result, and it uses a transformer architecture. I actually do think that it's fair to say that transformers have proven widely useful, especially in tasks that look like sequence modeling. It's a step change akin to the now-pervasive use of convolutional neural networks that started in the 2010s, and is deeply significant of course. This is also really different from "this is an LLM" The reason I want to specifically harp on this is because a lot of people are selling this narrative where "AI is becoming superintelligent" or whatever by making an amorphous blob out of a bunch of separate advances that use machine learning techniques. This has been happening for a while, is a great thing for science, and it's clear that machine learning methods are here to stay in science. I'm a machine learning researcher. I've understood, celebrated, and tried to help with this as best I can manage over the last 9 years of my life. And it's been going on for a lot longer than the general public has been in this AI hype wave. The entire modern field of bioinformatics is arguably built on the backbone of machine learning, and has been since before I went to grad school. This is really different from "We fed everything into a language model and now it's superintelligent and is making scientific advances all by itself" or even "scientists just ask chatGPT shit and it figures it out for them". The breathless tech press really makes it sound like anything that happens in AI research, which increasingly includes the entire usage of ML toolkits in the sciences (Which is pervasive, and expectedly so! ML is an extension of statistics and statistics has been the basis of science for like a century) is just some amorphous force called "AI" that's suddenly gained this aggregate body of competency. Imagine if we anthropomorphized statistics that way. Or Math for that matter. This kind of narrative gives me the overall impression that this is not being talked about honestly, and it's clear that this is profitable to do. I don't have to use charged words like "con" or "fraud" to think this deceptive framing is not a great thing

camdenreslink 2y ago

LLMs can do weather prediction? There is no way that is true. Considering ChatGPT sometimes insists 2+2=5, I sincerely doubt it is solving PDEs used to model weather systems.

famouswaffles 2y ago
Transformers can do weather prediction yes. https://arxiv.org/abs/2312.03876 https://arxiv.org/abs/2312.03876
- camdenreslink 2y ago
  This thread is about large language models and generative AI. AFAIK LLMs perform pretty badly at time series prediction. Maybe some specially trained weather prediction neural network could perform well, but that isn’t really what we’re talking about here.
  - famouswaffles 2y ago
    A Large Language Model is just a text predicting Transformer (and sometimes image and/or audio predicting if they're multimodal). Transformers are general sequence to sequence predictors. The only difference between this and GPT is the data it's been trained on. They're the same kind of neural network.
lblume 2y ago
Think base models. LLMs are extremely good at predicting the future when applied to human languages, as this is literally their only optimization goal. Why couldn't they also be good at predicting the future when applied to other complex forecasting tasks? Of course what can be mathematically calculated without inference is going to be. LLMs may however be able to interpret the results of these calculations better than humans or current stochastical evaluations.

scinadier 2y ago

Sounds like airlines need to start using ChatGPT to autopilot their airplanes.