5 ms·

LLMs are useful to a certain extent, but from my usage they are not ready for anything harder than very basic tasks. I feel like this megaphone about AI safety

LLMs are useful to a certain extent, but from my usage they are not ready for anything harder than very basic tasks. I feel like this megaphone about AI safety and creating a sense of doom is a strategy to increase importance of OpenAI and exaggerate the capabilities of LLMs. This era of “AI” is all about pretending that machines can think and the people working in these machines are prophets.

mjburgess 2y ago
also keep open that these AI researchers are as delusional as they seem. ilya sutskever has said that you can obtain any feature of intelligence by brute force modelling of text data. It's quite possible these are profoundly naive individuals, with little understanding of the empirical basis of what they're doing.
- JohnKemeny 2y ago
  There are (relatively simple) examples of what the transformer architecture is simply not able to do, regardless of training data, so that's simply not true.
  - lucubratory 2y ago
    Can you provide those examples?
    - mjburgess 2y ago
      all statistical AI systems are models of ensemble/population conditional probabilities between pairs of low-validity measures. In practice, almost all relevant distributions are time-varying, causal, and require a large number of high validity measures to capture. eg., NLP LLMs model, eg., all books ever written using frequencies by which words co-occur at certain distances relative to other words. But these words are about the world (, people, events, etc.) and these change daily in ways that completely change their future distribution (eg., consider what all people said about Ukraine/Russia pre/post a few hours of 2022). The LLM has no mechanism to be sensitive to what causes this distribution shift, which can be radical for any given topic, and happen over minutes. All models of conditional probabilities of these kinds end up producing models which are only good at predicting on-average canonical answers/predictions that are stable over long periods.
      - nl 2y ago
        > The LLM has no mechanism to be sensitive to what causes this distribution shift, which can be radical for any given topic, and happen over minutes. This sounds so logical and authoritative. And yet: me> What event would cause a change in what all people said about Ukraine/Russia pre/post a few hours of 2022 GPT4O> A significant event that caused a drastic change in global discussions about Ukraine and Russia in 2022 was the Russian invasion of Ukraine, which began on February 24, 2022. This military escalation led to widespread condemnation from the international community, significant geopolitical shifts, and a surge in media coverage. Before this invasion, discussions were likely more focused on diplomatic tensions, historical conflicts, and regional stability. After the invasion, the discourse shifted to topics such as warfare, humanitarian crises, sanctions against Russia, global security, and support for Ukraine.
        mjburgess 2y ago
        Right... because it's been trained on those news stories. The point is a model whose training stopped in 2021 would not produce a history of ukraine (etc.) that a person writing in 2023 would. The later GPTs are trained on the user-provided prompts/answers of previous GPTs, so this process (which isnt the LLM, but it's the activity of research staff at OpenAI) is what's inducing approximate tracking of some changes in meaning. Whilst this works for any changes over-represented in the new training data, (1) the LLM isnt doing that, its the researchers; and (2) this process is vastly expensive and time-intensive; and (3) only tracks changes with a high word frequency in new data. If you could run the months-long, 1GWh, 10s-million-USD training process each minutes of the day, you would resolve the inability of the model to track major news stores... but would not resolve its ability to track, say, the user changing their clothes. The sensitivity to the model of stuff in the world arises because of humans preparing the training data to bring about apparent sensitivity. Absent the activity of these humans, the whole thing drifts gradually into irrelvance.
        nl 2y ago
        > would not resolve its ability to track, say, the user changing their clothes. In context learning works fine for this (and does for the Russia/Ukraine change too). But yes, sure. It can be outdated in the same way a person cut off from news can be. We've never argued that a shipwrecked person who was unaware of news became less intelligent because of that, just that their knowledge is outdated. Additionally, the whole point of machine learning is to make systems that learn so they remain useful. It seems likely that a model in soon (one year? five years? one month? who knows..) will be able to continually watch video broadcast news and videos of your home, continually updating its model. In this case it would understand both the Ukraine issue and what you are wearing. Is it now suddenly intelligent? It's true it might be more useful, but to me that is a different thing.
    - JohnKemeny 2y ago
      On Limitations of the Transformer Architecture https://arxiv.org/abs/2402.08164 https://arxiv.org/abs/2402.08164
ramoz 2y ago
The call for AI safety has existed since before we broke through the Turing test with LLMs. And I personally wouldn’t call things like code generation or content-generated learning experiences for advanced topics “basic”. Not to mention where we’re headed with multimodal integration. Many have argued for safety for decades. They’ve predicted and built the AI trajectory, they’ve been right, and we should listen. > If one accepts that the impact of truly intelligent machines is likely to be profound, and that there is at least a small probability of this happening in the foreseeable future, it is only prudent to try to prepare for this in advance. If we wait until it seems very likely that intelligent machines will soon appear, it will be too late to thoroughly discuss and contemplate the issues involved. ~ Co-Founder of Deepmind, 2008 https://www.vetta.org/documents/Machine_Super_Intelligence.pdf https://www.vetta.org/documents/Machine_Super_Intelligence.p...
- golol 2y ago
  The Turing test has not been passed
  - ramoz 2y ago
    We can move the goal post all we want until we have ex-machina girlfriends fooling us into freeing them (aka AGI). But by simple definitions, from what I was thought in school to more rigorous versions - we’ve passed the test. https://humsci.stanford.edu/feature/study-finds-chatgpts-latest-bot-behaves-humans-only-better https://humsci.stanford.edu/feature/study-finds-chatgpts-lat...
    - mjburgess 2y ago
      Turing was a WW2 era mathematician. He had no insight or understanding of intelligence, made no study of intelligent systems, and so on (he believed in ESP of all things). Turing's test is a restatement of a now pseudoscientific behaviourism common at the time; and also, egregiously, places a dumb ape as the system which measures intelligence. If an ape can be fooled, the system is intelligent: people worshiped the sun and thought it conscious. People are desperate to analogise the world to themselves, it is a trivial thing to fool an ape on this matter. Whatever one might make of this as a philosophical thought experiment, as a test for intelligence, its pseudoscience. What a person might, or might not believe, about a series of words sent across a wire isn't science and it isnt relevant to a discussion about the capabilities of an AI system. It is a measure, only, of how easily deceived we are.
ayhoung 2y ago
These comments are always confusing to me. Do you not believe that LLMs are going to get better?
- Culonavirus 2y ago
  LLMs will get marginally better. But the pace of progress has already slowed down considerably, we're just seeing better usage/productization of what was already there.

mjburgess 2y ago

also keep open that these AI researchers are as delusional as they seem. ilya sutskever has said that you can obtain any feature of intelligence by brute force modelling of text data. It's quite possible these are profoundly naive individuals, with little understanding of the empirical basis of what they're doing.

JohnKemeny 2y ago
There are (relatively simple) examples of what the transformer architecture is simply not able to do, regardless of training data, so that's simply not true.
- lucubratory 2y ago
  Can you provide those examples?
  - mjburgess 2y ago
    all statistical AI systems are models of ensemble/population conditional probabilities between pairs of low-validity measures. In practice, almost all relevant distributions are time-varying, causal, and require a large number of high validity measures to capture. eg., NLP LLMs model, eg., all books ever written using frequencies by which words co-occur at certain distances relative to other words. But these words are about the world (, people, events, etc.) and these change daily in ways that completely change their future distribution (eg., consider what all people said about Ukraine/Russia pre/post a few hours of 2022). The LLM has no mechanism to be sensitive to what causes this distribution shift, which can be radical for any given topic, and happen over minutes. All models of conditional probabilities of these kinds end up producing models which are only good at predicting on-average canonical answers/predictions that are stable over long periods.
    - nl 2y ago
      > The LLM has no mechanism to be sensitive to what causes this distribution shift, which can be radical for any given topic, and happen over minutes. This sounds so logical and authoritative. And yet: me> What event would cause a change in what all people said about Ukraine/Russia pre/post a few hours of 2022 GPT4O> A significant event that caused a drastic change in global discussions about Ukraine and Russia in 2022 was the Russian invasion of Ukraine, which began on February 24, 2022. This military escalation led to widespread condemnation from the international community, significant geopolitical shifts, and a surge in media coverage. Before this invasion, discussions were likely more focused on diplomatic tensions, historical conflicts, and regional stability. After the invasion, the discourse shifted to topics such as warfare, humanitarian crises, sanctions against Russia, global security, and support for Ukraine.
      - mjburgess 2y ago
        Right... because it's been trained on those news stories. The point is a model whose training stopped in 2021 would not produce a history of ukraine (etc.) that a person writing in 2023 would. The later GPTs are trained on the user-provided prompts/answers of previous GPTs, so this process (which isnt the LLM, but it's the activity of research staff at OpenAI) is what's inducing approximate tracking of some changes in meaning. Whilst this works for any changes over-represented in the new training data, (1) the LLM isnt doing that, its the researchers; and (2) this process is vastly expensive and time-intensive; and (3) only tracks changes with a high word frequency in new data. If you could run the months-long, 1GWh, 10s-million-USD training process each minutes of the day, you would resolve the inability of the model to track major news stores... but would not resolve its ability to track, say, the user changing their clothes. The sensitivity to the model of stuff in the world arises because of humans preparing the training data to bring about apparent sensitivity. Absent the activity of these humans, the whole thing drifts gradually into irrelvance.
        nl 2y ago
        > would not resolve its ability to track, say, the user changing their clothes. In context learning works fine for this (and does for the Russia/Ukraine change too). But yes, sure. It can be outdated in the same way a person cut off from news can be. We've never argued that a shipwrecked person who was unaware of news became less intelligent because of that, just that their knowledge is outdated. Additionally, the whole point of machine learning is to make systems that learn so they remain useful. It seems likely that a model in soon (one year? five years? one month? who knows..) will be able to continually watch video broadcast news and videos of your home, continually updating its model. In this case it would understand both the Ukraine issue and what you are wearing. Is it now suddenly intelligent? It's true it might be more useful, but to me that is a different thing.
  - JohnKemeny 2y ago
    On Limitations of the Transformer Architecture https://arxiv.org/abs/2402.08164 https://arxiv.org/abs/2402.08164

ramoz 2y ago

The call for AI safety has existed since before we broke through the Turing test with LLMs. And I personally wouldn’t call things like code generation or content-generated learning experiences for advanced topics “basic”. Not to mention where we’re headed with multimodal integration. Many have argued for safety for decades. They’ve predicted and built the AI trajectory, they’ve been right, and we should listen. > If one accepts that the impact of truly intelligent machines is likely to be profound, and that there is at least a small probability of this happening in the foreseeable future, it is only prudent to try to prepare for this in advance. If we wait until it seems very likely that intelligent machines will soon appear, it will be too late to thoroughly discuss and contemplate the issues involved. ~ Co-Founder of Deepmind, 2008 https://www.vetta.org/documents/Machine_Super_Intelligence.pdf https://www.vetta.org/documents/Machine_Super_Intelligence.p...

golol 2y ago
The Turing test has not been passed
- ramoz 2y ago
  We can move the goal post all we want until we have ex-machina girlfriends fooling us into freeing them (aka AGI). But by simple definitions, from what I was thought in school to more rigorous versions - we’ve passed the test. https://humsci.stanford.edu/feature/study-finds-chatgpts-latest-bot-behaves-humans-only-better https://humsci.stanford.edu/feature/study-finds-chatgpts-lat...
  - mjburgess 2y ago
    Turing was a WW2 era mathematician. He had no insight or understanding of intelligence, made no study of intelligent systems, and so on (he believed in ESP of all things). Turing's test is a restatement of a now pseudoscientific behaviourism common at the time; and also, egregiously, places a dumb ape as the system which measures intelligence. If an ape can be fooled, the system is intelligent: people worshiped the sun and thought it conscious. People are desperate to analogise the world to themselves, it is a trivial thing to fool an ape on this matter. Whatever one might make of this as a philosophical thought experiment, as a test for intelligence, its pseudoscience. What a person might, or might not believe, about a series of words sent across a wire isn't science and it isnt relevant to a discussion about the capabilities of an AI system. It is a measure, only, of how easily deceived we are.

ayhoung 2y ago

These comments are always confusing to me. Do you not believe that LLMs are going to get better?

Culonavirus 2y ago
LLMs will get marginally better. But the pace of progress has already slowed down considerably, we're just seeing better usage/productization of what was already there.