4 ms·

To reframe the discussion a bit: LLMs are time series predictors. You give it a sequence and it predicts the next part of the sequence. As a society we've bee

To reframe the discussion a bit: LLMs are time series predictors. You give it a sequence and it predicts the next part of the sequence. As a society we've been dedicating a lot of resources to time series prediction for many years. What makes LLMs culturally significant is that they generate sequences that map to words that seem to humans like intelligent responses. Arguably, it has always been obvious that a sufficiently capable time series predictor would effectively be a super-weapon. Many technological advances that are currently in the realm of sci-fi could be classified similarly. However so could many technologies that are now widespread and largely harmless to the status quo. People worried that the internet would create massive social upheaval. But soon got algorithmic feeds which effectively filter out antisocial content. The masses got mobile phones with cameras, but after a few scandals about police brutality the only place we find significant content about police misconduct is CCP-afiliated TikTok. I think people get squeamish about AI because there are not clear authority structures other than what one can buy with a lot of A100s. So when people express concern about negative consequences, they are in effect asking whether we need yet another way that people can convert money + public resources into power while not contributing anything to society in return.

izzydata 3y ago
Maybe the internet did cause massive social upheaval. It just looks a lot more boring in reality than imagined. I get the feeling the most advanced LLMs won't be much different. In the future maybe that's just what we will call computers and life will go on.
- salynchnew 3y ago
  Exactly. What if the singularity happens and everything is still boring? I imagine our world would be mostly incomprehensible to someone from the 1400s (the lack of centricity of religion, assuming some infernal force is keeping airplanes aloft, etc., to say nothing of the internet). If superintelligent AI really does take over the world, I image the most uncomfortable part of it all will be explaining to future generations how we were just too lazy to stop it. Assuming climate change doesn't get us first.
  - tsunamifury 3y ago
    You mean like empty downtowns and no stores or social engagement and everyone just sitting at home in front of devices all day? That kind of boring singularity? Yea … what if…
    - kridsdale3 3y ago
      We marched right in to the Matrix ourselves, because it's fantastic at giving dopamine.
TerrifiedMouse 3y ago
> But soon got algorithmic feeds which effectively filter out antisocial content. You mean (we) got algorithmic feeds which feed us antisocial content for the sake of profit because such content drives the most engagement thus generating the most ad revenue.
- resters 3y ago
  I don't disagree, however I meant antisocial in the sense of being disruptive to the status quo
maebert 3y ago
I don’t disagree with you, but always think the “they’re just predicting the next token” argument is kind of missing the magic for the sideshow. Yes they do, but in order to do that, LLMs soak up the statistical regularities of just about every sentence ever written across a wide swath of languages, and from that infer underlying concepts common to all languages, which in turn, if you subscribe at least partially to the Sapir-Wharf hypothesis, means LLMs do encode concepts of human cognition. Predicting the next token is simply a task that requires an LLM to find and learn these structural elements of our language and hence thought, and thus serves as a good error function to train the underlying network. But it’s a red herring when discussing what LLMs actually do.
- srj 3y ago
  It's amazing but is it real intelligence? I listened to a radio segment last week where the hosts were lamenting that Europe was able to pass AI regulation but the US Congress was far from doing so. The fear and hype is fueling reaction to a problem that IMO does not exist. There is no AI. What we have is a wonder of what can be achieved through LLMs but it's still a tool rather than a being. Unfortunately there's a lot of money to be made pitching it as such.
- frozenwind 3y ago
  I am disappointed your comment did not have more responses because I'm very interested in deconstructing this argument I've heard over and over again. ("it just predicts the next words in the sentence"). While explanations of how GPT-style LLMs work involve a layering of structures which encode at the first levels some understanding of syntax, grammar etc. and then as the more levels of transformers are added, eventually some contextual and logical meanings are encoded. I really want to see a developed conversation about this. What are we humans even doing when zooming out? We're processing the current inputs to determine what best to do in the present, nearest future or even far future. Sometimes, in a more relaxed space (say a "brainstorming" meeting), we relax our prediction capabilities to the point our ideas come from a hallucination realm if no boundaries are imposed. LLMs mimic these things in the spoken language space quite well.
- resters 3y ago

izzydata 3y ago

Maybe the internet did cause massive social upheaval. It just looks a lot more boring in reality than imagined. I get the feeling the most advanced LLMs won't be much different. In the future maybe that's just what we will call computers and life will go on.

salynchnew 3y ago
Exactly. What if the singularity happens and everything is still boring? I imagine our world would be mostly incomprehensible to someone from the 1400s (the lack of centricity of religion, assuming some infernal force is keeping airplanes aloft, etc., to say nothing of the internet). If superintelligent AI really does take over the world, I image the most uncomfortable part of it all will be explaining to future generations how we were just too lazy to stop it. Assuming climate change doesn't get us first.
- tsunamifury 3y ago
  You mean like empty downtowns and no stores or social engagement and everyone just sitting at home in front of devices all day? That kind of boring singularity? Yea … what if…
  - kridsdale3 3y ago
    We marched right in to the Matrix ourselves, because it's fantastic at giving dopamine.

TerrifiedMouse 3y ago

> But soon got algorithmic feeds which effectively filter out antisocial content. You mean (we) got algorithmic feeds which feed us antisocial content for the sake of profit because such content drives the most engagement thus generating the most ad revenue.

resters 3y ago
I don't disagree, however I meant antisocial in the sense of being disruptive to the status quo

maebert 3y ago

I don’t disagree with you, but always think the “they’re just predicting the next token” argument is kind of missing the magic for the sideshow. Yes they do, but in order to do that, LLMs soak up the statistical regularities of just about every sentence ever written across a wide swath of languages, and from that infer underlying concepts common to all languages, which in turn, if you subscribe at least partially to the Sapir-Wharf hypothesis, means LLMs do encode concepts of human cognition. Predicting the next token is simply a task that requires an LLM to find and learn these structural elements of our language and hence thought, and thus serves as a good error function to train the underlying network. But it’s a red herring when discussing what LLMs actually do.

srj 3y ago
It's amazing but is it real intelligence? I listened to a radio segment last week where the hosts were lamenting that Europe was able to pass AI regulation but the US Congress was far from doing so. The fear and hype is fueling reaction to a problem that IMO does not exist. There is no AI. What we have is a wonder of what can be achieved through LLMs but it's still a tool rather than a being. Unfortunately there's a lot of money to be made pitching it as such.
frozenwind 3y ago
I am disappointed your comment did not have more responses because I'm very interested in deconstructing this argument I've heard over and over again. ("it just predicts the next words in the sentence"). While explanations of how GPT-style LLMs work involve a layering of structures which encode at the first levels some understanding of syntax, grammar etc. and then as the more levels of transformers are added, eventually some contextual and logical meanings are encoded. I really want to see a developed conversation about this. What are we humans even doing when zooming out? We're processing the current inputs to determine what best to do in the present, nearest future or even far future. Sometimes, in a more relaxed space (say a "brainstorming" meeting), we relax our prediction capabilities to the point our ideas come from a hallucination realm if no boundaries are imposed. LLMs mimic these things in the spoken language space quite well.
resters 3y ago