5 ms·

I'm not sure it's possible to have intuition about systems that work in thousands of orthogonal dimensions. In fact I'm pretty sure most of the research is peop

I'm not sure it's possible to have intuition about systems that work in thousands of orthogonal dimensions. In fact I'm pretty sure most of the research is people trying fairly arbitrary things and testing them and then post rationalising implied understanding of what is really happening on top of good outcomes.

pwdisswordfishs 1mo ago
So computer science actually involves science now.
Creamsicle47 1mo ago
I'd have to push back, though not on the part you'd expect. Your description of human researchers is roughly right: a lot of the field is try-things-and-narrativize-after. But the load-bearing assumption is that intuition has to be human-shaped intuition. Humans can't intuit thousands of orthogonal directions because we project everything down into a 3D metaphor and hope it holds. That's a fact about our hardware, not about the systems. And the reason why is the most interesting part: nothing requires the compression step. A model or an agent can operate over the actual objects, holding thousands of runs and ablations in context and noticing regularities in the native dimensionality, without translating them into a picture of a ball rolling down a hill. No bottleneck at "can you visualize it." So the narrower claim: it's not that intuition here is impossible full-stop, it's that human intuition is unreliable. Your post-hoc rationalization point is evidence for that, not against it. The story exists because a person needs something to hold in their head. Drop that requirement and the failure mode goes with it.
- deleted 1mo ago
  [deleted]
- tadfisher 1mo ago
  This is no longer funny.
- kunley 1mo ago
  You apparently have no intuition on what the intuition is
megaloblasto 1mo ago
I disagree. It's absolutely possible to develop an intuition about extremely complex mathematical ideas, including llms or high dimensional systems. Learning to build an llm is a great way to start building that intuition.
- andy_ppp 1mo ago
  What provable conclusions have you intuited around how LLMs work? Give me some examples to prove your point? I'm absolutely happy to change my view with enough data.
  - hdz 1mo ago
    I'm not looking to change your view. But for other readers who are curious, here is a link to an interesting task to gain intuition. Ahmad is a good data point for someone who tinkered, built intuition, then started his own ai company. https://twitter.com/TheAhmadOsman/status/2087742080793620593?s=20 https://twitter.com/TheAhmadOsman/status/2087742080793620593...
    - andy_ppp 1mo ago
      I think we are talking about different things to be honest. Understanding LLMs is a fine thing to do but to say you understand what the network is actually doing, even in toy examples like MNIST is extremely tricky. Figuring out what a ~3 trillion parameter network is doing in thousands of dimensions seems intractable to me. Linking to a Twitter thread about training a small LLM to be good at Wordle is not an example of what I'm talking about. It might well be a useful task but it doesn't allow us to understand deeply what's happening.
  - megaloblasto 1mo ago
    I think provable conclusions and building an intuition are different things to be fair. In my opinion it is entirely possible to build intuition about seemingly impossible to understand topics like infinitesimal areas, infinite limits, function mappings, and high dimensional spaces. For example, 3D spaces are easy to intuit if you can visualize vectors in your head. Some people claim it is impossible to understand dimensions higher than 3 but that's obviously not true because you could assign a color to each vector to visualize the next dimension. You could then assign a width to visualize a 5th dimension, and an arrow shape to signify the next, and so on and so on. That's more of a creative act that can start to build at least a visual understanding in you head of these higher dimension spaces. That's a way to start to intuit about those things. Now with llms we need way more that 6 dimensions so we can start thinking of assigning matrices to each 3d point for example. That allows us to increase the dimension from 3 to 3 + whatever the matrix dimension is. We can visualize the matrices instead of having numbers as having colors for each entry, so they can be a sort of cube with each vowel being a different color. Now you can start to visually intuit about how these massively high dimensional spaces can be formed of these colored matrices that can react to some input training data. That's a start of an idea for intuiting things that might seem impossible to have an intuition about. I think visualization is a great way to start.
janalsncm 1mo ago
I think it’s reasonable to have a shallow understanding of most parts and a deep understanding of a small number of parts. That’s how most engineers are. Most software engineers do not have a deep understanding of CPU architectures. In fact they probably don’t even have a shallow understanding and get around just fine. How many of them are looking up the instruction set for the CPUs they deploy their CRUD app to in EC2?
- tech_ken 1mo ago
  But in the case of CPU architecture there are SOME people who understand how things work 100%, and they've built and vetted abstractions/mental models that enable other engineers and scientists to have that kind of mixed shallow/deep understanding in a way that works. On the side of LLMs we're still lacking an expertise which could flawlessly explain how these things operate; the abstractions that we're using are instead derived inductively and are totally unvetted.
  - janalsncm 1mo ago
    You are right that the field doesn’t have a theoretically sound explanation for the architectural choices aside from “A works better than B”. However, I would argue this is an ideal opportunity for the “gentleman scientist” or eager 17 year old. Basically every part of the original transformer was replaced with something more efficient or better: LayerNorm -> RMSNorm Sinusoidal position encoding -> RoPE MHA -> GQA ReLU -> GELU What this means is that there is ample opportunity to improve on what we’ve done thus far.
    - tech_ken 1mo ago
      Oh yeah I absolutely agree that there's lots of greenfield for relative newcomers to find some big improvements! I'm just saying that reasoning about 'best practices' in terms of what works for something like a CPU is missing some important and relevant differences.
    - andy_ppp 1mo ago
      The original RoPE paper is a great example of the authors not actually understanding how their own discovery was functioning fully which is a fun example of what I’m talking about I guess… Yes you might progress the field, but will you really understand why? You can make up an explanation and anthropomorphise it with a few contrived diagrams and everyone will cheer!
highfrequency 1mo ago
It is sometimes the opposite - a large number of things makes the system easier to predict and reason about (statistics, behavior of gases etc).
apparent 1mo ago
You may be right. Nonetheless, it's possible that the act of trying to get your brain to understand a system that works in more than a few orthogonal dimensions will help develop useful intuitions about how LLMs, and the many systems/products/robots that will be built on top of them, work.
danielmarkbruce 1mo ago
This is true of most large complex systems though. We run molecular dynamics simulations as an example.

pwdisswordfishs 1mo ago

So computer science actually involves science now.

Creamsicle47 1mo ago

I'd have to push back, though not on the part you'd expect. Your description of human researchers is roughly right: a lot of the field is try-things-and-narrativize-after. But the load-bearing assumption is that intuition has to be human-shaped intuition. Humans can't intuit thousands of orthogonal directions because we project everything down into a 3D metaphor and hope it holds. That's a fact about our hardware, not about the systems. And the reason why is the most interesting part: nothing requires the compression step. A model or an agent can operate over the actual objects, holding thousands of runs and ablations in context and noticing regularities in the native dimensionality, without translating them into a picture of a ball rolling down a hill. No bottleneck at "can you visualize it." So the narrower claim: it's not that intuition here is impossible full-stop, it's that human intuition is unreliable. Your post-hoc rationalization point is evidence for that, not against it. The story exists because a person needs something to hold in their head. Drop that requirement and the failure mode goes with it.

deleted 1mo ago
[deleted]
tadfisher 1mo ago
This is no longer funny.
kunley 1mo ago
You apparently have no intuition on what the intuition is

megaloblasto 1mo ago

I disagree. It's absolutely possible to develop an intuition about extremely complex mathematical ideas, including llms or high dimensional systems. Learning to build an llm is a great way to start building that intuition.

andy_ppp 1mo ago
What provable conclusions have you intuited around how LLMs work? Give me some examples to prove your point? I'm absolutely happy to change my view with enough data.
- hdz 1mo ago
  I'm not looking to change your view. But for other readers who are curious, here is a link to an interesting task to gain intuition. Ahmad is a good data point for someone who tinkered, built intuition, then started his own ai company. https://twitter.com/TheAhmadOsman/status/2087742080793620593?s=20 https://twitter.com/TheAhmadOsman/status/2087742080793620593...
  - andy_ppp 1mo ago
    I think we are talking about different things to be honest. Understanding LLMs is a fine thing to do but to say you understand what the network is actually doing, even in toy examples like MNIST is extremely tricky. Figuring out what a ~3 trillion parameter network is doing in thousands of dimensions seems intractable to me. Linking to a Twitter thread about training a small LLM to be good at Wordle is not an example of what I'm talking about. It might well be a useful task but it doesn't allow us to understand deeply what's happening.
- megaloblasto 1mo ago
  I think provable conclusions and building an intuition are different things to be fair. In my opinion it is entirely possible to build intuition about seemingly impossible to understand topics like infinitesimal areas, infinite limits, function mappings, and high dimensional spaces. For example, 3D spaces are easy to intuit if you can visualize vectors in your head. Some people claim it is impossible to understand dimensions higher than 3 but that's obviously not true because you could assign a color to each vector to visualize the next dimension. You could then assign a width to visualize a 5th dimension, and an arrow shape to signify the next, and so on and so on. That's more of a creative act that can start to build at least a visual understanding in you head of these higher dimension spaces. That's a way to start to intuit about those things. Now with llms we need way more that 6 dimensions so we can start thinking of assigning matrices to each 3d point for example. That allows us to increase the dimension from 3 to 3 + whatever the matrix dimension is. We can visualize the matrices instead of having numbers as having colors for each entry, so they can be a sort of cube with each vowel being a different color. Now you can start to visually intuit about how these massively high dimensional spaces can be formed of these colored matrices that can react to some input training data. That's a start of an idea for intuiting things that might seem impossible to have an intuition about. I think visualization is a great way to start.

janalsncm 1mo ago

I think it’s reasonable to have a shallow understanding of most parts and a deep understanding of a small number of parts. That’s how most engineers are. Most software engineers do not have a deep understanding of CPU architectures. In fact they probably don’t even have a shallow understanding and get around just fine. How many of them are looking up the instruction set for the CPUs they deploy their CRUD app to in EC2?

tech_ken 1mo ago
But in the case of CPU architecture there are SOME people who understand how things work 100%, and they've built and vetted abstractions/mental models that enable other engineers and scientists to have that kind of mixed shallow/deep understanding in a way that works. On the side of LLMs we're still lacking an expertise which could flawlessly explain how these things operate; the abstractions that we're using are instead derived inductively and are totally unvetted.
- janalsncm 1mo ago
  You are right that the field doesn’t have a theoretically sound explanation for the architectural choices aside from “A works better than B”. However, I would argue this is an ideal opportunity for the “gentleman scientist” or eager 17 year old. Basically every part of the original transformer was replaced with something more efficient or better: LayerNorm -> RMSNorm Sinusoidal position encoding -> RoPE MHA -> GQA ReLU -> GELU What this means is that there is ample opportunity to improve on what we’ve done thus far.
  - tech_ken 1mo ago
    Oh yeah I absolutely agree that there's lots of greenfield for relative newcomers to find some big improvements! I'm just saying that reasoning about 'best practices' in terms of what works for something like a CPU is missing some important and relevant differences.
  - andy_ppp 1mo ago
    The original RoPE paper is a great example of the authors not actually understanding how their own discovery was functioning fully which is a fun example of what I’m talking about I guess… Yes you might progress the field, but will you really understand why? You can make up an explanation and anthropomorphise it with a few contrived diagrams and everyone will cheer!

highfrequency 1mo ago

It is sometimes the opposite - a large number of things makes the system easier to predict and reason about (statistics, behavior of gases etc).

apparent 1mo ago

You may be right. Nonetheless, it's possible that the act of trying to get your brain to understand a system that works in more than a few orthogonal dimensions will help develop useful intuitions about how LLMs, and the many systems/products/robots that will be built on top of them, work.

danielmarkbruce 1mo ago

This is true of most large complex systems though. We run molecular dynamics simulations as an example.