3 ms·

Oh dang, I am quite literally working on this as a side project (out of mere curiosity). Well, sort of ..., I'm refining an algo that takes several (carefully

Oh dang, I am quite literally working on this as a side project (out of mere curiosity). Well, sort of ..., I'm refining an algo that takes several (carefully calibrated) outputs from a given LLM and infers the most plausible set of parameters behind it. I was expecting to find clusters of parameters very much alike to what they observe. I informally call this problem inverting an LLM, and obv., it turns out to be non-trivial to solve. Not completely impossible, tho! as so far I've found some good approximations to it. Anyway, quite an interesting read, def. will keep an eye on what they publish in the future. Also, from the linked manuscript at the end, >Another hypothesis is that some features are actually higher-dimensional feature manifolds which dictionary learning is approximating. Well, you have something that behaves like a continuous, smooth space so you could define as many manifolds as you'd need to suit your needs, so yes :^). But, pedantry off, I get the idea and IMO that's definitely what's going on and the right framework to approach this problem from. One amazing realization one can get from this is, what is the conceptual equivalent of the transition functions that connect all different manifolds in this LLM space? When you see it your mind will be blown, not because of its complexity, but rather because of its exceptional simplicity.

herodoturtle 3y ago
At first I thought this was an ode to dang.
- stavros 3y ago
  Oh dang, a name so spry, A clever soul, with humor wry, In life's vast game, you do not shy, A friend to all, a bond we tie.
codethief 3y ago
> One amazing realization one can get from this is, what is the conceptual equivalent of the transition functions that connect all different manifolds in this LLM space? Could you elaborate on what you mean by "transition functions" here?
- evertedsphere 3y ago
  likely https://en.m.wikipedia.org/wiki/Atlas_(topology)#Transition_maps https://en.m.wikipedia.org/wiki/Atlas_(topology)#Transition_... although the quoted sentence does not make sense to me; transition maps connect different patches of one manifold. it's possible the "LLM space" gp is talking about is a parameter space of some nature each of whose points is a manifold, but that seems like a stretch
  - moralestapia 3y ago
    You're right, that's a bit ambiguous. Instead of "all different manifolds" I should have written "all different ways to define a manifold". Also, now that I think about it, it may not necessarily be all of them. The important thing here is that, however you define them and their transition maps, you'll find they're very much alike. As if there was some sort of general structure that is highly preferred over others ...
    - evertedsphere 3y ago
      "if you look at the (chart) transition maps for different atlases on this manifold, they tend to look similar for different atlases" what is the manifold here and what evidence do you have for this / what does it look like when you "define" them?
      - moralestapia 3y ago
        See, https://transformer-circuits.pub/2023/may-update/index.html#feature-manifolds https://transformer-circuits.pub/2023/may-update/index.html#.... They also reference, https://distill.pub/2020/circuits/equivariance/ https://distill.pub/2020/circuits/equivariance/. I'd also add, https://distill.pub/2021/multimodal-neurons/ https://distill.pub/2021/multimodal-neurons/.
  - codethief 3y ago
    Yeah, that's the definition I was thinking of, too, but the sentence didn't make sense to me, either.
LeonigMig 3y ago
Do you mean in the sense it’s hierarchical? Or am I missing the point entirely
- moralestapia 3y ago
  I'm not talking about that specifically but your intuition is also correct and there's a lot of research going around constructing/defining hierarchies of "learning" behavior.
bilsbie 3y ago
What was your approach to getting started doing this? I’m curious to learn more about how LLMs work too.
- moralestapia 3y ago
  Try to get something like tinygrad[1] running locally, that way you can tweak things a bit, run it again and see how it performs. While doing this you'll pick up most of the concepts and get a feeling of how things work. Also, take a look at projects like llama.cpp[2], you don't have to fully understand what's going on here, though. You may need some intermediate knowledge of linear algebra and this thing called "data science" nowadays, which is pretty much knowing how to mangle data and visualize it. Try creating a small model on your own, it doesn't have to be super fancy just make sure it does something you want it to do. And then ... you'll probably could go on your own then. 1: https://github.com/tinygrad/tinygrad https://github.com/tinygrad/tinygrad 2: https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp

herodoturtle 3y ago

At first I thought this was an ode to dang.

stavros 3y ago
Oh dang, a name so spry, A clever soul, with humor wry, In life's vast game, you do not shy, A friend to all, a bond we tie.

codethief 3y ago

> One amazing realization one can get from this is, what is the conceptual equivalent of the transition functions that connect all different manifolds in this LLM space? Could you elaborate on what you mean by "transition functions" here?

evertedsphere 3y ago
likely https://en.m.wikipedia.org/wiki/Atlas_(topology)#Transition_maps https://en.m.wikipedia.org/wiki/Atlas_(topology)#Transition_... although the quoted sentence does not make sense to me; transition maps connect different patches of one manifold. it's possible the "LLM space" gp is talking about is a parameter space of some nature each of whose points is a manifold, but that seems like a stretch
- moralestapia 3y ago
  You're right, that's a bit ambiguous. Instead of "all different manifolds" I should have written "all different ways to define a manifold". Also, now that I think about it, it may not necessarily be all of them. The important thing here is that, however you define them and their transition maps, you'll find they're very much alike. As if there was some sort of general structure that is highly preferred over others ...
  - evertedsphere 3y ago
    "if you look at the (chart) transition maps for different atlases on this manifold, they tend to look similar for different atlases" what is the manifold here and what evidence do you have for this / what does it look like when you "define" them?
    - moralestapia 3y ago
      See, https://transformer-circuits.pub/2023/may-update/index.html#feature-manifolds https://transformer-circuits.pub/2023/may-update/index.html#.... They also reference, https://distill.pub/2020/circuits/equivariance/ https://distill.pub/2020/circuits/equivariance/. I'd also add, https://distill.pub/2021/multimodal-neurons/ https://distill.pub/2021/multimodal-neurons/.
- codethief 3y ago
  Yeah, that's the definition I was thinking of, too, but the sentence didn't make sense to me, either.

LeonigMig 3y ago

Do you mean in the sense it’s hierarchical? Or am I missing the point entirely

moralestapia 3y ago
I'm not talking about that specifically but your intuition is also correct and there's a lot of research going around constructing/defining hierarchies of "learning" behavior.

bilsbie 3y ago

What was your approach to getting started doing this? I’m curious to learn more about how LLMs work too.

moralestapia 3y ago
Try to get something like tinygrad[1] running locally, that way you can tweak things a bit, run it again and see how it performs. While doing this you'll pick up most of the concepts and get a feeling of how things work. Also, take a look at projects like llama.cpp[2], you don't have to fully understand what's going on here, though. You may need some intermediate knowledge of linear algebra and this thing called "data science" nowadays, which is pretty much knowing how to mangle data and visualize it. Try creating a small model on your own, it doesn't have to be super fancy just make sure it does something you want it to do. And then ... you'll probably could go on your own then. 1: https://github.com/tinygrad/tinygrad https://github.com/tinygrad/tinygrad 2: https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp