3 ms·

>> Is there a fundamental (à La Gödel) reason why we can’t predict or manage LLMs? > We're building statistical models to make statistical predictions after tr

>> Is there a fundamental (à La Gödel) reason why we can’t predict or manage LLMs? > We're building statistical models to make statistical predictions after training with a sufficiently large and statistically diverse dataset. Aka it's random. This seems like a sort of hand wavy answer to what seems like a pretty deep and technical question. And to be fair, this isn’t the environment to ask that question, it seems like something a bunch of researchers would work out. Of course we build statistical models all the time and then use them to make pretty good predictions. Is there something actually fundamental about these LL modes that makes them… unmanageable? Well, we’ll have to define manageable first… etc etc.

rob_c 2y ago
So the question is so abstract as to not have a handle on the answer... Great, What is M theory? This also is missing a structure to allow us to unify existing fundamental theories. The only way to assess this is to stop treating the models as statistical beasts which simply only work with "enough" statistics and start talking about them from the direction of information theory. The answer is glib because the problem is the field requires a mix between stats, computing, and mathematics. 2 of these fields have their own languages for the problem (which differ, but exist) and computing has come along and made another set of names for the same complex things making the whole thing (for now) a mess... Especially with the main practicioners stuck in the view that big llm are simply money printers.
- bee_rider 2y ago
  I get the glibness, I think it is fine for here, but actually I really hope someone out there puts in the effort to figure out what the right question of to ask for this stuff. I think you are right that it isn’t well defined here, but getting a well defined question is like halfway to finishing your thesis, right? Haha.

rob_c 2y ago

So the question is so abstract as to not have a handle on the answer... Great, What is M theory? This also is missing a structure to allow us to unify existing fundamental theories. The only way to assess this is to stop treating the models as statistical beasts which simply only work with "enough" statistics and start talking about them from the direction of information theory. The answer is glib because the problem is the field requires a mix between stats, computing, and mathematics. 2 of these fields have their own languages for the problem (which differ, but exist) and computing has come along and made another set of names for the same complex things making the whole thing (for now) a mess... Especially with the main practicioners stuck in the view that big llm are simply money printers.

bee_rider 2y ago
I get the glibness, I think it is fine for here, but actually I really hope someone out there puts in the effort to figure out what the right question of to ask for this stuff. I think you are right that it isn’t well defined here, but getting a well defined question is like halfway to finishing your thesis, right? Haha.