4 ms·

I disagree, to keep it short, what you're describing is not understanding, it's superstition. And I think it's a wrong direction of engineering, relying on some

by js8 18d ago

I disagree, to keep it short, what you're describing is not understanding, it's superstition. And I think it's a wrong direction of engineering, relying on some sort of irreproducible expert intuition, one that has been successfully replaced by enlightenment and scientific method. There are 3 major obstacles in understanding LLMs: 1. They use inscrutable internal language of embeddings 2. They communicate in natural language which is itself ambiguous 3. The weights and training inputs are being hidden as a "trade secret" "with the added advantage of having been trained on a HUGE number of codebases" This doesn't really mean much unless we understand what is the quality and relevance of these sources for the problem at hand. Without this understanding it's just a superstition.

ActorNightly 17d ago
>They use inscrutable internal language of embeddings No different than the electrical signals in the intermediate neurons in your brain that comprises the latent space where all the processing happens >They communicate in natural language which is itself ambiguous They lack one-shot precision, sure, but it doesn't matter. They are precise enough with refinement over multiple prompts. >the weights and training inputs are being hidden as a "trade secret" For frontier models that make the company money through api pricing, sure. There are plenty of open source models that can be used for the same tasks, which have open weights. >This doesn't really mean much unless we understand what is the quality and relevance of these sources for the problem at hand. All of the modern models are RL trained on specific tasks when it comes to coding. I.e the initial training run learns to predict the next token based on context, from all the available texts, but then the RL runs specifically train the model in a harness where it produces code and RLed to produce correct code with specific formatting.

by js8 18d ago

ActorNightly 17d ago

>They use inscrutable internal language of embeddings No different than the electrical signals in the intermediate neurons in your brain that comprises the latent space where all the processing happens >They communicate in natural language which is itself ambiguous They lack one-shot precision, sure, but it doesn't matter. They are precise enough with refinement over multiple prompts. >the weights and training inputs are being hidden as a "trade secret" For frontier models that make the company money through api pricing, sure. There are plenty of open source models that can be used for the same tasks, which have open weights. >This doesn't really mean much unless we understand what is the quality and relevance of these sources for the problem at hand. All of the modern models are RL trained on specific tasks when it comes to coding. I.e the initial training run learns to predict the next token based on context, from all the available texts, but then the RL runs specifically train the model in a harness where it produces code and RLed to produce correct code with specific formatting.