4 ms·

> We have no idea how LLMs work on a high level This is just incredibly wrong. Do you think that LLM's were created out of thin air? A random combination of by

• frabcus 3y ago
  We understand the training algorithm, but not the network that was trained. There’s the whole interpretability sub-discipline which has limited but interesting results. Those just indicate even more that we really have no idea how something like GPT-4 works inside its trained model.
• p-e-w 3y ago
  Nobody knows how LLMs work on a high level. There's an algorithm that consumes Terabytes of raw text and produces 175 billion parameters, that when used as a statistical model magically reproduce human language. But the conceptual connection between those parameters and what the model can do is a complete mystery.