3 ms·

Congrats on the launch! I have an interesting dissonance with this. On one hand, I understand how huge parameter sets can and do model specific personas well.

• famouswaffles 3y ago
  >The other part of me is screaming at my laptop that language modeling is about aggregate statistics Yeah..but it's not. This is where people are having so much trouble. The erroneous belief that language model learn some "average" or "aggregate" of the training set. But when you get down to it, that doesn't make any sense at all. What help would some fictional aggregate distribution do with predicting the responses of decidedly non aggregate text ? None at all. So Language models don't learn an "average" distribution. They learn to predict every state that exists in the corpus in a sort of superposition. The perfect LLM would predict Einstein as well as it would predict the dumbass down the street. LLMs are biased but not uniformly so.
  • icyfox 3y ago
    I think we're using aggregate statistics in two different ways. Technically speaking - the whole idea with a language model is that you're learning to generalize underlying patterns of text, not just memorize the input text. Otherwise language models would be very good at echoing back training data but fail miserably during validation. If we go back to the training sequence - it's trying to maximize the posterior given the conditional probabilities in the sequence: P(y1, y2, ... yn ) = P(y1) * P(y2|y1) ... P(yn|y1...yn-1) That probability is by definition an aggregate; it's the best fit given potentially competing inputs of the training set that all have the same input conditional chains. Where generative LLMs have a leg-up is because they have such a large parameter space, large context windows, and coherent sampling strategies. This helps them stay internally consistent with their response data. But at the end of the day what they are learning are still patterns. That's why they aren't able to link content back to the exact source of origin; parameters fuse inputs from different places into one hybrid. Seeding a generative chain with Einstein or someone down the street doesn't change the fact that what's next is some fused learning from a lot of different training set inputs.
    • famouswaffles 3y ago
      The point i'm making is that i believe the "fused space" is not in fact very fused because that would be directly detrimental to reducing loss.
• timshell 3y ago
  Thank you! Agree A, B, and C are big hurdles. Re: A - we have started adding transparency (vis-a-vis the 'Investigate Results' and the tSNE plots + similarity scores) but we still have a ways to go Re: B - agree that the survey responses -> insights pipeline is nonlinear and it's not clear how to make that tighter Re: C - generally, we try to champion a human-AI interaction loop where people are needed to evaluate the outputs, generate insights, etc. All great points though and ones we are facing