3 ms·

> LLM drawing inspiration What does this even mean ? Aren’t LLM’s just predicting the next token with zero contextual knowledge of the world they inhabit As

• Manuel_D 3y ago
  The corpus of text used to train the AI is the contextual knowledge about the world. When you give it a word like "phasor" it's going to associate that word with the Star Trek universe (and probably angular representations of complex numbers from EE textbooks too). When you give it words like "Godswood" or "Andals" it's going to make associations with those words. Regardless of whether we think generative AI is empty regurgitation, the copyright claims are going to have difficulty sticking unless models are regularly reproducing copyright texts. Even when they are it's usually in short snippets, e.g. "It was the best of times, it was the worst of times..." not regurgitating a whole Dickens novel.
• mycall 3y ago
  > Aren’t LLM’s just predicting the next token with zero contextual knowledge of the world they inhabit Only partly true, especially with Q* learning coming onto the scene as it is much more dynamic, able to self-learn.