3 ms·
> when I speak am I just spitting out the next word that makes sense or is there more to it. There is more to it. Specifically you are doing so to advance towa
by TerrifiedMouse 3y ago
> when I speak am I just spitting out the next word that makes sense or is there more to it.
There is more to it. Specifically you are doing so to advance towards a specific goal. LLMs don't have goals. They just pick from a list of likely tokens - based on their training data - at random to generate the next token.
- brookst 3y agoSo the externally provided metaprompt doesn’t establish a goal that generation works toward?
- TerrifiedMouse 3y agoI don’t think LLMs work towards anything. It just picks from a list of likely “next tokens” at random.
- brookst 3y ago“Likely” as defined by…?
- TerrifiedMouse 3y agoStatistics as defined by training data. https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...