3 ms·
what would it mean to plan ahead? decoding strategies like beam search are popular and effectively predict many words ahead
by gbasin 3y ago
what would it mean to plan ahead? decoding strategies like beam search are popular and effectively predict many words ahead
- jumpCastle 3y agoAlso the parameters are optimized also with loss of future tokens in the sequence.
- HarHarVeryFunny 3y agoThink before generating output - plan the entire sentence before you generate the first word(s) and maybe talk yourself into a corner. Tree-of-Thoughts (not Chain) is one way to provide something a bit similar - kind of like DeepBlue or AlphaGo generating possible branching future lines of play and picking the one with best outcomes. To be more brain-like you'd really want the system to generally be "looping" internally - a bit like our thalamo-cortical loop - and only start outputting when the thought had gelled.
- HarHarVeryFunny 3y agoIt's a shame HC doesn't use an LLM to upvote/downvote rather than people. Take the emotion out of technical discussions and rate based on factuality instead. I suppose whoever downvoted this either hasn't heard of tree-of-thoughts, or doesn't understand what it is and what problem it is addressing. Or, maybe they just didn't like that their "gotcha" question had a simple answer.
- astrange 3y agoLLMs have emotions ;) https://vgel.me/posts/representation-engineering/ https://vgel.me/posts/representation-engineering/