3 ms·
Great read. I like how the author uses the analogy of scrabble word creation to describe LLM training but unfortunately the analogy didn't continue to inferenc
by ninju 19d ago
Great read.
I like how the author uses the analogy of scrabble word creation to describe LLM training but unfortunately the analogy didn't continue to inference and I got lost trying to keep up.
- mathisfun123 19d agoThe analogy is flawed - in Scrabble you get rewarded for unlikely word combos. Also causal attention attends to prior tokens (tiles already placed).