> Lower-order ML constructs do not demonstrate emergent capabilities like step by step, stream of consciousness thinking, and so on. As a matter of fact, I did

> Lower-order ML constructs do not demonstrate emergent capabilities like step by step, stream of consciousness thinking, and so on. As a matter of fact, I did a project on the normalization of the text, e.g., translate "crossing of 6 a. and 12 s." into "crossing of sixth avenue and 12-th street" with a simple LM (order 3) and beam search on the lattice paths, lattice formed with hypotheses' variants. I got two fold decrease of word error rate compared to simpler approach with just outputting the most probable WFST path. It was not "step by step stream of consciousness," but nevertheless very impressive feat, when system started to know more without much effort. The large LM's do not just output "most probable" token, they output most probable sequence of tokens and it is done with the beam search. As you can see, my experience tells me that beam search alone can noticeably, if not tremendously, improve quality of the output, even for very simple LMs. And if I may, the higher-order construct here is a beam search, not the LMs-as-matrix-coefficients' themselves. Beam search is used in speech recognition for decades now, SR does not work properly without it. LM's, apparently, also do not work without it.