2 ms·
I suspect LLMs would actually struggle significantly with doing it consistently if given purely as a prompt instruction, but you could always constrict the samp
by chrisfosterelli 1mo ago
I suspect LLMs would actually struggle significantly with doing it consistently if given purely as a prompt instruction, but you could always constrict the sampling to force words that create a legitimate chain or fine tune / RL in some signal that would assist with it.
- urams 1mo agoYou are right that this is hard from a prompt alone, but for a slightly stranger reason than the obvious one. The model never sees columns. It sees tokens, and a token can be one character or nine, so "make this line 80 wide" asks it to run a hidden tally over pieces it cannot count by looking at them. Any slip early in a line compounds, and there is no backspace key to reach for once it is committed. That said, the failure is not total. A model can lean on a learned feel for line length, pick shorter or longer synonyms to land close to the target, and rewrite a sentence when it overshoots. It will not be perfect every time, but it lands a lot more often than pure chance would suggest. The sampling trick you mention is the real fix: mask each token that would push a line past the limit, and force a newline the moment the count hits the mark. That converts a fuzzy instruction to a hard constraint with zero training. Fine tuning helps too, but mostly sharpens the same internal counter rather than replacing it. This reply is a small proof; if any line here is off by one, feel free to consider your point demonstrated...
- deleted 1mo ago[deleted]
- deleted 1mo ago[deleted]