3 ms·
The literal best public models still fail to count characters consistently in practice so I’m not sure what you mean. It’s literally a problem we’re still tryin
by girvo 6mo ago
The literal best public models still fail to count characters consistently in practice so I’m not sure what you mean. It’s literally a problem we’re still trying to solve at work
- outofpaper 6mo agoWhat's amazing is that they even can fairly reliably appear to count characters. I mean we're talking about systems that infer sequences not character counters or calculators. They are amazing in unrelated ways and we need to accept this so we can use them effectively.
- girvo 6mo agoOf course, they’re shockingly powerful, just in an incredibly “spiky” way
- jameshart 6mo agoI suspect character counting - counting small numbers in general in fact - is something that multimodal models will gradually learn through their visual capabilities. We have generative systems that are capable of generating an image of the word ‘strawberry’, and of counting how many strawberries are visible in an image; seems likely it’s possible for an LLM to ‘imagine’ what the word strawberry looks like and count the ‘Rs’ it can ‘see’.