3 ms·
Read the last paragraph. It doesn't take an essay.
by windowshopping 3y ago
Read the last paragraph. It doesn't take an essay.
- insanitybit 3y agoOK, I mean, you used the word essay so I used the word essay. But thanks for editing in that paragraph.
- stevenhuang 3y agoAh the classic "LLMs are limited bc they can't do character level manipulations". Buddy these are deliberate optimization trade-offs in the tokenizer, LLMs don't even see individual characters. Talk about a red herring. I highly suggest a more refined understanding of LLMs before emphatically stating what they are or aren't. It's frankly embarrassing.
- oivey 3y agoIt’s a performance critical optimization that has downsides. It’s a completely fair criticism. It’s like someone claimed a hammer is a universal construction tool, and then someone else pointed out hammers aren’t great for screwing in screws. Your position is that a universal construction tool doesn’t need to work with screws because hammers were never designed to do so.
- stevenhuang 3y agoA hammer is a finished product. LLMs as a technology are in their infancy and see groundbreaking developments every month. There are models trained on character level tokens that handle precisely these issues. https://arxiv.org/abs/2311.06158 https://arxiv.org/abs/2311.06158 https://www.reddit.com/r/LocalLLaMA/comments/17xj8wl/training_llms_to_follow_procedure_for_math_gives/ https://www.reddit.com/r/LocalLLaMA/comments/17xj8wl/trainin... The fact LLMs can't currently handle character-level changes tell us nothing about whether or not they truly understand meaning. The parent commentator is not probing the right things to answer his question.