4 ms·
LLMs need significant optimization or we get significant improvement on computing power while keeping the energy cost the same. It's similar with smartphone, wh
by fendy3002 1y ago
LLMs need significant optimization or we get significant improvement on computing power while keeping the energy cost the same. It's similar with smartphone, when at the start it's not feasible because of computing power, and now we have one that can rival 2000s notebooks.
LLMs is too trivial to be expensive
EDIT: I presented the statement wrongly. What I mean is the use case for LLM are trivial things, it shouldn't be expensive to operate
- trashchomper 1y agoCalling LLMs trivial is a new one. Yea just consume all of the information on the internet and encode it into a statistical model, trivial, child could do it /s
- hammyhavoc 1y ago> all of the information on the internet Total exaggeration—especially given Cloudflare providing free tools to block AI and now tools to charge bots for access to information.
- fendy3002 1y agowell I presented the statement wrongly. What I mean is the use case for LLM are trivial things, it shouldn't be expensive to operate
- lblume 1y agoImagine telling a person from five years ago that the programs that would basically solve NLP, perform better than experts at many tasks and are hard not to anthropomorphize accidentally are actually "trivial". Good luck with that.
- jrflowers 1y ago>programs that would basically solve NLP There is a load-bearing “basically” in this statement about the chat bots that just told me that the number of dogs granted forklift certification in 2023 is 8,472.
- lblume 1y agoSure, maybe solving NLP is too great a claim to make. It is still not at all ordinary that beforehand we could not solve referential questions algorithmically, that we could not extract information from plain text into custom schemas of structured data, and context-aware mechanical translation was really unheard of. Nowadays LLMs can do most of these tasks better than most humans in most scenarios. Many NLP questions at least I find interesting reduce to questions of the explanability of LLMs.
- clarinificator 1y agoYeah it solved NLP about 50% of the time, and also mangles data badly and in often hard-to-detect ways.
- Applejinx 1y ago"hard not to anthropomorphize accidentally' is a you problem. I'm unhappy every time I look in my inbox, as it's a constant reminder there are people (increasingly, scripts and LLMs!) prepared to straight-up lie to me if it means they can take my money or get me to click on a link that's a trap. Are you anthropomorphizing that, too? You're not gonna last a day.
- lblume 1y agoI didn't mean typical chatbot output, these are luckily still fairly recognizable due to stylistic preferences learned during fine-tuning. I mean actual base model output. Take a SOTA base model and give it the first two paragraphs of some longer text you wrote, and I would bet on many people being unable to distinguish your continuation from the model's autoregressive guesses.
- hyperbovine 1y agoIt still doesn't pass the Turing test, and is not close. Five years ago me would be impressed but still adamant that this is not AI, nor is it on the path to AI.
- killerstorm 1y agoLLM can give you thousands of lines of perfectly working code for less than 1 dollar. How is that trivial or expensive?
- zwnow 1y agoThousands of lines of perfectly working code? Did you verify that yourself? Last time I tried it produced slop, and I've been extremely detailed in my prompt.
- DSingularity 1y agoTry again.
- mrbungie 1y agoAny retries before nailing the prompt are still going to be billed, so this supports GP position about LLMs being expensive for trivial things.
- killerstorm 1y agoYes. I verified it myself. Best results from Opus 4 so far, Gemini might be OK too.
- fendy3002 1y agowell I presented the statement wrongly. What I mean is the use case for LLM are trivial things, it shouldn't be expensive to operate and the 1 dollar cost for your case is heavily subsidized, that price won't hold up long assuming the computing power stays the same.
- killerstorm 1y agoCheaper models might be around $0.01 per request, and it's not subsidized: we see a lot of different providers offering open source models, which offer quality similar to proprietary ones. On-device generation is also an option now. For $1 I'm talking about Claude Opus 4. I doubt it's subsidized - it's already much more expensive than the open models.
- jsnell 1y agoBut the thing is, LLMs are already incredibly cheap to operate compared to the alternatives. Both for trivial things and for complex things.
- fendy3002 1y agoWell recently cursor got a heat for rising price and having opaque usage, while anthropic's claude reported to be worse due to optimization. IMO the current LLMs are not sustainable, and prices are expected to increase sooner or later. Personally, until models comparable with sonnet 3.5 can be run locally on mid range setup, people need to wary that the price of LLM can skyrocket
- awuji 1y agoYou can already run a large LLM (like sonnet 3.5) locally on CPU with 128GB of ram which is <300 USD, but can be offset by swap space. Obviously, response speed is going to be slower, but I can't imagine people will pay much more than 20 USD for waiting 30-60 seconds longer for a response. And obviously consumer hardware is already being more optimized for running models locally.