4 ms·
They won't. These are not "issues", it's them trying to push the models to burn less compute. It will only get worse.
by aulin 6mo ago
They won't. These are not "issues", it's them trying to push the models to burn less compute. It will only get worse.
- criemen 6mo ago> it's them trying to push the models to burn less compute I'm curious, how does using more tokens save compute?
- shortstuffsushi 6mo agoI think that the idea is each action uses more tokens, which means that users hit their limit sooner, and are consequently unable to burn more compute.
- ryanschaefer 6mo agoWhat?
- bloppe 6mo agoIt could be the adaptive reasoning
- b65e8bee43c2ed0 6mo agoproductivity (tokens per second per hardware unit) increases at the cost of output quality, but the price remains the same. both Anthropic and OpenAI quantize their models a few weeks after release. they'd never admit it out loud, but it's more or less common knowledge now. no one has enough compute.
- cebert 6mo agoDo you have a source for that claim?
- b65e8bee43c2ed0 6mo agomy source is that people have been noticing this since GPT4 days. https://arxiv.org/pdf/2307.09009 https://arxiv.org/pdf/2307.09009 but of course, this isn't a written statement by a corporate spokespersyn. I don't think that breweries make such statements when they water their beer either.
- sthimons 6mo agoPretty bold claim - you have a source for that?
- Rapzid 6mo agoThere is no evidence TMK that the accuracy the models change due to release cycles or capacity issues. Only latency. Both Anthropic and OpenAI have stated they don't do any inference compute shenanigans due to load or post model release optimization. Tons of conspiracy theories and accusations. I've never seen any compelling studies(or raw data even) to back any of it up.
- BoorishBears 6mo agoI'm 99.9% sure Opus 4.7 is a smaller model than 4.6. Too many signs between the sudden jump in TPS (biggest smoking gun for me), new tokenenizer, commentary about Project Mythos from Ant employees, etc. It looks like their new Sonnet was good enough to be labeled Opus and their new Opus was good enough to be labeled Mythos. They'll probably continue post-training and release a more polished version as Opus 5
- rustyhancock 6mo agoIf you've not seen Common People Black Mirror episode I strongly recommend it. The only misprediction it makes is that AI is creating the brain dead user base... You have to hook your customers before you reel them in! https://www.netflix.com/gb/title/70264888?s=a&trkid=13747225&shareType=Title&shareUuid=42da6049-01e9-46aa-b0c6-b97354148313&trg=cp&unifiedEntityIdEncoded=Video%3A70264888&vlang=en&clip=81996971 https://www.netflix.com/gb/title/70264888?s=a&trkid=13747225...