5 ms·
People have been celebrating the fact that tokens got 100x cheaper and now here's a new system that will use 100x more tokens.
by wmf 2y ago
People have been celebrating the fact that tokens got 100x cheaper and now here's a new system that will use 100x more tokens.
- cowpig 2y agoIsn't that part of the point?
- jsheard 2y agoAlso you now have to pay for tokens you can't see, and just have to trust that OpenAI is using them economically.
- brookst 2y agoToken count was always an approximation of value. This may help break that silly idea.
- regularfry 2y agoI don't think it's much good as an approximation of value, but it seems ok as an approximation of cost.
- brookst 2y agoFair, cost and value are only loosely related. Trying to price based on cost always turns into a mess.
- regularfry 2y agoIts what you do when you're a commodity.
- seydor 2y agoIf it 's reasoning correctly, it shouldnt need a lot of tokens because you don't need to correct it. You only need to ask it to solve nuclear fusion once.
- deleted 2y ago[deleted]
- msp26 2y agoHave you seen how long the CoT was for the example. It's incredibly verbose.
- slt2021 2y agoI find there is an educational benefit in verbosity, it helps to teach user to think like a machine
- legel 2y agoWhich is why it is incredibly depressing that OpenAI will not publish the raw chain of thought. “Therefore, after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring, we have decided not to show the raw chains of thought to users. We acknowledge this decision has disadvantages. We strive to partially make up for it by teaching the model to reproduce any useful ideas from the chain of thought in the answer. For the o1 model series we show a model-generated summary of the chain of thought.”
- slt2021 2y agomaybe they will enable to show CoT for a limited uses, like 5 prompts a day for Premium users, or for Enterprise users with agreement not to steal CoT or something like that. if OpenAI sees this - please allow users to see CoT for a few prompts per day, or add it to Azure OpenAI for Enterprise customers with legal clauses not to steal CoT
- from-nibly 2y agoAs someone experienced with operations / technical debt / weird company specific nonsense (Platform Engineer). No, you have to solve nuclear fusion at <insert-my-company>. You gotta do it over and over again. If it were that simple we wouldn't have even needed AI we would have hand written a few things, and then everything would have been legos, and legos of legos, but it takes a LONG time to find new true legos.
- esafak 2y ago...while providing a significant advance. That's a good problem.
- mewpmewp2 2y agoIsn't that part of developing a new tech?
- zamadatix 2y agoThe new thing that can do more at the "ceiling" price doesn't remove your ability to still use the 100x cheaper tokens for the things that were doable on that version.
- digging 2y agoThat exact pattern is always true of technological advance. Even for a pretty broad definition of technology. I'm not sure if it's perfectly described by the name "induced demand" but it's basically the same thing.
- energy123 2y agoIt does dispel this idea that we are going to be flooded with too many GPUs.
- olalonde 2y ago"People have been celebrating the fact that RAM got 100x cheaper and now here's a new system that will use 100x more RAM."
- anticensor 2y agoKnown as Wirth's law.