9 ms·

The beginning of scarcity in AI

• byyoung3 7mo ago
  distillation is an equalizing force
  • eloisant 7mo ago
    Distillation doesn't give you an equivalent model.
• deleted 7mo ago
  [deleted]
• Lapalux 7mo ago
  "The first hit is free....."
• stupefy 7mo ago
  What limits LLM inference accelerators? I heard about Groq (https://groq.com/ https://groq.com/) not sure how much it pushes away the problem.
  • vessenes 7mo ago
    ASML only makes a certain number of machines a year that can do extreme ultra-violet lithography. Also - turbine blades limit power, according to Elon. Between them - we cannot chip fabs past a certain rate, and we cannot stand up the datacenter to run these desired chips past a certain rate. Different people believe one or the other is the 'true' current bottleneck. The turbine supply chain scaling looks much more tractable -- EUV is essentially the most complicated production process humans have ever devised.
    • ls612 7mo ago
      Presumably ASML can increase production if demand is high enough the question is over what time frame. 5 years seems plausible to me but I honestly don't know what that number is.
      • vessenes 7mo ago
        It's ... really long, according to Dylan Patel on the Dwarkesh Podcast. The supply chain is extremely deep and complex.
        • juliansimioni 7mo ago
          Yes. And the fab companies and their suppliers are deliberately and wisely slow to scale up production to meet short term changes in demand. They've seen the history of the semiconductor industry, it's constant boom and bust cycles. But they have the highest op-ex costs of anyone. So when the party's over they are the ones who pay for it the most.
    • deleted 7mo ago
      [deleted]
    • andai 7mo ago
      Is global compute bottlenecked by one company?
      • Tanjreeve 7mo ago
        Yes. At least, the manufacturing of compute is. And a lot of the chain has been bitten hard by increasing capacity prematurely in the past so they're reticent to increase bandwidth at vast cost.
    • Miraste 7mo ago
      If only there were some form of cheap, widely manufactured power generation technology that didn't use turbines... Are they really going to wait until 2030 to get more turbines rather than invest in solar?
      • stupefy 7mo ago
        I am clueless in this field, but solar seems to be unreliable and yield fraction of power required. Do you have a suggestion on something to read and learn more?
        • vessenes 7mo ago
          Google china solar deployments to read about the logistics end of it
        • Miraste 7mo ago
          Reliability: complete solar deployment includes some form of power storage. There are many variations, but chemical battery technology is improving the fastest, so it's gaining the most ground. Amount of power: World solar power generation capacity is in the terawatts and rapidly increasing, there's no issue with its potential ceiling. As a bonus, it tends to work best on land that's useless for other purposes.
    • utopiah 7mo ago
      Is ASML really the bottleneck? Do you believe anybody but TSMC and few fabs could really use and acquire those machines? I don't know the throughput of a EUV device from ASML but I imagine you need : - clean room, itself needing the infrastructure for it (size, airCo, filtering, electricity) and the staff to run and maintain that basically empty space - wafers to "print" on, so that's a lot of water and logistic to manipulate them (so infrastructure for clean water and all chemicals) also with dedicated staff - finally staff who would be able to design something significantly better than NVIDIA, Intel, Broadcom, IBM, etc while (and arguably that's the trickiest part IMHO) being able to get it good enough as at a scale that can be manufactured from their own fab. so I'm wondering who can afford this kind of setup that can only then make use of ASML machines.
      • Marazan 7mo ago
        > (so infrastructure for clean water and all chemicals) Fabs are some of the most complex chemical engineering sites (dealing with some of the most dangerous substances) in the world. So don't underestimate the complexity of this part.
        • utopiah 7mo ago
          Well that was part of my point, not everybody is TSMC. It's not "just" getting an ASML machine and voila, you're good to go.
• SadErn 7mo ago
  [dead]
• vessenes 7mo ago
  It seems very possible that we have at least five years of real limitations on compute coming up. Maybe ten, depending on ASML. I wonder what an overshoot looks like. I also wonder if there might be room for new entrants in a compute-scarce environment. For instance, at some point, could Coreweave field a frontier team as it holds back 10% of its allocations over time? Pretty unusual situation.
  • dist-epoch 7mo ago
    Jensen just said that if the signal/commitments are there, ASML can scale in 2-3 years.
    • vessenes 7mo ago
      With Anthropic buying compute in dark alleys I’d assume that day is coming..
• mattas 7mo ago
  This notion that "we don't have enough compute" does not cleanly reconcile with the fact that labs are burning cash faster than any cohort of companies in history. If I am a grocery store that pays $1 for oranges and sells them for $0.50, I can't say, "I don't have enough oranges."
  • earthnail 7mo ago
    If there were more oranges you’d pay less to buy them and your economics would work out.
    • 0x3f 7mo ago
      Not sure if this is a joke or not, but competitive pressure still exists. This only really holds if you're the only orange seller.
  • vessenes 7mo ago
    You misunderstand. "I built a ship to go to the Indies and bring back tea." "Bro, the ship cost 100,000 pounds sterling and only brought back 50,000 pounds of tea. I don't care if you paid 12,500 pounds for the tea itself, you're losing money." There is a very rational reason labs are spending everything they can get for more compute right now. The tea (inference) pays 60%+ margins. And that is rising. And that number is AFTER hyper scalars make their margins. There is an immense amount of profit floating around this system, and strategics at the edge believing they can build and control the demand through combined spend on training and inference in the proper ratios.
    • SpicyLemonZest 7mo ago
      60%+ margins according to numbers which are not published publicly and have not AFAICT been audited. Could they be accurate? Sure, I think people who claim this is impossible are overconfident. But I would encourage anyone who assumes they must be right to read a history of the Worldcom scandal. It's really quite easy for a person who wants to be making money (or an LLM who's been instructed to "run the accounts make no mistakes"!) to incorrectly categorize costs as capital investments when nobody's watching carefully.
• isawczuk 7mo ago
  It's artificial scarcity. LLM inference will soon be commodity as cloud. There is a 2-3years still before ASIC LLM inferences will catch up.
  • vessenes 7mo ago
    I don't think so. GB200 prices are GOING UP. A100s are still expensive. This implies massive utilization and demand, no? These machines are not sitting idle, or prices would drop in the very competitive hyperscaler environment.
    • Morromist 7mo ago
      Hard to say at this point. I'm sure you can run your LLM chips 24/7 for training and for the public to make weird thirst-trap videos about Judy Hopps but how real is the utilization and demand, really? Maybe very real, maybe not, I don't think we can know yet. Its like being back in 1850 and you build the world's first amusement park where the rides are free or very cheap. People are like Amusement parks are the next big thing since Steam Boats! And tons of other rich people start to build huge amusement parks everywhere. The people who are skilled at making amusement park rides will increase their prices, and since the first amusement parks are free so they can get the public going to them demand will be huge. But how sustainable is that? - well obviously we know from history that amusement parks did, in fact, take over the world and most people spent virtually all their time and money at amusement parks - I think the Crimean War was even fought over some religious-based theme park in Israel - until moving pictures came out, so it worked out for them, but for AI?
    • LogicFailsMe 7mo ago
      so much for all that hardware that was going to be obsolete in 3 years...
  • observationist 7mo ago
    The problem with this idea is that someone can, and likely will, come up with the next best architecture that leapfrogs the current frontier models at least once a year, likely faster, for the foreseeable future. This means by the time you've manufactured your LLM on an ASIC, it's 4-5 generations behind, and probably much less efficient than current SOTA model at scale. It won't make sense for ASIC LLMs to manifest until things start to plateau, otherwise it'll be cheaper to get smarter tokens on the cloud for almost all use cases. That said, a 10 trillion parameter model on a bespoke compute platform overcomes a lot of efficiency and FOOM aspects of the market fit, so the angle is "when will models that can be run on an asic be good enough that people will still want them for various things even if the frontier models are 10x smarter and more efficient" I think we're probably a decade of iteration on LLMs out, at least, and the entire market could pivot if the right breakthrough happens - some GPT-2 moment demonstrating some novel architecture that convinces the industry to make the move could happen any time now.
• dmazin 7mo ago
  Constraints can lead to innovation. Just two things that I think will get dramatically better now that companies have incentive to focus on them: * harness design * small models (both local and not) I think there is tremendous low hanging fruit in both areas still.
  • com2kid 7mo ago
    China already operates like this. Low cost specialized models are the name of the game. Cheaper to train, easy to deploy. The US has a problem of too much money leading to wasteful spending. If we go back to the 80s/90s, remember OS/2 vs Windows. OS/2 had more resources, more money behind it, more developers, and they built a bigger system that took more resources to run. Mac vs Lisa. Mac team had constraints, Lisa team didn't. Unlimited budgets are dangerous.
    • phist_mcgee 7mo ago
      Perhaps its because american hyperscalers want unlimited upside for their capital?
    • tasoeur 7mo ago
      Though I do agree with you, I just came back from a trip to China (Shanghai more specifically) and while attending a couple AI events, the overwhelming majority of people there were using VPNs to access Claude code and codex :-/
      • coldtea 7mo ago
        Parent's point was about deployment, not agentic coding.
    • busfahrer 7mo ago
      > Low cost specialized models Can you elaborate on this? Is this something that companies would train themselves?
      • 7mo ago
• czk 7mo ago
  "adaptive" thinking
• itmitica 7mo ago
  The current inference system is on a down slope. It remains to be seen what new wave of AI system or systems will replace it, making the whole current architecture obsolete. Meanwhile, they are milking it, in the name of scarcity.
• henry2023 7mo ago
  The US is bound by energy and China is bound by compute power. The one who solves its limitation first will end this “Scarcity Era”.
  • jakeinspace 7mo ago
    China is installing something like 500 GW of wind and solar per year now. Even if they're only able to build and otherwise access chips that have half the SoTA performance per watt, they will win.
    • odo1242 7mo ago
      Performance per dollar may be more important than performance per watt here, though
      • thelastgallon 7mo ago
        A dollar is an entirely fictional unit and trillions of it can be manufactured at no cost, while watts are constrained by the laws of physics, photons/electrons, supply chain of electricity and all that fun stuff in the real world.
        • ElFitz 7mo ago
          > A dollar is an entirely fictional unit and trillions of it can be manufactured at no cost It’s still a useful proxy for resources allocation and viability.
          • tucnak 7mo ago
            ..unless you're actually reasoning at nation-scale where OP's points apply
            • ElFitz 7mo ago
              I wouldn’t agree. Even at national scale, these projects cost resources. And the resources of all agents (org, countries) are constrained. While we could reason in "performance / watt" and "performance / people", "performance / whatever other resource involved", and "performance / opportunity cost of allocating these resources to this use case and not another", "performance / whatever unit of stable-ish currency" is a convenient and often "good enough" approximation that somewhat encapsulates them all. A simplification, like any model, but still useful.
• com2kid 7mo ago
  To bang on the same damn drum: Open Weight models are 6 months to a year behind SOTA. If you were building a company a year ago based on what AI could do then, you can build a company today with models that run locally on a user's computer. Yes that may mean requiring your customers to buy Macbooks or desktops with Nvidia GPUs, but if your product actually improves productivity by any reasonable amount, that purchase cost is quickly made up for. I'll argue that for anything short of full computer control or writing code, the latest Qwen model will do fine. Heck you can get a customer service voice chat bot running in 8GB of VRAM + a couple gigs more for the ASR and TTS engine, and it'll be more powerful than the hundreds of millions spent on chat bots that were powered by GPT 4.x. This is like arguing the age of personal computing was over because there weren't enough mainframes for people to telnet into. It misses the point. Yes deployment and management of personal PCs was a lot harder than dumb terminal + mainframe, but the future was obvious.
  • deleted 7mo ago
    [deleted]
  • space_fountain 7mo ago
    I've seen this claimed, but I'm not sure it's been true for my use cases? I should try a more involved analysis but so far open models seem much less even in their skills. I think this makes sense if a lot of them are built based on distillations of larger models. It seems likely that with task specific fine tuning this is true?
    • com2kid 7mo ago
      What are you trying to do? Write code? No. Use frontier models. They are subsidized and amazing and they get noticably better ever few months. Literally anything else? Smaller models are fine. Classifiers, sentiment analysis, editing blog posts, tool calling, whatever. They go can through documents and extract information, summarize, etc. When making a voice chat system awhile back I used a cheap open weight model and just asked it "is the user done speaking yet" by passing transcripts of what had been spoken so far, and this was 2 years ago and a crappy cheap low weight model. Be creative. I wouldn't trust them to do math, but you can tool call out to a calculator for that. They are perfectly fine at holding conversations. Their weights aren't large enough to have every book ever written contained in them, or the details of every movie ever made, but unless you need that depth and breadth of knowledge, you'll be fine.
• paulddraper 7mo ago
  This is wrong along multiple axes. 1. Supply can scale. You can point to COVID/supply-chain shocks, but the problem there is temporary changes. No one spins up a whole fab to address a 3 month spike. Whereas AI is not a temporary demand change. 2. Models are getting more efficient. DeepSeek V3 was 1/10th the cost of contemporary ChatGPT. Open weight models get more runnable or smarter every month. Cutting edge is always cutting edge, but if scarcity is real, model selection will adjust to fit it.
• yalogin 7mo ago
  Does this also mean ram prices are not coming down anytime soon?
  • stronglikedan 7mo ago
    they already are
  • dist-epoch 7mo ago
    yes, and it will keep increasing
  • i_think_so 7mo ago
    > Does this also mean ram prices are not coming down anytime soon? One person replies "yes". Another replies "no". This concludes our press conference. <3 HN
• wg0 7mo ago
  There's other side to it too. Whoever running and selling their own models with inference is invested into the last dime available in the market. Those valuations are already ridiculously high be it Anthropic or OpenAI to the tune of couple of trillion dollars easily if combind. All that investment is seeking return. Correct me if I'm wrong. Developers and software companies are the only serious users because they (mostly) review output of these models out of both culture and necessity. Anywhere else? Other fields? There these models aren't any useful or as useful while revenue from software companies by no means going to bring returns to the trillion dollar valuations. Correct me if I'm wrong. To make the matter worst, there's a hole in the bucket in form of open weight models. When squeezed further, software companies would either deploy open weight models or would resort to writing code by hand because that's a very skilled and hardworking tribe they've been doing this all their lives, whole careers are built on that. Correct me if I'm wrong. Eventually - ROI might not be what VCs expect and constant losses might lead to bankruptcies and all that build out of data centers all of sudden would be looking for someone to rent that compute capacity result of which would be dime a dozen open weight model providers with generous usage tiers to capitalize on that available compute capacity owners of which have gone bankrupt and can't use it any more wanting to liquidate it as much as possible to recoup as much investment as possible. EDIT: Typos
  • solenoid0937 7mo ago
    OpenAI has an absurdly high valuation given their cash burn vs RRR. Anthropic's is far more reasonable. It makes no sense to lump these two companies together when talking about valuation. They have completely different financial dynamics
    • wg0 7mo ago
      No matter how low and reasonably Anthropic is valued, don't think $200 Max plans are going to recoup the investment + some return on top because size of the software industry is not that huge and profit margins for AI inference aren't very high either.
      • 7mo ago
• 2001zhaozhao 7mo ago
  AKA, the beginning of big companies being able to roll over small companies with moar money (note: I don't expect this to actually happen until the AI gets good enough to either nearly entirely replace humans or solve cooperation, but the long term trend of scarce AI will go towards that direction)
• ttul 7mo ago
  Energy scarcity will drive more innovation in local silicon and local inference. Apple will be the unexpected beneficiary of this reality.
• piokoch 7mo ago
  Well it's in the books. O(n^2) algorithms are bad in the long run, transformers algorithm has such complexity, so not a big surprise we hit the limits.
• utopiah 7mo ago
  Initially I thought "Well... good for AI companies because they can then charge more" but IMHO that's a very tricky position because it means the cheap wave is behind us. It's one thing to "sell" free or symbolically cheap stuff, it's another to have an actual client who will do the math and compare expenditure vs actually delivered value.
  • classified 7mo ago
    > and compare expenditure vs actually delivered value Which means that the hype production will be driven up another few notches to make people doubt their rational findings and keep them in irrational territory just a tad longer. Every minute converts to dollars spent on tokens.
• keiferski 7mo ago
  We just had a realization during a demo call the other day: The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up. Not being dependent on LLMs for your fundamental product’s value will be a major advantage, at least in pricing.
  • andersmurphy 7mo ago
    Yup. Also regardless of price they need to spend more and more as the project collapses under the inevitable incidental complexity of 30k lines of code a day. It's similar to how if you know what you're doing you can manage a simple VPS and scale a lot more cost effectively than something like vercel. In a saturated market margins are everything. You can't necessarily afford to be giving all your margins to anthropic and vercel.
    • prox 7mo ago
      I also can’t wait for the time when few know how to code. Just like how many folks don’t know html from css when the homebrew website went away. Their might always be llms, but the dependence is an interesting topic.
      • Cthulhu_ 7mo ago
        Look no further to be honest; look at older generation programming languages like COBOL and how sought-after good developers for that language are. But I'm also afraid / certain that LLMs are able to figure out legacy code (as long as enough fits in their context window), so it's tenuous at best. Also, funny you mentioned HTML / CSS because for a while (...in the 90's / 2000's) it looked like nobody needed to actually learn those because of tools like Dreamweaver / Frontpage.
        • prox 7mo ago
          Well, on the 2nd paragraph, I have no illusion they’ll figure out more as they are being trained. I am more thinking of the custodians (as coders turn into that) Say you are a good coder now, but you are becoming a custodian, checking the llm work will slowly erode your skills. Maybe if you got a good memory or an amazing skillset it might be some time, but if you don’t use it, you lose it.
• throwaway290 7mo ago
  wasnt ai supposed to get us post-scarcity?
  • PessimalDecimal 7mo ago
    That worked out, for the founders of frontier labs at least.
• tim333 7mo ago
  >For the first time since the 2000s, technology companies are confronting the limits of their supply chain. I thought there'd been a shortage of cheap GPUs since ChatGPT took off and also before that in various crypto booms. I'm not sure it's a new thing.
  • the_gipsy 7mo ago
    But that concerned mostly only gamers and cryptominers. AI is supposed to be replacing traditional software development, which affects everything.
• sdevonoes 7mo ago
  It’s time to be AI-independent. It’s like AWS, for most of us, it’s not worth it.
• hemangjoshi37a 7mo ago
  [dead]
• KaiserPro 7mo ago
  one graph, One graph and the author is pinning an entire theory on it? Infra is always limited, even at hyper scalers. This leads to a bunch of tools dfofr caching, profiling and generally getting performance up, not to mention binpacking and all sorts of other "obvious" things.
  • malshe 7mo ago
    On X I had seen him mostly posting memes so this post seems par for the course
  • losvedir 7mo ago
    > Infra is always limited, even at hyper scalers I think maybe infra is limited only at hyperscalers. For the rest of us it's just how much capacity to we want to rent from the hyperscalars. It's kind of a recent cloud-native mindset, since back in the day when you ran your own hardware scaling and capacity was always top of mind. Looks like AI compute might be like that again, for the time being.
  • sph 7mo ago
    One graph, about 100 words, AI in the title: Hacker News front page. Not bad for a coffee break of effort.
• chatmasta 7mo ago
  Why is written with an assumption that we have finite hardware production capacity? Industrial processes can scale up, new factories can come online… it will take a while but the whole point of economics is that supply will scale to meet demand. The shortage is a temporary, point-in-time metric. And that’s not considering the software innovation that can happen in the meantime.
  • Bengalilol 7mo ago
    The economic hypothesis that has dominated the past hundred years is that economic growth is infinite because resources are infinite and (almost) free. We all know this is unrealistic and disconnected from our human condition. Regarding "innovation", I agree with your idea. I even think that the major innovation will be to transpose models locally, using reduced infrastructures that will still be sufficient for the majority of use cases.
• Bengalilol 7mo ago
  ... and I have this little idea in the back of my mind: when companies can no longer keep up with demand and people have (albeit more limited and reduced) local capacity, minds will start focusing on techniques (more humble and modest ones) to keep part of the system running locally, without dependency. I know it may sound ridiculous, but it could actually become a way to break away from the business models that have been developed over the past few decades. Broadly speaking, this even amounts to saying that the biggest victims of AI could be the companies that bet on AI as a service. Yet I know my vision is way too idealistic but I'm coming to imagine that a human brain, although less efficient in the long run, remains a reliable way to control the resulting costs and could even turn out to be more advantageous and more readily available than its silicon-based counterpart.
  • 20after4 7mo ago
    The human brain is incredibly efficient (Approximately 20W of energy consumption¹). These AI systems use many orders of magnitude more energy than human equivalents. 1. https://pmc.ncbi.nlm.nih.gov/articles/PMC8364152/ https://pmc.ncbi.nlm.nih.gov/articles/PMC8364152/
• bcjdjsndon 7mo ago
  Neither is this the first time nor are they really confronting it
• frigg 7mo ago
  The models have already plateaued, you don't need latest and greatest.
• latentframe 7mo ago
  This isn’t really looking like AI scarcity it’s more like compute becoming the bottleneck : when the access depends on chips energy and capital it stops being a pure software game and the winners are often whoever can secure capacity first
• siliconc0w 7mo ago
  Definitely feeling this - the subsidized subscription plans are already starting to buckle.
• 0xbadcafebee 7mo ago
  This isn't the first time they've dealt with scarcity, there's been supply chain scarcity four times since 2000. Post-dotcom boom, CDMA scarcity, HDD/flash scarcity, Pandemic scarcity. The scarcity isn't long-term. Like all manufactured products, they'll ramp up production and flood the market with hardware, people will buy too much, market will drop. Boom and bust. We're also still in the bubble. Eventually markets will no longer bear the lack of productivity/profit (as AI isn't really that useful) and there will be divestment and more hardware on the market as companies implode. Nobody is making 10x more from AI, they are just investing in it hoping for those profits which so far I don't think anyone has seen, other than in the companies selling the AI to other companies. But more importantly, the models and inference keeps getting more efficient, so less hardware will do more in the future. We already have multiple models good enough for on-device small-scale work. In 5 years consumer chips and model inference will be so good you won't need a server for SOTA. When that happens, most of the billions invested in SOTA companies will disappear overnight, which'll leave a sizeable hole in the market.
  • topherhunt 7mo ago
    > In 5 years consumer chips and model inference will be so good you won't need a server for SOTA. Naw man, you crazy. If you tell me that in 5 years, consumer chips will be so good that I can run GPT-5.4-level AI on my phone, I'd find that plausible (I buy cheap phones). If you're telling me that in 5 years we won't need _servers_ because our _phones and/or desktops_ will be powerful enough to run the biggest newest LLMs in existence, I question your judgment, I think that prediction shows a deep uncreativity about how massively compute-hungry SOTA models will get. The valuable things to do with inference will keep being a server niche because they'll keep being 1-2 OOM more compute-hungry than whatever consumer hardware can handle. Like gaming: my laptop can run games from 2015 at max settings no problem but the games actually worth getting excited about in 2026 still melt a $2k GPU, because whatever headroom the hardware gains, developers immediately spend on ray tracing and Nanite and modelling individual skin cells or whatever. I don't see any plausible reason to expect that the ceiling on "valuable server-side compute" or "inference capacity" will rise any more slowly than the on-device capability is rising. My assumption is that in 2031, SOTA top-intelligence AI will be hosted on cloud servers like it is today, offering dirt-cheap access to capabilities we can't even dream of today, while your Android will be running some open-source GPT-5+ equivalent.
• mystraline 7mo ago
  And folks are just now realizing the SaaS token provider rug-pull? How convenient, especially since everything has some LLM slop interaction. But that rug isnt going to pull itself!
• AtlasBarfed 7mo ago
  Pay for the latest AI for EXCLUSIVE POWERS ... Trying to up-tier fractional improvements in something that can't be quantified easily, and !BONUS! with gated access it can't be as easily analyzed by the (low profit) AI analyzers/benchmarkers. Foster paranoia among top executives that the fractional/debatable improvement is a MUST HAVE to STAY COMPETITIVE in your industry. Meanwhile, I have not seen any improvement in software in the now almost ?three to four? years than mainstream LLM and AI coding assistance has arrived on the scene. Although I will hold out the possibility that software has actually gotten far worse for the end user, because AI code is being dedicated to revenue enhancement and dark data collection.
• builderminkyu 7mo ago
  [flagged]
• NoSalt 7mo ago
  A few years ago, I purchased a handful of 250GB SSDs from amazon for $17.00 each. Last year, I purchased a few 8TB hard drives for $80.00 each. Today, I am sad. ;-(
• rafaelero 7mo ago
  Companies who could see it clearly and ignored the "AI is a bubble duh" crowd will ultimately get benefited by the GPUs they already acquired. The companies who acted cautiously will get burned.
• cowartc 7mo ago
  The scarcity framing assumes compute is the bottleneck. For most production deployment's Ive seen, the actual bottleneck is evaluation and knowing what to trust. You can throw cheaper models at a problem all day but, if you can't measure where the model fails on your data, You're just making mistakes faster at a lower cost. Compute gets cheaper. Reliable evaluation doesn't.
• sixhobbits 7mo ago
  There is a lot of demand still coming for sure but I think I'm more optimistic. Ready to eat my hat on this but - higher prices will result in huge demand destruction too. Currently we're burning a lot of tokens just because they're cheap, but a lot of heavy users are going to spend the time moving flows over to Haiku or onprem micro models the moment pricing becomes a topic. - data centers do not take that long to build, probably there are bottlenecks in weird places like transformers that will cause some hicups, but nvidia's new stuff is waay more efficient and the overall pipeline of stuff coming online is massive. - probably we will see some more optimization at the harness level still for better caching, better mix of smaller models for some use, etc etc. These companies have so much money and they at least anthropic and openai are playing winner takes it all stakes, with competition from the smaller players too. I think they're going to be feeding us for free to win favour for quite a while still. Let's see though.
  • mark_l_watson 7mo ago
    I agree and I am amazed at how much money some individuals and also a friend's company burn on token costs. I get huge benefits from this tech just using gemini-cli and Antigravity a few times a week, briefly. I also currently invest about $15/month in GLM-5.1 running Hermes Agent on a small dedicated VPS - fantastically good value for getting stuff done and this requires little of my time besides planning what I need done. I think the token burners are doing it wrong. I think that long term it is better to move a little slower, do most analysis and thinking myself, and just use AI when the benefits are large while taking little of my time and money to use the tools.
• jFriedensreich 7mo ago
  This is probably even the "fun" part of the whole picture. The purely dystopia starts when investment firms just silently grow bigger and bigger data centers like cancer. There will be no press releases, no papers, no chance anyone without billions will even know the details yet alone get access. One day we realise the worlds resources (maybe not as in the paperclip maximiser, but as in memory, energy, GPUs, water, locations) are consumed by trading models and the data centres are already guarded by robot armies. While we were distracted frighting with anthropic and openAI the real war was already over. Mythos is one sign in this direction but i also met a few people who were claiming to fund fairly large research and training operations just by internal models working on financial markets. I have no way to verify those claims but this happened 3 times now and the papers/research they were working on looked pretty solid and did not seem like they were running kimi openclaw on polymarket but actual models on some significant funds. Would be really interested if anyone here has some details on this reality. I would also not be surprised if this is a thing that people in SF just claim to sound dangerous and powerful.
  • nicbou 7mo ago
    The fun times will also end for consumers soon enough, when the oligopolies are established and investors start asking for returns.
• insane_dreamer 7mo ago
  Companies who become dependent on AI to "optimize" their offering/processes are going to be faced with some serious vendor lock-in unless they do it in such a way that they can swap out the foundation model.
• skybrian 7mo ago
  That’s a lot of speculation based on one graph. It doesn’t cover whatever Google is doing, for example.
• k8kraze 7mo ago
  [dead]