6 ms·

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Mo

• ozgrakkurt 3mo ago
  As someone who worked at multiple startups, I am pretty sure they get it but it doesn’t fit their goals. They want to moat where they don’t need to compete with other companies because they have something that other companies can’t have. In my opinion this is a short-sighted and greedy worldview. Haven’t seen it work personally. It is a different version of the month-to-month salary guy thinking he will be a billionaire and having that thrash “mindset”. The reality is that practically none of those companies will amount to anything and they would be better off weighing the usefulness aspect of their output more. Instead they are imagining they will be Google. Anthropic and openai ofc are the pinnacle of this greed culture and they correspond to FTX from the crypto trash hype so I don’t think they fit into the scale of sensibility. Coming from this perspective, it is pretty easy to see what they are.
• spongebobstoes 3mo ago
  the major labs want to advance science. current business use cases are a happy accident
  • cmiles8 3mo ago
    That party is over. They’re all on the clock now to show they can make money or the plug will be pulled.
• FranOntanaya 3mo ago
  Some specialized models may end solving themselves by helping fit the problem with the appropriate regular algorithms/formulas, which are a million times more efficient. So they are probably less attractive to dump money on. As of currently they still benefit from expressing lots of patterns that nobody bothered formalizing.
  • deleted 3mo ago
    [deleted]
• synergy20 3mo ago
  what are those cheap fine-tuning services these days?
  • bugglebeetle 3mo ago
    Tinker is pretty cheap. Prime Intellect if you want more flexibility.
    • synergy20 3mo ago
      unsloth for small use cases, tinker for enterprise level use cases?
      • bugglebeetle 3mo ago
        I haven’t used their platform so can’t speak to this, but super brilliant team and big fans of their work overall!
  • nolok 3mo ago
    I would not say cheap per say, but in the context of "where is the business success going to be" that's one of the reasons why Mistral focuses on providing tuned model on premises to their customers.
• stymaar 3mo ago
  > The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages I think they understand it but they think they can get away with it because they own the narrative. As long as they can make people believe they need such a model, it doesn't matter if it's true or not. They are playing the cloud playbook, it didn't matter that most companies didn't need 99,999% uptime and instantaneous horizontal scaling, as long as people believed they did they are happily paying 10-100x the cost to AWS instead.
• autuni 3mo ago
  seems that, like software engineering and other areas before, they also have to rediscover that one single monolithic solution that handles everything is too inflexible and not maintainable, it's just a bad approach. people don't need the models they use to generate their codebase to also be able to translate Shakespeare into gen-z slang
• vidarh 3mo ago
  Fine timing takes time and data, though. If smart enough models get cheap enough, then most people have lots of use cases that are cost insensitive enough that it's not worth the effort. Of course, "smart enough" is a low enough threshold for most uses that this is still a problem for the frontier labs. But at the same time, a truly smart enough closed model could also potentially command almost whatever they'd care to charge for it. Whether they can actually get to that level remains to be seen, but I can definitely see a situation where most people are perfectly happy with cheap middle of the tree models while large corporations pay magnitudes more than current API pricing for access to models never even marketed as a mass market product and keep the labs afloat. It's of course be a lot easier for them to find the path towards that of they didn't need to compete with open models in the meantime.
  • pbronez 3mo ago
    Yes. Fine tuning is an optimization. Like all optimizations, it’s downstream of figuring out your problem, implementing a naive solution, scaling the naive solution, and getting frustrated by SWAP-C. THEN you start poking at what optimizations are possible, choosing an approach, implementing the optimization, and redeploying. This cycle happens when you have a well defined use case with high volume. It is the far opposite end of the spectrum from the general purpose intelligence on tap that frontier AI models purport to deliver. I think focused fine tunes and big general models will coexist. Ideally with smart routing and caching to use small, specialized and local options when appropriate.
• ezst 3mo ago
  But then can't the case be made that narrower and stricter-defined use cases are better served by more conventional ML? If/Wherever efficiency is a concern, that is.
  • gchamonlive 3mo ago
    That's assuming conventional ML and fine-tuned SLMs are interchangeable with acceptable behavioral change/degradation for any specific case
    • cyanydeez 3mo ago
      LLMs fit between well defined and poorly defined data; if you have well defined data, you can just feed it to ML models and they'll do what you want. LLMs will take poorly defined data and "potentially" create well defined output. So ideal systems will likely be constructed with a LLM on one end and a ML on the other and some how, if you can feedback poor data from the ML back into the LLM to clean up the data, you have a magic layer that doesn't care as much about the structure of the data. That's a lot of supposition, but that's the difference I see between what we can do with LLMs and how ML models are structured. They might be classified as doing the same thing, but their data inputs are vastly different.
  • sciaijournal 3mo ago
    [flagged]
• torginus 3mo ago
  Yeah and these latest and greatest models universally suck hard for any use case outside of the few 'blessed' ones. Like I'm sure their ability to write prose and generally sound like a human being has regressed quite a bit, but even if not. Opus 5 is barely above GPT4 when it comes to stuff like home improvement advice.
• vkaku 3mo ago
  Very true. In fact, there's a lot more under the surface that will come to light once more efficient tiny models come out. P.S. Building something that proves that you don't need those many params even. https://github.com/guilt/tinytot https://github.com/guilt/tinytot
  • teodosin 3mo ago
    The premise of your project seems compelling. Is it novel or building on existing work? Anything one could read or watch to get introduced to that area of research?
    • dineshdb 3mo ago
      I took a look at the codebase and found that this is just pattern matching. There is no any novelty here. It looks good for knowledge-base retrieval system, but other than that, this is just nothing more than a pattern-matcher. Would love to be proven wrong with future updates.
      • vkaku 3mo ago
        Sure. Like throwing money at problems is novel. Why don't you go and actually build something novel? People need to understand better how current models work, and if retrieval indeed solves 99% of the problems people care about, then those other models aren't doing anything super novel either. I'm not arguing here: The code is out there, data is out there. Do better if you know something actually better. Thanks!
    • vkaku 3mo ago
      You would already be able to see RSI and loops being a thing. So there's going to be more and more research coming out telling you what size of models are really needed to solve a specific task. When you combine that with RSI, the next frontier would be to get this model as close to the real world with instant data and instant reasoning, no fine tuning at all. And see how this one could iterate itself. I'll also be trying out more tiny models and seeing if they can call each other, and be as effective; You will also be seeing me trying this, in addition to what I'd already mentioned in the other reply.
• samuelknight 3mo ago
  Going by the chart in the article, if your total workload is 1k cataloged items and your quality threshold is 70%, why wouldn't you just pay $19 to gemini API instead of $500 + time to make a custom fine tune?
  • somenameforme 3mo ago
    The resultant model much cheaper per call and outperforming frontier models. So you'd quickly make your money back at any sort of meaningful scale, continue to pay less from that point on, and have better results. And I think removing a third party dependency is also a benefit in and of itself.
• dominotw 3mo ago
  There is no such thing as extracting "task specific intelligence" into a cheaper model.
  • krapht 3mo ago
    News to me. What do you call mixture-of-experts and model distillation?
    • dominotw 3mo ago
      those are still general purpose models. do you have an example of task specific model?
• Aurornis 3mo ago
  > The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most generic use cases do benefit from a model that has been trained broadly. When you don’t know the specific use case ahead of time, you have to have world knowledge ready to go. Even when coding it’s helpful to have all that knowledge on tap so the model can understand product intent and use cases for the product you’re building. > As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles The real expensive part of fine tuning is gathering a good data set. That’s been the hard part of training anything for a long time. If you’re lucky enough to have a neatly organized and clean data set then you can attempt it, but you need to be in a position to run evals and measure quality. I’ve done it, but there are so many use cases where the engineering, data labeling, and ongoing quality review hours cost so much that it would be cheaper to continue using a frontier lab model that just works from the start.
  • amelius 3mo ago
    Can't you take a broadly trained model and throw away useless knowledge (like foreign languages, and 99% of wikipedia) and still end up with a model that is as intelligent and useful as the original but much smaller?
    • AussieWog93 3mo ago
      I'm not an AI researcher (would love to hear an actually informed answer on this!), but AFAIK the weights are more or less a black box - you can't easily pinpoint that weights x, y and z relate to Ancient Rome and can therefore be safely removed from a model that's optimised for coding.
      • choilive 3mo ago
        You can, and there are many techniques to do so - it's not super precise but you can essentially "delete" concepts out of a model.
    • 3mo ago
• mbesto 3mo ago
  This is basically the entire argument by Richard Sutton happening in real time: https://www.youtube.com/watch?v=21EYKqUsPfg https://www.youtube.com/watch?v=21EYKqUsPfg Right now, we keep creating new models with larger and larger weights and throwing hardware at it. His argument is that this is a dead end and ultimately we'll eventually go back to purpose fit algorithms like we've always done in the history of AI development.
  • sailfast 3mo ago
    The goal is still the singularity though right? Sure, for business, personal, etc it makes sense to build purpose built but that’s not what these companies are trying to achieve in the end. They want to own a nuclear weapon that outstrips everything else as soon as it exists so they “win”
  • moojacob 3mo ago
    Did you even watch the video? Purpose fit algorithms outperform is quite literally the OPPOSITE of Suttons argument. This is the man who wrote this: “The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.” His problem with LLMs are they aren’t general enough… In fact his essay the bitter lesson was what inspired the current LLM scaling. If you watched the video, you can even see the part where he talks about how AI will fully replace humans as the next stage of life. How wholesome!
  • suttontom 3mo ago
    It's kind of amazing how badly you've misrepresented Richard Sutton's position. This is the man who coined the term https://en.wikipedia.org/wiki/Bitter_lesson https://en.wikipedia.org/wiki/Bitter_lesson.
• DanielHB 3mo ago
  I actually prefer using less powerful models most of the time, I might use a stronger model initially and then switch to weaker models as I fine tune the output. Offloading too much of your task to models eliminates the human ownership, without ownership you can't move forward. Context sizes can't keep up with large codebases and markdown files with instructions and guidelines only take the LLM so far in the ownership aspect. My gut feeling is that to start offloading ownership to the LLM we would need to see at least two order of magnitude increases on context size.
• hectormalot 3mo ago
  I’m not sure it will go this direction. Using a generic (near) frontier model is often cheap enough that you need to talk really big volumes before it pays off to fine tune. My example: we were doing single digit millions of automated call summaries a few years back at a major bank with GPT-4o. Smaller model gave more rejected summaries (compliance not happy), so we briefly looked at fine tuning a smaller model and basically concluded that even at that scale the effort of data collection, management, fine tuning, hosting the model, etc didn’t have a sufficient business case vs picking up other projects.
• giancarlostoro 3mo ago
  I mostly agree, but at the same time I wouldn't blindly ignore the "it's China" angle at all. These are models you cannot open up and do a thorough audit of. I do think local compute will eventually be good enough that most people will just use local models. Microsoft might be the ultimate winner of that if they can stop ruining Windows and focus on making an OS similar to how straightforward and simple macOS is, no ads literally everywhere for Microsoft Office. Just make a good OS Microsoft, is that so much to ask? You HAD a really solid OS and you ruined it.
• billmalarky 3mo ago
  >As does the economic picture that justified the massive infrastructure building that’s now broadly funded by a complex network of debt. I have a genuine question and I'd like to hear people's good faith thoughts on this. There's a fair case that open models are threatening to institutions who spent a lot on training proprietary SOTA models. But to my understanding, the massive investment spend (much of it debt backed as you note) is on data centers, chips, physical infra. Yes, this infra is needed to train the models, but it is also needed to serve inference. Perhaps the costs associated with training SOTA models is ultimately a "bust" given open models eroding the SOTA closed model performance advantage. But demand for inference is skyrocketing and there seems to be no end in sight. The physical hardware underpinning inference is in fact a scarce good (currently, and this seems sustainable at least over mid-term). And inference is a scarce service as such. I know that cost of inference constantly goes down as models, technical infrastructure, and applied AI techniques become more efficient (specialized SLMs etc). So this puts downward pressure on prices. But still... demand for inference is just growing like crazy regardless. Putting upwards pressure on prices. Doesn't this mean that all the spending on AI infra is much better positioned to get positive ROI regardless of the type of model being served? Put another way, models seem to be commoditizing, but physical hardware is not (currently). The vast majority of the AI boom spend is on hardware to my understanding (even training capex can be repurposed for inference). Doesn't this suggest that the economics for the "railroads level of build-out spending" are healthier than they might seem at first glance?
  • fc417fc802 3mo ago
    Maybe. But you have to consider any difference in expected returns. Also account for expected efficiency and capability advances of new hardware. And perhaps most importantly, as of a few years ago the failure rate of these GPUs when run hard was in the ballpark of 10% per year (that might or might not change but regardless it definitely needs to be accounted for). It seems exceedingly unlikely that we will see growth sufficient to justify the level of investment. But as you say that doesn't necessarily mean a worst case scenario either. An investment can be bad without being ruinous after all.
• mandeepj 3mo ago
  > The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. I think that's true across the board. Along those lines, most companies all of a sudden started looking for AI researchers, instead of software engineers, which is what they still need :-). Just like a decade-ish ago, when the FAANG interview style got openly known, then everybody started to conduct interviews in that pattern :-)
• Yopolo 3mo ago
  Im pretty sure they know quite well but its not their current focus on creating or allowing others to create small finetuned and optimized models. In the race they are in, its still highly beneficial to be the frontier model. I use the frontier model every single day through my company and my company happily spends these tokens. It makes a huge difference if different people can use one interface to do everything. Big models give you fundamental things: A lot of facts/context, usability (you might be a native english speaker, don't underestimate how hard it is for A LOT of people to formulate what they want/need in english only AND a low complexity. No one needs to build a router and x sub models and a router architecture. You literaly just have an API, you might choose the model and the effort but thats it. I find this current state of the art a LOT more telling on the current progress we are in than anything else. I'm confident that small optimized models will become a lot more relevant like lets say java + english + a second language + spring boot + postgresql. It might also be beneficial in the long term to finetune with your project details. For a lot of very technical non human interfacing things, finetuning is happening left and right. But what i find very interesting is emerging complexity capabillity. I believe that fables skill to hold more topics and combine more complex solutions together is because of its parameter size. We will have to figure out if we can extract this complexity out of it while reducing the training data in a way that the training data focus more on thinking. Plenty of smaller thinking models show that this is doable. Btw. Mixture of Experts is for sure not optimal for this, but it already is a form of optimized sub models. Perhaps we might just have MoE with a million experts in the future. One per lanuage + area of expertise etc. Also don't forget: IF AGI is coming through a current frontier model, you will let it work for hours, days and weeks on one problem completly independent of any human input and it will be better than a human. If they reach this before a collapse, we are done and they 'won'. For this you need big frontier models.