3 ms·

This is one reason I've been trying to figure out tasks (and products based on those tasks) that can be pushed to the edge, either via small specialized models

This is one reason I've been trying to figure out tasks (and products based on those tasks) that can be pushed to the edge, either via small specialized models or small general purpose open models. I suspect the same desire to keep unit costs low is part of why Google is falling behind on the "frontier", but seemingly at the lead, or near it, on models that run on-device. I think they're just focused on making models for tasks that don't require boiling the ocean. But, it's a hard problem. The models that run locally on normal computers/phones are pretty terrible compared to the frontier, without specialization and fine-tuning. And, even with specialization and fine-tuning, often a high-end general purpose model is going to do a better job and people don't need a bunch of local tools installed to do their various tasks.

skinfaxi 2mo ago
> And, even with specialization and fine-tuning, often a high-end general purpose model is going to do a better job and people don't need a bunch of local tools installed to do their various tasks. I think this is the critical point that would be interesting to see if it holds. Technology seemingly tends towards increased specialization.
- antonvs 2mo ago
  > Technology seemingly tends towards increased specialization. The bitter lesson says the exact opposite.
  - ahartmetz 2mo ago
    The bitter lesson is rather specific and IMO vastly over-applied. It basically says that you can beat hand-tuning by waiting a few years and using "more of the same" generic method. It's not all that different from saying "Don't optimize software, just wait for faster hardware". Yet highly optimized software exist, and performance on currently available stacks is a competitive advantage. What gives? I say: Staying ahead of the "Don't optimize, just wait" curve can absolutely make sense. At worst, your advantage decays after a few years. At best, you stay ahead by n number of years and keep increasing the gap as you invest more.
    - SwellJoe 2mo ago
      The question is: How soon? If you make a specialist model and harness that takes six months to train and launch, will you have enough time to recoup your costs and make a profit before the generalists get good enough and cheap enough in your area of specialty and become the simpler option? We already see "skills" repos containing a bunch of specialist stuff being pushed to general models via prose instructions for how to do various things and use various tools; and those models will do the tasks much more expensively and slowly, but without needing the user to find/buy/learn something specific to do the job. The bitter lesson may be narrow, but the fact is people have been using spreadsheets for everything for 50 years, and only sometimes can a specialist tool effectively displace it in an organization. Very powerful generalist models are starting to feel like spreadsheets: The default way to solve problems in organizations. You might be able to pick off some of the specialist cases, but you might also miss.
      - ahartmetz 2mo ago
        I think the bitter lesson (from 2019, what is now seen as the tail end of an AI winter - probably wasn't at the time) made the most sense in academia during AI winter. There is no academic fame to be had by making something a little better for a limited time. When billions are at stake, there is potentially a lot of money (and utility for customers) to be gained by doing some extra work even if the work is thrown away soon after. Most commercial products contain a ton of extra work to be slightly better than the competition in one area or another. Perhaps we can call this "The sweet dollars lesson".

skinfaxi 2mo ago

> And, even with specialization and fine-tuning, often a high-end general purpose model is going to do a better job and people don't need a bunch of local tools installed to do their various tasks. I think this is the critical point that would be interesting to see if it holds. Technology seemingly tends towards increased specialization.

antonvs 2mo ago
> Technology seemingly tends towards increased specialization. The bitter lesson says the exact opposite.
- ahartmetz 2mo ago
  The bitter lesson is rather specific and IMO vastly over-applied. It basically says that you can beat hand-tuning by waiting a few years and using "more of the same" generic method. It's not all that different from saying "Don't optimize software, just wait for faster hardware". Yet highly optimized software exist, and performance on currently available stacks is a competitive advantage. What gives? I say: Staying ahead of the "Don't optimize, just wait" curve can absolutely make sense. At worst, your advantage decays after a few years. At best, you stay ahead by n number of years and keep increasing the gap as you invest more.
  - SwellJoe 2mo ago
    The question is: How soon? If you make a specialist model and harness that takes six months to train and launch, will you have enough time to recoup your costs and make a profit before the generalists get good enough and cheap enough in your area of specialty and become the simpler option? We already see "skills" repos containing a bunch of specialist stuff being pushed to general models via prose instructions for how to do various things and use various tools; and those models will do the tasks much more expensively and slowly, but without needing the user to find/buy/learn something specific to do the job. The bitter lesson may be narrow, but the fact is people have been using spreadsheets for everything for 50 years, and only sometimes can a specialist tool effectively displace it in an organization. Very powerful generalist models are starting to feel like spreadsheets: The default way to solve problems in organizations. You might be able to pick off some of the specialist cases, but you might also miss.
    - ahartmetz 2mo ago
      I think the bitter lesson (from 2019, what is now seen as the tail end of an AI winter - probably wasn't at the time) made the most sense in academia during AI winter. There is no academic fame to be had by making something a little better for a limited time. When billions are at stake, there is potentially a lot of money (and utility for customers) to be gained by doing some extra work even if the work is thrown away soon after. Most commercial products contain a ton of extra work to be slightly better than the competition in one area or another. Perhaps we can call this "The sweet dollars lesson".