3 ms·

Anyone who has designed circuits will consider CPUs wasteful compared to ASICs. This new FPGA technology is just a less efficient ASIC. That’s roughly what I’m

Anyone who has designed circuits will consider CPUs wasteful compared to ASICs. This new FPGA technology is just a less efficient ASIC. That’s roughly what I’m hearing. The fact that general purpose intelligent classifiers can be dynamically hacked together by an LLM in real time to allow them to build evolving labeled and understandable networks that perform substantially faster than the LLM, and can act as an intermediate sorting and organizing layer for caching context or handling simple tasks, and a complete layman like me can assemble a teachable layer of these in a few days from an inexpensive service… That’s wild! And then you can identify where an expert system needs a more specific ML technique for efficiency within this network that overlays the SOTA model. Or manually adjust the stored context in each secondary “neuron”. And paths forward can run programs or take actions at relative high speed. And you can share these with others and improve them as a group. You could insert this at the datacenters at scale with a local supervising expert to prune and encourage proper growth. You could identify specific gaps in capability that need more training, and patch over them temporarily. Then you train those corrections back into the general purpose model, or you identify highly efficient subsystems for specific purposes. And this is just one way to use it. High speed intelligent workflows can live in this. There’s a spot for a local LLM to learn on the fly. Maybe I’m way off base, but for the non-experts Jev seems extremely valuable.

NegativeLatency 17d ago
Also all of the mobile/embedded/resource constrained environments. Like sure my phone can run an LLM but it’s going to be bad and drain my battery.
gyanchawdhary 17d ago
Dude, this is gold!
bigyabai 17d ago
I don't think either of you are wrong. The parent's assertion is that we've known this for almost a decade. BERT was highly usable for classification and sentiment analysis a whopping 9 years ago, despite being less than 0.5B parameters large. Similar-scale models like FLAN-T5 showed that it could be improved without substantially scaling up. Today, we're extremely spoiled by trillion parameter-scale models. Our conceptualization of vibe coding relies on wasteful tool-calling paradigms, the one-size-fits-all mentality of LLMs is part of the marketing blitz to make people buy more tokens. It's lazy on the part of frontier labs, but also wastes electricity, time and money.
- AIorNot 17d ago
  Lol your argument is the same as programmers who complain about Javascript and internet browsers being the most common interface for all solutions on a computer You guys dont understand that the Lowest common denominator ALWAYS wins - its why excel is the linga franca for most companies LLMS and AI coding are the new javascript easy way to build amazing things and that trumps the tool specializers Years of Big Data and Data Engineers building fit for purpose ML pipelines expensively working in a shadowy corner of the company have been replaced by the PM vibe coding a tool to categorize his emails by relevance
  - bigyabai 17d ago
    Again - you are right, but it still doesn't refute the grandparent's claim that today's AI is lazy, wasteful and marketing-driven. There is room to improve, and if US labs don't take the initiative then Chinese ones will.
    - mediaman 16d ago
      It is lazy and wasteful if you ignore the costs of specialized skills in doing it the "right way." If you stop looking at things in a narrow technical frame, and look at it as an organization, it's not wasteful. And lazy is a useless pejorative used against products that let people do things easily. Lazy is good. When you learn how to make products that allow people to be more lazy, you will become successful.
      - intended 16d ago
        What results though. The people seeing measurable improvements to their core work with LLMs are coders. Everyone else is taking over intern level work from someone else’s team. They are reducing the friction costs of talking to someone else, for about a 30% productivity gain. Firms are trying desperately to automate their white collar workers, and that is following the same trend as all other automation projects, and ML/deep learning efforts in history.
  - 17d ago
deleted 17d ago
[deleted]
sscaryterry 17d ago
FPGA's are definitely not new. They've been mainstream for 20 odd years+. See https://www.eetimes.com/fpga-market-to-pass-2-7-billion-by-10-in-stat-says/?_ga https://www.eetimes.com/fpga-market-to-pass-2-7-billion-by-1...
slrainka 16d ago
That's the thing. LLM's can be used as a real-time teaching layer and build systems that can operate at deterministic speeds and escape hatch into a LLM when confidence levels drop. This approach has helped us save more than $1 Million annually against the straight LLM classifiers at scale (and pass that on to the customers) https://sureshsubasinghe.substack.com/p/how-to-cut-agent-llm-costs-by-85 https://sureshsubasinghe.substack.com/p/how-to-cut-agent-llm... https://sureshsubasinghe.substack.com/p/the-god-model-fallacy-how-to-double https://sureshsubasinghe.substack.com/p/the-god-model-fallac...

NegativeLatency 17d ago

Also all of the mobile/embedded/resource constrained environments. Like sure my phone can run an LLM but it’s going to be bad and drain my battery.

gyanchawdhary 17d ago

Dude, this is gold!

bigyabai 17d ago

I don't think either of you are wrong. The parent's assertion is that we've known this for almost a decade. BERT was highly usable for classification and sentiment analysis a whopping 9 years ago, despite being less than 0.5B parameters large. Similar-scale models like FLAN-T5 showed that it could be improved without substantially scaling up. Today, we're extremely spoiled by trillion parameter-scale models. Our conceptualization of vibe coding relies on wasteful tool-calling paradigms, the one-size-fits-all mentality of LLMs is part of the marketing blitz to make people buy more tokens. It's lazy on the part of frontier labs, but also wastes electricity, time and money.

AIorNot 17d ago
Lol your argument is the same as programmers who complain about Javascript and internet browsers being the most common interface for all solutions on a computer You guys dont understand that the Lowest common denominator ALWAYS wins - its why excel is the linga franca for most companies LLMS and AI coding are the new javascript easy way to build amazing things and that trumps the tool specializers Years of Big Data and Data Engineers building fit for purpose ML pipelines expensively working in a shadowy corner of the company have been replaced by the PM vibe coding a tool to categorize his emails by relevance
- bigyabai 17d ago
  Again - you are right, but it still doesn't refute the grandparent's claim that today's AI is lazy, wasteful and marketing-driven. There is room to improve, and if US labs don't take the initiative then Chinese ones will.
  - mediaman 16d ago
    It is lazy and wasteful if you ignore the costs of specialized skills in doing it the "right way." If you stop looking at things in a narrow technical frame, and look at it as an organization, it's not wasteful. And lazy is a useless pejorative used against products that let people do things easily. Lazy is good. When you learn how to make products that allow people to be more lazy, you will become successful.
    - intended 16d ago
      What results though. The people seeing measurable improvements to their core work with LLMs are coders. Everyone else is taking over intern level work from someone else’s team. They are reducing the friction costs of talking to someone else, for about a 30% productivity gain. Firms are trying desperately to automate their white collar workers, and that is following the same trend as all other automation projects, and ML/deep learning efforts in history.
- 17d ago

deleted 17d ago

[deleted]

sscaryterry 17d ago

FPGA's are definitely not new. They've been mainstream for 20 odd years+. See https://www.eetimes.com/fpga-market-to-pass-2-7-billion-by-10-in-stat-says/?_ga https://www.eetimes.com/fpga-market-to-pass-2-7-billion-by-1...

slrainka 16d ago

That's the thing. LLM's can be used as a real-time teaching layer and build systems that can operate at deterministic speeds and escape hatch into a LLM when confidence levels drop. This approach has helped us save more than $1 Million annually against the straight LLM classifiers at scale (and pass that on to the customers) https://sureshsubasinghe.substack.com/p/how-to-cut-agent-llm-costs-by-85 https://sureshsubasinghe.substack.com/p/how-to-cut-agent-llm... https://sureshsubasinghe.substack.com/p/the-god-model-fallacy-how-to-double https://sureshsubasinghe.substack.com/p/the-god-model-fallac...