4 ms·

Edit: it seems like this is likely one chip and not 10. I assumed 8B 16bit quant with 4K or more context. This made me think that they must have chained multipl

Edit: it seems like this is likely one chip and not 10. I assumed 8B 16bit quant with 4K or more context. This made me think that they must have chained multiple chips together since N6 850mm2 chip would only yield 3GB of SRAM max. Instead, they seem to have etched llama 8B q3 with 1k context instead which would indeed fit the chip size. This requires 10 chips for an 8 billion q3 param model. 2.4kW. 10 reticle sized chips on TSMC N6. Basically 10x Nvidia H100 GPUs. Model is etched onto the silicon chip. So can’t change anything about the model after the chip has been designed and manufactured. Interesting design for niche applications. What is a task that is extremely high value, only require a small model intelligence, require tremendous speed, is ok to run on a cloud due to power requirements, AND will be used for years without change since the model is etched into silicon?

teaearlgraycold 9mo ago
I'm thinking the best end result would come from custom-built models. An 8 billion parameter generalized model will run really quickly while not being particularly good at anything. But the same parameter count dedicated to parsing emails, RAG summarization, or some other specialized task could be more than good enough while also running at crazy speeds.
danpalmer 9mo ago
Alternatively, you could run far more RAG and thinking to integrate recent knowledge, I would imagine models designed for this putting less emphasis on world knowledge and more on agentic search.
- freeone3000 9mo ago
  Maybe; models with more embedded associations are also better at search. (Intuitively, this tracks; a model with no world knowledge has no awareness of synonyms or relations (a pure markov model), so the more knowledge a model has, the better it can search.) It’s not clear if it’s possible to build such a model, since there doesn’t seem to be a scaling cliff.
pjc50 9mo ago
Where are those numbers from? It's not immediately clear to me that you can distribute one model across chips with this design. > Model is etched onto the silicon chip. So can’t change anything about the model after the chip has been designed and manufactured. Subtle detail here: the fastest turnaround that one could reasonably expect on that process is about six months. This might eventually be useful, but at the moment it seems like the model churn is huge and people insist you use this week's model for best results.
- aurareturn 9mo ago
  > The first generation HC1 chip is implemented in the 6 nanometer N6 process from TSMC. Each HC1 chip has 53 billion transistors on the package, most of it very likely for ROM and SRAM memory. The HC1 card burns about 200 watts, says Bajic, and a two-socket X86 server with ten HC1 cards in it runs 2,500 watts. https://www.nextplatform.com/2026/02/19/taalas-etches-ai-models-onto-transistors-to-rocket-boost-inference/ https://www.nextplatform.com/2026/02/19/taalas-etches-ai-mod...
  - dakolli 9mo ago
    So it lights money on fire extra fast, AI focused VCs are going to really love it then!!
  - darkwater 9mo ago
    And what of that makes you assume that having a server with 10 HC1 cards is needed to run a single model version on that server?
- adityashankar 9mo ago
  This depends on how much better the models will get from now in, if Claude Opus 4.6 was transformed into one of these chips and ran at a hypothetical 17k tokens/second, I'm sure that would be astounding, this depends on how much better claude Opus 5 would be compared to the current generation
  - aurareturn 9mo ago
Shaanveer 9mo ago
ceo
- charcircuit 9mo ago
  No one would never give such a weak model that much power over a company.
thrance 9mo ago
> What is a task that is extremely high value, only require a small model intelligence, require tremendous speed, is ok to run on a cloud due to power requirements, AND will be used for years without change since the model is etched into silicon? Video game NPCs?
- aurareturn 9mo ago
  Doesn’t pass the high value and require tremendous speed tests.
  - thrance 9mo ago
    Video games are a huge market, and speed and cost of current models are definitely huge barriers to integrating LLMs in video games.
  - mike_hearn 9mo ago
    Speed = capacity = cost.
deleted 9mo ago
[deleted]
machiaweliczny 9mo ago
A lot of NLP tasks could benefit from this
tyushk 9mo ago
Data tagging? 20k tok/s is at the point where I'd consider running an LLM on data from a column of a database, and these <=100 token problems provide the least chance of hallucination or stupidity.

teaearlgraycold 9mo ago

I'm thinking the best end result would come from custom-built models. An 8 billion parameter generalized model will run really quickly while not being particularly good at anything. But the same parameter count dedicated to parsing emails, RAG summarization, or some other specialized task could be more than good enough while also running at crazy speeds.

danpalmer 9mo ago

Alternatively, you could run far more RAG and thinking to integrate recent knowledge, I would imagine models designed for this putting less emphasis on world knowledge and more on agentic search.

freeone3000 9mo ago
Maybe; models with more embedded associations are also better at search. (Intuitively, this tracks; a model with no world knowledge has no awareness of synonyms or relations (a pure markov model), so the more knowledge a model has, the better it can search.) It’s not clear if it’s possible to build such a model, since there doesn’t seem to be a scaling cliff.

pjc50 9mo ago

Where are those numbers from? It's not immediately clear to me that you can distribute one model across chips with this design. > Model is etched onto the silicon chip. So can’t change anything about the model after the chip has been designed and manufactured. Subtle detail here: the fastest turnaround that one could reasonably expect on that process is about six months. This might eventually be useful, but at the moment it seems like the model churn is huge and people insist you use this week's model for best results.

aurareturn 9mo ago
> The first generation HC1 chip is implemented in the 6 nanometer N6 process from TSMC. Each HC1 chip has 53 billion transistors on the package, most of it very likely for ROM and SRAM memory. The HC1 card burns about 200 watts, says Bajic, and a two-socket X86 server with ten HC1 cards in it runs 2,500 watts. https://www.nextplatform.com/2026/02/19/taalas-etches-ai-models-onto-transistors-to-rocket-boost-inference/ https://www.nextplatform.com/2026/02/19/taalas-etches-ai-mod...
- dakolli 9mo ago
  So it lights money on fire extra fast, AI focused VCs are going to really love it then!!
- darkwater 9mo ago
  And what of that makes you assume that having a server with 10 HC1 cards is needed to run a single model version on that server?
adityashankar 9mo ago
This depends on how much better the models will get from now in, if Claude Opus 4.6 was transformed into one of these chips and ran at a hypothetical 17k tokens/second, I'm sure that would be astounding, this depends on how much better claude Opus 5 would be compared to the current generation
- aurareturn 9mo ago

Shaanveer 9mo ago

ceo

charcircuit 9mo ago
No one would never give such a weak model that much power over a company.

thrance 9mo ago

> What is a task that is extremely high value, only require a small model intelligence, require tremendous speed, is ok to run on a cloud due to power requirements, AND will be used for years without change since the model is etched into silicon? Video game NPCs?

aurareturn 9mo ago
Doesn’t pass the high value and require tremendous speed tests.
- thrance 9mo ago
  Video games are a huge market, and speed and cost of current models are definitely huge barriers to integrating LLMs in video games.
- mike_hearn 9mo ago
  Speed = capacity = cost.

deleted 9mo ago

[deleted]

machiaweliczny 9mo ago

A lot of NLP tasks could benefit from this

tyushk 9mo ago

Data tagging? 20k tok/s is at the point where I'd consider running an LLM on data from a column of a database, and these <=100 token problems provide the least chance of hallucination or stupidity.