3 ms·

Imagine the size of chip needed to 'etch' something like Qwen 3.6 27B in size.

by walrus01 3mo ago

flog 3mo ago
If someone has that sort of knowledge; how big a chip would be required? Is it possible?
- mdp2021 3mo ago
  Well, given the data above, roughly a 220b transistors chip for the HC1 tech.
mdp2021 3mo ago
Not too dissimilar to the first HC1 (6nm 815mm² 53B Transistors embedding an 8b LLM): > Our second model, still based on Taalas’ first-generation silicon platform (HC1), will be a mid-sized reasoning LLM
golem14 3mo ago
Interesting thought, because it's a yield question. How tolerant are models today to a few broken weights. If tolerant, they could churn out many cheaper chips, some perhaps with slight abnormal tendencies ;)
- walrus01 3mo ago
  I wonder if you had a few percent of problems in the yield, if it would be functionally equivalent to the difference between a unsloth-published Q6 standard size GGUF vs. the nearly perfect precision of an unsloth Q8-K-XL. Or more like Q4 vs Q8 where a lot is lost.
- thepasch 3mo ago
  > How tolerant are models today to a few broken weights. Extremely! You can remove entire layers and the model will still work just fine, with barely perceptible capability losses. I've cut/bypassed ~15% of total parameters out of Gemma 4 31B on a pod once. Still got perfectly coherent responses out of it. Certain layers are a lot more important than others, particularly early and late ones; but it's honestly astonishing how much can be cut out from the middle without destroying the model's coherence. I didn't run any meaningful benchmarks, so I have no idea what the capability loss looks like exactly. But "produce coherent and sensible English in response to a wide variety of prompts" was definitely not among the things the model unlearned.
  - walrus01 3mo ago
    Brings to mind the scene in '2001' where Bowman is pulling out individual pieces of hardware that represent the mind of HAL, and it becomes increasingly incoherent as more physical hardware is detached. https://www.youtube.com/watch?v=UwCFY6pmaYY https://www.youtube.com/watch?v=UwCFY6pmaYY

Imagine the size of chip needed to 'etch' something like Qwen 3.6 27B in size.

flog 3mo ago

If someone has that sort of knowledge; how big a chip would be required? Is it possible?

mdp2021 3mo ago
Well, given the data above, roughly a 220b transistors chip for the HC1 tech.

mdp2021 3mo ago

Not too dissimilar to the first HC1 (6nm 815mm² 53B Transistors embedding an 8b LLM): > Our second model, still based on Taalas’ first-generation silicon platform (HC1), will be a mid-sized reasoning LLM

golem14 3mo ago

Interesting thought, because it's a yield question. How tolerant are models today to a few broken weights. If tolerant, they could churn out many cheaper chips, some perhaps with slight abnormal tendencies ;)

walrus01 3mo ago
I wonder if you had a few percent of problems in the yield, if it would be functionally equivalent to the difference between a unsloth-published Q6 standard size GGUF vs. the nearly perfect precision of an unsloth Q8-K-XL. Or more like Q4 vs Q8 where a lot is lost.
thepasch 3mo ago
> How tolerant are models today to a few broken weights. Extremely! You can remove entire layers and the model will still work just fine, with barely perceptible capability losses. I've cut/bypassed ~15% of total parameters out of Gemma 4 31B on a pod once. Still got perfectly coherent responses out of it. Certain layers are a lot more important than others, particularly early and late ones; but it's honestly astonishing how much can be cut out from the middle without destroying the model's coherence. I didn't run any meaningful benchmarks, so I have no idea what the capability loss looks like exactly. But "produce coherent and sensible English in response to a wide variety of prompts" was definitely not among the things the model unlearned.
- walrus01 3mo ago
  Brings to mind the scene in '2001' where Bowman is pulling out individual pieces of hardware that represent the mind of HAL, and it becomes increasingly incoherent as more physical hardware is detached. https://www.youtube.com/watch?v=UwCFY6pmaYY https://www.youtube.com/watch?v=UwCFY6pmaYY