Veedrac

78 ms·

841.
by Veedrac 7y ago
You basically have the option between SRAM, HBM (DRAM), and something new. You can imagine the risks with using new memory tech on a chip like this. The issue with HBM is that it's much slower, much more power hungry (per access, not p
842.
by Veedrac 7y ago
That's what the absurdly fast interconnect is for. You send the data to where the weights are.
843.
by Veedrac 7y ago
Indeed, but cores are only responsible for small fragments of the network, so don't need huge amounts of memory.
844.
by Veedrac 7y ago
Low-Cost 3D Chip Stacking with ThruChip Wireless by Dave Ditzel https://www.youtube.com/watch?v=S-hBSddgGY0 Fascinating idea, no idea about the practicalities.
845.
by Veedrac 7y ago
Brains run on computation. Computation is fungible. The highest bandwidth signal in the brain is through neuron spiking, and there isn't much space for other mechanisms to hide. Take a simple 10 virtual neurons:1 synapse ratio as a rou
846.
by Veedrac 7y ago
There are 100 trillion synapses in the brain. I'm not claiming Cerebras is human scale; I think it's a factor of 100-1000 off personally. I'm just saying I don't think the comparison is meaningless.
847.
by Veedrac 7y ago
> A few buckets of sand also contain more elements than there are synapses in the human brain. Actually you'd need about a thousand buckets. https://www.quora.com/How-many-grains-of-sand-could-fit-in-a... --- Ultima
848.
by Veedrac 7y ago
Most of the chip is already SRAM, I'm not really sure what else you would expect? 18 GiB × 6 transistors/bit ≈ .93 trillion transistors
849.
by Veedrac 7y ago
“full utilization at any batch size, including batch size 1” https://www.cerebras.net/
850.
by Veedrac 7y ago
The idea is that the whole model resides in the fast memory, so you don't need to ‘keep it fed’.
851.
by Veedrac 7y ago
18GB is huge! An NVIDIA V100 has 6MB of L2 memory. HBM is off-chip, and vastly (~100x) slower.
852.
by Veedrac 7y ago
> For example, the typical 300mm wafer from TSMC may contain “a modest hundred number of flaws,” said Feldman. Cerebras gave its Swarm interconnect redundant links to route around defective tiles and allocated “a little over 1% [of the t
853.
by Veedrac 7y ago
I think what Moore's law is depends on who you ask. Density makes sense and seems to be holding true, but so does cost/transistor, which hasn't.
854.
by Veedrac 7y ago
Major changes will include the introduction of more 3D packaging, which among others probably means much larger caches. I wouldn't be shocked if last level caches reached, say, half a gigabyte or more, if things go well, though it coul
855.
by Veedrac 7y ago
That looks like a pretty straight line to me, TBH. You're doing more subdivision than is fair to the data. I did my own graph of just density, which is also pretty clearly straight. https://docs.google.com/spreadsheets&
856.
by Veedrac 7y ago
This just seems like pondering, not anything concrete. I doubt there's much to expand on.
857.
by Veedrac 7y ago
For people actually interested in the topic rather than painfully bad pop-sci reenactments, you should read the simulation paper[1], or, for a less technical introduction, watch the video of Bostrom talking about it[2]. [1] https:/&#x
858.
by Veedrac 7y ago
I gave a straightforward argument: it keeps these complexities away from people who don't understand them as much as reasonably possible, while still leaving them within trivial reach when they are necessary. If you don't like the
859.
by Veedrac 7y ago
The point is that the most common thing people do when clicking the URL is typing something else in the search bar, and the second most is copying the URL. In both cases, having the URL change is a problem, because this includes people who
860.
by Veedrac 7y ago
http/https is already given by the lock/“Not secure” icon FWIW.
861.
by Veedrac 7y ago
Simple UX: don't move things around more than necessary. I honestly don't understand why people think this second click is unreasonable; the power-user shortcut (ctrl-l) already expands immediately anyway.
862.
by Veedrac 7y ago
I would not be surprised if there are many applications where removing 90% of the weight and bulk of the lens, or adding this much flexibility in the lens' functionality, would more than make up for having to double the sensor size or
863.
by Veedrac 7y ago
The press release has more info: https://engineering.columbia.edu/press-releases/nanfang-yu-f... I'm impressed at the images these produce.
864.
by Veedrac 7y ago
Worth noting that even the lowest power microcontroller will presumably run circles around an old 8-bit architecture.
865.
by Veedrac 7y ago
Have you considered the impact of superconducting circuitry? They're very niche and few people have heard of them, but they're not science fiction[1] and have a significant speed advantage. [1] https://www.hypres.com&#x
866.
by Veedrac 7y ago
Would you be less worried if the graph was still going up? If it was already falling?
867.
by Veedrac 7y ago
We'd only likely see huge consequences like that if we went outside the normal range of human variation. I agree that evolution would find single gene swaps with huge effects, but most of the variation available to us is over hundreds
868.
by Veedrac 7y ago
---
869.
by Veedrac 7y ago
This isn't a safe bet at all; we're the first arrivals, it would be a shock if we did everything right the first time.
870.
by Veedrac 7y ago
It's this core idea that compromise is equivalence that I was arguing against. You're comparing the fish with the birds, but you're forgetting the other fish that got infected with some specific disease before going extinct.

More ›