terafo

12 ms·

91.
by terafo 4y ago
You mean in contained app? It can already run on a phone. GPU acceleration would be nice at this point, though.
92.
by terafo 4y ago
No(llama.cpp is cpu-only) and no(you need to requantize the model).
93.
by terafo 4y ago
In-app purchases
94.
by terafo 4y ago
Dennard scaling died 2 decades ago. Moore's law is dead for less than a dexade. It took 4 years for density to double(from N7 to N3). And with density doubled the cost. It will get only worse from here.
95.
by terafo 4y ago
Closer to 4 months. It is much better than having a bunch of 2-4gb models laying around.
96.
by terafo 4y ago
It looks like LLaMa is undertrained, especially 65B(that's why it looses on a couple of important benchmarks to 30B).
97.
by terafo 4y ago
Model weights aren't deterministic due to FP math. But I think your argument still stands.
98.
by terafo 4y ago
Mill CPU never existed in silicon, so there are questions to that. But IIRC Ivan said that they didn't do anything that compilers didn't do already for DPSs. He specifically mentioned that they don't want to be like Itanic.
99.
by terafo 4y ago
It might, but in this specific instance some kind of fine-tuning would be better.
100.
by terafo 4y ago
You don't need to store it in weights. It would be too expensive to constantly update model with new data. It wouldn't be able to contain everything anyways. LLaMa 30B with access to 100TB of storage(you can store whole libgen plu
101.
by terafo 4y ago
But did he try to build AI? As far as I know MIRI is pure-play safety research lab.
102.
by terafo 4y ago
It fits, whisper.cpp uses 4 bit quantization, 13B model takes a little bit more than 8gb and around 9gb ram while inferencing.
103.
by terafo 4y ago
128 gigs is more than enough to load 13B model into. Pytorch has M1 support for some time now so CUDA isn't required.
104.
by terafo 4y ago
As far as I can tell you can do it right now, at least for small 13B model, not sure about bigger models.
105.
by terafo 4y ago
Basically all high-end CPU's in the last 20 years(almost 30 actually, it started with Pentium Pro in 1995) have small internal instruction set to which all of the instructions that are transmitted to CPU are converted. Part that conver
106.
by terafo 4y ago
Lithography isn't the only way to construct transistor, it is possible to to construct much smaller transistors using alternative tech. Not in scale, though
107.
by terafo 4y ago
They are negotiating DUV ban. EUV is banned for years now.
108.
by terafo 4y ago
It isn't that uncommon in countries with relatively high rate of inflation. For example in Ukraine government set some strict monetary controls and it's not really possible to get USD in cash form from official exchanges.
109.
by terafo 4y ago
Wonder how long we have until they make chiplet without L3$ and rely only upon 3D V-Cache. That would allow to make chiplet either roughly 2x smaller or pack twice as much cores.
110.
by terafo 4y ago
Why do you assume that compilation only happens in CI/CD?
111.
by terafo 4y ago
Isn't compilation mostly linker constrained in realistic scenarios? I mean first compilation time is quite long, but then it's often linker who is to blame for most of the compile time.
112.
by terafo 4y ago
Initial concept for mistborn was to show epic fantasy world live through different stages of technological progress. Initially he planned to do three trilogies, one pure epic fantasy, one set in early computer age and one in space age, but
113.
by terafo 4y ago
It will eventually be continued, just like Warbreaker. And it's not like it is forgotten, there are at least two characters from Sel in Lost Metal(his latest mistborn book).
114.
by terafo 4y ago
Metal raytraycing is a thing and it's the officially supported API. I think AMD just didn't bother to write the driver since their GPUs are present much viewed as legacy by Apple.
115.
by terafo 4y ago
Let's put it into perspective. A11 had 3x less bandwidth, 5x less flops, node that's a lot worse(since then we had N7, N7P, N6, N5 and it's many variations). Such big leaps are problematic in terms of project management. Just
116.
by terafo 4y ago
Not if we are talking game consoles. Sony and Microsoft sell them at a loss and then recoup that loss with a cut from game sales.
117.
by terafo 4y ago
Glad to see Tim Dodd selected. He deserved it. Wondering about timelines, though. 2023 seems unrealistic.
118.
by terafo 4y ago
I don't think it is possible to not complain about ANY modern development system a single time in a year.
119.
by terafo 4y ago
> Yeah, not Provide some evidence. I just ran tokei on freshly cloned tinygrad repo using arguments from[1]. Got 4854 lines of code, which is less than 5k lines. [1] tokei --exclude *.json --exclude accel/cherry --exclude test --
120.
by terafo 4y ago
Jax preallocates 90% of available GPU memory when first operation is run to minimize allocation overhead. Can PyTorch grab that VRAM for a similar reason?

More ›