bufo

14 ms·

31.
by bufo 3y ago
The neural engine has severe limitations at the moment. I tried using it for BERT about a year ago and kept crashing its API because of "out of memory" issues. The theoretical TOPs you mention also don't necessarily translate
32.
by bufo 3y ago
The memory bandwidth is still a bit lower than Nvidia's best cards, and it doesn't have the equivalent of Tensor Cores. If they wanted they could compete, but it's clearly not their desire. They build consumer end products.
33.
by bufo 3y ago
Deep learning work when optimizing inference.
34.
by bufo 3y ago
I confirm!
35.
by bufo 3y ago
Correct, but I don’t see Firefox or Chrome bothering to add tensor core support.
36.
by bufo 3y ago
It can be converted, and can be fairly fast, see Tensorflow.js or https://jott.live/markdown/m1_webgpu_perf . However the any “web” standard will always be a subset of what the hardware can truly provide because it has
37.
by bufo 3y ago
Yeah I think no one will likely have any edge for a simple thing like a matrix multiplication since all the right abstractions are supported in both languages and they end up in the LLVM code gen. Having Python 3 backwards compatibility and
38.
by bufo 3y ago
It's a superset of Python. What you call "Python" is actually CPython, a C implementation of the Python language. Many other Python implementations exist, though they typically don't support all packages well (for exampl
39.
by bufo 3y ago
This article is so misleading. First there’s nothing secret about all this. Also these chat bots don’t just “sound” smart, they are providing value to millions of users.
40.
by bufo 3y ago
I somewhat agree with this, note that Janet is not very functional, when I briefly used it I never used a single recursive function. It's meant to be used more in an imperative way.
41.
by bufo 3y ago
No, there are waaaay better models available these days like Flan (UL2 and the older T5), or Cerebras-GPT.
42.
by bufo 4y ago
This is like saying Fisher Price toy cars are equivalent to F1 cars.
43.
by bufo 4y ago
Perfectly put
44.
by bufo 4y ago
Rust already has runtime CPU feature detection. But that would require hand writing the SIMD code.
45.
by bufo 4y ago
There’s a good leaderboard for embedding focused models here: https://huggingface.co/spaces/mteb/leaderboard . Some of the sentence-transformers models are even smaller than gr-t5 while still having very close perf
46.
SolidGoldMagikarp (Plus, Prompt Generation)(lesswrong.com)
6 points by bufo 4y ago|1 comments
47.
by bufo 4y ago
"There are possible improvements still to be made on bigger buffers for example, where we could make better use of SIMD, but at the moment rustc still targets baseline x86-64 CPUs (SSE2) so that's a work item left for the future.&
48.
by bufo 4y ago
Ah ok I might run some tests on my RTX A5000 and report back!
49.
by bufo 4y ago
Have you tried torch.compile on the model with Torch 2.0? It should help a bunch!
50.
by bufo 4y ago
Came here to say this! Same for me.
51.
by bufo 4y ago
The BF16 and VNNI instructions are finally going to make AMD competitive for neural network inference.
52.
by bufo 4y ago
The iOS / Apple OCR Swift API is drastically better than the ones I’ve tried online (eg. Microsoft) or the open source ones (Tesseract). Highly recommended. You can get fairly high throughput with M1 chips. The CNN is accelerated by th
53.
by bufo 5y ago
Thanks I'll take a look!
54.
by bufo 5y ago
What are your favorite new developments in tooling? I haven’t used Haskell since 2014.
55.
OpenFold, trainable Pytorch port of AlphaFold 2(github.com)
3 points by bufo 5y ago|1 comments
56.
by bufo 6y ago
I recommend looking at the khash implementation https://github.com/attractivechaos/klib/blob/master/khash.h It's very competitive to optimized implementations like abseil and much, much better than
57.
by bufo 6y ago
Those Pixel 5 measurements are impressive if it’s a stock Android app.
58.
by bufo 6y ago
Thank you!
59.
by bufo 6y ago
How do your flutter and Android apps compare by the way?
60.
Visual C++ 1.0 Launch event(youtube.com)
1 points by bufo 6y ago|0 comments

More ›