yvdriess

28 ms·

331.
by yvdriess 7y ago
Seconding the question. More to the point, a language that only needs libc and not a separate runtime. It's a big boon that existing C projects can migrate one object file at a time to betterC, without needing to pull in extra dependen
332.
by yvdriess 7y ago
That makes the assumption that any refcounted object lifetime management can be replaced with manual management, which is not the case. As the paper states: it is not possible to manually determine the object lifetimes due to the nature of
333.
by yvdriess 7y ago
Untrue. Large memory nodes are par the course for genomics workloads. Only a few stages of the analysis pipeline can stream the in/out effectively. Even then, going to disk for the result only to be read back is going to bottleneck you
334.
by yvdriess 7y ago
+1 for the first point. Every major compiler can do the lambda-lifting transformation, either because of C++ lambda or OpenMP support. It's frustrating doing this manually while knowing the compiler supports it internally, but does not
335.
by yvdriess 7y ago
Correct, in GPUs can indeed do a better job at hiding latency through massive parallelism. My expertise might be outdated here, but the problem used to be that actually getting to that max bandwidth through divergent warps and uncoalesced r
336.
by yvdriess 7y ago
Surely you mean dense one-hot? Technically, the sparse one-hot encoding is the most efficient in terms of memory footprint. You simply store the non-zero coordinates. The problem in practice for GPUs is that sparse vector/matrix oper
337.
by yvdriess 7y ago
So you agree that the problem is fundamentally sparse? Embeddings are used to make sparse (e.g. categorical) data possible on GPUs, and real-world models are limited by how large they can make the embeddings to fit in GPU memory. Embedding
338.
by yvdriess 7y ago
The reverse is true, embeddings are both the performance and memory-footprint bottleneck of modern NN models. Check figure 6. of : https://arxiv.org/pdf/1906.00091.pdf Embeddings are used to lookup sparse features, so
339.
by yvdriess 7y ago
A slightly more elaborate answer than the sibling post to drive home how much happens on a simple read that is not cached : - request to L1D cache, misses - request to L2D cache, misses - packet is dropped on the mesh network to access L3D,
340.
by yvdriess 7y ago
Yes, a very good point. I am assuming the tables are quite large due to the workload. If it's large enough to give a benefit to large pages in reduced DTLB misses, it's likely too large for warp-local memory :)
341.
by yvdriess 7y ago
Since when is calculating the hash the bottleneck in hash table access? The bottleneck is the: ptr = h(x); // trivial, ~0.5-2 cycles bucket = *ptr; // --> ~200-300 cycles edit: reading the paper, it'
342.
by yvdriess 7y ago
No, their approach changes the fundamental access pattern into something anathema to GPU and TPU architectures. In ELI5 or layman's terms: current GPU/TPU accelerators are specialized in doing very regular and predictable calculat
343.
by yvdriess 7y ago
No, that is definitely not the case, not all memory-bounds are of the same type. Hash tables produce lots of data-dependent random accesses into DRAM which are definitely not better on GPUs. Warps divergence, bank conflicts, partial cache-l
344.
by yvdriess 7y ago
The quote in question: Have you heard the tunes by Rob Hubbard, Martin Galway, Tim Follin, Jeroen Tel, and all the other composers ? I'm afraid not, are recordings available in the US? I don't really know why, but th
345.
by yvdriess 7y ago
Yes, most have a variant where you can dump the image, maybe with a tree-shake even: http://www.sbcl.org/manual/#Saving-a-Core-Image
346.
by yvdriess 7y ago
Why are you sticking to desktop processors with that kind of workloads?
347.
by yvdriess 7y ago
Stuff that is really really really small behaves in weird and cool ways. We can sort of tell how it works, but it's really hard to do anything with it because it is so small. These smart people found a new way of working with really
348.
by yvdriess 7y ago
There is no language spec for Python, the CPython implementation defines the language.
349.
by yvdriess 7y ago
Before I forget: Henson's Labyrinth and The dark crystal are good examples. Especially with the latter inspiring a new generating to produce a modern version for TV.
350.
by yvdriess 7y ago
To give an Asian->West example: plenty of Manga and Anime are more popular in the west than in Japan.
351.
by yvdriess 7y ago
My personal theory is that much of asia did (or does) not have the disney-style cultural directive that children stories have to end happily. Old european children stories pre-Grimm definitely did not have that either. A well done sad momen
352.
by yvdriess 7y ago
D -betterC
353.
by yvdriess 7y ago
Nope, you're at the mercy of caches to keep the beast fed.
354.
by yvdriess 7y ago
Offloading to the GPU has a significant cost that needs to be amortized. DNNs work well because very little is moved over the PCIe bottleneck, and inputs can be buffered. Take modern compression, let's say H265, has a complex control f
355.
by yvdriess 7y ago
A modern chip design-to-production pipeline takes roughly five years, 2 years if you crunch it and dot try anything fancy. And yes moving from TSMC/GoFo production nodes to Intel would require significant redesign.
356.
by yvdriess 7y ago
There is this halo around function programming languages being trivial to parallelize, but how they are implemented really matters. Historically, only few functional languages have been implemented with parallelism in mind (SISAL, pH). The
357.
by yvdriess 7y ago
One of the strongest elements in that scene regarding the concept of 'identity' is completely missed by that video: Kusanagi briefly sees someone in an office building that looks exactly like her. It's a great nod to the boo
358.
by yvdriess 7y ago
A lot of people working during the PC revolution in the 80s that had a formal education would have worked on languages with exactly that kind of tech: various Lisp dialects, ML, Smalltalk. Same for all kinds of meta-programming, JIT or dyna
359.
by yvdriess 7y ago
(I also deal with sparse graph data in my dayjob.) A 1-step traversal is the same complexity for both a linear algebra and vertex- or edge-centered representation. However, you do absolutely need a sparse linear algebra representation and o
360.
by yvdriess 7y ago
Indeed, you can run a lot of new fridges in the developing world on the kWh budget of a heated private swimming pool.

More ›