ssivark

27 ms·

361.
Plenty of room at the top [how software can drive computer performance] [pdf](gwern.net)
3 points by ssivark 3y ago|0 comments
362.
by ssivark 3y ago
Suppose you are looking at a 52 card deck, and members of each of the four “shapes” self-identify (with some random noise, and maybe even systematic deviations — like sevens and aces are identified differently from just their shape, etc) as
363.
by ssivark 3y ago
> Game theory rises to the level of Solomon. Has it ever failed? There’s a reason we study the Prisoner’s dilemma and its failure modes. With regards to the cake example more specifically, what makes it somewhat trivial is that the state
364.
by ssivark 3y ago
What’s your take on the argument that fine tuning might be destroying model calibration and causing it to overfit to aspects of the finetuning dataset eg: causing it to appear more confident, rather than becoming more correct? See https:&#
365.
by ssivark 3y ago
> All the efforts to spot such an interaction seem to be based purely on wishing that it is not true, so that fancy expensive detectors can defensibly be promoted and somebody might get a prize. That's incredibly short-sighted. Ther
366.
by ssivark 3y ago
I found the paper easier to follow than the slides: https://www.maths.ed.ac.uk/~tl/mdiss.pdf (Less emphasis of the category theory, and more attention to the basic math behind entropy-like diversity measures) (TLDR) Th
367.
by ssivark 3y ago
Maybe actually a good thing in the long run — would be great for an application aiming for a federated architecture to be reasonably robust to things of this sort.
368.
by ssivark 3y ago
Loved the post. Just to add to the principal observation about the vocab point — One thing I’ve noticed is that experts who can communicate/teach well are very effective at getting you to the vocab point (to the extent possible witho
369.
by ssivark 3y ago
Can you, for example, subscribe to /r/linux on “all” Lemmy servers? Likewise, can you have a reasonable expectation that others would do the same — so it doesn’t matter which server you post your questions on?
370.
by ssivark 3y ago
The world is a very heterogeneous place, and we're all immersed in our own bubbles -- each with its own assumptions and needs. "Data related" is a fairly vague description. - It appears that there's substantial community
371.
by ssivark 3y ago
Thanks for the AMA. - Any comments/references on the potential applications where the Syndicated Actor Model is going to be a particularly good fit? (I'm curious, for example, about the potential in robotics) - Are you targeting a
372.
by ssivark 3y ago
There's a little more nuance: 1. Technically, the table you shared is better thought of as a two-dimensional tensor, rather than a "graph-like matrix" -- which as you point out must be a linear map from a (vector) space to it
373.
by ssivark 3y ago
If that were the case, I’d be extremely wary of their scientific output.
374.
by ssivark 3y ago
From that perspective, going through any kind of struggle (and building willpower/discipline) would be a good enough proxy for learning calculus. It’s a little sad that the best value that the OP can impute for learning calculus is m
375.
by ssivark 3y ago
> use it in prod? I think that’s the wrong goal. Langchain, with its kitchen-sink approach is great for newbies playing around with language models, to get a feel for the different things possible. The uniform interfaces make it very eas
376.
by ssivark 3y ago
Ah, I see. There’s XLA.jl for TPUs and a bunch of stuff from the JuliaGPU organization for GPUs — I wonder how they compare with TF/PyTorch/JAX. I’m curious to get perspectives on what remains to be done to solve the missing piece
377.
by ssivark 3y ago
> There's no TF / PyTorch / JAX replacement What do you mean by that? AFAICT for the most part Julia is already like the best that Jax could ever hope to be — since there is language level support for JIT compilation, grad
378.
by ssivark 3y ago
What even is a “known broken” technique — how do you establish that some technique “can’t work” rather than “just haven’t found a good execution” so far? Of course, there’s risk involved in straying off the beaten path and trying new things
379.
by ssivark 3y ago
It’s taking about “little languages” (DSLs), somewhat in the spirit of what one thinks of as libraries/APIs (on a continuum, all the way to something with its own syntax… Like regular expressions). https://chreke.com/li
380.
by ssivark 3y ago
The crassness in the terminology feels... somewhat appropriate for the phenomenon. The experience is an extremely unpleasant, and it would be nice for users to recognize it as such -- so they avoid repeating the cycle of network effects
381.
by ssivark 3y ago
> "the sooner you can get to the real work of creating something that the market itself will validate — or not." I think the "or not" (that you've skipped) is crucial. In my reading, it emphasizes the irony tha
382.
by ssivark 3y ago
Since the whole discussion (post + comments here) is a little abstract, I wonder whether this short comic sketch might serve to exemplify through caricature what we’re trying to get at: “The expert” https://youtu.be/BKorP55A
383.
by ssivark 3y ago
My guess is that any product/brand that wishes to grow beyond a small niche market has to do so by agglomerating features I.e. trying to swallow multiple niches. The push towards economies of scale (and “growth”) makes it really hard f
384.
by ssivark 3y ago
> Python is the visually most elegant language. Having program login buried in a nauseating amount of boilerplate doesn’t feel very “visually elegant” :-P
385.
by ssivark 3y ago
I hate the fact that if I choose Numba, then I can’t easily compose with other tools like autodiff, etc. Maybe Jax improves the situation in some aspects. Generally though… While Python might have many distinct popular/well-developed t
386.
by ssivark 3y ago
I've run into some pretty straight-forward problems (eg: modeling physical systems, with linear algebra and some other basic math) where avoiding python for loops meant allocating memory up front for massive numpy, and then doing multi
387.
by ssivark 3y ago
Right — that’s certainly surprising & intriguing. So they discuss this in section 4, and offer a theoretical argument why the rate of convergence might be independent of the (large) number of parameters. I haven’t grokked that yet, but
388.
by ssivark 3y ago
You evaluate the function at two neighboring points, and the difference of function values (divided by the distance between the two points) gives you an estimate of the gradient projected along the direction between the two points. It’s lit
389.
by ssivark 3y ago
Some context, for those who might be getting carried away: The essential idea here is estimating gradients using numerical (forward pass) function evaluation. The reason it’s called “zeroth order” is that it’s a much much worse approximatio
390.
by ssivark 3y ago
Did you try using “substitution servers”? They supply pre-built artifacts.

More ›