radarsat1

6 ms·

31.
My Calculator Is a Transformer(sinclairs.gitlab.io)
4 points by radarsat1 6mo ago|1 comments
32.
by radarsat1 6mo ago
Recently I got interested in how to "compile" a program definition into the weights of a Transformer. I settled for distilling the MLPs individually, but the attention weights are fully "calculated". The example program
33.
by radarsat1 6mo ago
you gave me the idea of using it to explore weird random scifi ideas, ended up spending way too much time clicking through details about the role of astrophage in the development of intelligence in deep sea life. Fun!
34.
by radarsat1 6mo ago
Wow, love this idea.
35.
by radarsat1 6mo ago
That was a fascinating read! Thanks for the link.
36.
by radarsat1 6mo ago
This reminds me a lot of the tricks to turn BERT into a generative model. I guess the causal masking that keeps it to essentially be autoregressive is an important difference though. Kind of best of both worlds.
37.
by radarsat1 6mo ago
Because the nature of transformers is that running a bunch of pregenerated tokens through them is a parallel operation, not autoregressive. That's how it works at training time, but speculative decoding uses it at inference time. So if
38.
by radarsat1 7mo ago
Consider also that even reattaching nerves that are supposed to be there is not exactly a walk in the park. Look into finger reattachment surgery and post operation care. Think pain, tingling, a year or more of physiotherapy.. and that&#x
39.
by radarsat1 7mo ago
> "Redefine" is a favorite word of AI. Honestly no need to read further. You're not wrong, but it certainly is an annoying outcome of AI that we're not allowed to use.. words.. anymore.
40.
by radarsat1 7mo ago
> Should you be counting on confusion of an underpowered text-merge to catch such problems? This does not really follow from my statement. I said that underpowered text merge should not silently accept such situations, not that it is t
41.
by radarsat1 7mo ago
Is it a good thing to have merges that never fail? Often a merge failure indicates a semantic conflict, not just "two changes in the same place". You want to be aware of and forced to manually deal with such cases. I assume the pr
42.
by radarsat1 7mo ago
Something I want to add to the discussion is that the only time I've encountered this was not with a specific company but with an "AI recruitment agency", which I'm seeing getting more and more popular. And while I get t
43.
by radarsat1 7mo ago
> There are no details about training my understanding was that they are not training at all, which would explain that. they are compiling an interpreter down to a VM that has the shape of a transformer. ie they are calculating the trans
44.
by radarsat1 7mo ago
> Is it speed? > Is it that you can backprop through this computation? Do you do so? With respect, I feel that you may not have read the article. > Because the execution trace is part of the forward pass, the whole process remains
45.
by radarsat1 7mo ago
I'm curious if 1-bit params can be compared to 4- or 8-bit params. I imagine that 100B is equivalent to something like a 30B model? I guess only evals can say. Still, being able to run a 30B model at good speed on a CPU would be amazin
46.
by radarsat1 7mo ago
it also reminds me a bit of this diffusion paper [1] which proposes having an encoding layer and a decoding layer but repeats the middle layers until a fixed point is reached. but really there is a whole field of "deep equilibrium mode
47.
by radarsat1 7mo ago
This is interesting because I've been considering a similar project. I maintain a package for a scientific simulation codebase, it's all in Fortran and C++ with too much template code, which takes ages to build and is very error
48.
by radarsat1 8mo ago
I think your last point raises the following question: how would you change your answer if you know they read all about guns and death and how one causes the other? What if they'd seen pictures of guns? And pictures of victims of guns
49.
by radarsat1 8mo ago
> The hardest part was figuring out OpenLDAPs configuration syntax, especially the correct ldif incantations .. As a long time Linux user on personal machines, I found myself for the first time a couple of years ago needing to support a
50.
by radarsat1 8mo ago
> and the quality of the result can be measured automatically this part is nontrivial though
51.
by radarsat1 8mo ago
I didn't say that 2.5gb is unreasonable for an LLM. I said it's an unreasonable payload size for a website. Not the same.
52.
by radarsat1 8mo ago
It sounds good, but I'm not sure that in practice sites will want to "let go" of control this way, knowing that some random model can be used. Usually sites with chatbots want a lot of control over the model behaviour, and
53.
by radarsat1 8mo ago
Ah fair enough. But then it seems the bot completely ignored the discussion in question, there's a reason they spent time evaluating and discussing it instead of just making the change. Having a bot push on the issue that the humans ar
54.
by radarsat1 8mo ago
> I opened PR #31132 to address issue #31130 — a straightforward performance optimization replacing np.column_stack() with np.vstack().T(). > The technical facts: - np.column_stack([x, y]): 20.63 µs - np.vstack([x, y]).T: 13.18 µs - 3
55.
by radarsat1 8mo ago
It's cool but do I really want a single browser tab downloading 2.5 GB of data and then just leaving it to be ephemerally deleted? I know the internet is fast now and disk space is cheap but I have trouble bringing myself around to thi
56.
by radarsat1 8mo ago
> The ai tooling reverses this where the thinking is outsourced to the machine and the user is borderline nothing more than a spectator, an observer and a rubber stamp on top. I find it a bit rare that this is the case though. Usually I
57.
by radarsat1 8mo ago
Additionally it's not like you're constrained to write it in bash. You could use Python or any other language. The author talks about how you're now redeveloping a shitty CI system with no tests? Well, add some tests for it!
58.
by radarsat1 9mo ago
> The "dump" on their end was to use this as marketing bait and a way to inflate their valuation. Maybe a bit different but I think it's worth pointing out how this parallels the state of the job market right now. It is so
59.
by radarsat1 9mo ago
Thanks, really interesting to think about these trade-offs.
60.
by radarsat1 9mo ago
> Experts can be swapped in and out of VRAM for each token. I've often wondered how much it happens in practice. What does the per-token distribution of expert selection actually look like during inference? For example does it act l

More ›