EGreg

14 ms·

271.
by EGreg 6mo ago
Since yesterday, I put up the source code btw: https://github.com/Safebots/KV
272.
by EGreg 6mo ago
I would add this: https://magarshak.com/blog/perfection-is-the-enemy-of-the-go...
273.
by EGreg 6mo ago
the k forward passes for different candidate tokens share all their prefix computation -- the KV cache up to position i-1 is identical for all candidates, so you run one pass through the shared layers and then k cheap single-token extension
274.
by EGreg 6mo ago
The predicted KV vector is the expected KV vector under the model's distribution over next tokens, i.e. a weighted average over the vocabulary, not an actual sampled token. So no forward pass with a sampled token is involved. Yes, the
275.
by EGreg 6mo ago
"This holds for almost all matrices" is actually something you'd want to know if we're talking about probabilities, no?
276.
by EGreg 6mo ago
Well yeah, I just wrote it as a lemma, but it's basically close to tautological. Its only job is to formally ground the entropy argument that follows it. The interesting claim is what comes after: because KV vectors are deterministic f
277.
by EGreg 6mo ago
The prediction being used is the model's prediction of the next token's KV vector, given all previous KV vectors. Because the model was trained on language, it has strong priors about what comes next. The residual, i.e the differe
278.
by EGreg 6mo ago
No, the 914,000x in the paper is talking about the ratio between two entropy floors, it's not a claim about practical compression. The point is that per-vector quantization has been chasing the wrong theoretical limit: the sequential e
279.
by EGreg 6mo ago
You're right, I'm not a well-known researcher, simply an entrepreneur who started to publish academic papers. However, I do have a long history of diving deep into fields and building practical, open-source solutions to major prob
280.
by EGreg 6mo ago
Haha, yes I always used -- when I typed an em-dash manually. What bot detection extensions? :-P
281.
by EGreg 6mo ago
Author here. Since starting to teach AI at IENYC, I started publishing my papers recently on arXiv, and considering submitting them to a journal. This is based on my original "PLT" paper: Probablistic Language Tries ( https:/
282.
KV Cache Compression 900000x Beyond TurboQuant and Per-Vector Shannon Limit(arxiv.org)
46 points by EGreg 6mo ago|42 comments
283.
by EGreg 6mo ago
Is war2.ru next? I'm glad Blizzard doesn't mess with servers of its older games. Warcraft 2 was such a classic! Even more than Starcraft. The original granddaddy that people play 25 years later. That, and Myth 2 TFL was my favorit
284.
by EGreg 6mo ago
Actually, prolly trees are probably best for intersections. You can use bloom filters as a first pass
285.
by EGreg 6mo ago
Can Cloudflare do an SMS service? That would be something :)
286.
by EGreg 6mo ago
This just proves that we should stop using old environments and operating systems for mission-critical work, and build a completely new environment from the ground up, that's secure by default. Instead of trying to fix leaky buckets.
287.
by EGreg 6mo ago
They're just a thin layer to be replaced last. They're just arrogant enough to think they're the company, but ultimately the endgame is -- all humans become economically insignificant compared to the automated economy. https
288.
by EGreg 6mo ago
Under GDPR, can someone send an email to people who 1) downloaded our app, 2) voluntarily entered their email, viewing a message that says if they do that, we can send them a newsletter about how to be more efficient with the app, and 3) se
289.
by EGreg 6mo ago
You don’t need to be a crypto expert, blockchain is just to avoid the double-spend problem for the currency that is needed in the ecosystem. If you want everything to be free, you don’t need it. If you want everything to be centralized, you
290.
by EGreg 6mo ago
To quote your own words: > So here's where I'm a hard-ass and maybe even worse than google's rules. If I see the RFC8058 unsubscribe link, it is too late. I only notice that link after I've decided to mark your email
291.
by EGreg 6mo ago
So basically — you are affirming the point of the OP whose article was shared. Wait too long — respect people’s attention and time so much that you don’t send them anything unless it is ready and benefits them - and apparently it’s spam whe
292.
by EGreg 6mo ago
Is there an actual regulation or case law showing what the cutoff time is du jure? I would be glad to respect it if there was. As it is, laws do allow for things they didn’t explicitly prohibit, and especially good-faith things like welcomi
293.
by EGreg 6mo ago
So by your own description, ANYONE sending you a newsletter, by complying with Google’s rules, they piss you off and make you mark their email as SPAM because, according to you, they made “javascript execute on your computer”. Actually, gma
294.
by EGreg 6mo ago
People originally opted in and provided it expecting to get a newsletter on how to use the app. We never seemed to have the bandwidth to create a good enough one, so we never sent it. We kept improving the app until it became very good and
295.
by EGreg 6mo ago
That one-time announcement is called an email. And therefore that first announcement itself can be flagged as spam. And naturally, unless they click a link in the first email, gmail should consider anything subsequent to be spam anyway. The
296.
by EGreg 6mo ago
No, the consent was given, recorded, and never acted upon. It had no expiration date. Many of those people are still using the app today.
297.
by EGreg 6mo ago
I have done the opposite We had a million people enter their email over the last decade We haven’t messaged a single one. Now we plan to start sending out a newsletter. For many, they may have forgotten downloading the app, but they might s
298.
by EGreg 6mo ago
True, it's not so much a dictatorship, more like just a certain type of mindset. Shared by Trump, Vance (I guess), Elon, and the AfD party, Le Pen (maybe) and more. What do we call this mindset? Right wing authoritarianism? Anti-immig
299.
Probabilistic Language Tries: Unified Framework for Compression and AI Execution(arxiv.org)
2 points by EGreg 6mo ago|0 comments
300.
by EGreg 6mo ago
Just for everyone to make up their own minds based on the data, here are multiple studies: 2025: https://www.sciencedirect.com/science/article/pii/S126236362... 2023: https://pmc.ncbi.nlm.nih.gov&#

More ›