yorwba

20 ms·

451.
by yorwba 7mo ago
The root, not the leaves.
452.
by yorwba 7mo ago
Any other benchmark at that sample size would have similarly huge error bars. Unless Anthropic makes a model that works 100% of the time or writes a bug that brings it all the way to zero, it's going to work sometimes and fail sometime
453.
by yorwba 7mo ago
Real world usage is unlikely to give you the large sample sizes needed to reliably detect the differences between models. Standard error scales as the inverse square root of sample size, so even a difference as large as 10 percentage points
454.
by yorwba 7mo ago
The Mapterhorn data for the area does have some obvious artifacts: https://mapterhorn.com/viewer/#map=16.15/37.806698/-122.2773...
455.
by yorwba 7mo ago
More trainable parameters, more data, higher quality data.
456.
by yorwba 7mo ago
Multi-Head Latent attention is a redesigned attention mechanism that produces lower-dimensional KV-cache entries. Vector quantization can store KV-cache entries using a small number of bits per dimension while ensuring that the resulting at
457.
by yorwba 7mo ago
It's feasible to put the expert routing logic in a previous layer. People have done it: https://arxiv.org/abs/2507.20984
458.
by yorwba 7mo ago
It's called "suppressive fire": https://en.wikipedia.org/wiki/Suppressive_fire " The primary intended effect of suppressive fire is psychological. Rather than directly trying to kill enemy soldiers,
459.
by yorwba 7mo ago
CPI tracks a weighted average of a large basket of different goods, of which eggs are only a small part. It would be extremely surprising if the change in egg prices over time closely matched CPI.
460.
by yorwba 7mo ago
Those are not sources for the statement you were asked to back up with a source.
461.
by yorwba 7mo ago
Their latency measurements comparing Mamba-2 and Mamba-3 are done with a batch size of 128. It doesn't seem like Mamba-2 was compute-bound even at that batch size.
462.
by yorwba 7mo ago
The figures for students graduating primary school (18.57 million) and entering junior high school (18.49 million) match up quite well, though. Do you think primary schools and junior high schools manage to coordinate massive student number
463.
by yorwba 7mo ago
Of ≈17 million Chinese students who graduated junior high school after grade 9 in 2024, ≈10 million were admitted to a high school, ≈4 million to a vocational school and the remaining ≈3 million disappear from education statistics, presumab
464.
by yorwba 8mo ago
Related: Discussion on the initial NanoGPT Slowrun announcement: https://news.ycombinator.com/item?id=47251259 (185 points 15 days ago, 39 comments)
465.
by yorwba 8mo ago
It's an indication that it's one of the many bot accounts currently doing the same thing https://hn.algolia.com/?query=this%20is%20more%20nuanced%20t... So the reason the comment appears weirdly disconnected from
466.
by yorwba 8mo ago
After reading both the original post and this submission, what do you think is new here?
467.
by yorwba 8mo ago
If you have showdead on, you can see that this account posts generic oneliners: https://news.ycombinator.com/threads?id=balinha_8864
468.
by yorwba 8mo ago
Previous discussion: https://news.ycombinator.com/item?id=47385935 (439 points 3 days ago, 388 comments)
469.
by yorwba 8mo ago
It's just setting the font-family in the style attribute of a <span>. (As you can see by inspecting the text/html content of your clipboard, e.g. with `xclip -selection clipboard -o -t text/html`)
470.
by yorwba 8mo ago
Well, the weights are accumulated in full precision and are multiplied by a full-precision scale factor after quantization, and the activations and backward pass are computed in full precision as well, so it's not quite true 4-bit prec
471.
Reasons to be pessimistic (and optimistic) on the future of biosecurity(owlposting.com)
3 points by yorwba 8mo ago|0 comments
472.
by yorwba 8mo ago
Iran does have an oil pipeline bypassing the Strait of Hormuz https://www.worldoil.com/news/2021/5/31/iran-opens-new-oil-e... but it doesn't have infinite capacity, so that doesn't mean there a
473.
by yorwba 8mo ago
Nice work! The paper feels verbose at times and could use some editing to slim it down (also, equation 6 is just equation 5 in a box) but I enjoyed it a lot nonetheless.
474.
by yorwba 8mo ago
There are a few pictures of truncated icosahedra in the article, alongside several other shapes that are not icosahedra. The point is that they have icosahedral symmetry. The L is important.
475.
by yorwba 8mo ago
Flowering plants (angiosperms) appeared during the Cretaceous before dinosaurs got wiped out, and there is fossil evidence of insects pollinating non-flowering plants (gymnosperms) like ferns and confers even earlier than that: https:&#x
476.
by yorwba 8mo ago
It makes it easier to compare with other papers. If two different papers apply different methods to different models and get different results, how do you know which method is better? Once you have identified the best method and want to pro
477.
by yorwba 8mo ago
It's part of the official origin story that was published alongside the introduction of the script, so students will learn about it at some point long after they're already fluent readers and writers, and then promptly forget abou
478.
by yorwba 8mo ago
For x = -i, square(x) = -1, sqrt(square(x)) = i. Meanwhile, abs(x) = 1. You're right that it simplifies to abs(x) for real x, but that no longer holds for arbitrary complex values.
479.
by yorwba 8mo ago
Treating attention as a lookup operation is popular among computational complexity theorists (e.g. https://arxiv.org/abs/2310.03817 ) because it's easier to work with when you're explicitly constructing a tra
480.
by yorwba 8mo ago
The approach here is very bad for training though, because unlike softmax attention, average-hard attention is not differentiable with respect to the keys and queries, and if you try to fix that e.g. with straight-through estimation, the ba

More ›