pavpanchekha

4 ms·

1.
by pavpanchekha 14d ago
Last author here. We've talked to the Chrome folks and they are interested, but it's difficult work. Chrome is big enough that emitting different sequences is hard and would requiring changing a lot of internal abstractions. Skia
2.
by pavpanchekha 14d ago
I am a big fan of DB-style thinking, very much on the same wavelength as you :)
3.
by pavpanchekha 14d ago
Let me also add that nanobench was a huge help, not just because it was a good benchmarking tool but also because it gave us some confidence that we're measuring the right thing. It's easy to make _something_ faster but hard to kn
4.
by pavpanchekha 14d ago
Thank you! The SkRecord system was _perfect_ for doing these optimizations. I don't think it would have been possible to do this project without it.
5.
by pavpanchekha 14d ago
Last author here, this is very much what I've worked on for most of my career. In this project, I had the idea of optimizing rendering instructions years ago, while I was writing https://browser.engineering/ , but the h
6.
by pavpanchekha 14d ago
Hi folks! Last author here, happy to answer questions, very surprised to see this on HN. We had a blast working on this. Let me add that the Skia team at Google was super supportive, met with us many times to explain a lot of stuff. I had t
7.
by pavpanchekha 2mo ago
I do a substantial amount of coding in Racket, including maintaining the Herbie numerical compiler ( https://herbie.uwplse.org/ ) over the last decade. Racket is great! The runtime is reasonably fast, and the standard library
8.
by pavpanchekha 2mo ago
It's about chips with a large enough scale up domain. Larger domain allows for bigger model, which is what's driving this jump. You've got to get the chips, test them, tune kernels, then start a big pre train, mid & post-
9.
by pavpanchekha 2mo ago
A lot of algorithmic improvement in AI is ultimately bottlenecked by compute. It is very easy to come up with ideas that could improve models! But to prove that they do, especially at scale, is expensive and takes a long time .
10.
by pavpanchekha 2mo ago
Making Luna, which was already very cheap and extremely capable, 5x cheaper is crazy. I use Sol at work but Luna at home, and while there's definitely a difference, it doesn't feel like night-and-day. After a year of ever-increasi
11.
by pavpanchekha 3mo ago
For compiler work I found that Sol is noticably better than 5.5 (and I generally use OAI models because I like the Codex app), but Fable was still obviously better.
12.
by pavpanchekha 4mo ago
OpenAI used to make Codex-specific models, but they stopped. What I've gathered from interviews and similar is that training two models isn't worth the (small) lift from having a coding-specific model. You're pre-training on
13.
Can LLMs accelerate science? An experiment(pavpanchekha.com)
2 points by pavpanchekha 6mo ago|1 comments
14.
by pavpanchekha 6mo ago
University of Washington Programming Languages and Software Engineering (research group). I'm not at UW any more, I'm now at Utah, but some of the Herbie team is at UW and they provide the infrastructure
15.
by pavpanchekha 6mo ago
Documented here but yes it's an average, of something similar to but not exactly the same as relative error: https://herbie.uwplse.org/doc/latest/error.html It's true that averages can be misleading but
16.
by pavpanchekha 6mo ago
Author here. The speed up is modeled throughput, though the model is relatively naive. It's possible to disable branches by turning off the regimes flag, see https://herbie.uwplse.org/doc/1.0/options.html
17.
by pavpanchekha 6mo ago
Author here. I've got a few papers about this problem (including one in submission), but it is very very hard to do, especially with acceptable overhead. The state of the art is maybe 100x overhead.
18.
by pavpanchekha 6mo ago
It is, there's a page in the documentation about how errors are defined. Let me also add: Herbie generally gives the most accurate option it found first, and then the other stuff might be useful for speed (0.5x is way faster than two s
19.
by pavpanchekha 6mo ago
Author here! Yes, the float distribution isn't what you want in practice, but distribution selector isn't really the right thing either, because a low probability bad result can still be pretty bad! Hence the range selector; the f
20.
by pavpanchekha 6mo ago
It was me. Damn it you're right! Will fix!
21.
by pavpanchekha 6mo ago
In calculus the core issue is that the concept of a "function" was undefined but generally understood to be something like what we'd call today an "expression" in a programming language. So, for example, "x^2 +
22.
by pavpanchekha 7mo ago
They're cheap but not free, especially at the front end of the CPU where it's just a lot more instructions to churn through. What the branch predictor gets you is it turns branches, which would normally cause a pipeline bubble, to
23.
by pavpanchekha 7mo ago
Horner's form is typically also more accurate, or at least, it is not bit-identical, so the compiler won't do it unless you pass -funsafe-math, and maybe not even then.
24.
by pavpanchekha 7mo ago
Deterministic output is incompatible with batching, which in turn is critical to high utilization on GPUs, which in turn is necessary to keep costs low.
25.
The Token Production System(pavpanchekha.com)
1 points by pavpanchekha 8mo ago|0 comments
26.
by pavpanchekha 8mo ago
Frontier models are now much bigger than an individual query, hence batching, MoE, etc. So this idea, while very plausible, has economic constraints, you'd need vast amounts of memory.
27.
by pavpanchekha 10mo ago
That pretty much is how CSS works! At the most basic level, Flow level is about widths down, heights up. But this basic model doesn't let you do a lot of things some people want to do, like distributing left-over space in a container
28.
by pavpanchekha 10mo ago
Author here—it is from Tufte CSS. I have a blog post [1] about how floats work. It is a nice example of there being unintuitive and also more-intuitive ways to achieve things in CSS. These days I believe CSS Anchor Positioning provides a si
29.
by pavpanchekha 10mo ago
Author here. You're right that a lot of CSS's edge cases and implicit rules stem from other choices and implicit rules that maybe need to be reconsidered. But take this logic a step further. The way text with mixed font sizes is l
30.
by pavpanchekha 10mo ago
Author here. I suppose it depends on what "rely on" means, but... have you ever used CSS to center text? Did you think much at all about what happens if the zoom level is high enough and the screen size small enough that the text

More ›