gyrovagueGeist

5 ms·

1.
by gyrovagueGeist 6mo ago
Electricity (on continental US) is pretty cheap assuming you already have the hardware: Running at a full load of 1000W for every second of the year, for a model that produces 100 tps at 16 cents per kWh, is $1200 USD. The same amount of to
2.
by gyrovagueGeist 7mo ago
Not wrong, but I think it's more accurate to say: Mamba is an architecture for the middle layers of the network (the trunk) which assumes decoding takes place through an autoregressive sequence (popping out tokens in order). This is th
3.
by gyrovagueGeist 7mo ago
This is just standard Fourier theory of being able to apply dense global convolutions with pointwise operations in frequency space? There’s no mystery here. It’s no different than a more general learnable parameterization of “Efficient Cha
4.
by gyrovagueGeist 8mo ago
They also didn't have nuclear weapons to use in global conflicts over resources.
5.
by gyrovagueGeist 9mo ago
I've always found curriculum learning incredibly hard to tune and calibrate reliably (even more so than many other RL approaches!). Reward scales and horizon lengths may vary across tasks with different difficulty, effectively explorin
6.
by gyrovagueGeist 11mo ago
Thief of Time by Terry Prachett has a great minor bit about characters who are naming themselves after colors running out of human made labels, as they have to get increasingly esoteric with the names. It's fun to see that visualized.
7.
by gyrovagueGeist 1y ago
It's from the depth of the computation, not the work
8.
by gyrovagueGeist 1y ago
In the middle term, I almost feel less productive using modern GPT-5/Claude Sonnet 4 for software dev than prior models, precisely because they are more hands off and less supervised. Because they generate so much code, that often pass
9.
by gyrovagueGeist 1y ago
Useful and interesting but likely still dangerous in production without connecting to formal verification tools. I know o3 is far from state of the art these days but it's great at finding relevant literature and suggesting inequalitie
10.
by gyrovagueGeist 1y ago
For people who are interested Kokkos (a C++ library for writing portable kernels) also has a naming scheme for hierarchical parallelism. They use ThreadTeam, Thread (for individual threads within a group), and ThreadVector (for per thread S
11.
by gyrovagueGeist 1y ago
I can't reproduce this, possibly it's another environment error or the problem has been fixed in the versions I am using. (uv 0.7.11, zsh 5.9, python 3.13.5, MacOS 15.3.1)
12.
by gyrovagueGeist 1y ago
It does? What problems have you had?
13.
by gyrovagueGeist 1y ago
Yep! Optimize (solve in infinite dimensions) and then discretize onto a finite basis has typically led to much better and stable methods than a discretize and then optimize approach. Time-scale calculus is a pretty niche theoretical field t
14.
by gyrovagueGeist 1y ago
Does anyone know why they added minibatch advantage normalization (or when it can be useful)? The paper they cite "What matters in on-policy RL" claims it does not lead to much difference on their suite of test problems, and (mean
15.
by gyrovagueGeist 1y ago
I am not sure why HN has mostly LANL posts. Otherwise though it is a combination of things. Machine learning applications for NATSec & fundamental research have become more important (see FASST, proposed last year), the current politic
16.
by gyrovagueGeist 1y ago
All the problems of reproducibility in Python notebooks ( https://arxiv.org/abs/2308.07333 , https://leomurta.github.io/papers/pimentel2019a.pdf ) with the power of a terminal.
17.
by gyrovagueGeist 1y ago
How many people still play centaur chess?
18.
by gyrovagueGeist 2y ago
ah, yes, the spooky similarities of hilbert spaces and probability theory
19.
by gyrovagueGeist 2y ago
They likely mean on any of the current era of NISQ-like devices ( https://en.wikipedia.org/wiki/Noisy_intermediate-scale_quant... ) like this one or quantum annealers.
20.
by gyrovagueGeist 2y ago
Yep, this is also the problem that self-driving cars have with the "accidents per mile" metric.
21.
by gyrovagueGeist 2y ago
I love this paper, but something I think is often missed when it comes up is that you CAN hear the shape of many drums if you restrict the shape space, for example with a prior of "what a drum should look like" Zelditch proved sp
22.
by gyrovagueGeist 2y ago
This problem seems like it would be very similar to the Low-Rank + Sparse decompositions that used to be popular in audio-visual filtering.
23.
by gyrovagueGeist 2y ago
These were two separate and mostly unrelated PEPs but otherwise that's correct.
24.
by gyrovagueGeist 2y ago
This is neat and links to some great papers. I wish the comparison was with OpenMP tasks though; I’ve heard Rayon has a reputation for being a bit slow
25.
by gyrovagueGeist 2y ago
Typically no, although BLAS software engineers occasionally write HPC strassen's implementations and papers about them. In my opinion there's a few reasons why they're not common in BLAS libraries: - They're a bit less n
26.
by gyrovagueGeist 2y ago
- "In this world, Elwood, you must be oh so smart or oh so pleasant." Well, for years I was smart. I recommend pleasant"
27.
by gyrovagueGeist 2y ago
+1 for this. I use rga all the time. it's a "simple" solution but often enough for what I actually needed.
28.
by gyrovagueGeist 2y ago
They genuinely do not sound that similar to me
29.
by gyrovagueGeist 2y ago
The interconnect and network topology is also a big component of the hardware where you can't "fake The Real Thing" in practice. You can often get fairly confident in program correctness for toy problem runs by scaling 1-~40
30.
by gyrovagueGeist 3y ago
It's been commonly used as a noun in the HPC / scientific computing space for a long time (relevant to this thread) typically talking about "compute bound" and "memory bound" algorithms. It's spilled over

More ›