sdenton4

7 ms·

91.
by sdenton4 6mo ago
The randomness (and exploration) encouraged by batch training also helps avoid 'real' minima, if they exist.
92.
by sdenton4 6mo ago
https://www.neh.gov/article/1921-tulsa-massacre "After an all-night battle on the Frisco Tracks, many residents of Greenwood were taken by surprise as bullets ripped through the walls of their homes in the predawn
93.
by sdenton4 6mo ago
I think the first two paragraphs of the post are exactly saying that the bottleneck is memory... Long contexts, bigger but less flop-intensive models (moe's). The funny thing about scaling laws is that as soon as they were known, the w
94.
by sdenton4 6mo ago
/me looks in the direction of Washington dc ... But what else is new?
95.
by sdenton4 6mo ago
Hey, I also read that book, and came to basically the opposite conclusion! The point of the book is that we've been very bad at testing animal intelligence because of a vast stack of human biases, including things like language and the
96.
by sdenton4 6mo ago
Political power is the bottleneck for most shit that matters, not computational power. Most of the stuff that sucks on the us sucks because of entrenched institutions with perverse interests (health insurers, tax filing companies) and congr
97.
by sdenton4 6mo ago
It's the next step removed from the tablet based ordering that has taken over in restaurants. Like those tablets, it won't be everywhere, but its easy to imagine it being ubiquitous, especially in chain stores.
98.
by sdenton4 6mo ago
Block auto regressive generation can give you big speedups. Consider that outputting two tokens at a time will be a (2-epsilon)x speedup over running one token at a time. As your block size increases, you quickly get to fast enough that it
99.
by sdenton4 6mo ago
You're not even wrong, as they say... I'm tempted to add 'seeing like a state' to your reading list. "Understandable in market terms" doesn't mean the thing is actually understood, and in fact may be dange
100.
by sdenton4 6mo ago
The court is stacked with so called originalists - history stopped in the eighteenth century.
101.
by sdenton4 6mo ago
Grow, Scott, grow!
102.
by sdenton4 6mo ago
Yah, I think the question really comes down to: what's so special about a block chain vs any other public record or database? For example, it's just as easy - and more accessible - to put the code and example configurations with o
103.
by sdenton4 6mo ago
When you resolve bottlenecks, new bottlenecks become apparent. Right now, it's looking like assessment and evaluation are massive bottlenecks.
104.
by sdenton4 6mo ago
It's far more sane to review a complete PR than to verify every small change. They are like dicey new interns - do you want to look over their shoulder all day, or review their code after they've had time to do some meaningful qua
105.
by sdenton4 7mo ago
That's funny. In my study it was 70%. Nah, make that 85%.
106.
by sdenton4 7mo ago
Or maybe press the timer and let it float...
107.
by sdenton4 7mo ago
Sure, it might try to subtly steer you towards fascism, but other than that, it's great.
108.
by sdenton4 7mo ago
It indicates that there's a good chance that they have trained on the test set, making the eval scores useless. Even if you have given up on the dream of generalization entirely, you can't meaningfully compare models which have tr
109.
by sdenton4 7mo ago
Doing great on public datasets and underperforming on private benchmarks is not a good look.
110.
by sdenton4 7mo ago
It's the statistics equivalent of 'no one needs more than 640kb of RAM'
111.
by sdenton4 7mo ago
And that energy costs money, both at the training/cgi stage and at the inference/consumption stage. It's not even an externality. CGI renders do use a lot of electricity relative to playing back the movie for individual viewe
112.
by sdenton4 7mo ago
Idk, kernel anti cheat is a pretty clear sign to me that I should pick a different game to play anyway...
113.
by sdenton4 7mo ago
Right idea, but the application is incorrect. Model training is similar to the creation of the cgi for the movie. Both happen before anyone consumes the output, and represent the up front cost for the producer. Both a movie and a language m
114.
by sdenton4 7mo ago
What are ai and robots other than excess labor, waiting to be allocated? Why does the wealth have to come from the meat bags?
115.
by sdenton4 7mo ago
Yah, I'm a bit skeptical - ime humans tend to under explore due to incorrect assumptions. Often this is due to forming a narrative to explain some result, and then over attaching to it. Also, agents aren't actually good at reasoni
116.
by sdenton4 7mo ago
For raw hyperparameter search, though, I would expect a proper Bayesian framework to be much better. Eg, vizier.
117.
by sdenton4 7mo ago
The gist of these things is you point them at an eval metric and say 'make it go better.' so, you can point it at anything you can measure. The example in the blog post here is bonding boxes on wood cut images.
118.
by sdenton4 7mo ago
On the one hand, a search engine is not heroin... It's a pretty broken analogy. On the other hand, we could probably convince Cory Doctorow to write a piece about how fentanyl is really about the enshitification of opiates.
119.
by sdenton4 7mo ago
However, D_A is moving, while D_B can be stationary.
120.
by sdenton4 7mo ago
Harris was the vice president, and was therefore the closest thing to a small-d democratic choice amongst the available options. Otherwise... Why Harris and not Newsom? The better choice would have been Biden stepping out earlier and having

More ›