trott

7 ms·

31.
by trott 2y ago
The plot was just showing where the solid lines were trending (see prior messages), and that happened to predict the performance at 400k samples (red dot) very well. An exponential scaling curve would steer a bit more to the right, but it w
32.
by trott 2y ago
> But if someone has already hit the human-level threshold There is some controversy over what the human-level threshold is. A recent and very extensive study measured just 60.2% using Amazon Mechanical Turkers, for the same setup [1]. B
33.
by trott 2y ago
> The probability of not hitting bullseye at least once ... I added a clarification.
34.
by trott 2y ago
> Maybe he figured out a model that beats ARC-AGI by 85%? People have, I think. One of the published approaches (BARC) uses GPT-4o to generate a lot more training data. The approach is scaling really well so far [1], and whether you ex
35.
The surprising effectiveness of test-time training for abstract reasoning [pdf](mit.edu)
87 points by trott 2y ago|27 comments
36.
by trott 2y ago
> Turing machine or simple virtual machine in the layers of a neural network There's the Neural Turing Machine and the Differentiable Neural Computer, among others.
37.
by trott 2y ago
> Marcus has always been a mouth just trying to take down neural networks. This isn't true. Marcus is against "pure NN" AI, especially in situations where reliability is desired, as would be the case with AGI/ASI. He
38.
by trott 2y ago
And the problems are much harder now.
39.
AI Mathematical Olympiad – Progress Prize 2(kaggle.com)
80 points by trott 2y ago|6 comments
40.
by trott 2y ago
You originally wrote: > very, very, few drugs are "novel" as opposed to being analogues of something naturally in the body But "analog" means "structural analog" in this context (see https://en.wi
41.
by trott 2y ago
https://en.wikipedia.org/wiki/Functional_analog_(chemistry) explains the difference between structural and functional analogs: fentanyl is quite dissimilar from morphine, but binds the same targets.
42.
by trott 2y ago
> It's not a weird coincidence that helps ML; it's inherent in the problem. This depends on the application. If you are trying to design new proteins for something, unconstrained by evolution, you may want a method that does we
43.
by trott 2y ago
> There are few approaches that will accelerate the field of drug development and chemistry as a whole in a way that the works of these three people will. As the author of one such approach, I'm skeptical. AlphaFold 2 just predicts
44.
by trott 2y ago
> The big-O here is irrelevant for the architectures since it's all in the configuration & implementation of the model; i.e. there is no relevant asymptote to compare. ?! NNs are like any other algorithm in this regard. Heck, lo
45.
by trott 2y ago
> Transformers actually have an quantifiable state size Are you griping about my writing O(X^2) above instead of precisely 2X^2, like this paper? The latter implies the former. > So a sufficiently sized RNN could have the same state c
46.
by trott 2y ago
> Simplistic thinking. An RNN hidden parameter space of high dimension provides plenty of room for linear projections of token histories. I think people just do not realize just how huge R^N can be. 16N bits as hard limit, but more reali
47.
by trott 2y ago
> This is no different than a transformer, which, after all, is bound by a finite state, just organized in a different manner. It's not just a matter of organizing things differently. Suppose your network dimension and sequence leng
48.
by trott 2y ago
People did something similar to what you are describing 10 years ago: https://arxiv.org/abs/1409.0473 But it's trained on translations, rather than the whole Internet.
49.
by trott 2y ago
My feeling is that the answer is "no", in the sense that these RNNs wouldn't be able to universally replace Transformers in LLMs, even though they might be good enough in some cases and beat them in others. Here's why. A
50.
by trott 2y ago
TLDR: Physicians scored 73.7. Physicians armed with GPT-4 scored 76.3. But GPT-4 alone scored 89.2. The authors think it's unlikely that the materials are in the GPT-4 training data, because the cases have never been publicly released.
51.
LLMs and Diagnostic Reasoning: A Randomized Clinical Vignette Study [pdf](medrxiv.org)
3 points by trott 2y ago|3 comments
52.
O1 Test-Time Compute Scaling Laws(github.com)
1 points by trott 2y ago|0 comments
53.
OpenAI Pitched White House on 5-7 5GW datacenters(bloomberg.com)
3 points by trott 2y ago|0 comments
54.
What It's Like to Solve a Math Olympiad Problem(amistrongeryet.substack.com)
2 points by trott 2y ago|0 comments
55.
by trott 2y ago
I'm the author of AutoDock Vina (the most cited docking program, and the "runner-up" in the AlphaFold 3 paper) Docking software is used to scan millions and billions of drug-like molecules looking for new potential binders.
56.
by trott 2y ago
> Google was already sold on the idea, having concluded that its Rust developers are twice as productive as its C++ engineers. Not so. The actual study compared the cost of writing something in C++ to the cost of porting it to Rust.
57.
by trott 2y ago
> I don't think that ability to acquire knowledge from other people is our most most characteristic trait. Creativity is. Chimps (and some other animals) can be very creative: https://www.youtube.com/watch?v=fPz6uvIb
58.
Learning a new language with ChatGPT Advanced Voice Mode [video](youtube.com)
2 points by trott 2y ago|1 comments
59.
by trott 2y ago
> "AI" every 5 words That's not peak AI yet. Wait until "AI" is a verb (meaning to apply an AI to, or to ask an AI about) and an adjective. https://en.wikipedia.org/wiki/Buffalo_buffalo_Buffa
60.
From sci-fi to state law: California's plan to prevent AI catastrophe(arstechnica.com)
4 points by trott 2y ago|0 comments

More ›