mxwsn

9 ms·

61.
by mxwsn 3y ago
(2020)
62.
by mxwsn 3y ago
Closely related, but not a duplicate. This post explicitly calls out services and products that will continue to be supported, like world's, musical collaborations like KDA, the Arcane anime, and upcoming Project L.
63.
by mxwsn 3y ago
Whoa hold on, table 1 of stopping distances is calculated for 2.5 second reaction time which is 10x longer than conventional human reaction time. Then the stopping distances are compared to radar/lidar/camera to argue that AVs can
64.
by mxwsn 3y ago
https://archive.is/KUSO6
65.
by mxwsn 3y ago
Flagged for misleading title - the four strong rejects are from a single author. It's listed four times for some unknown reason but likely an openreview quirk. The actual status described by the page is: 2 unknown (with accompanying lo
66.
by mxwsn 3y ago
It's an island area called billionaire bunker. The article does not say that he's living in an actual underground bunker.
67.
by mxwsn 3y ago
Twitter thread from Albert Gu, a primary author: https://nitter.net/_albertgu/status/1731727672286294400?s=20 "Quadratic attention has been indispensable for information-dense modalities such as language... u
68.
My Techno-Optimism(vitalik.ca)
3 points by mxwsn 3y ago|0 comments
69.
by mxwsn 3y ago
I agree this is likely disastrous for OpenAI's current corporate structure, and Microsoft will be very unhappy. But as an ML researcher, I suspect lots of researchers are going to continue to follow Ilya, unless his reputation has sign
70.
by mxwsn 3y ago
I first read about understanding LLMs as simulators generating simulacra a while ago through this post: https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators
71.
by mxwsn 3y ago
They're just defining grokking in a different way. It's reasonable to me though - grokking suggests elements of intuitive understanding, and a sudden, large increase in understanding. These mirror what happens to the loss.
72.
by mxwsn 3y ago
Link is NSFW
73.
by mxwsn 3y ago
Speculation: a very low probability token was sampled, shifting the context to unfamiliar area that increased the probability of further unlikely tokens. This can be seen by multiple strange punctuation and capitalisation that don't ob
74.
by mxwsn 3y ago
The example figure shows a key hit every half second, which suggests a pecking style of typing at around 24 wpm. This way the model gets very clean waveforms. I wonder how their approach would work with average or fast typists. The sound pr
75.
by mxwsn 3y ago
Nah. Speaking from experience as a scientist, drama increases with the magnitude of impact. This is far from any confirmation that the result is real, but it's quite consistent with the authors believing it's real.
76.
by mxwsn 3y ago
Related: ICML 2023 Orals. https://icml.cc/virtual/2023/events/oral
77.
by mxwsn 3y ago
This is an important study and I've been waiting for something like this ever since Alpaca and the following wave of imitating models that have had lackluster, non rigorous evaluation.
78.
by mxwsn 3y ago
Yeah, RETRO [0] embeds all an entire question/prompt, and searches for similar text passages with k-NN, then does further processing. This can kind of be understood as attention on paragraphs. This preprint instead does k-NN and calls
79.
by mxwsn 3y ago
https://archive.is/FjDbo
80.
by mxwsn 3y ago
1. This is not exact attention, but an approximation of it. Specifically, they use k-nearest neighbors to retrieve the top-k most similar tokens, out of an "unlimited-length input" say of size N, where k << N. 2. This idea i
81.
by mxwsn 3y ago
This title is confusing due to the automatic removal of "How". Should be "How physics advanced generative AI".
82.
by mxwsn 3y ago
Maybe for general consumers. But deep learning is really a fundamental advance that has impacted science in important ways, e.g. AlphaFold. In general, deep learning learns function approximators that scale smoothly with data and resources,
83.
by mxwsn 4y ago
An important use-case is federated learning, which Google, and many healthcare / pharmaceutical companies are very interested in. In federated learning, multiple companies or groups with their own private data come together to train a
84.
by mxwsn 4y ago
Loess curves are very useful in data visualization and in my opinion, underused and underappreciated.
85.
by mxwsn 4y ago
This contributions list, and much more, are in the GPT-4 technical report: https://cdn.openai.com/papers/gpt-4.pdf
86.
by mxwsn 4y ago
How so? Deep neural nets and ChatGPT are very much statistical models.
87.
by mxwsn 4y ago
There's a subfield called symbolic regression that considers this problem. https://www.science.org/doi/10.1126/sciadv.aay2631 The Kepler equation is relatively simple compared to other equations re-discovered
88.
by mxwsn 4y ago
https://archive.is/CK5Be
89.
by mxwsn 4y ago
> overjustification effect Fair enough but not a flaw. > There is no reason why a game would accept any non-worthless NFT object minted by a different publisher No reason? Surely you can be more creative than that. > The longer the
90.
by mxwsn 4y ago
I agree it's illuminating to understand diffusion models in relation to VAEs, but I personally consider them different models, but the line in the sand is definitely subjective. I think this because (reasons I'm sure you're f

More ›