gwern

15 ms·

211.
by gwern 10mo ago
> which while describes a curve that shows the same behavior, it's more "accurate" of the actual post creation. I would say that this graph looks a lot more extreme, actually!
212.
by gwern 10mo ago
I am guessing you are on mobile and are referring to the togglebar 'demo mode', illustrating its existence. I wish we didn't have that, but we have to. That demo of the theme togglebar exists to educate users, because we foun
213.
by gwern 10mo ago
To clarify the point here for people who didn't read OP: the oral exams here are customized and tailored to the student's individual unique project, that's the point and why they are not written: > In our new "AI&#x2F
214.
by gwern 10mo ago
Note: this is different from https://blog.glyphdrawing.club/font-with-built-in-syntax-hig... https://news.ycombinator.com/item?id=41245159 which recently resurfaced, and is cited as prior art.
215.
by gwern 10mo ago
So if it's not using attention and it processes the entire input into an embedding to process in one go, I guess this is neither a Transformer nor a RNN but just a MLP?
216.
by gwern 10mo ago
Note: from May 2025. https://gwern.net/doc/genetics/editing/2025-musunuru.pdf
217.
by gwern 10mo ago
Isn't even thoughtful either. > The question was never “how do we align AI with human values?” The question was always “which humans get to define those values?” Grok answered that question: the ones with the most money. Grok is rou
218.
by gwern 10mo ago
> I would also welcome someone writing a short takedown where they fix the prompts and get better-than-2022 results from nbp NBP (and the new ChatGPT generator) are integrated with LLMs to various degrees, so seems like the obvious start
219.
by gwern 10mo ago
Penn & Teller formulation: https://www.smithsonianmag.com/arts-culture/teller-reveals-h...
220.
by gwern 10mo ago
There's a vein of research which interprets self-attention as a kind of gradient descent and says that LLMs have essentially pre-solved indefinitely large 'families' or 'classes' of tasks, and the 'learning&#x2
221.
by gwern 10mo ago
There's also the efficiency argument from new capability: even a tiny bit better weather forecast is highly economically valuable (and saves a lot of wasted energy) if it means that 1 city doesn't have to evacuate because of an er
222.
by gwern 10mo ago
No, the search space is tiny: you can just attack 1 BPE at a time! Stuff like password guessing is almost trivial when you get to do a timing attack on each successive character. So that lets you quickly exfiltrate arbitrary numbers of prom
223.
by gwern 11mo ago
While we're at it: my own work 2 years ago in creating an entire workflow for turning Midjourney or DALL-E dropcaps into attractive, lightweight, easy-to-create dropcaps for web pages: https://gwern.net/dropcap We use
224.
by gwern 11mo ago
No. I think the need for adversarial losses in order to distill diffusion models into one-step forward passes has provided additional evidence that GANs were much more viable than diffusimaxis loudly insisted. (Although I'm not really
225.
by gwern 11mo ago
This works surprisingly well. If you look into enough dark corners of Unicode, it turns out that you can do a shocking amount of typography, going far beyond the obvious italics and bolds: https://gwern.net/utext In fact, I
226.
by gwern 1y ago
> but I think fixing the verbal tics is still important and useful. I don't. I think they're useful for flagging the existence of mode-collapse and also providing convenient tracers for AI-written prose. Erasing only the verb
227.
by gwern 1y ago
This is the epitome of patching symptoms rather than treating the disease. Even if you suppress the obvious syntactic slop like 'it's not X but Y', you have no reason to believe you've fixed mode-collapse on higher more
228.
by gwern 1y ago
You're correct, but when the worst the ChatGPTisms get is turns of phrases like "LeetCode youth finally paid off: turns out all those "rebalance a binary search tree" problems were preparing me for salami, not FAANG inte
229.
by gwern 1y ago
My point is more that Schmidhuber is saying that the gates or the initialization are the innovation solely because they produce well-behaved gradients, which is why Hochreiter's 1991 thing is where he starts and nothing before that c
230.
by gwern 1y ago
> Note again that a residual connection is not just an arbitrary shortcut connection or skip connection (e.g., 1988)[LA88][SEG1-3] from one layer to another! No, its weight must be 1.0, like in the 1997 LSTM, or in the 1999 initialized
231.
by gwern 1y ago
This apparently doesn't apply here, but in fact, pixels can be generated independently of each other. There are architectures where you can generate an arbitrary pixel or element of the image without generating the others; they are j
232.
by gwern 1y ago
Requires a login?
233.
by gwern 1y ago
I think OP is an instance that proves the point this is a lazy, clumsy editing attempt, done through a document registration service which exists to prevent exactly this and yet, you have to be an experienced nerd (eg https://e
234.
by gwern 1y ago
One of the most surprising Gwern.net bug reports was from a compulsive highlighter who noted that the skip-ink implementation (which uses the old text-shadow trick, because frustratingly, the recently standardized skip-ink CSS still manag
235.
by gwern 1y ago
Demonstrating that Rich Sutton was never really on the 'LLM bus' in the first place. Note the remarkable absence from the essay of language models & large language models from that essay despite BERT and GPT-2 and 'unreas
236.
by gwern 1y ago
I don't have any particularly canonical reference I'd cite here, but self-preference bias in LLMs is well-established. (Just search Arxiv.)
237.
by gwern 1y ago
They can, but they are known to have a self-favoring bias, and in this case, the error is so easily identified that it raises the question of why GPT-5 would both come up with it & preserve it when it can so easily identify it; while if
238.
by gwern 1y ago
It is lacking in URLs or references. (The systematic error in the self-reference blog post URLs is also suspicious: outdated system prompt? If nothing else, shows the human involved is sloppy when every link is broken.) The assertions are b
239.
by gwern 1y ago
It is especially not obvious because this was written using ChatGPT-5. One appreciates the (deliberate?) irony, at least. (Or at least, surely if they had asymptoted, OP should've been able to write this upvoted HN article with an old
240.
by gwern 1y ago
https://gwern.net/doc/statistics/causality/1926-yule.pdf

More ›