gwern

53 ms·

751.
by gwern 3y ago
I don't know anyone has properly analyzed this, nor how robust such methods are if one is trying to cover it up. Also, I doubt anyone has analyzed the scenario where the warm-started model is then extensively trained for trillions of t
752.
by gwern 3y ago
They did not, in part because it would reveal the data-filtering routines (particularly the political censorship - Chinese LLM papers sometimes mention the ban list but never reveal it), and also in part because it might reveal things they&
753.
by gwern 3y ago
Does AlphaZero get inbred?
754.
by gwern 3y ago
But we're not discussing ordinary savants like Peek. Such savants often hone their skills over decades, sometimes inventing their own algorithms or tricks and memorizing vast reams of data. Those are amazing, but not sudden . To analo
755.
by gwern 3y ago
This is a very sloppy comment. How do you know that about 'the new board', exactly? Are you already an expert on these 3 women and know their background and all possible connections to Altman? I doubt it. (I'm not.) They coul
756.
by gwern 3y ago
Context: https://news.ycombinator.com/item?id=39635058 You could predict this yesterday from the leaks that were coming out like https://www.nytimes.com/2024/03/07/technology/openai-execu
757.
by gwern 3y ago
Your take is the bizarre one. Who is 'the board'? Who is 'they' and why should they wait for the lawyers? There's at least 2 factions on the temporary board, an unknown successor board, and the old board had at leas
758.
by gwern 3y ago
> I have found that any tests that include a negative are highly likely to fail unless you word it in a specific way. Well yes, that's the point. This is an easy, objective, simple task not in any benchmarks I know of that much stu
759.
by gwern 3y ago
Fails my usual RLHFing test of "write a non-rhyming poem". It initially starts well but falls back into rhyming. I then ask "are there any rhymes in that?" and it correctly identifies several but then justifies itself wi
760.
by gwern 3y ago
An important revelation in this article is that the investigation is over and the report is almost done. The fact that this is the worst that is leaking shows that Altman has won and will be put in full control of OA, IMO: https:/&#x2
761.
by gwern 3y ago
> Actually, my first thought was to just feed these pgn strings to a network and create an LLM-esque sort of network. But I wasn't convinced that I was going to get that anywhere within reasonable time. A number of people have done
762.
by gwern 3y ago
The quote at the end from Eiseley is more interesting than it sounds: https://www.101bananas.com/library2/eiseley1.html It's about a crow being terrified to nearly fly into Eiseley's head on an extremely fogg
763.
by gwern 3y ago
How do you measure any of these improvements?
764.
by gwern 3y ago
> And we are using the older Tesla T4. That's an undemanding and well-debugged chip by this point (6 years ago!). So you aren't experiencing any of the pain people using A100s or H100s (never mind people who have to stand up cl
765.
by gwern 3y ago
(These are just the ones I recall offhand from personal experience on gwern.net using SVG in quite limited ways, mostly just for the link-icons.)
766.
by gwern 3y ago
More on M5: https://www.sciencedirect.com/science/article/pii/S016920702...
767.
by gwern 3y ago
Langsec https://langsec.org/ would like to have a word with you about your view that only Turing-completeness is a problem and you can solve all security issues with overpowered config formats by just chucking them in a san
768.
by gwern 3y ago
So many bugs. The background will break in dark-mode, or the foreground. The bounding box will cut off the image contents. You export in SVG to get nice text rendering and everything carefully aligned, and it looks in Chromium the way it di
769.
by gwern 3y ago
> While AI is unique in directly augmenting human thought, we expect its impact to be more akin to the printing press Which, among its undoubted benefits, also led directly to centuries of religious warfare worldwide and the deaths of te
770.
by gwern 3y ago
The obvious one is a parrot, but looks like they've already used a lot of parrots: https://www.oreilly.com/animals.csp?x-search=parrot&x-sort=a... Still some room for superintelligent cephalopods https:/&#x2F
771.
by gwern 3y ago
No, that can't explain his result showing the witch gene going extinct despite, by stipulation, zero fitness impact on individuals but 100% transmission. Because the human population did grow over that time, so every deadend must have
772.
by gwern 3y ago
> As tempting as it might be to predict that we’ll have a similar changing of the guard in the future, the following graphic provides a needed corrective to this thinking, showing the most valuable companies over the last three decades (
773.
by gwern 3y ago
> “People just haven’t realized the financial rewards” of investing in startups with diverse leadership, Judge said. “Minority-led companies consistently outperform.” The market doesn't seem to agree. And where are all of the minori
774.
by gwern 3y ago
See also: https://danluu.com/diseconomies-scale/
775.
"Chance and Order", Stanisław Lem (1984 Autobiography)(gwern.net)
3 points by gwern 3y ago|0 comments
776.
by gwern 3y ago
It was included because it was hilarious. The mental image of a worker ducking under a nozzle and their stomach bloating like some sort of ice cream mosquito justifies itself. (Watch out for the ice cream headache. That may not be covered u
777.
by gwern 3y ago
Yes, that's the opposite of _Dracula_, though. Dracula isn't overwhelmed by the modern world, he's defeated by it. That's not the same thing. He's no Lestat in _Interview_, driven to catatonia by change.* Dracula
778.
by gwern 3y ago
I haven't, sorry. I didn't realize either until August last year!
779.
by gwern 3y ago
If grids are very conservative, that would also seem to suggest that grid operators would want existing solar farms topped up rather than replaced - old debugged solar farms declining, to be replaced by excitingly untested new ones, seems l
780.
by gwern 3y ago
The complement play for Mistral was that they'd be the low-cost SaaS serving their own models or any better ones that the FLOSS releases created and the upsell: you prototype and develop on Mistral models like Mixtral and then when you

More ›