ainch

8 ms·

91.
by ainch 5mo ago
They're two different models - you can use the world model to train (or test like Wayve) a different car-driving model. The world model is basically intended as a more true-to-life simulator.
92.
by ainch 5mo ago
And importantly it's got nice properties like being differentiable and monotonic, unlike eg. taking |x|.
93.
by ainch 5mo ago
I'm generally suspicious of IQ but still - 2 standard deviations above average would include about 2.5% of the population. Hassabis is in a significantly more exclusive slice. His parents weren't particularly wealthy. More likely:
94.
by ainch 6mo ago
On tabular datasets less than ~250k samples, tabular foundation models now outperform boosting. Of course it remains to be seen how they'll scale to significantly larger datasets as the models improve. https://huggingface.co
95.
by ainch 6mo ago
Check out Andrew Gordon Wilson's excellent paper "Deep Learning is Not so Mysterious or Different" for a discussion of the ways in which existing learning theory does and doesn't work neural nets. https://arxi
96.
by ainch 6mo ago
I would expect to see a significant wall clock improvement if that was the case - Meta's Coconut paper was ~3x faster than tokenspace chain-of-thought because latents contain a lot more information than individual tokens. Separately, I
97.
by ainch 6mo ago
I think that should be true, but doesn't hold up in practice. I work with a good editor from a respected political outlet. I've tried hard to get current models to match his style: filling the context with previous stories, classi
98.
by ainch 6mo ago
Germany has an anonymous support programme for people who feel paedophilic urges but don't wish to offend. I believe they've used that network for research, but I think it's probably quite a limited, and potentially biased, s
99.
by ainch 6mo ago
Oh sure, I see what you mean - thanks for clarifying. On top of your point, it's true that CO2 has a prolonged impact on global temperature even after it's been 'removed' from the atmosphere, so even once solar pays back
100.
by ainch 6mo ago
Sorry, I'm not picking up on the connection - could you expand? Do you think they should also pay for offsets alongside developing energy infrastructure?
101.
by ainch 6mo ago
Carbon offsets are a sham, but you could just require them to directly pay for the actual energy infrastructure required. If you need 1GW of electricity, develop 1GW of solar.
102.
by ainch 6mo ago
Transformers do have a fixed input/output size though - that's what a context window is. It's just that, via scaling and algorithmic improvements, the length of usable context windows has increased to the point that they&#x27
103.
by ainch 6mo ago
This opens up an interesting new avenue for corporate FOMO. What if you don't partner with Anthropic, miss out on access to their shiny new cybersec model, and then fall prey to a vuln that the model would have caught?
104.
by ainch 6mo ago
Great piece. And a good excuse to read up on the use of diaeresis in English (eg. coördination, reëlection) to distinguish repeated vowels - I hadn't seen the New Yorker's usage before.
105.
by ainch 6mo ago
Randall Munroe of xkcd? I like his work but I'm not sure I'd call him a philosopher...
106.
by ainch 6mo ago
Sure, it's clearly marketing. I think a private company pursuing marketing via open research with open source code (including datasets) is a good trade. A hypey blogpost + research is better than no blogpost and no research.
107.
by ainch 6mo ago
The Gaussian Processes underpinning this work are hardly a product of the 'AI Hype Machine' - they've been around for decades, have strong statistical underpinnings, and are being widely explored for experimental design acros
108.
by ainch 6mo ago
A sidenote along these lines - I've recently done an MSc, and found that the default approach to lectures is now to present slide decks. One of the profs, however, delivers a more traditional lecture, writing everything on a blackboard
109.
by ainch 6mo ago
It's definitely true that they've increased their revenue rapidly. But at the same time the 'scaling laws' that the labs were first built around require exponentially-scaling cost (10x flops for a fixed reduction in trai
110.
by ainch 6mo ago
Do you have any evidence that inference revenue is growing faster than training costs? RLVR is significantly less compute-efficient than token-prediction pretraining - especially as labs are trying to train models to achieve agentic tasks w
111.
by ainch 7mo ago
As a PhD student doing my fair share of midnight paper-reading I think I'm the exact target market - thank you for sharing!
112.
by ainch 7mo ago
On that note, Terence Tao gave a good interview to Dwarkesh Patel talking about Kepler. He pointed out that the previous geocentric models were actually more accurate than Kepler's at the time, in part because they'd had so much c
113.
by ainch 7mo ago
I think it depends whether you can leverage some knowledge. It's possible for a person/LLM to look at a loss curve and say "oh that's undertraining, let's bump the lr" - whereas a Bayesian method doesn't n
114.
by ainch 7mo ago
I'd like see a system like this take more inspiration from the ES literature, similar to AlphaEvolve. Let's see an archive of solutions, novelty scoring and some crossover rather than purely mutating the same file in a linear fash
115.
by ainch 7mo ago
Microsoft released a report with some numbers on Deepseek adoption globally. They say it's got ~90% market share in China, and is growing in popularity across Africa. https://www.microsoft.com/en-us/corporate-respo
116.
by ainch 7mo ago
I was a little surprised to see a Telegram integration rather than Slack or Teams, given Anthropic's enterprise-first posture. But then I looked it up, and it turns out Telegram dwarfs both, at around 1bn MAUs, vs 50m and 300m respecti
117.
by ainch 7mo ago
The human genome contains around 1.5GB of information and DeepSeek v3 weighs in at around 800GB, so it's a bit apples-to-oranges. As you say, what's been evolved over hundreds of millions of years is the learning apparatus and arc
118.
by ainch 7mo ago
What competition was OpenAI likely to face from a team working on fast Python tooling?
119.
by ainch 7mo ago
I got Codex to whip me up a Chrome extension that autoswaps back to Pro whenever I reload the page. It's made Gemini significantly less irritating to use.
120.
by ainch 7mo ago
I don't think it's just an engineering problem - decades of research have failed to produce a convincing, general definition of intelligence, capability or agency. You can try to form proxy metrics by combining benchmarks, but exi

More ›