ainch

5 ms·

31.
by ainch 2mo ago
No I think you're right that the amount of compute spent on office work is lower than coding - although I don't have any sense for the right share. The best source I could find was an OpenAI report [1] which mentions that ~64% of
32.
by ainch 2mo ago
Agreed, Kimi is cheaper for coding - I say that explicitly in the post too. However I'd have to disagree with you on the "office task" front. General office work is one of the big frontiers the labs are pushing on, and it&#x2
33.
by ainch 2mo ago
That's a very fair critique. I don't mean to imply that Kimi is not at all cheaper than U.S frontier models. I more wrote this because I believe - since Chinese LLMs entered the public consciousness via DeepSeek R1, which was genu
34.
Kimi K3 is not cheap(alexinch.com)
23 points by ainch 2mo ago|25 comments
35.
by ainch 2mo ago
It's telling that in their release post, Moonshot themselves said that K3 lags Fable and GPT-5.6 in "user experience". I took that to mean the stuff you can't push directly via RL, what some people call "big model s
36.
by ainch 2mo ago
I appreciate people thinking this is a marketing ploy, but at the same time, OpenAI have just had to delay a model release because of government attention on cybersecurity risk. This incident will increase the attention on them specifically
37.
by ainch 2mo ago
HuggingFace posted an incident report a week ago, which makes it much more likely that this happened. I understand people are suspicious of OpenAI, but I don't think there's any reason to believe this is a made-up event. https:&#
38.
The Boring Frontier of Robotics(alexinch.com)
2 points by ainch 2mo ago|0 comments
39.
by ainch 2mo ago
I've been sharing music on SoundCloud for a decade and I am genuinely interested in this topic, but I cannot stand how strongly this post reeks of LLM writing. Do people releasing this kind of content not notice all the clichés?
40.
by ainch 2mo ago
There is a field of hierarchical RL in which the optimisation occurs over a range of time scales/abstraction. But I'm not aware of much practical success for these approaches so far.
41.
by ainch 2mo ago
Do you have a good source on this information theory framing? I don't remember it being covered in Sutton & Barto.
42.
by ainch 2mo ago
We're still far from solving real-world physical interaction, even with the world knowledge of LLMs incorporated into VLAs. It's the reason robotics startups trying to deploy into people's homes are so reliant on teleop; the
43.
by ainch 3mo ago
One of the lead authors, Daniel Kokotajlo, worked at OpenAI for years before quitting. In 2021 he wrote a remarkably accurate forecast of how LLMs would develop over the following 5 years[0]. I think it should be obvious that he understands
44.
by ainch 3mo ago
Yann is a big SSL guy but I don't think he was involved in the original DINO - he's not listed as a co-author or anything.
45.
by ainch 3mo ago
The most basic machine learning-related query gets flagged for me. For example: In flax nnx, what's the idiomatic way to store state on a Module. For example, if I'm handling the carry manually for an nnx.RNN. Or one asking
46.
by ainch 3mo ago
There have been papers about model collapse, but the underlying assumption is that you constantly train on only the outputs of the previous model. Later research has shown that as long as you retain some "real" data, training on l
47.
by ainch 3mo ago
I think you'll be waiting a while for the former, unless you're ok with strangers teleoperating a robot around your house whenever it gets confused.
48.
by ainch 3mo ago
Wouldn't modern SLAM or VSLAM address that problem?
49.
by ainch 3mo ago
Thank you for expanding! I come from more of an ML background so still learning a lot on these topics: Agreed that the neocortex uses fewer layers because of looping - I also suspect it's partly because neurons are more complex than th
50.
by ainch 3mo ago
I'm afraid the precise connection you're making isn't totally obvious to me. As far as prediction - I mean sure the cortex and LLMs do prediction, but then so can RNNs or diffusion models or any other generative model. Really
51.
by ainch 3mo ago
Could you provide more detail? My understanding is that the neocortex is predominantly focused on forwards simulation, which seems distinct to how transformers operate.
52.
by ainch 3mo ago
As an ML researcher, I know box2d because it underpins many of the standard reinforcement learning environments (in OpenAI Gym) that we use to benchmark methods, like Lunar Lander or Car Racing: https://gymnasium.farama.org/
53.
by ainch 3mo ago
This method is cool and the post explains it well. It would, however, be good to get more detail on the energy efficiency they flag as their motivation: is this model actually more energy efficient than the comparators they highlight?
54.
by ainch 3mo ago
They've said that Mojo is still on track to be open-sourced this year, post-acquisition.
55.
by ainch 3mo ago
Indeed. The world models research many labs are now chasing was to some degree ignited by David Ha and Schmidhuber's 2018 paper. More broadly, Sakana is pursing a refreshingly distinct research path, with their focus on evolutionary me
56.
by ainch 3mo ago
Basically all the images and videos on their website are AI-generated. It doesn't inspire much confidence.
57.
by ainch 4mo ago
Tri Dao's lab must have saved countless watts with FlashAttention. Great to see them continuing to open-source massive efficiency gains.
58.
by ainch 4mo ago
Here's one that was flagged for me: a question about a niche Reinforcement Learning paper from 2012 I've been reading the option-option model paper by David Silver. It appears that they achieved quite an effective result. Why has
59.
by ainch 4mo ago
I agree it's an oversimplification. The example I think of is something like Newton's law of gravitation vs Ptolemaic epicycles: one simple explanation replaced many layers of tweaks. It's also a relevant example for AI - one
60.
by ainch 4mo ago
Anthropic's claim was that Deepseek collected ~150k conversations. https://www.anthropic.com/news/detecting-and-preventing-dist... I think the extent of distillation by Deepseek specifically is overstated. For com

More ›