alyxya

5 ms·

31.
by alyxya 7mo ago
Fundamentally I don't believe second-order methods get better data efficiency by itself, but changes to the optimizer can because the convergence behavior changes. ML theory lags behind the results in practice.
32.
by alyxya 8mo ago
This looks extremely impressive, really deserves more attention here. Are the inverse dynamics and forward dynamics models trained separately? It sounds like if the inverse dynamics model is meant to extrapolate more training data, then per
33.
by alyxya 8mo ago
The interesting thing to me about their world models is that it's like a static point cloud model, as opposed to deepmind's genie 3 which feels more like a video generation model at its base. I see the video generation base as gen
34.
by alyxya 8mo ago
I read that they only have 30 people maintaining the whole app. It must be difficult holding everything together with so few people, and probably insufficient redundancy with oncall.
35.
by alyxya 8mo ago
The best and proven linear attention is the Gated DeltaNet or variations of it, used by Kimi and Qwen. Anyone who thinks linear attention can't work is forgetting that models are a fixed size so attention should always be compressable
36.
by alyxya 9mo ago
I think the standard way to convert repeating decimals or decimals that appear to have a certain repeating pattern to fractions is to take the first repeating period and divide by 0.999.. with the number of 9s matching the period length. 0
37.
by alyxya 9mo ago
Unlike most improvements to LLMs that modify the architecture or optimizer or something about the model, this paper discusses a novel technique that relies on some external lookup table in the forward pass computation, with the external loo
38.
by alyxya 9mo ago
Given the decrease in the benchmark score from the correction, I don't think you can assume they didn't check a single output. Clearly the model is still very capable and the model cheating its results didn't affect most of t
39.
by alyxya 9mo ago
It'll steadily continue the same way Moore's law has continued for a while. I don't think people question the general trend in Moore's law besides the point where it's nearing the limit of physics. It's a lot h
40.
by alyxya 9mo ago
I don't get the anti-LLM sentiment because plenty of trends continue to show steady progress with LLMs over time. Sure, you can poke at some dumb things LLMs do as evidence of some fundamental issue, but the frontier capabilities conti
41.
by alyxya 10mo ago
I think the most neutral solution right now is having multiple competing models as different perspectives. We already see this effect in social media algorithms amplifying certain biases and perspectives depending on the platform.
42.
by alyxya 10mo ago
I don’t think the two kinds of vibe coding are entirely separate. There’s a spectrum of how much context you care to understand yourself, and it’s feasible to ask a lot of questions to gain more understanding or let loose and give more disc
43.
by alyxya 10mo ago
Minor thing that bothers me is that I can't scroll through the things like in the deep sea or space elevator.
44.
by alyxya 10mo ago
I’ve had a hard time parsing what exactly the paper is trying to explain. So far I’ve understood that their comparison seems to be models within the same family and same weight tensor dimensions, so they aren’t showing a common subspace whe
45.
by alyxya 10mo ago
The hardest part about making a new architecture is that even if it is just better than transformers in every way, it’s very difficult to both prove a significant improvement at scale and gain traction. Until google puts in a lot of resourc
46.
by alyxya 10mo ago
There generally aren't new techniques when optimizing something ubiquitous. Instead, there are a lot of ways to apply existing techniques to create new and better results. Most ideas are built on top of the same foundational principles
47.
by alyxya 10mo ago
The chart confused me because I expected to see performance numbers of CUDA-L2 compared to the others, but instead it shows a chart showing the speedup percentage of CUDA-L2 over the others. In some sense, the bar chart effectively inverts
48.
by alyxya 11mo ago
Fair, I think it would be more appropriate to say higher capacity.
49.
by alyxya 11mo ago
That's true, I expect more inference time scaling and hybrid inference/training time scaling when there's continual learning rather than scaling model size or pretraining compute.
50.
by alyxya 11mo ago
They have a moat defined by being well known in the AI industry, so they have credibility and it wouldn't be hard for anything they make to gain traction. Some unknown player who replicates it, even if it was just as good as what SSI d
51.
by alyxya 11mo ago
The impactful innovations in AI these days aren't really from scaling models to be larger. It's more concrete to show higher benchmark scores, and this implies higher intelligence, but this higher intelligence doesn't necessa
52.
by alyxya 11mo ago
Making a VSCode fork is probably the wrong direction at this point in time. The future of agentic coding should need less support for code editor related functionality, and could eventually primarily support viewing code rather than editing
53.
by alyxya 11mo ago
I think Google probably cares more about a strong generalist model rather than solely optimizing for coding.
54.
by alyxya 11mo ago
People are bad at things that don't have quick and clear feedback. It's hard to improve at something if you just reinforce your own wrong ideas.
55.
by alyxya 11mo ago
> Briefly, the argument goes that if there is a 0.001% chance of AGI delivering an extremely large amount of value, and 99.999% chance of much less or zero value, then the EV is still extremely large because (0.001% * very_large_value) +
56.
by alyxya 11mo ago
Unironically this is how most of us think about the world. I believe as humans we’re innately selfish and want to feel like we’re usually good or at least justified. Everyone is the protagonist of their own life and sometimes the antagonist
57.
by alyxya 11mo ago
I was disappointed to find out it’s not an AI model dedicated to generating ketchup images.
58.
by alyxya 11mo ago
I used to think that Cursor would get killed by the major AI labs owning their coding models, but with the best open source models being on par with the proprietary models, I can realistically see Cursor winning the coding AI race. Once the
59.
by alyxya 11mo ago
What's the point of this show hn if there's hardly any info and no demo?
60.
by alyxya 11mo ago
I don't think this is a bug specifically with Claude Code, rather it's due to Claude Code having javascript in the backend. The interesting thing to me is that the numeric string was interpreted as an integer.

More ›