antirez

31 ms·

541.
by antirez 2y ago
the S1 paper did the same a few days ago, basically. 1000 total CoT with SFT. I believe that all this shows that pre-training stage already creates the representations needed for CoT reasoning, so they are very simple to uncover. Either wit
542.
by antirez 2y ago
They are one liner maxims about things that are impossible to don't understand. You may argue, for instance, why it happened, what are (if any) the good sides, for instance, of this absurd chain of dependencies, but if you don't
543.
by antirez 2y ago
Living is serverless world.
544.
by antirez 2y ago
The word had moved past Redis already when I started it. Redis is an experiment in minimalism, and because of that was an industry anomaly. Wohpe is probably the novel that sold the most copies in the recent years italian sci-fi history. Th
545.
by antirez 2y ago
Sometimes, to write a post that makes people think about complicated processes, you need to write statements that may sound dissonant. Reality is complicated and multi-sided, and I want to ignite a reflection on the state of the software wh
546.
by antirez 2y ago
I think likewise. Semver in theory was a good idea (at least in principle: I never liked the exact form, since the start), and the original idea was that the version number was a form of documentation. But what it became is very different:
547.
by antirez 2y ago
If you read the post as: software should not evolve, you are not reading it right. Also I'm very enthusiastic with modern AI and definitely open to new things that make a difference. The complexity I point my finger to, in the post, is
548.
by antirez 2y ago
I never lost the joy of coding since I simply do what I want and refuse to do what I don't enjoy. But unfortunately most people can't because of many reasons and I know many that are now very disappointed with the state of softwar
549.
by antirez 2y ago
Well put. And even when we think that writing doc is a less important thing.
550.
by antirez 2y ago
Why this is a result of software complexity? I'm not in favor of less capable software.
551.
by antirez 2y ago
> When they want to tinker, they should write their own compiler. But they shouldn't use that in production. Why? We should stop saying others how they want to write/use their code ASAP. Many established technologies are a tota
552.
by antirez 2y ago
> Overall as a whole we are creating things, more than we are destroying. I don't see the need to take a negative stance. Fair point: each one of us can think about the balance and understand if it's positive or negative. But a
553.
by antirez 2y ago
The rewrite line is about "Let's rewrite this big system from X to Y", not about experimenting creating new variants.
554.
We are destroying software(antirez.com)
1012 points by antirez 2y ago|622 comments
555.
by antirez 2y ago
Copy-right is not learn/train-right. That said Meta full its mouth with open source while they release models that are not SOTA nor usable for commercial purposes.
556.
by antirez 2y ago
Sure, but you have multiple thoughts tokens in the context the model sees to process the next token.
557.
by antirez 2y ago
In recent weeks what's happening is exactly the contrary.
558.
by antirez 2y ago
Cool, but isn't this encoding a potentially very long thinking process into a fixed embedding? Intuitively should not work as well.
559.
by antirez 2y ago
Yep, it's a terrible materialization of a theoretically awesome idea. Both because of these details and because of the robotic tone of the voice compared to the demos they had released.
560.
by antirez 2y ago
I believe you can disable it clicking on the gear.
561.
by antirez 2y ago
Indeed. YouTube is rolling this out to more and more folks, and it gets really interesting as you can kinda produce videos in your language but still have an international audience. The quality is not outstanding as the voice is a bit robot
562.
by antirez 2y ago
LLMs using CoT are also decoder-only, it's not a paradigm shift like people want to claim now to don't say they were wrong: it's still next token prediction, that is forced to explore more possibilities in the space it contai
563.
by antirez 2y ago
I read the article: it does not look like very good research: It's simple to find flaws in LLMs reasoning / compositional capabilities looking at problems that are at the limit of what they can do now, or just picking problems tha
564.
by antirez 2y ago
Yes, the stochastic parrots story is one of the most strong instances in recent times where experts in a field are made blind by their own expertise (the mental model they have of certain things) to the point of being incapable of seeing tr
565.
by antirez 2y ago
Yes there is another part of the community that overhypes everything. But I can expect that from a CEO of an AI company (especially if he is Altman), but from researches? Also the fact that LLMs may reach superhuman expertise in certain fie
566.
by antirez 2y ago
LLMs keep showing more and more they are the wonder of AI that we awaited for decades: talking machines that every two months do progresses that two months before were anticipated impossible because of <put here some limit that was actua
567.
by antirez 2y ago
For me other than severance: Halt and catch fire and Better call Saul.
568.
by antirez 2y ago
Just tested two complicated coding tasks, and surprisingly o3-mini-high nailed it while Sonnet 3.5 failed it. Will do more tests tomorrow.
569.
by antirez 2y ago
Thanks! I'll try. I didn't bother believing that if this was developed heavily on CUDA, it was likely going to use kernels that were missing in MPS.
570.
by antirez 2y ago
No cpu / mps support to train on Macs, apparently.

More ›