5 ms·

I don't get the comments trashing this. If it slightly beats or even matches Opus 4.6, it means Meta is capable of building a model competitive with the leading

by tty456 6mo ago

I don't get the comments trashing this. If it slightly beats or even matches Opus 4.6, it means Meta is capable of building a model competitive with the leading AI company. Sure, they spent a lot of money and will have on-going costs. But how much more work would it take to turn that into a coding agent people are willing to try (and pay for) along side their usage of a collection of agents (Claude, Codex, etc)? Also means Meta doesn't have to pay another company to use a SATA model across all their products (including IG and WhatsApp, vr) which will matter to their balance sheet long term (despite the constant r&d spend).

prodigycorp 6mo ago
Comments trashing this are rightly correct skeptics who remember the benchmaxxing of llama 4. This model was out in the woods as early as like a couple months ago but they didn't release it because it was at gemini 2.5 pro levels.
- zozbot234 6mo ago
  The llama4 series was one of the earliest large MoE's to be made publically available. People just ignored it because they were focused on running smaller and denser models at the time, we should know better these days.
  - prodigycorp 6mo ago
    the models were objectively horrible
    - NitpickLawyer 6mo ago
      They really weren't horrible. They were ~gpt4o, with the added benefit that you could run them on premise. Just "regular" models, non "thinking". Inefficient architecture (number of active out of total) but otherwise "decent" models. They got trashed online by bots and chinese shills (I was online that weekend when it happened, it's something to behold). Just because they were non-thinking when thinking was clearly the future doesn't make them horrible. Not SotA by any means, but still.
      - prodigycorp 6mo ago
        Nah I remember how disgusted I felt trying llama 4 maverick and scout. They were both DOA.. couldn't even beat much smaller local models.
        refulgentis 6mo ago
        I'll cosign what you said, simultaneously, yr interlocutor's point is also well-founded and it depresses me it's not better known and sounds so...off...due to conventional wisdom x God King Zuck's misunderstanding his own company and resulting overreaction. They beat Gemini 2.5 Flash and Pro handily on my benchmark suite. (tl;dr: tool calling and agentic coding). Llama 4 on Groq was ~GPT 4.1 on the benchmark at ~50% the cost. They shouldn't have released it on a Saturday. They should have spent a month with it in private prerelease, working with providers.[1] The rushed launch and ensuing quality issues got rolled into the hypebeast narrative of "DeepSeek will take over the world" I bet it was super fucking annoying to talk to due to LMArena maxxing. [1] my understanding is longest heads up was single-digit days, if any. Most modellers have arrived at 2+ weeks now, there's a lot between spitting out logits and parsing and delivering a response.
        alex1138 6mo ago
        Your comments seem to imply the engineers made a great product but Zuck intervened so now it's shit
        refulgentis 6mo ago
        I don't know how Zuck intervening could change float32s in a trained model, so I don't think I think that, but maybe I'm parsing your words incorrectly.
        pixel_popping 6mo ago
        failing non-stop at tool calls on top of that.
      - refulgentis 6mo ago
  - dilap 6mo ago
    Deepseek R1 was a publically-available, MoE model that was getting a ton of attention before llama4. Llama4 didn't get much attention because it wasn't good.
    - jychang 6mo ago
      Also, Gemini 2.5 Pro launched a week before Llama 4. It was Gemini 2.5 Pro that redeemed Google in the eyes of most people as a valid competitor to OpenAI instead of as a joke, so Meta dropping the ball with Llama 4 was extra bad.
- refulgentis 6mo ago
  > 4. This model was out in the woods as early as like a couple months ago but they didn't release it because it was at gemini 2.5 pro levels. Source? (Even if rumor)
  - prodigycorp 6mo ago
    It was from a techmeme ride home podcast where the host discussed "sources at the company said". I don't remember which day's episode it was.
  - nl 6mo ago
    NYTimes had a story about this (March 12): > Meta’s new foundational A.I. model, which the company has been working on for months, has fallen short of the performance of leading A.I. models from rivals like Google, OpenAI and Anthropic on internal tests for reasoning, coding and writing, said the people, who were not authorized to speak publicly about confidential matters. > The model, code-named Avocado, outperformed Meta’s previous A.I. model and did better than Google’s Gemini 2.5 model from March, two of the people said. But it has not performed as strongly as Gemini 3.0 from November, they said. > They added that the leaders of Meta’s A.I. division had instead discussed temporarily licensing Gemini to power the company’s A.I. products, though no decisions have been reached. https://www.nytimes.com/2026/03/12/technology/meta-avocado-ai-model-delayed.html https://www.nytimes.com/2026/03/12/technology/meta-avocado-a... https://archive.is/uUV5h#selection-715.98-715.277 https://archive.is/uUV5h#selection-715.98-715.277
    - o10449366 6mo ago
      [flagged]
      - mupuff1234 6mo ago
        Does Meta not harvest data on a massive scale? Not sure what exactly is the issue with doing a series on that.
      - nl 6mo ago
        If you are trying to come up with anti-media conspiracies there are always plenty of ways to do it against any media company. The idea that NY Times is particularly anti-Meta seems a stretch. They - like most traditional media companies - are anti-tech in general. The fact they also collect data doesn't make their reporting untrue. Personally I think a much more interesting rumor to make up would be that Yann Lecun (who famously had his reporting lines rearranged to go through Alexander Wang after Scale.ai acquihire) works at New York University. New York University is in the same place as the New York Times. There's a conspiracy for you. I made it up, but I mean it could be true I guess? (Of course Lecun also publicly congratulated Wang on the launch of the model. But maybe that's a ruse to hide everything.. blah blah)
redox99 6mo ago
> If it slightly beats or even matches Opus 4.6 It doesn't though
- ryeguy_24 6mo ago
  Curious on why you think this. Any data points that led you to this?
  - howdareme 6mo ago
    The benchmarks they released
    - johnfn 6mo ago
      What do you mean? In most cases, the benchmarks show a larger number for Muse and a smaller number for Opus.
      - spprashant 6mo ago
        In Multimodal yes, but Opus is definitely edging out in Text/Reasoning and Agentic benchmarks. I think the general skepticism is because they are late to race, and they are releasing a Opus-4.6-equivalent model now, when Anthropic is teasing Mythos.
ChipopLeMoral 6mo ago
> I don't get the comments trashing this. People like to hate on Meta regardless of anything, and regardless of whether it's justified or not. Not saying it isn't, just that it's many people's default bias.
- jatora 6mo ago
  That is not the case here. Nobody hated on llama 1,2,3 at all. They justifiably felt burned by the benchmaxxing of llama 4. Trust broken must be re-earned, and benchmarks alone cannot do that.
modeless 6mo ago
It's a decent model if the benchmarks are to be believed, but it won't be close to Opus in usefulness for programming. None of these benchmarks completely capture what makes a model useful for day-to-day coding tasks, unfortunately. It will take time for them to catch up, and Opus will keep improving in the meantime. But it's good to have more competition.
- ai5iq 6mo ago
  Benchmarks miss the thing that actually matters for agentic use: how does behavior change over a multi-day horizon? A model that scores well on one-shot coding tasks can still make terrible decisions when it has persistent state and resource constraints. That's where you see the real gaps between models.
  - andai 6mo ago
    Is there a benchmark for these long tasks? That kind of seems like the only number worth measuring. (Of course at that point it involves memory and context management and so on, so you're testing the harness as well as the model.)
    - ai5iq 6mo ago
      [dead]
blazespin 6mo ago
Because bots and trillion dollar ipos and even bigger stakes. People need to better appreciate the level of manipulation going on. Social media has an outsized impact. Bots and even people are getting paid to post and upvote/downvote narratives.
- asdfman123 6mo ago
  > people are getting paid to post and upvote/downvote narratives This problem will be solved shortly with better AI (if it hasn't essentially been solved already). No more humans in the loop, much lower costs for social media manipulation. Welcome to the future!
canes123456 6mo ago
Why go into coding agents? Both anthropic and OpenAI are going all in on that. The opportunity is customer facing AI now. OpenAI has the mindshare but they going to have to decide if they allocate their limited compute for free users or go all in trying to keep up with Anthropic in enterprise.
- foobiekr 6mo ago
  Because coding agents are where the revenue is.
- kaycey2022 6mo ago
  you can do way more than just coding with the coding agents.
- refulgentis 6mo ago
  If you squint at coding agents you see the next OS. Maybe better phrasing is “HCI paradigm”, but that somehow manages to say everything and nothing.
  - whattheheckheck 6mo ago
    Programming was always about designing rube goldberg systems that did a complicated state machine akin to dominos but now we have a probabalistic and nondeterministic domino that has a huge amount of dominos inside amd can dynamically generate many different paths of dominos sometimes not even leading to the intended final domino you wanted to fall. I see it more like a compiler
    - RealStupidity 6mo ago
      I agree that it's more like a compiler (turns higher level language into machine code) but I also think that's only half the story - a compiler could never turn requirements into functional software, generate boilerplate or debug. It's also a development tool
  - deleted 6mo ago
    [deleted]

by tty456 6mo ago

prodigycorp 6mo ago

Comments trashing this are rightly correct skeptics who remember the benchmaxxing of llama 4. This model was out in the woods as early as like a couple months ago but they didn't release it because it was at gemini 2.5 pro levels.

zozbot234 6mo ago
The llama4 series was one of the earliest large MoE's to be made publically available. People just ignored it because they were focused on running smaller and denser models at the time, we should know better these days.
- prodigycorp 6mo ago
  the models were objectively horrible
  - NitpickLawyer 6mo ago
    They really weren't horrible. They were ~gpt4o, with the added benefit that you could run them on premise. Just "regular" models, non "thinking". Inefficient architecture (number of active out of total) but otherwise "decent" models. They got trashed online by bots and chinese shills (I was online that weekend when it happened, it's something to behold). Just because they were non-thinking when thinking was clearly the future doesn't make them horrible. Not SotA by any means, but still.
    - prodigycorp 6mo ago
      Nah I remember how disgusted I felt trying llama 4 maverick and scout. They were both DOA.. couldn't even beat much smaller local models.
      - refulgentis 6mo ago
        I'll cosign what you said, simultaneously, yr interlocutor's point is also well-founded and it depresses me it's not better known and sounds so...off...due to conventional wisdom x God King Zuck's misunderstanding his own company and resulting overreaction. They beat Gemini 2.5 Flash and Pro handily on my benchmark suite. (tl;dr: tool calling and agentic coding). Llama 4 on Groq was ~GPT 4.1 on the benchmark at ~50% the cost. They shouldn't have released it on a Saturday. They should have spent a month with it in private prerelease, working with providers.[1] The rushed launch and ensuing quality issues got rolled into the hypebeast narrative of "DeepSeek will take over the world" I bet it was super fucking annoying to talk to due to LMArena maxxing. [1] my understanding is longest heads up was single-digit days, if any. Most modellers have arrived at 2+ weeks now, there's a lot between spitting out logits and parsing and delivering a response.
        alex1138 6mo ago
        Your comments seem to imply the engineers made a great product but Zuck intervened so now it's shit
        refulgentis 6mo ago
        I don't know how Zuck intervening could change float32s in a trained model, so I don't think I think that, but maybe I'm parsing your words incorrectly.
      - pixel_popping 6mo ago
        failing non-stop at tool calls on top of that.
    - refulgentis 6mo ago
- dilap 6mo ago
  Deepseek R1 was a publically-available, MoE model that was getting a ton of attention before llama4. Llama4 didn't get much attention because it wasn't good.
  - jychang 6mo ago
    Also, Gemini 2.5 Pro launched a week before Llama 4. It was Gemini 2.5 Pro that redeemed Google in the eyes of most people as a valid competitor to OpenAI instead of as a joke, so Meta dropping the ball with Llama 4 was extra bad.
refulgentis 6mo ago
> 4. This model was out in the woods as early as like a couple months ago but they didn't release it because it was at gemini 2.5 pro levels. Source? (Even if rumor)
- prodigycorp 6mo ago
  It was from a techmeme ride home podcast where the host discussed "sources at the company said". I don't remember which day's episode it was.
- nl 6mo ago
  NYTimes had a story about this (March 12): > Meta’s new foundational A.I. model, which the company has been working on for months, has fallen short of the performance of leading A.I. models from rivals like Google, OpenAI and Anthropic on internal tests for reasoning, coding and writing, said the people, who were not authorized to speak publicly about confidential matters. > The model, code-named Avocado, outperformed Meta’s previous A.I. model and did better than Google’s Gemini 2.5 model from March, two of the people said. But it has not performed as strongly as Gemini 3.0 from November, they said. > They added that the leaders of Meta’s A.I. division had instead discussed temporarily licensing Gemini to power the company’s A.I. products, though no decisions have been reached. https://www.nytimes.com/2026/03/12/technology/meta-avocado-ai-model-delayed.html https://www.nytimes.com/2026/03/12/technology/meta-avocado-a... https://archive.is/uUV5h#selection-715.98-715.277 https://archive.is/uUV5h#selection-715.98-715.277
  - o10449366 6mo ago
    [flagged]
    - mupuff1234 6mo ago
      Does Meta not harvest data on a massive scale? Not sure what exactly is the issue with doing a series on that.
    - nl 6mo ago
      If you are trying to come up with anti-media conspiracies there are always plenty of ways to do it against any media company. The idea that NY Times is particularly anti-Meta seems a stretch. They - like most traditional media companies - are anti-tech in general. The fact they also collect data doesn't make their reporting untrue. Personally I think a much more interesting rumor to make up would be that Yann Lecun (who famously had his reporting lines rearranged to go through Alexander Wang after Scale.ai acquihire) works at New York University. New York University is in the same place as the New York Times. There's a conspiracy for you. I made it up, but I mean it could be true I guess? (Of course Lecun also publicly congratulated Wang on the launch of the model. But maybe that's a ruse to hide everything.. blah blah)

redox99 6mo ago

> If it slightly beats or even matches Opus 4.6 It doesn't though

ryeguy_24 6mo ago
Curious on why you think this. Any data points that led you to this?
- howdareme 6mo ago
  The benchmarks they released
  - johnfn 6mo ago
    What do you mean? In most cases, the benchmarks show a larger number for Muse and a smaller number for Opus.
    - spprashant 6mo ago
      In Multimodal yes, but Opus is definitely edging out in Text/Reasoning and Agentic benchmarks. I think the general skepticism is because they are late to race, and they are releasing a Opus-4.6-equivalent model now, when Anthropic is teasing Mythos.

ChipopLeMoral 6mo ago

> I don't get the comments trashing this. People like to hate on Meta regardless of anything, and regardless of whether it's justified or not. Not saying it isn't, just that it's many people's default bias.

jatora 6mo ago
That is not the case here. Nobody hated on llama 1,2,3 at all. They justifiably felt burned by the benchmaxxing of llama 4. Trust broken must be re-earned, and benchmarks alone cannot do that.

modeless 6mo ago

It's a decent model if the benchmarks are to be believed, but it won't be close to Opus in usefulness for programming. None of these benchmarks completely capture what makes a model useful for day-to-day coding tasks, unfortunately. It will take time for them to catch up, and Opus will keep improving in the meantime. But it's good to have more competition.

ai5iq 6mo ago
Benchmarks miss the thing that actually matters for agentic use: how does behavior change over a multi-day horizon? A model that scores well on one-shot coding tasks can still make terrible decisions when it has persistent state and resource constraints. That's where you see the real gaps between models.
- andai 6mo ago
  Is there a benchmark for these long tasks? That kind of seems like the only number worth measuring. (Of course at that point it involves memory and context management and so on, so you're testing the harness as well as the model.)
  - ai5iq 6mo ago
    [dead]

blazespin 6mo ago

Because bots and trillion dollar ipos and even bigger stakes. People need to better appreciate the level of manipulation going on. Social media has an outsized impact. Bots and even people are getting paid to post and upvote/downvote narratives.

asdfman123 6mo ago
> people are getting paid to post and upvote/downvote narratives This problem will be solved shortly with better AI (if it hasn't essentially been solved already). No more humans in the loop, much lower costs for social media manipulation. Welcome to the future!

canes123456 6mo ago

Why go into coding agents? Both anthropic and OpenAI are going all in on that. The opportunity is customer facing AI now. OpenAI has the mindshare but they going to have to decide if they allocate their limited compute for free users or go all in trying to keep up with Anthropic in enterprise.

foobiekr 6mo ago
Because coding agents are where the revenue is.
kaycey2022 6mo ago
you can do way more than just coding with the coding agents.
refulgentis 6mo ago
If you squint at coding agents you see the next OS. Maybe better phrasing is “HCI paradigm”, but that somehow manages to say everything and nothing.
- whattheheckheck 6mo ago
  Programming was always about designing rube goldberg systems that did a complicated state machine akin to dominos but now we have a probabalistic and nondeterministic domino that has a huge amount of dominos inside amd can dynamically generate many different paths of dominos sometimes not even leading to the intended final domino you wanted to fall. I see it more like a compiler
  - RealStupidity 6mo ago
    I agree that it's more like a compiler (turns higher level language into machine code) but I also think that's only half the story - a compiler could never turn requirements into functional software, generate boilerplate or debug. It's also a development tool
- deleted 6mo ago
  [deleted]