5 ms·

Once again, we see Meta being more open than OpenAI. I’m loving that their business incentive is aligned with open sourcing and commodifying state-of-the-art LL

Once again, we see Meta being more open than OpenAI. I’m loving that their business incentive is aligned with open sourcing and commodifying state-of-the-art LLM technology. Keep em coming

fragmede 2y ago
don't buy their bullshit. it's not open source.
- speedgoose 2y ago
  Yes it’s more about open weights. I also think that you would need the training data to consider it open source. Open weights is still appreciated and they probably train on data they don’t have the license to open source.
- astrange 2y ago
  I'm not sure open source is a useful concept for something that takes millions of dollars to compile from it.
blackeyeblitzar 2y ago
I mean they have no way to monetize LLMs as well as others, so they’re working on it and giving it away to not look irrelevant and to weaken anyone who may make money off this tech and threaten them in the future. Meanwhile there is a danger they impose their long standing invisible “moderation” on everyone else once they starve all the startups of revenue by giving this away. We’ll just be left with the same big tech overlords to choose from. Oh and it still isn’t open source even though people like Yann LeCun dishonestly claim it is. Only OLMo is truly open source among competitive models, as far as I know: https://allenai.org/blog/olmo2 https://allenai.org/blog/olmo2
- spencerflem 2y ago
  Facebook would rather do no moderation, it's an expense for them. They do it to make the platform more pleasant so that people stay on it
  - blackeyeblitzar 2y ago
    No they do it to support their owners’ and employees’ biases. It doesn’t make the platform more pleasant for the half that gets censored. That’s leaving aside the feed not remembering the choice to view chronologically ordered posts, the inability to easily track actual people in my life, the addictive algorithms, the clickbait that causes mental health issues for teens, etc.
    - creato 2y ago
      99% of FB's moderation has nothing to do with "biases", unless you think FB is biased against spam, scams, and all the other dregs of the internet that incessantly pops up anywhere users can post content.
      - TheOtherHobbes 2y ago
        Quite a few people left Threads for Bluesky because progressive posts were being removed while far-right, antivax, etc content was allowed to stand even though it was reported. At best the algo is imperfect. At worst it really does seem oddly selective.
      - dudeinjapan 2y ago
        I am a humble Cialis salesman, like my father and grandfather before me. I confirm Facebook is biased against our profession. (My grandfather also moonlighted as a Barrister representing the estates of deceased African royalty—it was always so difficult to track down their heirs.)
    - roywiggins 2y ago
      The stuff that Facebook moderators are actually tasked with removing is really awful, bad enough to produce severe psychological effects in the moderators. Facebook pays people to look at and remove this stuff because the platform would not survive if it wasn't removed before you or I saw it. Do they also enforce other corporate values? Yeah, probably. That doesn't seem to be the main job though, they have their hands full dealing with the worst content in the world. https://amp-theguardian-com.cdn.ampproject.org/v/s/amp.theguardian.com/media/2024/dec/18/kenya-facebook-moderators-sue-after-diagnoses-of-severe-ptsd https://amp-theguardian-com.cdn.ampproject.org/v/s/amp.thegu... > The images and videos including necrophilia, bestiality and self-harm caused some moderators to faint, vomit, scream and run away from their desks... > Some reported marriage breakdown and the collapse of desire for sexual intimacy, and losing connection with their families. Some whose job was to remove videos uploaded by terrorist and rebel groups were afraid they were being watched and targeted, and that if they returned home they would be hunted and killed.
- BoorishBears 2y ago
  > they have no way to monetize LLMs as well as others Random nobodies are putting together companies to monetize generative AI and getting bought out a couple of years later, you think Meta couldn't figure out how to deploy their own models to an API and stick up a billing interface if they really wanted to? (or even buy a company that does already?) > they starve all the startups of revenue by giving this away Would you say startups like Deepseek have been hurt or help by their (even partial) openness? In fact, how does this track with your first statement? They're not monetizing this: so their startup competition can actually serve their models to gain revenue which they then turn around use to train competitor models (we've already seen this with Fireworks.ai) You seem to underestimate how much of the value in LLMs is productizing them. The margins on per-token usage are insane, Meta not taking that margin is creating a huge opportunity for a wave of startups in so many directions... > Only OLMo is truly open source among competitive models Synthetic data from competitor models was a huge part of that. It would seem no one is fighting the startups as hard as you're claiming they are.
  - bongodongobob 2y ago
    All the LLM companies are going to eat those "product companies" lunch in a few years. Why would I use product X when it's going to be inevitably be baked into the actual tech itself? Those product companies are just wrappers and have even less of a moat than the LLM companies. The very fact that random nobodies are doing this should signal there isn't a lot of real value there. Yes, there is some money to be made right now but it reminds me a lot of the videogame bust and dotcom bust. A LOT of companies are wasting a crazy amount of money on "solutions" that will be obsolete in a few years.
    - BoorishBears 2y ago
      Productization in this context is creating APIs for Meta's models. Fireworks.ai, Together.ai, and literal boatloads of other startups are making real money just efficiently serving up these models that Meta is supposedly using to... choke out startups. The comment I replied to is under the mistaken idea that the presence of free models from Meta has a chilling effect on startups trying to build their own models, but right now the biggest barriers are capital and data. Meta updated Llama to allow for synthetic generation, and they're even partnering with these startups give them distribution and day 0 access to the models. - If anything I'd say Meta is actively fighting against the big tech overlords the comment thinks they're trying to join. Even before Ilya mentioned it, it was clear to me that the power of post-training was going to become more and more important (I've literally built a business on it). Llama represents a real ongoing chance for tiny startups with minimal resources to get into the fray very affordably (through either offering inference, or post-training for a specific task, etc.), scale revenue, and then start to compete against much larger, resource rich companies.
- jayd16 2y ago
  Is there any vendor lock-in with this conspiracy? Even if startups are pushed out of the spotlight, what stops them from competing? If the meta model is bad, won't it be even easier to make an alternative in the future?
- scarface_74 2y ago
  They are definitely making some money off of their licensing to AWS as part of the bedrock offering. Facebook’s licensing is such that they aren’t going to let happen to them what happened to ElasticSearch, Redis, etc. I’m okay with that.
- rlupi 2y ago
  In the agentic era, the new Ads eyeballs are the LLMs training corpus (IMHO).

Once again, we see Meta being more open than OpenAI. I’m loving that their business incentive is aligned with open sourcing and commodifying state-of-the-art LLM technology. Keep em coming

fragmede 2y ago

don't buy their bullshit. it's not open source.

speedgoose 2y ago
Yes it’s more about open weights. I also think that you would need the training data to consider it open source. Open weights is still appreciated and they probably train on data they don’t have the license to open source.
astrange 2y ago
I'm not sure open source is a useful concept for something that takes millions of dollars to compile from it.

blackeyeblitzar 2y ago

I mean they have no way to monetize LLMs as well as others, so they’re working on it and giving it away to not look irrelevant and to weaken anyone who may make money off this tech and threaten them in the future. Meanwhile there is a danger they impose their long standing invisible “moderation” on everyone else once they starve all the startups of revenue by giving this away. We’ll just be left with the same big tech overlords to choose from. Oh and it still isn’t open source even though people like Yann LeCun dishonestly claim it is. Only OLMo is truly open source among competitive models, as far as I know: https://allenai.org/blog/olmo2 https://allenai.org/blog/olmo2

spencerflem 2y ago
Facebook would rather do no moderation, it's an expense for them. They do it to make the platform more pleasant so that people stay on it
- blackeyeblitzar 2y ago
  No they do it to support their owners’ and employees’ biases. It doesn’t make the platform more pleasant for the half that gets censored. That’s leaving aside the feed not remembering the choice to view chronologically ordered posts, the inability to easily track actual people in my life, the addictive algorithms, the clickbait that causes mental health issues for teens, etc.
  - creato 2y ago
    99% of FB's moderation has nothing to do with "biases", unless you think FB is biased against spam, scams, and all the other dregs of the internet that incessantly pops up anywhere users can post content.
    - TheOtherHobbes 2y ago
      Quite a few people left Threads for Bluesky because progressive posts were being removed while far-right, antivax, etc content was allowed to stand even though it was reported. At best the algo is imperfect. At worst it really does seem oddly selective.
    - dudeinjapan 2y ago
      I am a humble Cialis salesman, like my father and grandfather before me. I confirm Facebook is biased against our profession. (My grandfather also moonlighted as a Barrister representing the estates of deceased African royalty—it was always so difficult to track down their heirs.)
  - roywiggins 2y ago
    The stuff that Facebook moderators are actually tasked with removing is really awful, bad enough to produce severe psychological effects in the moderators. Facebook pays people to look at and remove this stuff because the platform would not survive if it wasn't removed before you or I saw it. Do they also enforce other corporate values? Yeah, probably. That doesn't seem to be the main job though, they have their hands full dealing with the worst content in the world. https://amp-theguardian-com.cdn.ampproject.org/v/s/amp.theguardian.com/media/2024/dec/18/kenya-facebook-moderators-sue-after-diagnoses-of-severe-ptsd https://amp-theguardian-com.cdn.ampproject.org/v/s/amp.thegu... > The images and videos including necrophilia, bestiality and self-harm caused some moderators to faint, vomit, scream and run away from their desks... > Some reported marriage breakdown and the collapse of desire for sexual intimacy, and losing connection with their families. Some whose job was to remove videos uploaded by terrorist and rebel groups were afraid they were being watched and targeted, and that if they returned home they would be hunted and killed.
BoorishBears 2y ago
> they have no way to monetize LLMs as well as others Random nobodies are putting together companies to monetize generative AI and getting bought out a couple of years later, you think Meta couldn't figure out how to deploy their own models to an API and stick up a billing interface if they really wanted to? (or even buy a company that does already?) > they starve all the startups of revenue by giving this away Would you say startups like Deepseek have been hurt or help by their (even partial) openness? In fact, how does this track with your first statement? They're not monetizing this: so their startup competition can actually serve their models to gain revenue which they then turn around use to train competitor models (we've already seen this with Fireworks.ai) You seem to underestimate how much of the value in LLMs is productizing them. The margins on per-token usage are insane, Meta not taking that margin is creating a huge opportunity for a wave of startups in so many directions... > Only OLMo is truly open source among competitive models Synthetic data from competitor models was a huge part of that. It would seem no one is fighting the startups as hard as you're claiming they are.
- bongodongobob 2y ago
  All the LLM companies are going to eat those "product companies" lunch in a few years. Why would I use product X when it's going to be inevitably be baked into the actual tech itself? Those product companies are just wrappers and have even less of a moat than the LLM companies. The very fact that random nobodies are doing this should signal there isn't a lot of real value there. Yes, there is some money to be made right now but it reminds me a lot of the videogame bust and dotcom bust. A LOT of companies are wasting a crazy amount of money on "solutions" that will be obsolete in a few years.
  - BoorishBears 2y ago
    Productization in this context is creating APIs for Meta's models. Fireworks.ai, Together.ai, and literal boatloads of other startups are making real money just efficiently serving up these models that Meta is supposedly using to... choke out startups. The comment I replied to is under the mistaken idea that the presence of free models from Meta has a chilling effect on startups trying to build their own models, but right now the biggest barriers are capital and data. Meta updated Llama to allow for synthetic generation, and they're even partnering with these startups give them distribution and day 0 access to the models. - If anything I'd say Meta is actively fighting against the big tech overlords the comment thinks they're trying to join. Even before Ilya mentioned it, it was clear to me that the power of post-training was going to become more and more important (I've literally built a business on it). Llama represents a real ongoing chance for tiny startups with minimal resources to get into the fray very affordably (through either offering inference, or post-training for a specific task, etc.), scale revenue, and then start to compete against much larger, resource rich companies.
jayd16 2y ago
Is there any vendor lock-in with this conspiracy? Even if startups are pushed out of the spotlight, what stops them from competing? If the meta model is bad, won't it be even easier to make an alternative in the future?
scarface_74 2y ago
They are definitely making some money off of their licensing to AWS as part of the bedrock offering. Facebook’s licensing is such that they aren’t going to let happen to them what happened to ElasticSearch, Redis, etc. I’m okay with that.
rlupi 2y ago
In the agentic era, the new Ads eyeballs are the LLMs training corpus (IMHO).