5 ms·

I can't fathom how anyone thought this was a sensible thing. It's so bad I have to wonder if there's a different angle here, maybe they think that releasing so

I can't fathom how anyone thought this was a sensible thing. It's so bad I have to wonder if there's a different angle here, maybe they think that releasing something so terribly bad will make it easier when they release something less comically bad? Idk.

brookst 2y ago
Fact is it is hard to make a great product. I don’t think we need complex conspiracy theories to explain bad ones. Odds are the product team cherry picked conversations for internal demos, their management wanted press and promotions, KPIs were around engagement and not quality, and nobody had perspective and authority to say “this sucks, we can’t release it”.
- rsynnott 2y ago
  It’s not hard to identify, before launching, “this is unacceptably bad”. Like, I think this is actually quite a bit worse than Microsoft Tay, which others mentioned, as an example of “how could you possibly launch this”? Tay was reckless, and you’ve got to assume that they knew there was at least a change it could work out as it did, but in this case the product _as launched_ was clearly unacceptable; it didn’t require input from 4chan to make it so. I mean, just very basic QA, having someone talk to the damn things for a bit, would’ve shown the problem, in this case.
  - TheNewsIsHere 2y ago
    I’ve always believed Microsoft Tay was a really idealistic, genuine, if bombastically naïve situation.
  - brookst 2y ago
    I wish I agreed. But I have seen many products launched where only true believers who internalized the limitations were involved in testing. Yes, a good org with self-critical people and an independent red team and execs who cared about quality would not have made this blunder. Getting that org stuff right sounds easy but is nigh impossible if it doesn’t align to the company culture.
    - ryandrake 2y ago
      Critics and naysayers tend to leave teams where they don't believe in the product, and eventually teams are left with only true believers. Their entire purpose is to ship Project X, and if they were to admit Project X is a blunder, then they are admitting that their purpose is a blunder. Few teams at few companies are willing to do that.
      - brookst 2y ago
        Exactly. That’s why formal and independent compliance / security / red team signoffs are so important. A good product team wants those checks and balances.
      - lukan 2y ago
        "and eventually teams are left with only true believers." Or with nihilists who learned to say yes to everything, to at least get paid.
  - prymitive 2y ago
    Isn’t it as simple as management wanting to be first one across the release line, with the wishful thinking of “we’ll fix everything later”?
giantrobot 2y ago
The next step will be to release bots that aren't labeled as bots. They'll be influencers (advertisers) without having to pay a person. They'll produce hyper targeted influencer slop, hocking products directly to individuals using Facebook's knowledge graphs of users to be incredibly manipulative. Companies will pay Facebook to make people siphon money directly to them. It'll be like entirely automated catfishing.
- nonfamous 2y ago
  1. Create AI bot 2. Wait until it gets lots of followers 3. Sell mentions of products by popular AI bot. Profit. Sure, they faltered at Step 2, but how was this not the plan?
  - giantrobot 2y ago
    The future bots won't need followers. They will reach out to engage users based on their ad graph. Like how catfish accounts work. They'll reach out to users and talk to them about whatever they're interested in. They'll then make influencer-style native advertisements. If you're a middle aged man that likes video games when you see their picture posts they'll be "wearing" some vintage game t-shirt (that's for sale). If you're instead a twenty year old woman into yoga the same bit will be "wearing" some new Lululemon yoga pants. The first pass of these bots failed because they used the follower mechanism. The next version will just follow you to push ads or scrape more data about you.
TheNewsIsHere 2y ago
I find myself generally wondering the same thing about half the things coming out of SV over the past several years. I have worked with competent product organizations. I know they exist. A few even exist in SV. But for some reason the largest players have just absolutely lost the plot beyond what can get them to favorable quarterly earnings, and that game eventually and fairly consistently doesn’t end well in the long term. Meta in particular is clearly rudderless, lacking in vision and strategic leadership, and throwing whatever it can find at the wall hoping to find something that sticks. Facebook turned into a cesspool, Instagram isn’t as popular with the newest demographics Meta has traditionally wanted to court (young people), and Threads turned out to be a nothing burger. Their grand quest to unify messaging was a disaster, and mores the putty because they’ve basically delineated their ecosystem by demographic generation in what has to be an unintentional series of missteps. Their “metaverse” projects weren’t even compelling to the teams working on them, their foray into crypto was over almost as soon as it began, and their business products are a nearly unusable mess for those of us who have had the misfortune of using them. You’d think eventually someone at Meta would hit upon an idea that goes somewhere. It’s like a more pathetic version of Google’s stagnation.
- ryandrake 2y ago
  Stonk prices are going up, so whatever they are doing, the only real feedback they care about is positive. These AI products are not made for users. They're made for Wall Street. Wall Street expects big companies to 1. talk about AI in their earnings calls, and then 2. do something, anything, with AI. All of BigTech seems to be doing this now, and investors are rewarding them by buying their stock. So they're going to continue the cycle of building useless AI products and then canceling them when they have served their purpose (pleasing investors).
- jpc0 2y ago
  We aren't a big org and I know we spend several hundred if not thousands of USD a month on meta ads and WhatsApp business. For larger orgs I can imagine that number is larger and we definitely get a ton of positive interaction from that. That might be swayed by demographics but I can tell you age isn't one of them because our interaction (on WhatsApp Business which is mostly where I interact with Meta products) is decently in the 20-40 range.
NeutralCrane 2y ago
Working in ML Engineering this doesn’t surprise me in the least. The ratio between “impressiveness of technology” to “the number of practical problems actually solved” is probably higher for LLMs than just about any tech to come out in my career, probably longer. Every executive at every company is tripping over themselves to incorporate “AI” (which is almost exclusively defined as LLMs, for some reason) into their products. Problem is that many, if not most, companies really don’t have meaningful use cases for LLMs. So you end up with a bunch of problems being invented so they can be solved with AI. What feature can Facebook provide for their users utilizing LLMs that users actually, genuinely care about? I can’t think of any. And I assume Facebook can’t either, which is how they arrived at this.
- rsynnott 2y ago
  > Every executive at every company is tripping over themselves to incorporate “AI” (which is almost exclusively defined as LLMs, for some reason) into their products I think a hangover from the last AI winter, after which everything that would previously have been called ‘AI’ got called ‘ML’, the term ‘AI’ then being shareholder poison. Anything older than the current AI bubble mostly gets called ‘ML’ still, with a possible exception for non-LLMish CV stuff.

brookst 2y ago

Fact is it is hard to make a great product. I don’t think we need complex conspiracy theories to explain bad ones. Odds are the product team cherry picked conversations for internal demos, their management wanted press and promotions, KPIs were around engagement and not quality, and nobody had perspective and authority to say “this sucks, we can’t release it”.

rsynnott 2y ago
It’s not hard to identify, before launching, “this is unacceptably bad”. Like, I think this is actually quite a bit worse than Microsoft Tay, which others mentioned, as an example of “how could you possibly launch this”? Tay was reckless, and you’ve got to assume that they knew there was at least a change it could work out as it did, but in this case the product _as launched_ was clearly unacceptable; it didn’t require input from 4chan to make it so. I mean, just very basic QA, having someone talk to the damn things for a bit, would’ve shown the problem, in this case.
- TheNewsIsHere 2y ago
  I’ve always believed Microsoft Tay was a really idealistic, genuine, if bombastically naïve situation.
- brookst 2y ago
  I wish I agreed. But I have seen many products launched where only true believers who internalized the limitations were involved in testing. Yes, a good org with self-critical people and an independent red team and execs who cared about quality would not have made this blunder. Getting that org stuff right sounds easy but is nigh impossible if it doesn’t align to the company culture.
  - ryandrake 2y ago
    Critics and naysayers tend to leave teams where they don't believe in the product, and eventually teams are left with only true believers. Their entire purpose is to ship Project X, and if they were to admit Project X is a blunder, then they are admitting that their purpose is a blunder. Few teams at few companies are willing to do that.
    - brookst 2y ago
      Exactly. That’s why formal and independent compliance / security / red team signoffs are so important. A good product team wants those checks and balances.
    - lukan 2y ago
      "and eventually teams are left with only true believers." Or with nihilists who learned to say yes to everything, to at least get paid.
- prymitive 2y ago
  Isn’t it as simple as management wanting to be first one across the release line, with the wishful thinking of “we’ll fix everything later”?

giantrobot 2y ago

The next step will be to release bots that aren't labeled as bots. They'll be influencers (advertisers) without having to pay a person. They'll produce hyper targeted influencer slop, hocking products directly to individuals using Facebook's knowledge graphs of users to be incredibly manipulative. Companies will pay Facebook to make people siphon money directly to them. It'll be like entirely automated catfishing.

nonfamous 2y ago
1. Create AI bot 2. Wait until it gets lots of followers 3. Sell mentions of products by popular AI bot. Profit. Sure, they faltered at Step 2, but how was this not the plan?
- giantrobot 2y ago
  The future bots won't need followers. They will reach out to engage users based on their ad graph. Like how catfish accounts work. They'll reach out to users and talk to them about whatever they're interested in. They'll then make influencer-style native advertisements. If you're a middle aged man that likes video games when you see their picture posts they'll be "wearing" some vintage game t-shirt (that's for sale). If you're instead a twenty year old woman into yoga the same bit will be "wearing" some new Lululemon yoga pants. The first pass of these bots failed because they used the follower mechanism. The next version will just follow you to push ads or scrape more data about you.

TheNewsIsHere 2y ago

I find myself generally wondering the same thing about half the things coming out of SV over the past several years. I have worked with competent product organizations. I know they exist. A few even exist in SV. But for some reason the largest players have just absolutely lost the plot beyond what can get them to favorable quarterly earnings, and that game eventually and fairly consistently doesn’t end well in the long term. Meta in particular is clearly rudderless, lacking in vision and strategic leadership, and throwing whatever it can find at the wall hoping to find something that sticks. Facebook turned into a cesspool, Instagram isn’t as popular with the newest demographics Meta has traditionally wanted to court (young people), and Threads turned out to be a nothing burger. Their grand quest to unify messaging was a disaster, and mores the putty because they’ve basically delineated their ecosystem by demographic generation in what has to be an unintentional series of missteps. Their “metaverse” projects weren’t even compelling to the teams working on them, their foray into crypto was over almost as soon as it began, and their business products are a nearly unusable mess for those of us who have had the misfortune of using them. You’d think eventually someone at Meta would hit upon an idea that goes somewhere. It’s like a more pathetic version of Google’s stagnation.

ryandrake 2y ago
Stonk prices are going up, so whatever they are doing, the only real feedback they care about is positive. These AI products are not made for users. They're made for Wall Street. Wall Street expects big companies to 1. talk about AI in their earnings calls, and then 2. do something, anything, with AI. All of BigTech seems to be doing this now, and investors are rewarding them by buying their stock. So they're going to continue the cycle of building useless AI products and then canceling them when they have served their purpose (pleasing investors).
jpc0 2y ago
We aren't a big org and I know we spend several hundred if not thousands of USD a month on meta ads and WhatsApp business. For larger orgs I can imagine that number is larger and we definitely get a ton of positive interaction from that. That might be swayed by demographics but I can tell you age isn't one of them because our interaction (on WhatsApp Business which is mostly where I interact with Meta products) is decently in the 20-40 range.

NeutralCrane 2y ago

Working in ML Engineering this doesn’t surprise me in the least. The ratio between “impressiveness of technology” to “the number of practical problems actually solved” is probably higher for LLMs than just about any tech to come out in my career, probably longer. Every executive at every company is tripping over themselves to incorporate “AI” (which is almost exclusively defined as LLMs, for some reason) into their products. Problem is that many, if not most, companies really don’t have meaningful use cases for LLMs. So you end up with a bunch of problems being invented so they can be solved with AI. What feature can Facebook provide for their users utilizing LLMs that users actually, genuinely care about? I can’t think of any. And I assume Facebook can’t either, which is how they arrived at this.

rsynnott 2y ago
> Every executive at every company is tripping over themselves to incorporate “AI” (which is almost exclusively defined as LLMs, for some reason) into their products I think a hangover from the last AI winter, after which everything that would previously have been called ‘AI’ got called ‘ML’, the term ‘AI’ then being shareholder poison. Anything older than the current AI bubble mostly gets called ‘ML’ still, with a possible exception for non-LLMish CV stuff.