4 ms·

Important clarification: consumers are turned off by AI marketing material. I'd be interested to see real studies into consumer satisfaction with AI features i

Important clarification: consumers are turned off by AI marketing material. I'd be interested to see real studies into consumer satisfaction with AI features in existing products. My gut feeling is that people don't like (visible) AI in things they use, but that's biased by me reading online reporting about the failings of these features, I wouldn't be too surprised if it turns out people mostly like them.

afarviral 2y ago
I tend to agree. If you asked about a feature which uses some kind of ML under the hood to deliver that feature you'd different results. I think companies should urgently pull back on mentioning AI at all if they want any credibility at this point.
- viraptor 2y ago
  Yup. "do you like that you can search for a name in your photo album" will get different answers then asking about AI.
  - deleted 2y ago
    [deleted]
  - ksaj 2y ago
    That is a problem with Facebook, where people can tag you in images, even if you don't notice or approve it (you can opt out, but that's the opposite of what is optimal). Funny to see that come up again, but in a different context.
    - viraptor 2y ago
      There's lots of different implementations. For example Immich (https://immich.app/ https://immich.app/) is fully self-hosted and does local recognition. It doesn't have to be an online service stealing your data.
  - rsynnott 2y ago
    That was never previously marketed as 'AI', though (I suspect because when CV started seeing this sort of application, the memory of the _previous_ AI winter was still fairly fresh, so everyone carefully avoided the term). In marketing, 'AI' tends to mean _generative_ AI these days.
wokwokwok 2y ago
> I'd be interested to see real studies into consumer satisfaction with AI features in existing products mmm... I guess.... I mean, a feature that is not 'obviously AI' is just a feature; even if you normalized it against 'normal' features with no AI as part of them, surely the deviance from random noise would be negligible? In a double blind test, you would have three cohorts; control, placebo and treatment (ie. AI). Since the placebo and treatment groups would receive the same feature with or without AI, you'd be looking at absolutely nothing meaningful in the data you collect. > I wouldn't be too surprised if it turns out people mostly like them. How can users POSSIBLY have a positive / negative / ANY KIND of meaningful response to a feature based on the unknown backend implementation? If you put a chatGPT interface on your product and don't put an AI label on it, I guess most people, not being completely stupid, will not meaningfully distinguish between it and the equivalent feature with "Artificial Intelligence" painted on the side. An AI chat bot is an AI chat bot. You're not fooling anyone by calling it an 'intelligent assistant' instead of 'chatGPT'. :P I mean, I'm just guessing, you could study it. ...but, I guess it's probably not worth bothering. It's probably more likely the take-away here is: Make your product amazing; if it uses AI, hide that fact that it uses AI if you can.
- viraptor 2y ago
  > would receive the same feature with or without AI, I'm not sure how you'd do that, given many features wouldn't exist without AI. We don't know how to implement them that way. The only existing choices are "no feature" and "AI feature".
amluto 2y ago
I am certainly turned off by “AI” customer support. The ones I’ve encountered are actively terrible, and I’d rather click through a tree of first-level fix-it-yourself advice than get walked through it by a stunningly poor AI. But sure, it would be spiffy if Siri was more capable, as long as it didn’t become susceptible to injection attacks from my photo album in the process…
- ksaj 2y ago
  When The Weather Network first included an AI chat function, one of their sample questions was about stargazing. I asked the exact same question to see what kind of response it would give, and it said that it couldn't answer because weather does not affect star gazing. Really? I took a screencap, because this was such an epic example of why companies should actually test things before giving public access to them.
  - Piskvorrr 2y ago
    Can you actually test a component that is, by design, a black box? ("by design" of the current models. Yes, the systems could be re-made traceable. But then it would also turn out on how much pirated material the LLM has trained, and hoo boy would the IP lawyers show up. That's the con: "if it's a blackbox, you can't prove whence the corpus, nyah nyah!")
    - sensanaty 2y ago
      Our "testing" involves sending it 50 prompts and seeing if it bullshits too much and revising the prompt until it bullshits to acceptable levels. I suspect that's how the model makers test it as well, just fling crap at the wall and see which wall lets the most crap slide off of it (but never all of it)
      - ksaj 2y ago
        This is called Fuzzing. And it is a major component of software testing. Especially in infosec, but I digress. The reasoning behind it is exactly the same.
DelightOne 2y ago
Every time I hear it contains AI it sounds like the features' outcome will be uncertain. Especially with more experience. If your feature were good you wouldn't have to mention AI to show it is awesome. More than not AI is used as an excuse for the feature to be bad.
- deleted 2y ago
  [deleted]
keybored 2y ago
Okay then at what point do the users become reliable narrators? The AI cheerleaders can take comfort in that AI is in the hands of the average person now—but they also have to contend with any negative reactions as well. AI isn’t some behind the scenes technology (any more). And companies with their “AI” gimmick often highlight it with a star or something. People can chat with these things as easily as doing a Google search.
rsynnott 2y ago
I'd buy that people like "AI" features writ large; automatic image classification on your phone, say. However, I'm not convinced anyone much likes the existing applications of LLMs (mostly terrible obstructive chatbots, and blogspam).

afarviral 2y ago

I tend to agree. If you asked about a feature which uses some kind of ML under the hood to deliver that feature you'd different results. I think companies should urgently pull back on mentioning AI at all if they want any credibility at this point.

viraptor 2y ago
Yup. "do you like that you can search for a name in your photo album" will get different answers then asking about AI.
- deleted 2y ago
  [deleted]
- ksaj 2y ago
  That is a problem with Facebook, where people can tag you in images, even if you don't notice or approve it (you can opt out, but that's the opposite of what is optimal). Funny to see that come up again, but in a different context.
  - viraptor 2y ago
    There's lots of different implementations. For example Immich (https://immich.app/ https://immich.app/) is fully self-hosted and does local recognition. It doesn't have to be an online service stealing your data.
- rsynnott 2y ago
  That was never previously marketed as 'AI', though (I suspect because when CV started seeing this sort of application, the memory of the _previous_ AI winter was still fairly fresh, so everyone carefully avoided the term). In marketing, 'AI' tends to mean _generative_ AI these days.

wokwokwok 2y ago

> I'd be interested to see real studies into consumer satisfaction with AI features in existing products mmm... I guess.... I mean, a feature that is not 'obviously AI' is just a feature; even if you normalized it against 'normal' features with no AI as part of them, surely the deviance from random noise would be negligible? In a double blind test, you would have three cohorts; control, placebo and treatment (ie. AI). Since the placebo and treatment groups would receive the same feature with or without AI, you'd be looking at absolutely nothing meaningful in the data you collect. > I wouldn't be too surprised if it turns out people mostly like them. How can users POSSIBLY have a positive / negative / ANY KIND of meaningful response to a feature based on the unknown backend implementation? If you put a chatGPT interface on your product and don't put an AI label on it, I guess most people, not being completely stupid, will not meaningfully distinguish between it and the equivalent feature with "Artificial Intelligence" painted on the side. An AI chat bot is an AI chat bot. You're not fooling anyone by calling it an 'intelligent assistant' instead of 'chatGPT'. :P I mean, I'm just guessing, you could study it. ...but, I guess it's probably not worth bothering. It's probably more likely the take-away here is: Make your product amazing; if it uses AI, hide that fact that it uses AI if you can.

viraptor 2y ago
> would receive the same feature with or without AI, I'm not sure how you'd do that, given many features wouldn't exist without AI. We don't know how to implement them that way. The only existing choices are "no feature" and "AI feature".

amluto 2y ago

I am certainly turned off by “AI” customer support. The ones I’ve encountered are actively terrible, and I’d rather click through a tree of first-level fix-it-yourself advice than get walked through it by a stunningly poor AI. But sure, it would be spiffy if Siri was more capable, as long as it didn’t become susceptible to injection attacks from my photo album in the process…

ksaj 2y ago
When The Weather Network first included an AI chat function, one of their sample questions was about stargazing. I asked the exact same question to see what kind of response it would give, and it said that it couldn't answer because weather does not affect star gazing. Really? I took a screencap, because this was such an epic example of why companies should actually test things before giving public access to them.
- Piskvorrr 2y ago
  Can you actually test a component that is, by design, a black box? ("by design" of the current models. Yes, the systems could be re-made traceable. But then it would also turn out on how much pirated material the LLM has trained, and hoo boy would the IP lawyers show up. That's the con: "if it's a blackbox, you can't prove whence the corpus, nyah nyah!")
  - sensanaty 2y ago
    Our "testing" involves sending it 50 prompts and seeing if it bullshits too much and revising the prompt until it bullshits to acceptable levels. I suspect that's how the model makers test it as well, just fling crap at the wall and see which wall lets the most crap slide off of it (but never all of it)
    - ksaj 2y ago
      This is called Fuzzing. And it is a major component of software testing. Especially in infosec, but I digress. The reasoning behind it is exactly the same.

DelightOne 2y ago

Every time I hear it contains AI it sounds like the features' outcome will be uncertain. Especially with more experience. If your feature were good you wouldn't have to mention AI to show it is awesome. More than not AI is used as an excuse for the feature to be bad.

deleted 2y ago
[deleted]

keybored 2y ago

Okay then at what point do the users become reliable narrators? The AI cheerleaders can take comfort in that AI is in the hands of the average person now—but they also have to contend with any negative reactions as well. AI isn’t some behind the scenes technology (any more). And companies with their “AI” gimmick often highlight it with a star or something. People can chat with these things as easily as doing a Google search.

rsynnott 2y ago

I'd buy that people like "AI" features writ large; automatic image classification on your phone, say. However, I'm not convinced anyone much likes the existing applications of LLMs (mostly terrible obstructive chatbots, and blogspam).