5 ms·

Seems like a mostly reasonable list of things to not let AI do without better safety evals. > AI that tries to infer people’s emotions at work or school I won

Seems like a mostly reasonable list of things to not let AI do without better safety evals. > AI that tries to infer people’s emotions at work or school I wonder how broadly this will be construed. For example if an agent uses CoT and they needs emotional state as part of that, can it be used in a work or school setting at all?

stavros 2y ago
The EU generally (so far) has passed reasonable legislation about these things. I'd be surprised if it was taken more broadly than the point where a reasonable person would feel comfortable with it.
Zenst 2y ago
I would imagine that such a tool to infer emotional states would be most useful for autistic people who are as I can attest, somewhat handicapped upon that front. Maybe that will get challenged as disability discrimination by some Autistic group. Which would be interesting. As with most things, there are rules, and exceptions to those rules - no shoe fits everyone, though forcing people to wear the wrong shoe size, can do more harm than good.
- danielheath 2y ago
  > I would imagine that such a tool to infer emotional states would be most useful for autistic people who are as I can attest, somewhat handicapped upon that front. It might well be a useful tool to point at yourself. It's an entirely inappropriate one to point at someone else. If you can't imagine having someone estimate your emotional state (usually incorrectly), and use that as a basis to disregard your opinion, you've lived a very different life to mine. Don't let them hide behind "the AI agreed with my assessment".
- cwillu 2y ago
  On the other hand, as someone who's emotional state is routinely incorrectly assessed by people, I can't imagine a worse hell than having that misassessment codified into an ai that I am required to interact with.
- Mordisquitos 2y ago
  > I would imagine that such a tool to infer emotional states would be most useful for autistic people who are as I can attest, somewhat handicapped upon that front. The regulation explicitly provides an exception for medical reasons: Article 5: 1. The following AI practices shall be prohibited: [...] (f) the placing on the market, the putting into service for this specific purpose, or the use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions, except where the use of the AI system is intended to be put in place or into the market for medical or safety reasons;
- pjc50 2y ago
  I can definitely find you autistic people who would hate having such a device pointed at them, because they don't mask the ""correct"" emotional state well enough.
  - Zenst 2y ago
    Was on about Autistic people being able to use it to understand the other person, not the other way around, which would be a nightmare for them, which sadly is what they get in real life anyhow already. Though might be useful to warn them that they are sending the wrong signals, so a mixed bag on that way around.
layer8 2y ago
This quote is inaccurate. The actual wording is: "the placing on the market, the putting into service for this specific purpose, or the use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions, except where the use of the AI system is intended to be put in place or into the market for medical or safety reasons;" and it links to https://artificialintelligenceact.eu/recital/44/ https://artificialintelligenceact.eu/recital/44/ for rationale. So, this targets the use case of a third party using AI to detect the emotional state of a person.
- dmix 2y ago
  Is this just based on hypothetical scenario they sat in a room coming up with or has such a thing been tried and harmed people?
  - nielsole 2y ago
    While this is not primarily emotion, I hope it would squarely be covered by it: https://bigthink.com/the-present/attention-headbands/ https://bigthink.com/the-present/attention-headbands/
  - jorisboris 2y ago
    I once tried building a Macbook app which lowers my music volume when I'm not smiling, to try to stimulate more happy emotions (forcing yourself to smile induces happy feelings) Then I started thinking how this could be used in restaurants to see if waiters smile to the people they are serving Or in customer service (you can actually hear it when people smile on the phone) Then I realised that this kind of tech would definitely lead to abuse (btw that's not the reason I didn't build it, it was just not that easy to build)
    - zelphirkalt 2y ago
      We cannot extend your work contract, based on your performance the last week. You definitely did not smile enough. Our systems indicate, that you were also not happy enough a sufficient portion of the time you worked.
deleted 2y ago
[deleted]
sofixa 2y ago
> Seems like a mostly reasonable list of things to not let AI do without better safety evals. Yes. This is how you know that all the people screaming about the EU overregulating and how the EU will miss all that AI innovation haven't even bothered to Google or ask their preferred LLM about the legislation. It's mostly just common sense to avoid EU citizens having their rights or lives decided by blackbox algorithms nobody can explain, be it in a Post Office (UK) scandal style, or US healthcare style.
__MatrixMan__ 2y ago
A difficulty I have with customer service folk is that usually I'm just trying to report a bug. I'm not upset. Please stop trying to give me coupons. I'm not trying to cancel my account I just want to help your engineers fix this bug (and later on, I want to see that it has actually gone away). If I must intact with an AI for this, I'd prefer that it infer my emotions correctly.

stavros 2y ago

The EU generally (so far) has passed reasonable legislation about these things. I'd be surprised if it was taken more broadly than the point where a reasonable person would feel comfortable with it.

Zenst 2y ago

I would imagine that such a tool to infer emotional states would be most useful for autistic people who are as I can attest, somewhat handicapped upon that front. Maybe that will get challenged as disability discrimination by some Autistic group. Which would be interesting. As with most things, there are rules, and exceptions to those rules - no shoe fits everyone, though forcing people to wear the wrong shoe size, can do more harm than good.

danielheath 2y ago
> I would imagine that such a tool to infer emotional states would be most useful for autistic people who are as I can attest, somewhat handicapped upon that front. It might well be a useful tool to point at yourself. It's an entirely inappropriate one to point at someone else. If you can't imagine having someone estimate your emotional state (usually incorrectly), and use that as a basis to disregard your opinion, you've lived a very different life to mine. Don't let them hide behind "the AI agreed with my assessment".
cwillu 2y ago
On the other hand, as someone who's emotional state is routinely incorrectly assessed by people, I can't imagine a worse hell than having that misassessment codified into an ai that I am required to interact with.
Mordisquitos 2y ago
> I would imagine that such a tool to infer emotional states would be most useful for autistic people who are as I can attest, somewhat handicapped upon that front. The regulation explicitly provides an exception for medical reasons: Article 5: 1. The following AI practices shall be prohibited: [...] (f) the placing on the market, the putting into service for this specific purpose, or the use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions, except where the use of the AI system is intended to be put in place or into the market for medical or safety reasons;
pjc50 2y ago
I can definitely find you autistic people who would hate having such a device pointed at them, because they don't mask the ""correct"" emotional state well enough.
- Zenst 2y ago
  Was on about Autistic people being able to use it to understand the other person, not the other way around, which would be a nightmare for them, which sadly is what they get in real life anyhow already. Though might be useful to warn them that they are sending the wrong signals, so a mixed bag on that way around.

layer8 2y ago

This quote is inaccurate. The actual wording is: "the placing on the market, the putting into service for this specific purpose, or the use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions, except where the use of the AI system is intended to be put in place or into the market for medical or safety reasons;" and it links to https://artificialintelligenceact.eu/recital/44/ https://artificialintelligenceact.eu/recital/44/ for rationale. So, this targets the use case of a third party using AI to detect the emotional state of a person.

dmix 2y ago
Is this just based on hypothetical scenario they sat in a room coming up with or has such a thing been tried and harmed people?
- nielsole 2y ago
  While this is not primarily emotion, I hope it would squarely be covered by it: https://bigthink.com/the-present/attention-headbands/ https://bigthink.com/the-present/attention-headbands/
- jorisboris 2y ago
  I once tried building a Macbook app which lowers my music volume when I'm not smiling, to try to stimulate more happy emotions (forcing yourself to smile induces happy feelings) Then I started thinking how this could be used in restaurants to see if waiters smile to the people they are serving Or in customer service (you can actually hear it when people smile on the phone) Then I realised that this kind of tech would definitely lead to abuse (btw that's not the reason I didn't build it, it was just not that easy to build)
  - zelphirkalt 2y ago
    We cannot extend your work contract, based on your performance the last week. You definitely did not smile enough. Our systems indicate, that you were also not happy enough a sufficient portion of the time you worked.

deleted 2y ago

[deleted]

sofixa 2y ago

> Seems like a mostly reasonable list of things to not let AI do without better safety evals. Yes. This is how you know that all the people screaming about the EU overregulating and how the EU will miss all that AI innovation haven't even bothered to Google or ask their preferred LLM about the legislation. It's mostly just common sense to avoid EU citizens having their rights or lives decided by blackbox algorithms nobody can explain, be it in a Post Office (UK) scandal style, or US healthcare style.

__MatrixMan__ 2y ago

A difficulty I have with customer service folk is that usually I'm just trying to report a bug. I'm not upset. Please stop trying to give me coupons. I'm not trying to cancel my account I just want to help your engineers fix this bug (and later on, I want to see that it has actually gone away). If I must intact with an AI for this, I'd prefer that it infer my emotions correctly.