20 ms·

AI systems with 'unacceptable risk' are now banned in the EU

• whereismyacc 2y ago
  sounds good
• adityamwagh 2y ago
  [flagged]
  • OKRainbowKid 2y ago
    That doesn't seem obvious to me. Would you mind elaborating?
  • dijksterhuis 2y ago
    > unacceptable risk applications — the focus of this month’s compliance requirements — will be prohibited entirely. Some of the unacceptable activities include: … what follows is a list of some pretty nasty and insidious use cases. it’s not “AI is completely banned”, it’s “consider the use cases you are working on responsibly”. only for those specific use cases, mind you. for all other use cases not in the list, which is a significantly larger subset of development, just ensure you do the required safety/regulatory sign off work. just like when we get our SaaS webapps evaluated for compliance to security standards, its just a box ticking exercise for the most part.
    • IMTDb 2y ago
      > AI that tries to infer people’s emotions at work or school. When I talk to ChatGPT advanced voice mode with a happy and upbeat tone, it replies similarly. If I talk to it in a lot neutral way it does adapt. The AI thus infers my emotions. I use ChatGPT at work, my company pays for it. Sounds like I should sue. Also, I am trying to implement a new policy for pull request in my tech team. We send an anonymous form to gather feedback. I sent all the responses in one block to ChatGPT and asked it to summarize the feedback. The AI indicated that “generally people seem pretty happy about the new policy”. Should I go to jail now for being clearly a deranged madmen according to the EU ?
      • dijksterhuis 2y ago
        from the actual act, which is linked in the article > the placing on the market, the putting into service for this specific purpose, or the use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions emphasis mine. chatgpt was not specifically put into service to monitor your emotions at work. so it’s fine i’d say. and your pull request thing is fine. also, you’re not trying to infer the emotions of any specific natural person. you’re trying to guage satisfaction of a process. that’s different to working out whether someone is feeling sad or feeling lonely in the workplace because they “aren’t smiling enough”. unfortunately that’s means you can’t sue and get a pay day. edit — i find it kind of funny that people are knee jerk reacting emotionally to this. kind of ironic when you consider the example at hand.
        • infecto 2y ago
          And its kind of funny when non-legal experts attempt to say something is fine or not. It depends highly on not only how its written but the spirit of what the EU is attempting to do. The knee jerk reaction is probably that historically institutions do a terrible job of writing rules and especially rules around new technology.
          • dijksterhuis 2y ago
            you are right to call me out. IANAL. on me.
            • IMTDb 2y ago
              An actual lawyer will probably agree with you. And also charge me to give me the same opinion. They will also say “but if you end up being sued, it’s not my fault”. So basically I just need to second guess everything I do, until someone somewhere gets sued and loses and another dude gets sued and loses. At that point we will have some idea about what the law really entails (at which point they change it and the cycle restarts) In the meantime, my US competitors are just moving full steam ahead.
            • 2y ago
  • user432678 2y ago
    I actually like it, moving to EU is currently my back-up plan when “great advances in AI” renders the rest of the world to make people loose jobs, give up privacy and overall make dystopian future a present. I’ll be happily coding my boilerplate code by hand, enjoying my boring life without AI.
    • joshstrange 2y ago
      The idea of the EU wouldn’t just follow suit seems kind of laughable. Or rather that the EU would be some bastion of safety from AI. If these “great advances in AI” happen then EU companies will just outsource their jobs, there will be no boilerplate/CRUD jobs.
      • user432678 2y ago
        Well, EU is not a single country. I can imagine Germany outsourcing such jobs to Poland. Oh well, I can move back “home” from UK then.
  • troupo 2y ago
    [flagged]
• _heimdall 2y ago
  What I don't see here is how the EU is actually defining what is and is not considered AI. > AI that manipulates a person’s decisions subliminally or deceptively. That can be a hugely broad category that covers any algorithmic feed or advertising platform. Or is this limited specifically to LLMs, as OpenAI has so successfully convinced us that LLMs really are Aai and previous ML tools weren't?
  • dijksterhuis 2y ago
    the actual text in the ~act~ guidance states: > Exploitation of vulnerabilities of persons, manipulation and use of subliminal techniques techcrunch simplified it. from my reading, it counts if you are intentionally setting out to build a system to manipulate or deceive people. edit — here’s the actual text from the act, which makes more clear it’s about whether the deception is purposefully intended for malicious reasons > the placing on the market, the putting into service or the use of an AI system that deploys subliminal techniques beyond a person’s consciousness or purposefully manipulative or deceptive techniques, with the objective, or the effect of materially distorting the behaviour of a person or a group of persons by appreciably impairing their ability to make an informed decision, thereby causing them to take a decision that they would not have otherwise taken in a manner that causes or is reasonably likely to cause that person, another person or group of persons significant harm
    • Bjartr 2y ago
      Seems like even a rudimentary ML model powering ad placements would run afoul of this.
      • dijksterhuis 2y ago
        > In addition, common and legitimate commercial practices, for example in the field of advertising, that comply with the applicable law should not, in themselves, be regarded as constituting harmful manipulative AI-enabled practices. https://artificialintelligenceact.eu/recital/29/ https://artificialintelligenceact.eu/recital/29/
• jalapenos 2y ago
  [flagged]
  • deleted 2y ago
    [deleted]
  • mdhb 2y ago
    [flagged]
    • Bancakes 2y ago
      Remind me of the European Silicon Valley and its greatest accomplishments?
      • deleted 2y ago
        [deleted]
      • micromacrofoot 2y ago
        the US currently has a tech oligarch employing teenagers to audit government finance software
      • deleted 2y ago
        [deleted]
      • HeatrayEnjoyer 2y ago
        Not actively overthrowing democracy and infecting society with an attention addiction industry.
      • ben_w 2y ago
        In addition to the european groups bought by US big tech, and all the European divisions of American-parented big tech firms? A quick search reveals DeepMind, Skype, SwiftKey, Shazam, Moodstocks for the former. Bit of overlap with the latter, too, as e.g. AlphaFold is from DeepMind after getting bought. Quick look on the Apple App store also gets me Komoot (Germany), Trade Republic (Germany), Revolut (UK), Babbel (Germany). Aside from them, ETH Zürich and CERN are doing pretty good work, too, the latter inventing the modern hypertext based web on which you are currently reading this. Cambridge has some decent digital tech, also has Metalysis and The Welding Institute and was where the double helix structure of DNA were found and where Stephen Hawking chose to work.
• mhitza 2y ago
  > AI that attempts to predict people committing crimes based on their appearance. Should have been > AI that attempts to predict people committing crimes
  • troupo 2y ago
    Instead of relying on Techcrunch and speculating, you could read sections (33), (42), and (59) of the EU AI Act yourself.
    • mhitza 2y ago
      Article 59 seems relevant, other two on a quick skim don't seem to relate to the subject. > 2. For the purposes of the prevention, investigation, detection or prosecution of criminal offences or the execution of criminal penalties, including safeguarding against and preventing threats to public security, under the control and responsibility of law enforcement authorities, the processing of personal data in AI regulatory sandboxes shall be based on a specific Union or national law and subject to the same cumulative conditions as referred to in paragraph 1. https://artificialintelligenceact.eu/article/59/ https://artificialintelligenceact.eu/article/59/ Seems like it allows pretty easily for national states to add in laws that allow them to skirt around
      • hcfman 2y ago
        Yep, they want to be able to continue to violate human rights and do the dirty.
      • cccbbbaaa 2y ago
        There is the same kind of language in GDPR (see article 2(2)D), but it still did not prevent this decision: https://www.laquadrature.net/en/2025/01/31/justice-finally-finds-briefcam-unlawful/ https://www.laquadrature.net/en/2025/01/31/justice-finally-f...
      • shortsunblack 2y ago
        EU law largely does not regulate national security matters of the states. Though that justification is limited per international law (such as ECHR, which is the basis of many anti-government surveillance rulings by CJEU). All European Union members are part of ECHR because that is a pre-requisite for EU membership. But ECHR is not part of EU law, especially it is not binding on the European Commission (in the context of it being a federal or seemingly federal political executive). This creates a catch-22 where member states might be violating ECHR but are mandated by EU law, though this is a very fringe consequence arising out of legal fiction and failed plans to federalize EU. Most recently, this legal fiction has become relevant in Chat Control discourse. Great Britain and Poland have explicit opt-outs out of some European law.
• ItsBob 2y ago
  > AI that attempts to predict people committing crimes based on their appearance. FTFY: AI that attempts to predict people committing crimes. By "appearance" are they talking about a guy wearing a hoodie must be a hacker or are we talking about race/colour/religious garb etc? I'd rather they just didn't use it for any kind of criminal application at all if I have a say in it! Just my $0.02
  • troupo 2y ago
    > I'd rather they just didn't use it for any kind of criminal application at all if I have a say in it! Instead of relying on Techcrunch and speculating, you could read sections (33), (42), and (59) of the EU AI Act yourself.
  • Oarch 2y ago
    It will be hard to gauge this too if it's all just vectors?
    • deleted 2y ago
      [deleted]
  • layer8 2y ago
    The actual wording is: "based solely on the profiling of a natural person or on assessing their personality traits and characteristics". The Techcrunch article oversimplifies and is borderline misleading.
  • stared 2y ago
    There was a joke: - Could you tell from an image if a man is gay? - Depending on what he is doing.
• rustc 2y ago
  Does this affect open weights AI releases? Or is the ban only on the actual use for the listed cases? Because you can use open weights Mistral models to implement probably everything on that list.
  • ben_w 2y ago
    Use and development. I know how to make chemical weapons in two distinct ways using only items found in a perfectly normal domestic kitchen, that doesn't change the fact that chemical weapons are in fact banned. """The legal framework will apply to both public and private actors inside and outside the EU as long as the AI system is placed on the Union market, or its use has an impact on people located in the EU. The obligations can affect both providers (e.g. a developer of a CV-screening tool) and deployers of AI systems (e.g. a bank buying this screening tool). There are certain exemptions to the regulation. Research, development and prototyping activities that take place before an AI system is released on the market are not subject to these regulations. Additionally, AI systems that are exclusively designed for military, defense or national security purposes, are also exempt, regardless of the type of entity carrying out those activities.""" - https://ec.europa.eu/commission/presscorner/detail/en/qanda_21_1683 https://ec.europa.eu/commission/presscorner/detail/en/qanda_...
  • hkwerf 2y ago
    The law specifically refers to putting a product in to service or placing a product on the market if they are on the prohibited (article 5) or high-risk (article 6, annex III) list and don't have an applicable exception. IANAL, but it's pretty obvious that a set of weights alone clearly is not such a product. Also note that the law has explicit exceptions for research, development, open source and personal use.
• dijksterhuis 2y ago
  link to the actual act: https://artificialintelligenceact.eu/article/5/ https://artificialintelligenceact.eu/article/5/ link to the q&a: https://ec.europa.eu/commission/presscorner/detail/en/qanda_21_1683 https://ec.europa.eu/commission/presscorner/detail/en/qanda_... (both linked in the article)
• DoingIsLearning 2y ago
  I am not expert but there seems to be an overlap in the article between 'AI' and well ... just software, or signal processing: - AI that collects “real time” biometric data in public places for the purposes of law enforcement. - AI that creates — or expands — facial recognition databases by scraping images online or from security cameras. - AI that uses biometrics to infer a person’s characteristics - AI that collects “real time” biometric data in public places for the purposes of law enforcement. All of the above can be achieved with just software, statistics, old ML techniques, i.e. 'non hype' AI kind of software. I am not familiar with the detail of the EU AI pact but it seems like the article is simplifying important details. I assume the ban is on the purpose/usage rather than whatever technology is used under the hood, right?
  • amelius 2y ago
    You can even replace AI by humans. For example, it is not legal for e.g. police officers to engage in racial profiling.
    • uniqueuid 2y ago
      No you cannot, see Article 2: Scope
  • spacemanspiff01 2y ago
    From the laws text: For the purposes of this Regulation, the following definitions apply: (1) ‘AI system’ means a machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments; Related: Recital 12 https://artificialintelligenceact.eu/article/3/ https://artificialintelligenceact.eu/article/3/ https://artificialintelligenceact.eu/recital/12/ https://artificialintelligenceact.eu/recital/12/ So, it seems like yes, software, if it is non-deterministic enough would qualify. My impression is that software that simply takes "if your income is below this threshold, we deny you a credit card." Would be fine, but somewhere along the line when your decision tree grows large enough, it probably changes.
• hcfman 2y ago
  So is a burglar alarm that uses AI to see a person on your property and alert the owner (Same function as the old PIR sensors) now AI tech that is predicting that there is a person that might be wanting to commit a crime and thus a banned use of AI now? Or is it something that is open to interpretation, let the courts sort of out and fine you 15,000,000 euros if someone in the EU has leverage on the courts and doesn't like you? Oh and the courts will already kill any small startup.
  • dijksterhuis 2y ago
    From the act: > to assess or predict the risk of a natural person committing a criminal offence, based solely on the profiling of a natural person or on assessing their personality traits and characteristics; more details in recital 42: https://artificialintelligenceact.eu/recital/42/ https://artificialintelligenceact.eu/recital/42/
    • DocTomoe 2y ago
      In all fairness, that would still ban a system that understands my front door being approached by a guy in a postal uniform wearing a parcel is likely unproblematic, while a guy wearing a hoodie and a crowbar is likely to commit a crime.
      • WrongAssumption 2y ago
        How so?
      • dathinab 2y ago
        no it doesn't because here you are accessing their actions (approaching your front door in some unusual context like holding a crow bar) in addition, having front doors idk. calling the police on people just because it's a "unusual" situation would be quite dystopian and would most likely for society as a whole lead to far more damage then it would prevent so instead of your door trying to "detect maybe soon to happen crime" it could "try to detect unusual situation which might require human actions" and then have the human (you someone or in a call center if you aren't available) do the actions (which might be just one or two button presses, nothing prevents you to take the action by directing the AI to do it for you) and lets not forget we are speaking about before the break in (and maybe no break in at all because it's actually a Halloween costume or similar), if the system detects someone braking in we have a action
• hcfman 2y ago
  Laws that are open to interpretation with drastic consequences if it's interpreted against your favour pose unacceptable risk to business investors and stifle innovation.
  • dns_snek 2y ago
    People said that about GDPR. Laws that don't leave any room for interpretation are bound to have loopholes that pose unacceptable risk to the population.
    • daedrdev 2y ago
      I think its quite clear gdpr has indeed lead to lower investment and delayed or cancelled products in Europe
      • deleted 2y ago
        [deleted]
      • jeffgreco 2y ago
        Yet better privacy protections than we in the States enjoy.
        • DocTomoe 2y ago
          The EU in a nutshell: You can't have nice things, but on the bright side Google/Apple/Facebook won't know what you had for dinner. Now give us your whole financial transaction and travel history, so we can share it with the US, a hostile country, citizen!
          • fredoliveira 2y ago
            Can we get some sources that back the narrative for that last sentence?
            • 2y ago
• hcfman 2y ago
  I would like to see a new law that puts any member of government found obstructing justice is put in jail. Except that the person responsible for travesty of justice framing 9 innocent people in this Dutch series is currently the president of the court of Maastricht. https://npo.nl/start/serie/de-villamoord https://npo.nl/start/serie/de-villamoord Remember. The courts have the say as to who wins and looses in these new vague laws. The ones running the courts have to not be corrupt. But the case above shows that this situation is in fact not the case.
  • deleted 2y ago
    [deleted]
  • bmicraft 2y ago
    Yes sometimes stuff like this happens. Still, I'd like to think the EU is a prime example for how "reasonable" legislation has benefits over extremely specific legislation. Reasonable wins almost every time with how it fares under changing circumstances and how it's pretty much loop hole proof by design.
  • _bin_ 2y ago
    surely EU courts will not unfairly penalize US-developed models...
  • cactusplant7374 2y ago
    Is there somewhere I can read more about this?
• waltercool 2y ago
  Oh nooo, that will stop everyone from using AI systems. Do European politicians understand that those laws are usually dead? There is no way a law like that can be enforced except by large companies. Also, this kind of laws would make Europeans to stay at the loser side of the AI competition as China and mostly every US corporation doesn't care about that.
  • kandesbunzler 2y ago
    European politicians dont understand anything. We are led by utter morons. And redditors here complain about trump, I'd much rather have him than these mouthbreathers we have.
    • mattnewton 2y ago
      not to derail us into politics, but with Trump's team I'd take that trade offer for almost any EU nation's representatives in a heartbeat, if for no other reason than they would respect the process and take years to break what the US is now breaking in days.
    • naabb 2y ago
      Yeah trump is not a total utter moron, you're totally right, i'd rather have the insurrectionist who hired a gaggle of oligarchs to run the country. that's a much better alternative to people who actually protect your rights and the environment with regulation.
  • jampekka 2y ago
    The usage covering "unacceptable risk" is stuff that's either illegal or heavily regulated already in human operation too. > Also, this kind of laws would make Europeans to stay at the loser side of the AI competition as China and mostly every US corporation doesn't care about that. Not sure that's a game I want to win.
    • apwell23 2y ago
      Europe is still riding high on the wealth they stole from the colonies. That well is going to run dry soon.
• theptip 2y ago
  Seems like a mostly reasonable list of things to not let AI do without better safety evals. > AI that tries to infer people’s emotions at work or school I wonder how broadly this will be construed. For example if an agent uses CoT and they needs emotional state as part of that, can it be used in a work or school setting at all?
  • stavros 2y ago
    The EU generally (so far) has passed reasonable legislation about these things. I'd be surprised if it was taken more broadly than the point where a reasonable person would feel comfortable with it.
  • Zenst 2y ago
    I would imagine that such a tool to infer emotional states would be most useful for autistic people who are as I can attest, somewhat handicapped upon that front. Maybe that will get challenged as disability discrimination by some Autistic group. Which would be interesting. As with most things, there are rules, and exceptions to those rules - no shoe fits everyone, though forcing people to wear the wrong shoe size, can do more harm than good.
    • danielheath 2y ago
      > I would imagine that such a tool to infer emotional states would be most useful for autistic people who are as I can attest, somewhat handicapped upon that front. It might well be a useful tool to point at yourself. It's an entirely inappropriate one to point at someone else. If you can't imagine having someone estimate your emotional state (usually incorrectly), and use that as a basis to disregard your opinion, you've lived a very different life to mine. Don't let them hide behind "the AI agreed with my assessment".
    • cwillu 2y ago
      On the other hand, as someone who's emotional state is routinely incorrectly assessed by people, I can't imagine a worse hell than having that misassessment codified into an ai that I am required to interact with.
• kazinator 2y ago
  OK, so certain documents are allowed to exist out there; they are not banned. But if you train a mathematical function to provide a very clever form of access to those documents, that is banned. That is similar to, say, some substance being banned above a certain concentration. Information from AI is like moonshine. Too concentrated; too dangerous. There could be methyl alcohol in there that will make you go blind. Must control.
  • hkwerf 2y ago
    Training that function is not banned. Concentrating information is not banned. I have no idea where you're taking that from. Only making use (i.e. putting into service a product containing it or placing that product on the market) of that function in a manner that is listed in article 5 (which is quite terse and reasonable) is prohibited unless covered by an exception. Making use of that function in a manner that may be high-risk (see article 6 and annex III, also quite terse and reasonable) leads to the requirement of either documenting why it isn't high-risk or employing measures to ensure that the risk is acceptable (see article 9, item 5). IANAL
• Havoc 2y ago
  For once that doesn’t seem overly broad. Pretty much agree with all of the list
  • johndhi 2y ago
    The "high risk" list is where the breadth comes in
    • hkwerf 2y ago
      The "high risk" list, though, is essentially traditional safety functions (article 6) and functions that affect fundamental rights and access to basic services (annex III)? It's not that broad at all either.
      • johndhi 2y ago
        "fundamental rights" is a very broad concept in eu.
• hnburnsy 2y ago
  If AI is outlawed then only outlaws will have AI.
  • bmicraft 2y ago
    That take doesn't at all engage with the main point of the article: Unacceptable risk.
    • _bin_ 2y ago
      it doesn't actually define unacceptable risks. could be used for the development of biological, chemical, or nuclear agents? sure, an intelligent general-purpose model could be. so can wikipedia versus having to trawl through old library books. so can control-F on a PDF.
      • fredoliveira 2y ago
        It provides examples of unacceptable risks — they're in the article. And the obvious whataboutism is obvious. Yes, you can find other sources for information on, say, developing bio weapons elsewhere. Does that mean you should have systems that aid you in collecting, synthesizing and displaying that information? That with the right interfaces and actuators can actually help you move towards that goal? There's a line somewhere, that is very hard to draw, and yet should be drawn regardless.
        • _bin_ 2y ago
          Wikipedia collects, synthesizes, and displays such information, and provides a references section for further exploration. How the heck can anyone justifying something that does that but in a chat format? The threshold to building any of these save nukes is extremely low, and nukes are only high because there are fewer use cases for radioactive material so it's simply less available.
  • Mistletoe 2y ago
    I know you are trying to reference this quote with regard to guns but I think you are actually disproving the point you are trying to make. https://vpc.org/press/states-with-strong-gun-laws-and-lower-gun-ownership-have-lowest-gun-death-rates-in-the-nation-new-data-for-2021-confirms/ https://vpc.org/press/states-with-strong-gun-laws-and-lower-... I'll gladly live in a country with no AI at all. Give me Dune post-Butlerian jihad levels of AI outlawing and I'll move there. I strongly believe that myself and all the people living there will be much happier.
• okokwhatever 2y ago
  EU has become a true dystopia.
  • bmicraft 2y ago
    Ah yes, the worst of all the dystopias: The one where companies can't fuck people over as much as they'd like to. Truly a horror scenario for any tech billionaire.
  • dathinab 2y ago
    ah yes countries doing their job of trying to protect their are a dystopia, sure
  • sunaookami 2y ago
    It's really sad what happened to EU after von der Leyen took over. Someone must stop here and the EU commission or we will fall even more into irrelevance.
  • BDPW 2y ago
    Can you elaborate specifically what you think makes this law so detrimental? And to who?
  • 127 2y ago
    It's great Americans criticize the EU. I just wish they did more than (paraphrasing) "EU sucks, lol." Wouldn't hurt being a bit more constructive.
  • naabb 2y ago
    You might be looking in the mirror
• sunshine-o 2y ago
  Never forget the EU is run by lobbies by design. So usually those scary regulations are not what they seem to be. Here is what happened in most corporations when GDPR came out: - An new Chief Privacy Officer would be appointed, - A series of studies would be conducted by big consulting firms with a review of all processes and data flow across the organisation, - After many meeting they would conclude that a move to the cloud (one of the big ones) is the best and safest approach. The Chief Privacy and Legal Officer would put their stamp on it with some reservations, - This would usually accelerate a lot of outsourcing and/or workforce reduction in IT, - Bonus if a big "data governance" platform is bought and half implemented.
  • bmicraft 2y ago
    > Here is what happened in most corporations when GDPR came out Do you have a source on that, or is this what you feel like may have happened? The move to the cloud was in full swing way before GDPR came out in 2016 and got enacted in 2018. Same for outsourcing.
    • sunshine-o 2y ago
      I can assure you I have witnessed it in dozens of organisations and have been involved. In terms of timeline I can tell you: - by 2012 I already heard about that regulation but only knew it was gonna be about data protection. At that time some "Big tech" lobbying groups were already organising events in Brussels raising awareness about how important is data privacy and protection. I have been to some of those events and I even witnessed very heated exchanges between some EU people and and lobbyists about that. Proof is a lot of people knew way before that time. - by 2014 many big corporations were already preparing for GDPR, big budgets have already been validated. At that time they already knew it would be at least reasonably disruptive and they had to start early to prepare. Also remember before 2014 "Windows Azure" (what would become the most successful cloud for most European corporations) was absolutely not ready as a enterprise product. So those are not Silicon Valley startups on AWS since 2006, for many decision makers in those big corporations the GDPR upcoming problem predate the cloud solution.
  • watwut 2y ago
• hirokio123 2y ago
  In the EU, no risk is ever tolerated. They keep creating laws to regulate things, but do they ever measure the actual impact? With so many regulations, it must be hard to move freely. There should also be a system for repealing regulations.
  • watwut 2y ago
    More free then America, unless you are billionaire. I mean, American billionaires are trying to export their form of anti-democraric policies, Russia is trying ro export their own fascism etc. It is not like it was safe democracy. But, it is still one and one that cares more about own citizens then the rest. Maybe except Canada.
    • jart 2y ago
      [flagged]
  • barrenko 2y ago
    They don't, Brussels will just produce some 600k undecipherable pages on this topic, waste honest workers money to keep it's bureaucrat caste afloat. With time this is worsening, the caste is ever bigger, and the system will not change until a WW2 type situation.
• lyime 2y ago
  Do the same rules apply to the government entities luke law enforcement and security services?
  • lksaar 2y ago
    > For example, the Act permits law enforcement to use certain systems that collect biometrics in public places if those systems help perform a “targeted search” for, say, an abduction victim, or to help prevent a “specific, substantial, and imminent” threat to life. This exemption requires authorization from the appropriate governing body, and the Act stresses that law enforcement can’t make a decision that “produces an adverse legal effect” on a person solely based on these systems’ outputs. yes
• campl3r 2y ago
  This is great news! let's hope it's as successful as GDPR and applied in many more countries
• octacat 2y ago
  Some of the unacceptable activities include: AI used for social scoring (e.g., building risk profiles based on a person’s behavior) - Oh, so insurance, and credit score is banned now? And background checks. AI that manipulates a person’s decisions subliminally or deceptively. - Oh, so no more ads? AI that exploits vulnerabilities like age, disability, or socioeconomic status. - Oh, are we banning facebook now? AI that attempts to predict people committing crimes based on their appearance. - pretty sure that exists somewhere too. AI that uses biometrics to infer a person’s characteristics, like their sexual orientation. - oh, my, tiktok does not even needs biometrics, just a couple of swipes. Google actually too, just where you visit. AI that collects “real time” biometric data in public places for the purposes of law enforcement. - but cameras everywhere are ok. AI that tries to infer people’s emotions at work or school. - like every social network, right? or a company with toxic marketing, but without ai (hello, apple with green bubbles) AI that creates — or expands — facial recognition databases by scraping images online or from security cameras. - oh, this also probably exists. So companies could track clients.
  • fredoliveira 2y ago
    I fail to see where you stand based on your line by line commentary. Are we not supposed to be against these obvious negatives? Regulation against undesired outcomes needs to start somewhere. Do you believe we should not regulate, simply because we already do some of the things that seem to fall under these individual buckets?
    • octacat 2y ago
      Laws are nice, when they work, clear and applicable. It is probably would be as useful, as GDPR. Like of course, it sounds nice on the paper, but in reality it will get drown in a lot of legalize. Like with tracking consent in forms nowadays. Do you know which companies you gave consent and when? - me neither. The issue with such laws, is that they are extremely wide and hard to regulate/enforce/check. But making regulation would make a few political points. While probably not so useful in real life. We already do a lot falling under these baskets for years, big tech uses AI for algorithms left and right. "Ooopsie, we removed your youtube channel / application, because our AI system said so. You can talk to another AI system next." - we already have these, but I don't hear any reasonable feedback from EU for this. Basically, big companies with strong legal departments would find the way around the rules. Small startups would be forced to move.
• atoav 2y ago
  The (well known) problematic bit about AI is that it furthers the diffusion of responsibility that already became so commonplace with computers. People can just handwave catastrophic decisions away with a "the computer made an error, nothing we can do". This has been the case before AI, the differrnce AI makes is just that more decisions are going to be affected by this. What we need is to make the (legal) buck stop somewhere, ideally in a place that can positively change things. If you’re a civil engineer and your bridge collapses, because you fucked up the material selection, you go to jail. If you are a software engineer and you make design decisions in 2025 that would have had severe security implications in the 80s — and then this leads to the leaking of millions of medical records you can still YOLO it off somehow and go to work on the next thing. The buck has to stop somewhere with software and it doesn't really. I know that is a feature for a certain type of person, but it actively makes the world worse.
  • hkwerf 2y ago
    That is explicitly addressed by the regulation. If an AI product is used in a way you describe it, there need to be measures put in place to reduce the risk to an acceptable level. If these measures are missing, it cannot even be put into service. This places the responsibility for the use of AI clearly with either the entity that places the product on the market (i.e. if you sell something in the EU) or that puts it into service (if a EU company uses an AI product across borders remotely). With the required employee AI training, this is further shifted onto the users if done correctly or stays with the employer if not.
• _uxvx 2y ago
  [dead]
• wtcactus 2y ago
  I guess that’s how the EU is expecting to catch to the rest of the world in AI… by imposing even more regulation. It’s maddening that a group on non elected politicians and their friends have this kind of power and are using it to destroy Europe and our future.
• throw310822 2y ago
  The title is a bit absurd as it's the law that defines what's acceptable and what isn't. Anything that this regulation doesn't allow is by definition "unacceptable", even if it's tea with biscuits.
  • hkwerf 2y ago
    > Anything that this regulation doesn't allow is by definition "unacceptable", That's not true. The regulation first defines high-risk products with a narrow scope (see article 5 and annex III). It then requires risk management to be implemented. It does explicitly state what risks are acceptable, it only requires the "adoption of appropriate and targeted risk management measures" that are effective to the point that the "overall residual risk" of the is "judged to be acceptable". IANAL, the whole story is a bit more complex. But not by much.
• thiago_fm 2y ago
  Love the EU, living here is the best. There are plenty of companies in the EU using and developing AI even with the fact that Americans say we have "heavy regulation", it isn't just in the same ballpark as the US and China, which both have much bigger potential markets and a stronger VC base with of course, more money. The lack of regulations from the US in AI creates a very harsh atmosphere for the population. It's so naive to think that Meta/Google(Youtube) doesn't have power to manipulate people's opinion by showing content based on their algorithms. That's all manipulation through the use of AI. They are thinking for you. Making you depressed, making you buy useless stuff. Look on research on this subject and you will be surprised how much the likes of Meta and Google are getting away with. Hope to see more EU fines for American Big Tech firms using AI to abuse people's weaknesses.
  • Ekaros 2y ago
    Actually thinking what advanced AI could do with say Meta. It could be used to rewrite the content for each user. And not even tell that they are doing it. AI proponents see nothing wrong with it. Why not allow Meta for instance change negative experiences with restaurants to positive ones. Or modify political statements made by some sides...
  • profeatur 2y ago
    > It's so naive to think that Meta/Google(Youtube) doesn't have power to manipulate people's opinion We have that here too, except in our case it’s the government using the good old fashioned medium of television.
• mjw_byrne 2y ago
  As with GDPR, the spirit is admirable but the fundamental definitions have been hand-waved. So for the foreseeable future, the expensive lawyers you hire are going to answer the important questions with "well, we don't have much case law yet..." Also, the definition of AI seems to exclude anything that doesn't "exhibit adaptiveness after deployment". So, a big neural network doing racist facial recognition crime prediction isn't AI as long as it can't learn on-the-fly? Is my naive HTTP request rate limiter "exhibiting adaptiveness" by keeping track of each customer's typical request rate in a float32? Laws that regulate tech need to get into the weeds of exactly what is meant by the various terms up-front, even if that means loads of examples, clarification etc.
• frereubu 2y ago
  I always sigh when I see these threads on HN because many of the comments (although not all, thankfully) devolve into US / EU name-calling and wild overgeneralisations. I would really love to see a Q&A thread like https://news.ycombinator.com/item?id=42770125 https://news.ycombinator.com/item?id=42770125 from someone who's actually read the documents, practices law in the area, and also understands the difference between US and EU law.
  • frantzmiccoli 2y ago
    Not a lawyer, not versed in US and EU Law, but ... I read (part) of the regulation. https://outofthecomfortzone.frantzmiccoli.com/thoughts/2024/12/29/eu-ai-act-notes-on-the-regulation.html https://outofthecomfortzone.frantzmiccoli.com/thoughts/2024/... and here is my shameless plug.
    • 2rsf 2y ago
      Not a lawyer, only an engineer starting to assess our AI models. Your comparison to GDPR seems to be correct in a way, both are quite vague and wide. The implementation of GDPR is still unclear in certain situations and it was even worse when it was launched, the EU AI act have very little references to work with and except for very obvious area it is still a lot of a guesswork
      • Ylpertnodi 2y ago
        >...GDPR seems to be correct in a way, both are quite vague and wide. How is the gdpr vague?
        • mimsee 2y ago
          Are IP addresses considered PII or not? I remember there being multiple conflicting conclusions on that
          • sdefresne 2y ago
• deleted 2y ago
  [deleted]
• thrance 2y ago
  > AI that manipulates a person’s decisions subliminally or deceptively. Can we use this to ban X and other American algorithmic political propaganda tools?
  • 2rsf 2y ago
    Are they using AI for that? algorithmic doesn't necessarily mean AI
    • thrance 2y ago
      Right after Musk bought twitter, the code for the algorithm was open sourced and I took a look at it. I don't know if it's still open today but anyways, there was some ML stuff in it at the time. I guess it would depend on what constitutes "AI" to European legislators.
      • NekkoDroid 2y ago
        > I don't know if it's still open today but anyways I am p sure that they made like major major changes after they code dumped it. Considering "verified boosts" and "elon boosts" are very noticable, with the first being an confirmed "feature", I doubt the algorithm would even remotely work with data from nowadays. Anyway, what I wanna say is that the last commit was over 2 years ago.
        • thrance 2y ago
          Yes, the AI behind X's "algorithm" has been heavily skewed towards favoring far right content. That much is obvious to anyone taking a look at the X front page. This should give some more grounds to a continent-wide ban on this platform, threatening our democracies with relentless propaganda.
  • olivierduval 2y ago
    Actually that could be an interesting case... except if Twitter can show that its algorithm is "deterministic"
• kamma4434 2y ago
  I would frankly love if EU bodies were to plunge 10 K into the Realtime ad tracking industry as explained here https://news.ycombinator.com/item?id=42909921 https://news.ycombinator.com/item?id=42909921 , buy a lot of data of European citizens so that they have proof of sharing illegally, and then proceed suing the hell out of all parties involved. That would make me feel better about them doing something, Instead of regulating and regulating stuff that does not exist yet
• olivierduval 2y ago
  "Companies that are found to be using any of the above AI applications in the EU will be subject to fines, regardless of where they are headquartered. They could be on the hook for up to €35 million (~$36 million), or 7% of their annual revenue from the prior fiscal year, whichever is greater." US did a real gift to the world with "extra-territorial" laws: now EU use it everywhere too !!!! :-) Sooooo... the GAFAM either will have to "limit" some of their AI system when used in EU (NOT including EU citizen that may be abroad, but including foreign citizen in the EU) or to be fined. And I guess that this kind of fines may accumulate with GDPR for example...
• mrfinn 2y ago
  Hope this law doesn't become into a peasant-trap. But my gut is telling me that... that's exactly what it is. ("This and that is forbidden EXCEPT to us because blah blah blah")
  • profeatur 2y ago
    That’s what nearly all UE regulation is. Most of these regulators are on the payroll of 200 year old companies who maintain their control of the economy by preventing any real challengers from rising.
• DaveMcMartin 2y ago
  Does this only apply to usage, or does it include training the model as well? Training a model is extremely expensive, and it’s hard to imagine a company investing a huge amount of money to develop two different models just to comply with regulations (though maybe it’s worth it, I’m just guessing here). I think it’s more likely that companies would adhere to EU regulations and use the same model everywhere or implement some kind of filter.
  • drakonka 2y ago
    Not a lawyer. When I attended a conference about this I remember the distinction between "Provider" and "Deployer" being discussed. Providers are manufacturers developing a tool, deployers are professional users making a service available using the tool. A deployer may deploy a provided AI tool/model in a way that falls within the definition of unacceptable risk, and it is (also) the deployer's responsibility to ensure compliance. The example given was of a university using AI for grading. The university is a deployer, and it is their responsibility to conduct a rights impact assessment before deploying the tool to its internal users. This was compared to normal EU-style product safety regulation, which is directed at the manufacturer (what would be the provider here): if you make a stuffed toy, don’t put in such and such chemicals, etc. Here, the _application_ of the tool is under scrutiny as well vs just the tool itself. Note that this is based on very hasty notes[0] from the talk - I'm not sure to what extent the provider vs deployer responsibility divide is actually codified in the act. [0] https://liza.io/ai-act-conference-2024-keynote-notes-navigating-the-ai-act/ https://liza.io/ai-act-conference-2024-keynote-notes-navigat...
• yieldcrv 2y ago
  so just launch everything in Liechtenstein and access the single market anyway
• deleted 2y ago
  [deleted]
• mgdev 2y ago
  The EU once again races toward premature regulation. Europe's tech sector will continue to wither as America and others surge ahead. You can't regulate your way to technological leadership.
  • justonceokay 2y ago
    I think in the medium-term future we will have our own real-life Butlerian Jihad against thinking machines. Maybe they won’t be outright banned, but there needs to be some conservative force that pushes back against progress for the sake of “because we can”. Is this legislation premature? Maybe. But I don’t think it will be the last or the the most comprehensive within our lifetimes.
    • kvgr 2y ago
      And China will have none of that and will remove us from the face of earth.
      • justonceokay 2y ago
        Why would they do that? Are we trying to remove Chinese people from earth and they might retaliate?
    • Nasrudith 2y ago
      Why the fuck do people cite the Butlerian jihad as a good thing instead of what it is: stupidity writ large? "There must always be a lich king" logic in play. No, there really fucking doesn't.
      • justonceokay 2y ago
        Who is the lich king? I’m not following
  • qoez 2y ago
    If you read the list of unacceptable things they're actually super reasonable and kinda crazy to think american companies will be allowed to (and probably will) participate in
• a3w 2y ago
  Can we ban any projects with "unacceptable risk", too? Independent of technology.
• skapadia 2y ago
  Not using AI will soon be an unacceptable risk.
  • xigency 2y ago
    With thinking like that it's not a far bridge to, "not leaking all your private keys will soon be an unacceptable risk."
    • skapadia 2y ago
      I mean the other side of this argument, which I also support, is "Using the Internet will soon be an unacceptable risk".
• bArray 2y ago
  > Under the bloc’s approach, there are four broad risk levels: (1) Minimal risk (e.g., email spam filters) will face no regulatory oversight; (2) limited risk, which includes customer service chatbots, will have a light-touch regulatory oversight; (3) high risk — AI for healthcare recommendations is one example — will face heavy regulatory oversight; and (4) unacceptable risk applications — the focus of this month’s compliance requirements — will be prohibited entirely. I think this is a massive oversight, for a few reasons: 1. Things will continue to be done, just elsewhere. The EU could find itself scrambling to catch-up (again) because of their own regulation. 2. Increased oversight is only part of the picture, the real challenge is that even with the oversight, proving that AI is acceptably safe, or that the risk is acceptable. 3. Some things are inherently not safe, e.g. war. I know many (almost all) military tech companies using AI, and the EU is about to become an impossible investment zone for these guys. I think this will make investment into the EU tough, given tonnes of investment is now focused around AI. AI is and will likely remain the fuel to economic growth for quite some time, and the EU adding a time/money tax to the fuel.
• gqgs 2y ago
  >AI that manipulates a person’s decisions subliminally or deceptively. This is a strange one. Arguably this is the objective of marketing in general. Therefore, I'm not sure why draw the line only when AI is involved.