7 ms·

Mamdani to kill the NYC AI chatbot caught telling businesses to break the law

180 points by jyunwai 8mo ago

sylens 8mo ago
> The bot, built using Microsoft’s cloud computing platform When is the last time there was positive news involving Microsoft? This bot could've easily been on AWS or GCP but I find it hilarious that here they are, getting dragged yet again
- embedding-shape 8mo ago
  https://iet.ucdavis.edu/content/microsoft-releases-xpsp2 https://iet.ucdavis.edu/content/microsoft-releases-xpsp2
  - fragmede 8mo ago
    golf clap
  - walterbell 8mo ago
    MS 2004
- paxys 8mo ago
  Even if the capability of each platform was exactly the same, Microsoft cloud users skew heavily towards governments, large non-tech corporations and really anyone who you sell to using large sales teams, fancy dinners and kickbacks rather than quality of software. And the end result follows.
toomuchtodo 8mo ago
> A spokesperson for the mayor, Dora Pekec, confirmed in a text message that the new administration plans to take down the chatbot. She said a member of the Mamdani transition team had seen reporting on the bot from The Markup and THE CITY and presented it to the mayor as a possible place to save funds. Journalism works.
- atq2119 8mo ago
  It does. And it works best if you elect politicians who are willing to listen.
- andrewflnr 8mo ago
  Journalism teed up an easy way for an incoming politician to dunk on his predecessor, if you'll forgive the mixed metaphor. Not that I'm opposed to any part of it, just that this was an easy scenario for "journalism" to "work" in.
  - toomuchtodo 8mo ago
    If you'd like other examples, 404media and adjacent journalism grinding against Flock across the country, as well as perfectunion working against datacenter siting. I admit the egregious nature of the Adams NYC administration and his fraud makes this particular scenario straightforward. https://en.wikipedia.org/wiki/Investigations_into_the_Eric_Adams_administration https://en.wikipedia.org/wiki/Investigations_into_the_Eric_A...
    - nate_meurer 8mo ago
      Yes the Flock coverage is a very good example.
andsoitis 8mo ago
Why did NYC release it in the first place? Did they not QA it? Or was it perhaps one of those cases where they found issues, but the only way to really know for sure that the deleterious impact is significant enough by pushing it to prod?
- JohnTHaller 8mo ago
  It was implemented by our scammy, grifting, Republican in a Democratic lawmaker suit former mayor Eric Adams who should probably be in prison but who made a deal with Trump to not be prosecuted.
- erxam 8mo ago
  > Why did NYC release it in the first place? Perhaps a big fat check was involved.
  - kevin_thibedeau 8mo ago
    Usually it's a manila envelope.
  - Eric_WVGG 8mo ago
    Yeah… no offense, but only a person who didn't know anything about Mayor Eric Adams would ask a question like that. Just days out of office, he made a few million off a crypto scam. Buffoonishly corrupt. https://finance.yahoo.com/news/eric-adams-promoted-memecoin-crashes-160110674.html https://finance.yahoo.com/news/eric-adams-promoted-memecoin-...
    - worthless-trash 8mo ago
      Not op. That wasn't a question.
- elgenie 8mo ago
  QA efforts can whack-a-mole some issues, but the mismatch of problem and solution is inherent in any situation in which a generator of plausible-sounding text gets pointed at an area where correctness matters.
monero-xmr 8mo ago
[flagged]
- geoffeg 8mo ago
  To ride NYC's free busses, you must have a two minute conversation with a chat bot. (/s)
terespuwash 8mo ago
What else to expect from Eric Adams.
- greekrich92 8mo ago
  This is the only comment worth making. Virtually everything he did should be heavily audited and/or undone.
cmiles8 8mo ago
We’ll likely see a lot of these AI pet projects get axed in the coming year or two… especially things rushed out in the early phases of the AI bubble when folks were desperate to appear to be using AI.
- chasd00 8mo ago
  yeah i hope the problems stay to somewhat humorous themes like convincing a car sales bot to sell you a car for $1 and not more serious issues like convincing a bot to metaphorically launch the ICBMs.
  - toomuchtodo 8mo ago
    "The WOPR did a better job avoiding thermonuclear war than most humans would" is my hot take.
    - jjk166 8mo ago
      Thinks through possibilities -> realizes what it is proposing is a bad idea Hell put WOPR in charge of everything
kittikitti 8mo ago
Being in and around the NYC area, while also knowing plenty of small businesses, I'm glad Mamdani killed this bot. Telling bosses to steal tips from their employees is run-of-the-mill corruption and common over here. The vibe for businesses is that everyone has to be exploiting someone else or have a schtick. If you were to talk about morals, you would be ridiculed. Most lawyers wouldn't even prosecute small businesses for this. It's probably why the agent was put into production, the level of business ethics in NYC is cartoonishly evil.
- patrickmay 8mo ago
  In the case of stealing tips, that's wage theft and the New York State Department of Labor has zero sense of humor about that. They will definitely investigate all claims on that topic. It might be too little and too late for the individual affected, but the business will pay.
hashberry 8mo ago
> The Office of Technology and Innovation spent nearly $600,000 to build out the foundations of the MyCity chatbot, which will be used for future chatbot offerings on MyCity. [0] This was experimental tech... while I admire cities attempting to implement AI, it seems they did not spend enough tax dollars on it! [0] https://abc7ny.com/post/ai-artificial-intelligence-eric-adams-nyc/13922640/ https://abc7ny.com/post/ai-artificial-intelligence-eric-adam...
1970-01-01 8mo ago
He is turning out to be a benevolent, law-abiding mayor that just happens to be communist.
- direwolf20 8mo ago
  What's that supposed to mean?
  - 1970-01-01 8mo ago
    The previous mayors were none of these things
    - georgemcbay 8mo ago
      Mamdani is a socialist, not a communist. And Fiorello La Guardia was (in terms of beliefs and enacted policy) even more socialist than Mamdani is even though he was technically a Republican when elected.
  - Izikiel43 8mo ago
    Some of it is good, some of it is bad.
    - direwolf20 8mo ago
      Can you be more specific?
- hydrogen7800 8mo ago
  To some, anything sufficiently resembling functioning government is indistinguishable from communism.
Neywiny 8mo ago
I always ask this question about these bots: is the literature the training data or is the understanding of literature the training data? Meaning, sure you trained the bot on the current rules and regulations. But does that mean the model weights contain them? Such that really is a guess at legal accuracy? Or is it trained to be a lawyer and understand the docs which sit outside the model? Every time I've asked the answer is the former, and to me that's the wrong approach. But I'm not an AI scientist so I don't know how hard my theoretically perfect solution is. What I do know is that if it was done my way it would be pretty easy for it to do what the Google AI does. Say it's not responsible, give links for humans to fact check it. I've noticed a dramatic drop in hallucinations after it had to provide links to its sources. Still not 0, though.
- sdwr 8mo ago
  > pretty easy to do what the Google AI does I thought Gemini just started providing citations in the last few months. Are you saying they should have beaten Google to the punch on this? As part of the $500,000 budget?
  - Neywiny 8mo ago
    Correct. Much in the same way that videos were online before YouTube, social networks existed before Facebook, and messaging existed before WhatsApp and co, they should have understood their problem set better instead of just following the leaders. Because Gemeni is not this chatbot on steroids, it's a different problem entirely that happens to now employ the same technique. Also, search says they did links in 2024 for the Google AI. So there's that.
- acdha 8mo ago
  > I've noticed a dramatic drop in hallucinations after it had to provide links to its sources. Still not 0, though. I’ve noticed that Google does a fair job at linking to relevant sources but it’s still fairly common for it to confabulate something that source doesn’t say or even directly contradicts. It seems to hit the underlying inability to reason where if the source covers more than one thing it’s prone to taking an input “X does A while Y does B” and emitting “Y does A” or “X does A and B”. It’s a fascinating failure mode which seems to be insurmountable.
SanjayMehta 8mo ago
That's not the only thing he's achieved. https://organiser.org/2026/02/10/339366/world/zohran-mamdani-first-100-days-as-new-york-mayor-antisemitism-definition-scrapped-jews-on-edge-as-hate-crimes-surge/ https://organiser.org/2026/02/10/339366/world/zohran-mamdani...

180 points by jyunwai 8mo ago

> The bot, built using Microsoft’s cloud computing platform When is the last time there was positive news involving Microsoft? This bot could've easily been on AWS or GCP but I find it hilarious that here they are, getting dragged yet again

embedding-shape 8mo ago
https://iet.ucdavis.edu/content/microsoft-releases-xpsp2 https://iet.ucdavis.edu/content/microsoft-releases-xpsp2
- fragmede 8mo ago
  golf clap
- walterbell 8mo ago
  MS 2004
paxys 8mo ago
Even if the capability of each platform was exactly the same, Microsoft cloud users skew heavily towards governments, large non-tech corporations and really anyone who you sell to using large sales teams, fancy dinners and kickbacks rather than quality of software. And the end result follows.

toomuchtodo 8mo ago

> A spokesperson for the mayor, Dora Pekec, confirmed in a text message that the new administration plans to take down the chatbot. She said a member of the Mamdani transition team had seen reporting on the bot from The Markup and THE CITY and presented it to the mayor as a possible place to save funds. Journalism works.

atq2119 8mo ago
It does. And it works best if you elect politicians who are willing to listen.
andrewflnr 8mo ago
Journalism teed up an easy way for an incoming politician to dunk on his predecessor, if you'll forgive the mixed metaphor. Not that I'm opposed to any part of it, just that this was an easy scenario for "journalism" to "work" in.
- toomuchtodo 8mo ago
  If you'd like other examples, 404media and adjacent journalism grinding against Flock across the country, as well as perfectunion working against datacenter siting. I admit the egregious nature of the Adams NYC administration and his fraud makes this particular scenario straightforward. https://en.wikipedia.org/wiki/Investigations_into_the_Eric_Adams_administration https://en.wikipedia.org/wiki/Investigations_into_the_Eric_A...
  - nate_meurer 8mo ago
    Yes the Flock coverage is a very good example.

andsoitis 8mo ago

Why did NYC release it in the first place? Did they not QA it? Or was it perhaps one of those cases where they found issues, but the only way to really know for sure that the deleterious impact is significant enough by pushing it to prod?

JohnTHaller 8mo ago
It was implemented by our scammy, grifting, Republican in a Democratic lawmaker suit former mayor Eric Adams who should probably be in prison but who made a deal with Trump to not be prosecuted.
erxam 8mo ago
> Why did NYC release it in the first place? Perhaps a big fat check was involved.
- kevin_thibedeau 8mo ago
  Usually it's a manila envelope.
- Eric_WVGG 8mo ago
  Yeah… no offense, but only a person who didn't know anything about Mayor Eric Adams would ask a question like that. Just days out of office, he made a few million off a crypto scam. Buffoonishly corrupt. https://finance.yahoo.com/news/eric-adams-promoted-memecoin-crashes-160110674.html https://finance.yahoo.com/news/eric-adams-promoted-memecoin-...
  - worthless-trash 8mo ago
    Not op. That wasn't a question.
elgenie 8mo ago
QA efforts can whack-a-mole some issues, but the mismatch of problem and solution is inherent in any situation in which a generator of plausible-sounding text gets pointed at an area where correctness matters.

monero-xmr 8mo ago

[flagged]

geoffeg 8mo ago
To ride NYC's free busses, you must have a two minute conversation with a chat bot. (/s)

terespuwash 8mo ago

What else to expect from Eric Adams.

greekrich92 8mo ago
This is the only comment worth making. Virtually everything he did should be heavily audited and/or undone.

cmiles8 8mo ago

We’ll likely see a lot of these AI pet projects get axed in the coming year or two… especially things rushed out in the early phases of the AI bubble when folks were desperate to appear to be using AI.

chasd00 8mo ago
yeah i hope the problems stay to somewhat humorous themes like convincing a car sales bot to sell you a car for $1 and not more serious issues like convincing a bot to metaphorically launch the ICBMs.
- toomuchtodo 8mo ago
  "The WOPR did a better job avoiding thermonuclear war than most humans would" is my hot take.
  - jjk166 8mo ago
    Thinks through possibilities -> realizes what it is proposing is a bad idea Hell put WOPR in charge of everything

kittikitti 8mo ago

Being in and around the NYC area, while also knowing plenty of small businesses, I'm glad Mamdani killed this bot. Telling bosses to steal tips from their employees is run-of-the-mill corruption and common over here. The vibe for businesses is that everyone has to be exploiting someone else or have a schtick. If you were to talk about morals, you would be ridiculed. Most lawyers wouldn't even prosecute small businesses for this. It's probably why the agent was put into production, the level of business ethics in NYC is cartoonishly evil.

patrickmay 8mo ago
In the case of stealing tips, that's wage theft and the New York State Department of Labor has zero sense of humor about that. They will definitely investigate all claims on that topic. It might be too little and too late for the individual affected, but the business will pay.

hashberry 8mo ago

> The Office of Technology and Innovation spent nearly $600,000 to build out the foundations of the MyCity chatbot, which will be used for future chatbot offerings on MyCity. [0] This was experimental tech... while I admire cities attempting to implement AI, it seems they did not spend enough tax dollars on it! [0] https://abc7ny.com/post/ai-artificial-intelligence-eric-adams-nyc/13922640/ https://abc7ny.com/post/ai-artificial-intelligence-eric-adam...

1970-01-01 8mo ago

He is turning out to be a benevolent, law-abiding mayor that just happens to be communist.

direwolf20 8mo ago
What's that supposed to mean?
- 1970-01-01 8mo ago
  The previous mayors were none of these things
  - georgemcbay 8mo ago
    Mamdani is a socialist, not a communist. And Fiorello La Guardia was (in terms of beliefs and enacted policy) even more socialist than Mamdani is even though he was technically a Republican when elected.
- Izikiel43 8mo ago
  Some of it is good, some of it is bad.
  - direwolf20 8mo ago
    Can you be more specific?
hydrogen7800 8mo ago
To some, anything sufficiently resembling functioning government is indistinguishable from communism.

Neywiny 8mo ago

I always ask this question about these bots: is the literature the training data or is the understanding of literature the training data? Meaning, sure you trained the bot on the current rules and regulations. But does that mean the model weights contain them? Such that really is a guess at legal accuracy? Or is it trained to be a lawyer and understand the docs which sit outside the model? Every time I've asked the answer is the former, and to me that's the wrong approach. But I'm not an AI scientist so I don't know how hard my theoretically perfect solution is. What I do know is that if it was done my way it would be pretty easy for it to do what the Google AI does. Say it's not responsible, give links for humans to fact check it. I've noticed a dramatic drop in hallucinations after it had to provide links to its sources. Still not 0, though.

sdwr 8mo ago
> pretty easy to do what the Google AI does I thought Gemini just started providing citations in the last few months. Are you saying they should have beaten Google to the punch on this? As part of the $500,000 budget?
- Neywiny 8mo ago
  Correct. Much in the same way that videos were online before YouTube, social networks existed before Facebook, and messaging existed before WhatsApp and co, they should have understood their problem set better instead of just following the leaders. Because Gemeni is not this chatbot on steroids, it's a different problem entirely that happens to now employ the same technique. Also, search says they did links in 2024 for the Google AI. So there's that.
acdha 8mo ago
> I've noticed a dramatic drop in hallucinations after it had to provide links to its sources. Still not 0, though. I’ve noticed that Google does a fair job at linking to relevant sources but it’s still fairly common for it to confabulate something that source doesn’t say or even directly contradicts. It seems to hit the underlying inability to reason where if the source covers more than one thing it’s prone to taking an input “X does A while Y does B” and emitting “Y does A” or “X does A and B”. It’s a fascinating failure mode which seems to be insurmountable.

SanjayMehta 8mo ago

That's not the only thing he's achieved. https://organiser.org/2026/02/10/339366/world/zohran-mamdani-first-100-days-as-new-york-mayor-antisemitism-definition-scrapped-jews-on-edge-as-hate-crimes-surge/ https://organiser.org/2026/02/10/339366/world/zohran-mamdani...