Microsoft left Bing Chat online long after they knew it would inevitably lead to headlines like this one - and, from the NYT homepage, "A Talk With Bing’s Chatb

Microsoft left Bing Chat online long after they knew it would inevitably lead to headlines like this one - and, from the NYT homepage, "A Talk With Bing’s Chatbot Left Our Columnist ‘Deeply Unsettled, Even Frightened.’" Imagine how much pressure must exist within Microsoft for that to happen. Back in 2016, Microsoft took the chatbot Tay down in 1-2 days (https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...). This time, some team made the calculation that it's better to learn on end users, even with the press hit, than to learn on paid testers (like with more RLHF). And that's despite being behind a waiting list; Bing Chat's waiting list would grow either way. I'm not an AI risk zealot or even expert, but next time you hear "AI risk," think "Humans deciding to release products that they know have major user-facing flaws." Almost nobody is going to knowingly release something that will cause major real-world damage, but many will intentionally release things that they know have gaping flaws. The effective AI risk margin isn't "We thought <new product> was great, so it was impossible to imagine that it could cause real-world damage," it's "We thought <new product> only had major user-facing flaws, not that it would cause real-world damage."