3 ms·
Is this what happens when you let large-scale language models into the wild? It seems like as soon as you start modelling your responses to queries on real-worl
by briga 5y ago
Is this what happens when you let large-scale language models into the wild? It seems like as soon as you start modelling your responses to queries on real-world text data you're going to run into this quality control problem. What happens when bad actors start polluting public textual data sets to make these kinds of responses more common?
- chrismarlow9 5y agoAlready has happened. Microsoft had to take it's chat AI offline a few years ago because it became a racist asshole almost instantly.
- throwaway2037 5y agoNice reference. Wiki: https://en.wikipedia.org/wiki/Tay_(bot) https://en.wikipedia.org/wiki/Tay_(bot)
- vikasnair 5y agoThis is exactly what we’re trying to solve at Unbox (S21). Tons of potholes like this exist in AI we use everyday. We used to be ML engineers at Siri and had to invest millions into monitoring tools to stay on top. This is fine and all, but what’s better is to catch them before you ship and before your users suffer (sometimes literally, as in this case). We think that better tools for QA-ing models, which allow more people (not just ML engineers) to get eyes on the model, might help catch mistakes proactively rather than retroactively.
- pkz 5y agoThis made me happy. The state of language models paired with the overoptimistic ideas about AI a lot of people have sets the scene for a number of train wrecks. I hope more people critically evaluate their models before releasing them in mass scale.
- giardini 5y agovikasnair says:>"Tons of potholes like this exist in AI we use everyday..."< Maybe "potholes" is not the correct analogy. Maybe AI is going down the wrong road.