5 ms·

That isn't sufficient justification to force these tools into things that require some strict determinism. We've already seen cases of companies being forced to

That isn't sufficient justification to force these tools into things that require some strict determinism. We've already seen cases of companies being forced to compensate users because their fancy AI toy was unpredictable and gave a user incorrect information: https://www.theguardian.com/world/2024/feb/16/air-canada-chatbot-lawsuit https://www.theguardian.com/world/2024/feb/16/air-canada-cha... That was only customer service with little harm done, hammering these kinds of tools into medicine, defense or law is going to risk even worse consequences, and I fully expect people drawing false equivalences between animal intelligence and current generation AI will just attempt to dodge responsibility for their reprehensible decisions when they inevitably cause losses of life. Lowering the stakes a bunch and coming back down to say, apps, if I have to talk to my phone to get it to do stuff that I used to just be able to do without having to hold a conversation with it, it'd be a clear downgrade. The reason Google Assistant and Siri on phones remained mostly just gimmicks is that it's just faster to search for something or enter an appointment manually than to ask for it to schedule it when you have a physical interface in your hand already. Forcing things like that might work out in the short term for our tech oligopolies, but it's only going to further increase the push for breaking them up. As an example, I presented on the current state of LLMs to a group of non-techies at my workplace about how we could leverage LLMs to enhance our productivity, and the discussion with others who had tried them also concluded that they were simply too unpredictable to trust for anything where a human isn't immediately checking the output. I fully expect that all these irresponsible false equivalencies will lead to software, and especially AI usage, being heavily regulated in the same way that construction or flight is.

emseetech 3y ago
> too unpredictable to trust for anything where a human isn't immediately checking the output. Not just any human either, but one with enough specific domain knowledge to understand subtle failures. It’s why I strongly recommend against junior developers or anyone new to a language or framework to not use Copilot. I call this the Babysitter Problem. There’s probably a more official academic term, but in general I don’t any of these vc funded ai startups are asking the right questions when it comes to UX.
- dotnet00 3y ago
  Yep, this was a presentation to scientists so concerns about subtle failures came up. Even when discussing the potential of applying LLMs to help make it easier for non-native speakers to fix grammar errors, there was the major caveat that the output would have to be carefully checked to ensure that it did not subtly change the meaning, which is something even I struggled with on my early papers, despite the benefits of being a native speaker, having natural general intelligence and domain knowledge.
famouswaffles 3y ago
>That isn't sufficient justification to force these tools into things that require some strict determinism. By and large, LLMs are not being used for endeavours that strictly require or utilize determinism. >We've already seen cases of companies being forced to compensate users because their fancy AI toy was unpredictable and gave a user incorrect information: Regular old human customer service has given me incorrect information before and will continue to do so. If the harm is great enough, it has always been the company responsible for damages. >That was only customer service with little harm done, hammering these kinds of tools into medicine, defense or law is going to risk even worse consequences Medicine, law or defense are not areas that require or utilize strict determinism and in fact, incorrect diagnoses and botched operations kill at least thousands of people every year. Perfect does not exist here so you don't need perfect to improve upon existing methods. If GPT-4 can give more accurate diagnostics than the average doctor then you are only killing more people by not utilizing it as part of the diagnosis process. >if I have to talk to my phone to get it to do stuff that I used to just be able to do without having to hold a conversation with it, it'd be a clear downgrade. Ok...where has that happened ? What things do you now have to do with an LLM that you could on your own before? >The reason Google Assistant and Siri on phones remained mostly just gimmicks The reason they remained gimmicks is that google assistant and Siri are not competent enough to do anything that isn't strictly hard-coded in.
- dotnet00 3y ago
  >Regular old human customer service has given me incorrect information before and will continue to do so. If the harm is great enough, it has always been the company responsible for damages. >Medicine, law or defense are not areas that require or utilize strict determinism and in fact, incorrect diagnoses and botched operations kill at least thousands of people every year. >Perfect does not exist here so you don't need perfect to improve upon existing methods. If GPT-4 can give more accurate diagnostics than the average doctor then you are only killing more people by not utilizing it as part of the diagnosis process. When a human makes a mistake, the human can be found responsible and in the vast majority of cases just having been informed about the mistake will ensure that they don't make that mistake again. Through our empathy we intrinsically understand enough of human intelligence to consider people to be relatively predictable. However the same cannot be said of AI. A current generation medical AI could simultaneously ace exams for getting an MD and confidently lie to patients about basic health knowledge for no discernable reason, on par with intentionally giving an untreated dysfunctional schizophrenic an MD. See the recent discourse around the use of AI by Israel when it comes to defense, where clearly many people have concerns about using the AI to "morality wash", because all the humans involved can claim to just be following orders. Even if the AI is picking targets more precisely than a human would, if a human messes up, you can sack them, when an AI messes up, you just blame it on the AI, claim to fix it, and then move on to the next time it makes a mistake. You can't really hold the creators of the AI responsible, because they couldn't have predicted the failure mode, and you can't hold the person who executed the fire order responsible because they're just following the orders they've been told to follow and the information they've been told is supposed to hold merit. >Ok...where has that happened ? What things do you now have to do with an LLM that you could on your own before? The comment chain started with asking how such "probablistic UI" could be done. It hasn't been done yet, but it isn't exactly a stretch to believe that it'll eventually happen. We've already had several examples of companies similarly getting rid of "legacy" interfaces to push their "modern" interface. Similarly with UX "designers" and their shitty design languages, eg the way so many websites are optimized for mobile screens, wasting most of the screen space on desktops/laptops. >The reason they remained gimmicks is that google assistant and Siri are not competent enough to do anything that isn't strictly hard-coded in. I see that we'll just have to agree to disagree since I, and everyone I know disagrees on this. Even if Google Assistant or Siri were literal human secretaries, they'd be completely useless if they required being within arms reach of a device where you can just type and read. Speech is simply not an efficient enough communication method and if you can type, it's far more efficient to tap a few buttons (which most of us can do even learn to do with little conscious thought) and fill out some fields than to effectively chat with the software.

emseetech 3y ago

> too unpredictable to trust for anything where a human isn't immediately checking the output. Not just any human either, but one with enough specific domain knowledge to understand subtle failures. It’s why I strongly recommend against junior developers or anyone new to a language or framework to not use Copilot. I call this the Babysitter Problem. There’s probably a more official academic term, but in general I don’t any of these vc funded ai startups are asking the right questions when it comes to UX.

dotnet00 3y ago
Yep, this was a presentation to scientists so concerns about subtle failures came up. Even when discussing the potential of applying LLMs to help make it easier for non-native speakers to fix grammar errors, there was the major caveat that the output would have to be carefully checked to ensure that it did not subtly change the meaning, which is something even I struggled with on my early papers, despite the benefits of being a native speaker, having natural general intelligence and domain knowledge.

famouswaffles 3y ago

>That isn't sufficient justification to force these tools into things that require some strict determinism. By and large, LLMs are not being used for endeavours that strictly require or utilize determinism. >We've already seen cases of companies being forced to compensate users because their fancy AI toy was unpredictable and gave a user incorrect information: Regular old human customer service has given me incorrect information before and will continue to do so. If the harm is great enough, it has always been the company responsible for damages. >That was only customer service with little harm done, hammering these kinds of tools into medicine, defense or law is going to risk even worse consequences Medicine, law or defense are not areas that require or utilize strict determinism and in fact, incorrect diagnoses and botched operations kill at least thousands of people every year. Perfect does not exist here so you don't need perfect to improve upon existing methods. If GPT-4 can give more accurate diagnostics than the average doctor then you are only killing more people by not utilizing it as part of the diagnosis process. >if I have to talk to my phone to get it to do stuff that I used to just be able to do without having to hold a conversation with it, it'd be a clear downgrade. Ok...where has that happened ? What things do you now have to do with an LLM that you could on your own before? >The reason Google Assistant and Siri on phones remained mostly just gimmicks The reason they remained gimmicks is that google assistant and Siri are not competent enough to do anything that isn't strictly hard-coded in.

dotnet00 3y ago
>Regular old human customer service has given me incorrect information before and will continue to do so. If the harm is great enough, it has always been the company responsible for damages. >Medicine, law or defense are not areas that require or utilize strict determinism and in fact, incorrect diagnoses and botched operations kill at least thousands of people every year. >Perfect does not exist here so you don't need perfect to improve upon existing methods. If GPT-4 can give more accurate diagnostics than the average doctor then you are only killing more people by not utilizing it as part of the diagnosis process. When a human makes a mistake, the human can be found responsible and in the vast majority of cases just having been informed about the mistake will ensure that they don't make that mistake again. Through our empathy we intrinsically understand enough of human intelligence to consider people to be relatively predictable. However the same cannot be said of AI. A current generation medical AI could simultaneously ace exams for getting an MD and confidently lie to patients about basic health knowledge for no discernable reason, on par with intentionally giving an untreated dysfunctional schizophrenic an MD. See the recent discourse around the use of AI by Israel when it comes to defense, where clearly many people have concerns about using the AI to "morality wash", because all the humans involved can claim to just be following orders. Even if the AI is picking targets more precisely than a human would, if a human messes up, you can sack them, when an AI messes up, you just blame it on the AI, claim to fix it, and then move on to the next time it makes a mistake. You can't really hold the creators of the AI responsible, because they couldn't have predicted the failure mode, and you can't hold the person who executed the fire order responsible because they're just following the orders they've been told to follow and the information they've been told is supposed to hold merit. >Ok...where has that happened ? What things do you now have to do with an LLM that you could on your own before? The comment chain started with asking how such "probablistic UI" could be done. It hasn't been done yet, but it isn't exactly a stretch to believe that it'll eventually happen. We've already had several examples of companies similarly getting rid of "legacy" interfaces to push their "modern" interface. Similarly with UX "designers" and their shitty design languages, eg the way so many websites are optimized for mobile screens, wasting most of the screen space on desktops/laptops. >The reason they remained gimmicks is that google assistant and Siri are not competent enough to do anything that isn't strictly hard-coded in. I see that we'll just have to agree to disagree since I, and everyone I know disagrees on this. Even if Google Assistant or Siri were literal human secretaries, they'd be completely useless if they required being within arms reach of a device where you can just type and read. Speech is simply not an efficient enough communication method and if you can type, it's far more efficient to tap a few buttons (which most of us can do even learn to do with little conscious thought) and fill out some fields than to effectively chat with the software.