3 ms·

This seems to apply to all areas of AI in its current form and in my experience 70% may be a bit generous. AI is great at getting you started or setting up sca

by dkrich 2y ago

This seems to apply to all areas of AI in its current form and in my experience 70% may be a bit generous. AI is great at getting you started or setting up scaffolds that are common to all tasks of a similar kind. Essentially anything with an identifiable pattern. It’s yet another abstraction layer sitting on an abstraction layer. I suspect this is the reason we are really only seeing AI agents being used in call centers, essentially providing stand ins for chatbots- because chatbots are designed to automate highly repetitive, predictable tasks like changing an address or initiating a dispute. But for things like “I have a question about why I was charged $24.38 on my last statement” you will still be escalated to an agent because inquiries like that require a human to investigate and interpret an unpredictable pattern. But creative tasks are designed to model the real world which is inherently analog and ever changing and closing that gap of identifying what’s missing between what you have and the real world and coming up with creative solutions is what humans excel at. Self driving, writing emails, generating applications- AI gets you a decent starting point. It doesn’t solve problems fully, even with extensive training. Being able to fill that gap is true AI imo and probably still quite a ways off.

jeremyjh 2y ago
Yeah, its more like it can generate 70% of the code by volume, rather than get you 70% of the way to a complete solution. 12 week projects don't become 4 week projects, at best they are 9-10 week projects.
adamc 2y ago
Great analysis, and I agree it's Fred Brooks' point all over again. None of these tools hurt, but you still need to comprehend the problem domain and the tools -- not least because you have to validate proposed solutions -- and AI cannot (yet) do that for you. In my experience, generating code is a relatively small part of the process.
marcosdumay 2y ago
> because chatbots are designed to automate highly repetitive, predictable tasks like changing an address or initiating a dispute You know what is even cheaper, more scalable, more efficient, and more user-friendly than a chatbot for those use cases? A run of the mill form on a web page. Oh, and it's also more reliable.
- mistersquid 2y ago
  > You know what is even cheaper, more scalable, more efficient, and more user-friendly than a chatbot for those use cases? > A run of the mill form on a web page. Oh, and it's also more reliable. Web-accessible forms are great for asynchronous communication and queries but are not as effective in situations where the reporter doesn't have a firm grasp on the problem domain. For example, a user may know printing does not work but may be unable to determine if the issue is caused by networking, drivers, firmware, printing hardware, etc. A decision tree built from the combinations of even a few models of printer and their supported computers could be massive. In such cases, hiring people might be more effective, efficient, and scalable than creating and maintaining a web form.
  - marcosdumay 2y ago
    > but are not as effective in situations where the reporter doesn't have a firm grasp on the problem domain Hum... Your point is that LLMs are more effective? Because, of course people are, but that's not the point. Oh, and if you do create that decision tree, do you know how you communicate it better than with a chatbot? You do that by writing it down, as static text, with text-anchors on each step.
    - bsder 2y ago
      > Because, of course people are, but that's not the point. Are they? If the LLMs could talk to grandma for 40 minutes until it figures out what her problem actually is as opposed to what she thinks it is and then transfer her over to a person with the correct context to resolve it, I think that's probably better than most humans in a customer service role. Chatting to grandma being random for an extended amount of time is not something that very many customer service people can put up with day in and day out. The problem is that companies will use the LLMs to eliminate customer service roles rather than make them better.
      - marcosdumay 2y ago
        > If the LLMs could talk to grandma for 40 minutes until it figures out what her problem actually is as opposed to what she thinks it is Ok, but it can't. If we had superhuman AGI, it would beat people in customer service, yes.
nottorp 2y ago
> But for things like “I have a question about why I was charged $24.38 on my last statement” you will still be escalated to an agent because inquiries like that require a human to investigate and interpret an unpredictable pattern. Wishful thinking? You'll just get kicked out of the chat because all the agents have been fired.
- banku_brougham 2y ago
  Wishful thinking you will have a bank account when you already were downsized

by dkrich 2y ago

jeremyjh 2y ago

Yeah, its more like it can generate 70% of the code by volume, rather than get you 70% of the way to a complete solution. 12 week projects don't become 4 week projects, at best they are 9-10 week projects.

adamc 2y ago

Great analysis, and I agree it's Fred Brooks' point all over again. None of these tools hurt, but you still need to comprehend the problem domain and the tools -- not least because you have to validate proposed solutions -- and AI cannot (yet) do that for you. In my experience, generating code is a relatively small part of the process.

marcosdumay 2y ago

> because chatbots are designed to automate highly repetitive, predictable tasks like changing an address or initiating a dispute You know what is even cheaper, more scalable, more efficient, and more user-friendly than a chatbot for those use cases? A run of the mill form on a web page. Oh, and it's also more reliable.

mistersquid 2y ago
> You know what is even cheaper, more scalable, more efficient, and more user-friendly than a chatbot for those use cases? > A run of the mill form on a web page. Oh, and it's also more reliable. Web-accessible forms are great for asynchronous communication and queries but are not as effective in situations where the reporter doesn't have a firm grasp on the problem domain. For example, a user may know printing does not work but may be unable to determine if the issue is caused by networking, drivers, firmware, printing hardware, etc. A decision tree built from the combinations of even a few models of printer and their supported computers could be massive. In such cases, hiring people might be more effective, efficient, and scalable than creating and maintaining a web form.
- marcosdumay 2y ago
  > but are not as effective in situations where the reporter doesn't have a firm grasp on the problem domain Hum... Your point is that LLMs are more effective? Because, of course people are, but that's not the point. Oh, and if you do create that decision tree, do you know how you communicate it better than with a chatbot? You do that by writing it down, as static text, with text-anchors on each step.
  - bsder 2y ago
    > Because, of course people are, but that's not the point. Are they? If the LLMs could talk to grandma for 40 minutes until it figures out what her problem actually is as opposed to what she thinks it is and then transfer her over to a person with the correct context to resolve it, I think that's probably better than most humans in a customer service role. Chatting to grandma being random for an extended amount of time is not something that very many customer service people can put up with day in and day out. The problem is that companies will use the LLMs to eliminate customer service roles rather than make them better.
    - marcosdumay 2y ago
      > If the LLMs could talk to grandma for 40 minutes until it figures out what her problem actually is as opposed to what she thinks it is Ok, but it can't. If we had superhuman AGI, it would beat people in customer service, yes.

nottorp 2y ago

> But for things like “I have a question about why I was charged $24.38 on my last statement” you will still be escalated to an agent because inquiries like that require a human to investigate and interpret an unpredictable pattern. Wishful thinking? You'll just get kicked out of the chat because all the agents have been fired.

banku_brougham 2y ago
Wishful thinking you will have a bank account when you already were downsized