3 ms·

I see people frame generative AI this way a lot, as if they're strictly worse than alternatives in all dimensions (inaccurate, low quality). But I think there's

by tibbar 2y ago

I see people frame generative AI this way a lot, as if they're strictly worse than alternatives in all dimensions (inaccurate, low quality). But I think there's several important dimension where they're strictly better, including flexibility, interactivity, and accessibility. I can ask an LLM a very specific question about my specific development environment, codebase, and tools and get a specific answer back that works. If it doesn't work, I can chat with the LLM about what happened to make changes. I don't even reach for LLMs that often compared to most engineers these days (maybe a couple times a month), but when I do, it's basically always the fastest and best way to deal with my particular situation.

deleted 2y ago
[deleted]
bruce511 2y ago
I concur. I'm notca huge user of AI, but it does make a good search tool when my search term is verbose. For example I was looking for the name of a windows API command. I "knew" the command must exist, but didn't gave a clue what it was called. Asked Cgpt for an example program, and there's the name of the API. (Which I can then Google for docs.) I also had a complicated-to-ask question about sun movement which it explained to me, along with site links to actual data. I'm not using it as a Google replacement, but more of a Google supplement when the question is long-winded to write.
caseyy 2y ago
I think a lot of people drastically underestimate how much the LLMs mislead the user with plausible-sounding answers. I would say that whatever % of queries you found them not to give you correct answers, the actual percentage is materially higher. In my work niche, which at least 10,000 other software engineers do each day, ChatGPT 4 and 4o almost never give correct answers. Usually, they are misleading. I do find myself trying the LLMs when I'm faced with a challenging problem, but in that scenario they have not been helpful once. Granted, there are areas of work that are much more popular and LLMs will be more helpful there. But these are individual scenarios, and we can find many where LLMs are fantastic and many where LLMs are awful if we wanted to cherry-pick. Overall, the quality of the content is lower than what a human professional would do in their area of work, including flexibility, interactivity, and accessibility. This extends to books written by professionals and lectures given by them, as well as work carried out by them. It applies to home carpentry as much as it does to neurosurgery. The more specific the knowledge has to be, the more this is true. The more generic, the less. But all-in-all, I think it's still very evident humans produce higher quality knowledge and content. And any quantized model of that content and knowledge will be unable to reproduce it at the same fidelity or quality. With that said, I hope I expressed this enough — I do see your point in some circumstances. Just not overall.
- tibbar 2y ago
  It feels like you mostly view generative AI as a system that produces finished products, which you don’t like. And for some reason that is how they are marketed, but it’s a terrible use for them. To me the point of generative AI is as a tool for an expert user to use en route to achieving some larger goal. I can personally attest that it works really well for me in this regard, as can many others. I think if I tried to use LLMs as surrogate experts I would be very frustrated, since they are totally unready for that purpose.
  - caseyy 2y ago
    This conversation has drifted slightly off-topic. It was originally about the efficiency of producing good quality outputs akin to products or services with generative AI. I work in an industry adjacent to generative AI, so my views are very specific, and kind of beside the point. But I think the broader public does think what you describe. Much (and probably most) of AI-generated content that ends up on the internet is very barely changed by a human. I am not saying gen AI is not useful, but that it's not efficient in the process of making high quality content. It's simply not steerable enough in practice. In creative industries, people are going pretty wild about how much work AI can replace, but all I've seen is mediocrity and failure when it is involved. Or frustration, as you say, that what it outputs is very difficult to turn into a high-quality product.
    - tibbar 2y ago
      I agree. The state of the art of gen AI systems is well below a typical human expert in every domain. Here human expert is well below the bar of 'world class', more like 'some one with 2-5 years of professional experience in that area.' So, if you have access to someone like that, it's a no-brainer to use their work instead of gen AI system. The interesting use case which has emerged is that there are a lot of times where it would be really helpful to me to have a short conversation with an expert on a topic adjacent to my own expertise. And it turns out that for those conversations, talking to ChatGPT is much better than talking to no one; it can help me with the kind of things someone would learn in the first few months on the job in that area, things a little too hard to google but where a human expert is not readily available. I think this is the best, maybe the only, professional use case for GenAI right now -- advice and limited assistance in areas just outside your area of expertise, such that you don't need to depend directly on its output and can easily check/integrate the work.

by tibbar 2y ago

deleted 2y ago

[deleted]

bruce511 2y ago

I concur. I'm notca huge user of AI, but it does make a good search tool when my search term is verbose. For example I was looking for the name of a windows API command. I "knew" the command must exist, but didn't gave a clue what it was called. Asked Cgpt for an example program, and there's the name of the API. (Which I can then Google for docs.) I also had a complicated-to-ask question about sun movement which it explained to me, along with site links to actual data. I'm not using it as a Google replacement, but more of a Google supplement when the question is long-winded to write.

caseyy 2y ago

I think a lot of people drastically underestimate how much the LLMs mislead the user with plausible-sounding answers. I would say that whatever % of queries you found them not to give you correct answers, the actual percentage is materially higher. In my work niche, which at least 10,000 other software engineers do each day, ChatGPT 4 and 4o almost never give correct answers. Usually, they are misleading. I do find myself trying the LLMs when I'm faced with a challenging problem, but in that scenario they have not been helpful once. Granted, there are areas of work that are much more popular and LLMs will be more helpful there. But these are individual scenarios, and we can find many where LLMs are fantastic and many where LLMs are awful if we wanted to cherry-pick. Overall, the quality of the content is lower than what a human professional would do in their area of work, including flexibility, interactivity, and accessibility. This extends to books written by professionals and lectures given by them, as well as work carried out by them. It applies to home carpentry as much as it does to neurosurgery. The more specific the knowledge has to be, the more this is true. The more generic, the less. But all-in-all, I think it's still very evident humans produce higher quality knowledge and content. And any quantized model of that content and knowledge will be unable to reproduce it at the same fidelity or quality. With that said, I hope I expressed this enough — I do see your point in some circumstances. Just not overall.

tibbar 2y ago
It feels like you mostly view generative AI as a system that produces finished products, which you don’t like. And for some reason that is how they are marketed, but it’s a terrible use for them. To me the point of generative AI is as a tool for an expert user to use en route to achieving some larger goal. I can personally attest that it works really well for me in this regard, as can many others. I think if I tried to use LLMs as surrogate experts I would be very frustrated, since they are totally unready for that purpose.
- caseyy 2y ago
  This conversation has drifted slightly off-topic. It was originally about the efficiency of producing good quality outputs akin to products or services with generative AI. I work in an industry adjacent to generative AI, so my views are very specific, and kind of beside the point. But I think the broader public does think what you describe. Much (and probably most) of AI-generated content that ends up on the internet is very barely changed by a human. I am not saying gen AI is not useful, but that it's not efficient in the process of making high quality content. It's simply not steerable enough in practice. In creative industries, people are going pretty wild about how much work AI can replace, but all I've seen is mediocrity and failure when it is involved. Or frustration, as you say, that what it outputs is very difficult to turn into a high-quality product.
  - tibbar 2y ago
    I agree. The state of the art of gen AI systems is well below a typical human expert in every domain. Here human expert is well below the bar of 'world class', more like 'some one with 2-5 years of professional experience in that area.' So, if you have access to someone like that, it's a no-brainer to use their work instead of gen AI system. The interesting use case which has emerged is that there are a lot of times where it would be really helpful to me to have a short conversation with an expert on a topic adjacent to my own expertise. And it turns out that for those conversations, talking to ChatGPT is much better than talking to no one; it can help me with the kind of things someone would learn in the first few months on the job in that area, things a little too hard to google but where a human expert is not readily available. I think this is the best, maybe the only, professional use case for GenAI right now -- advice and limited assistance in areas just outside your area of expertise, such that you don't need to depend directly on its output and can easily check/integrate the work.