6 ms·

Says more about how useless BCG consultants are.

vorticalbox 3y ago
Somewhat agree, I know LLM have boosted my programming output mostly in writing jsdocs and pr descriptions. The things I don't really like doing
- cube00 3y ago
  If your docs and PR descriptions can be generated off file diffs everyone's time could be better spent scanning the diff to come to the same conclusions. Consider using your PRs and docs to capture the answers to the usual why questions which LLM won't be able to do.
  - devin 3y ago
    Ah yes, but that would require actual effort, and in the end is only going to serve to improve someone else’s model.
  - vorticalbox 3y ago
    The why is largely in the ticket and the what in the pr.
    - dbalatero 3y ago
      Huh, your tickets aren't just a single vague title sentence and no description body?
      - vorticalbox 3y ago
        Sometimes this is the case but most tickets have a detailed info of the bug or links to a confluence page of design specs.
    - zten 3y ago
      I've seen code bases survive three different ticket management systems. Meanwhile, the tickets never made it between the different systems, so if the 'why' isn't in the commit message, then it got lost to time. I will admit that a lot of the really old decisions don't have much relevance to the current business, but the historical insight is sometimes nice.
olivierduval 3y ago
Agreed: the study only shows that BCG consultant's work is 40% noise without real added value... I guess that customers should now ask for a 40% rebates !!! ;-)
dopylitty 3y ago
I’m starting to think there’s an LLM equivalent to the old saying about how everything the media writes is accurate except on the topics you’re an expert in. All LLM output looks to be good quality except when it’s output you’re an expert in. People who have no background in writing or editing think LLMs will revolutionize those fields. Actual writers and editors take one look at LLM output and can see it’s basically valueless because the time taken to fix it would be equivalent to the time taken to write it in the first place. Similarly people who are poor programmers or have only a surface level understanding of a topic (especially management types who are trying to appear technical) look at LLM output and think it’s ready to ship but good programmers recognize that the output is broken in so many ways large and small that it’s not worth the time it would take to fix compared to just writing from scratch.
- catchnear4321 3y ago
  everyone you described share something in common. they aren’t good at using language models.
  - mikro2nd 3y ago
    Nor are 99.9% of humanity. I think that's the point.
- skybrian 3y ago
  Programmers don't think that, though, or least not all the time. You could say similar things about Stack Overflow, and yet we use it.
  - jprete 3y ago
    Stack Overflow responses are well known to be misranked. I’ve heard a rule of thumb that the actual correct answer is typically about #3.
    - moffkalast 3y ago
      And #1 is usually broken or wrong, due to its (typically) old age. The longer it has to accumulate upvotes the less relevant it becomes.
famouswaffles 3y ago
Says more about how people will parrot the same phrase over and over for anything at all. It's just funny how you can predict a comment like this in every thread regardless of what it does. "It says more about [insert]" anytime GPT does something just makes the phrase lose all meaning. Surely you have something meaningful to say?
- jprete 3y ago
  Often effortposts aren’t worth it because someone will come along and Gish Gallop the post with opaquely nonsensical bad-faith counterarguments that are a lot of work to refute. I agree with you in an ideal world, but sadly this isn’t one.
wqtz 3y ago
Most management consultants are useless. But there are some realities you must accept. Number 1. In a team of 20-30 engineers there is only one extremely god "why is he with us" engineers who is great at technical stuff and being a people person. However, no matter how nice he is his approach to his job, it is a job and I will only drop hints how the management should be done. He doesn't care about where the company is headed because he plays video games, has a family and has a literal life. He doesn't care about management and taking on undue responsibilities. Moreover, the people up to has a label for him as an "engineer" does not see as a "manager". For the rest of the engineers and managers, have also adopted the approach of "not my problem", you see a bizarre communication gap. Engineers working closesly with the product don't want to talk to their managers, becase the conversation goes like "if you know this so much, why don't you.... <a description of something results in more work that goes outside their JD>" and managers don't want to talk with engineers because "if you are you so interested, why don't you.... <a description of something results in more work that goes outside their JD>" From this progressive distance between managers and engineers comes the "manaegment consultant". Management consultant have the upper management given flexibility of going back and forth between engineers and managers. They can have conversations with full flexibility but they are not bound to "why don't you...." phrases. They can talk with anyone and submit a report and take home 1 years worth of salary of managers/engineers in 1 month. The conversation gap between product and business where management consultants come in. And the funny thing is that, management consultants target those "I don't want to but I should" work things and report to the upper management. They can do this so well, because they are not burdened with the "work" part. Seriously, if you do some introspection, you will see there is plenty of things you know your company should do, but you don't want to voice them because it results in more work and in fact more risk. There comes a "good" management consultant who will discover those things and report to upper management who will create the system to get those jobs done. That is my pitch if anyone wants a management consultant hire me. I am going to tell them why their company sucks in 20 different ways with 18 of those points being generated by ChatGPT.
- jprete 3y ago
  Needs an /s.
ftxbro 3y ago
So when AI is better at humans at everything, the takeaway will be that humans weren't so great after all?
lagrange77 3y ago
As I understand it, they have a very specific purpose. The customer needs someone to blame in making difficult decisions. The difficult decision process itself is secondary.
momirlan 3y ago
perfect tool for a consultancy: take a fresh graduate, pair it with a LLM tool and charge big bucks. not much different from current but the client will get a much more confident consultant and will be happy to fork more money.
amelius 3y ago
And how even more useless they will be in the near future.
fluidcruft 3y ago
Yeah, that was my thought too... alternative headline: "ChatGPT-4 significantly decreases the need for business consultants".

Somewhat agree, I know LLM have boosted my programming output mostly in writing jsdocs and pr descriptions. The things I don't really like doing

cube00 3y ago
If your docs and PR descriptions can be generated off file diffs everyone's time could be better spent scanning the diff to come to the same conclusions. Consider using your PRs and docs to capture the answers to the usual why questions which LLM won't be able to do.
- devin 3y ago
  Ah yes, but that would require actual effort, and in the end is only going to serve to improve someone else’s model.
- vorticalbox 3y ago
  The why is largely in the ticket and the what in the pr.
  - dbalatero 3y ago
    Huh, your tickets aren't just a single vague title sentence and no description body?
    - vorticalbox 3y ago
      Sometimes this is the case but most tickets have a detailed info of the bug or links to a confluence page of design specs.
  - zten 3y ago
    I've seen code bases survive three different ticket management systems. Meanwhile, the tickets never made it between the different systems, so if the 'why' isn't in the commit message, then it got lost to time. I will admit that a lot of the really old decisions don't have much relevance to the current business, but the historical insight is sometimes nice.

olivierduval 3y ago

Agreed: the study only shows that BCG consultant's work is 40% noise without real added value... I guess that customers should now ask for a 40% rebates !!! ;-)

dopylitty 3y ago

I’m starting to think there’s an LLM equivalent to the old saying about how everything the media writes is accurate except on the topics you’re an expert in. All LLM output looks to be good quality except when it’s output you’re an expert in. People who have no background in writing or editing think LLMs will revolutionize those fields. Actual writers and editors take one look at LLM output and can see it’s basically valueless because the time taken to fix it would be equivalent to the time taken to write it in the first place. Similarly people who are poor programmers or have only a surface level understanding of a topic (especially management types who are trying to appear technical) look at LLM output and think it’s ready to ship but good programmers recognize that the output is broken in so many ways large and small that it’s not worth the time it would take to fix compared to just writing from scratch.

catchnear4321 3y ago
everyone you described share something in common. they aren’t good at using language models.
- mikro2nd 3y ago
  Nor are 99.9% of humanity. I think that's the point.
skybrian 3y ago
Programmers don't think that, though, or least not all the time. You could say similar things about Stack Overflow, and yet we use it.
- jprete 3y ago
  Stack Overflow responses are well known to be misranked. I’ve heard a rule of thumb that the actual correct answer is typically about #3.
  - moffkalast 3y ago
    And #1 is usually broken or wrong, due to its (typically) old age. The longer it has to accumulate upvotes the less relevant it becomes.

famouswaffles 3y ago

Says more about how people will parrot the same phrase over and over for anything at all. It's just funny how you can predict a comment like this in every thread regardless of what it does. "It says more about [insert]" anytime GPT does something just makes the phrase lose all meaning. Surely you have something meaningful to say?

jprete 3y ago
Often effortposts aren’t worth it because someone will come along and Gish Gallop the post with opaquely nonsensical bad-faith counterarguments that are a lot of work to refute. I agree with you in an ideal world, but sadly this isn’t one.

wqtz 3y ago

Most management consultants are useless. But there are some realities you must accept. Number 1. In a team of 20-30 engineers there is only one extremely god "why is he with us" engineers who is great at technical stuff and being a people person. However, no matter how nice he is his approach to his job, it is a job and I will only drop hints how the management should be done. He doesn't care about where the company is headed because he plays video games, has a family and has a literal life. He doesn't care about management and taking on undue responsibilities. Moreover, the people up to has a label for him as an "engineer" does not see as a "manager". For the rest of the engineers and managers, have also adopted the approach of "not my problem", you see a bizarre communication gap. Engineers working closesly with the product don't want to talk to their managers, becase the conversation goes like "if you know this so much, why don't you.... <a description of something results in more work that goes outside their JD>" and managers don't want to talk with engineers because "if you are you so interested, why don't you.... <a description of something results in more work that goes outside their JD>" From this progressive distance between managers and engineers comes the "manaegment consultant". Management consultant have the upper management given flexibility of going back and forth between engineers and managers. They can have conversations with full flexibility but they are not bound to "why don't you...." phrases. They can talk with anyone and submit a report and take home 1 years worth of salary of managers/engineers in 1 month. The conversation gap between product and business where management consultants come in. And the funny thing is that, management consultants target those "I don't want to but I should" work things and report to the upper management. They can do this so well, because they are not burdened with the "work" part. Seriously, if you do some introspection, you will see there is plenty of things you know your company should do, but you don't want to voice them because it results in more work and in fact more risk. There comes a "good" management consultant who will discover those things and report to upper management who will create the system to get those jobs done. That is my pitch if anyone wants a management consultant hire me. I am going to tell them why their company sucks in 20 different ways with 18 of those points being generated by ChatGPT.

jprete 3y ago
Needs an /s.

ftxbro 3y ago

So when AI is better at humans at everything, the takeaway will be that humans weren't so great after all?

lagrange77 3y ago

As I understand it, they have a very specific purpose. The customer needs someone to blame in making difficult decisions. The difficult decision process itself is secondary.

momirlan 3y ago

perfect tool for a consultancy: take a fresh graduate, pair it with a LLM tool and charge big bucks. not much different from current but the client will get a much more confident consultant and will be happy to fork more money.

amelius 3y ago

And how even more useless they will be in the near future.

fluidcruft 3y ago

Yeah, that was my thought too... alternative headline: "ChatGPT-4 significantly decreases the need for business consultants".