4 ms·
"Consultants across the skills distribution benefited significantly from having AI augmentation, with those below the average performance threshold increasing b
by datadrivenangel 3y ago
"Consultants across the skills distribution benefited significantly from having AI augmentation, with those below the average performance threshold increasing by 43% and those above increasing by 17% compared to their own scores. For a task selected to be outside the frontier, however, consultants using AI were 19 percentage points less likely to produce correct solutions compared to those without AI."
So AI makes consultants faster but worse. As a consultant, this should make my expertise even more valuable.
- tempest_ 3y agoOnly if being wrong actually matters.
- datadrivenangel 3y agoThe research was partially conducted by Boston Consulting Group, so in this case being wrong doesn't really matter as long as the Partners are smooth and the associates create billable hours.
- CharlesW 3y ago> So AI makes consultants faster but worse. That was just for tasks not suited to GPT-4 (and I assume, LLMs in general). > "For a task selected to be outside the frontier, however, consultants using AI were 19 percentage points less likely to produce correct solutions compared to those without AI." For tasks suited to GPT-4, consultants produced 40% higher quality results by leveraging GPT-4. My take on that is that people who have at least some understanding of how LLMs work will be at a significant advantage, while people who think LLMs "think" will misuse LLMs and get predictably worse results.
- ParetoOptimal 3y ago> while people who think LLMs "think" will misuse LLMs and get predictably worse results. I've found that now that I see LLM's less as thinking and more as stochiastic parrot I get a lot less use from them because I give 3-5 turns at most versus 10 turns.
- NemoNobody 3y agoWell, maybe a balance between a stoic parrot and a "thinking machine" might be the best approach. You know, everything in moderation and what not.
- ParetoOptimal 3y agoYes, ideally. The problem for me is it used to be an addition to my energy and confidence, but when you "take the magic out of it" it doesn't help out there so much. In antirez's blog he mentioned how it made things "not worth it, worth it" and I agreed, but that happens much less often if you think an LLM helping you is a fluke.
- NemoNobody 3y agoSeriously, you cherry picked the paragraph you use to cite your opinion, which contrary to the finding of the article itself and completely ignore the the first part of the second sentence of your paragraph "for a task selected outside the frontier" - so, something beyond the capabilities of AI at this time ONLY decreased their correct solutions 19 percentage points. 43% better it made below average people, 17% better it made above average people. Same paragraph.
- steve1977 3y agoBCG consultants. Not average people.
- datadrivenangel 3y agoYou know, I missed that part. Oops. Looking into the full paper, it looks like the task identified inside the frontier is much more broken down, with 18 numbered steps. The outside the frontier task is two bullet points. So for clearly scoped tasks, ChatGPT is hands down good. For less well scoped work it reduces quality. Interestingly, the authors also looked at the amount of text retained from ChatGPT, and found that people who received ChatGPT training ended up retaining a larger portion of the ChatGPT output in their final answers and performed better in the quality assessment.
- quickcheque 3y ago' As a consultant, this should make my expertise even more valuable. ' Consultant logic at its finest.
- j4yav 3y agoMore garbage outputs being produced faster, paired with expertise which makes it credibly sound like more fast garbage will solve any remaining issues, probably _is_ approaching peak consultant value.
- datadrivenangel 3y agoOnly if that's what the clients want or need to justify actual improvements! Usually the goal is to produce less garbage outputs slower.
- j4yav 3y agoThe goal isn’t to generate billable hours?
- datadrivenangel 3y agoWe like to find win-win situations.