13 ms·
ChatGPT-4 significantly increased performance of business consultants
- Projectiboga 3y agoLet the Turbocharged En-Shitifications commence.
- seydor 3y ago"Productivity returned to 100% after consultant was eliminated"
- mgaunard 3y agoSays more about how useless BCG consultants are.
- vorticalbox 3y agoSomewhat agree, I know LLM have boosted my programming output mostly in writing jsdocs and pr descriptions. The things I don't really like doing
- cube00 3y agoIf your docs and PR descriptions can be generated off file diffs everyone's time could be better spent scanning the diff to come to the same conclusions. Consider using your PRs and docs to capture the answers to the usual why questions which LLM won't be able to do.
- devin 3y agoAh yes, but that would require actual effort, and in the end is only going to serve to improve someone else’s model.
- vorticalbox 3y agoThe why is largely in the ticket and the what in the pr.
- dbalatero 3y agoHuh, your tickets aren't just a single vague title sentence and no description body?
- vorticalbox 3y agoSometimes this is the case but most tickets have a detailed info of the bug or links to a confluence page of design specs.
- zten 3y agoI've seen code bases survive three different ticket management systems. Meanwhile, the tickets never made it between the different systems, so if the 'why' isn't in the commit message, then it got lost to time. I will admit that a lot of the really old decisions don't have much relevance to the current business, but the historical insight is sometimes nice.
- olivierduval 3y agoAgreed: the study only shows that BCG consultant's work is 40% noise without real added value... I guess that customers should now ask for a 40% rebates !!! ;-)
- dopylitty 3y agoI’m starting to think there’s an LLM equivalent to the old saying about how everything the media writes is accurate except on the topics you’re an expert in. All LLM output looks to be good quality except when it’s output you’re an expert in. People who have no background in writing or editing think LLMs will revolutionize those fields. Actual writers and editors take one look at LLM output and can see it’s basically valueless because the time taken to fix it would be equivalent to the time taken to write it in the first place. Similarly people who are poor programmers or have only a surface level understanding of a topic (especially management types who are trying to appear technical) look at LLM output and think it’s ready to ship but good programmers recognize that the output is broken in so many ways large and small that it’s not worth the time it would take to fix compared to just writing from scratch.
- catchnear4321 3y agoeveryone you described share something in common. they aren’t good at using language models.
- mikro2nd 3y agoNor are 99.9% of humanity. I think that's the point.
- skybrian 3y agoProgrammers don't think that, though, or least not all the time. You could say similar things about Stack Overflow, and yet we use it.
- jprete 3y agoStack Overflow responses are well known to be misranked. I’ve heard a rule of thumb that the actual correct answer is typically about #3.
- moffkalast 3y agoAnd #1 is usually broken or wrong, due to its (typically) old age. The longer it has to accumulate upvotes the less relevant it becomes.
- famouswaffles 3y agoSays more about how people will parrot the same phrase over and over for anything at all. It's just funny how you can predict a comment like this in every thread regardless of what it does. "It says more about [insert]" anytime GPT does something just makes the phrase lose all meaning. Surely you have something meaningful to say?
- jprete 3y agoOften effortposts aren’t worth it because someone will come along and Gish Gallop the post with opaquely nonsensical bad-faith counterarguments that are a lot of work to refute. I agree with you in an ideal world, but sadly this isn’t one.
- wqtz 3y agoMost management consultants are useless. But there are some realities you must accept. Number 1. In a team of 20-30 engineers there is only one extremely god "why is he with us" engineers who is great at technical stuff and being a people person. However, no matter how nice he is his approach to his job, it is a job and I will only drop hints how the management should be done. He doesn't care about where the company is headed because he plays video games, has a family and has a literal life. He doesn't care about management and taking on undue responsibilities. Moreover, the people up to has a label for him as an "engineer" does not see as a "manager". For the rest of the engineers and managers, have also adopted the approach of "not my problem", you see a bizarre communication gap. Engineers working closesly with the product don't want to talk to their managers, becase the conversation goes like "if you know this so much, why don't you.... <a description of something results in more work that goes outside their JD>" and managers don't want to talk with engineers because "if you are you so interested, why don't you.... <a description of something results in more work that goes outside their JD>" From this progressive distance between managers and engineers comes the "manaegment consultant". Management consultant have the upper management given flexibility of going back and forth between engineers and managers. They can have conversations with full flexibility but they are not bound to "why don't you...." phrases. They can talk with anyone and submit a report and take home 1 years worth of salary of managers/engineers in 1 month. The conversation gap between product and business where management consultants come in. And the funny thing is that, management consultants target those "I don't want to but I should" work things and report to the upper management. They can do this so well, because they are not burdened with the "work" part. Seriously, if you do some introspection, you will see there is plenty of things you know your company should do, but you don't want to voice them because it results in more work and in fact more risk. There comes a "good" management consultant who will discover those things and report to upper management who will create the system to get those jobs done. That is my pitch if anyone wants a management consultant hire me. I am going to tell them why their company sucks in 20 different ways with 18 of those points being generated by ChatGPT.
- jprete 3y agoNeeds an /s.
- ftxbro 3y agoSo when AI is better at humans at everything, the takeaway will be that humans weren't so great after all?
- lagrange77 3y agoAs I understand it, they have a very specific purpose. The customer needs someone to blame in making difficult decisions. The difficult decision process itself is secondary.
- momirlan 3y agoperfect tool for a consultancy: take a fresh graduate, pair it with a LLM tool and charge big bucks. not much different from current but the client will get a much more confident consultant and will be happy to fork more money.
- amelius 3y agoAnd how even more useless they will be in the near future.
- fluidcruft 3y agoYeah, that was my thought too... alternative headline: "ChatGPT-4 significantly decreases the need for business consultants".
- _pferreir_ 3y agoMaybe this tells more about BCG consultants than its does about GPT-4?
- brabel 3y agoThat's what you would like to think, isn't it? I'm afraid this would be just as much true with any other kind of subjects, and as far as I know, there's no evidence either way so this is just a cheap stab you're having at them.
- mawadev 3y agoAfter all the cheap stabs I had to take as a programmer... I allow myself to experience schadenfreude, even if there is no evidence...
- refurb 3y agoMeh.. I mean a lot of consulting is tasks like writing or idea generation. Using something like chat GPT to do it [faster, better] doesn't negate the value in what they do, since they are hired to do those tasks, those tasks are required for the broader work.
- awestroke 3y agoNot surprised. It's frighteningly good, and a perfect match for programming. I often ask GPT4 to write code for something, and try if it works, but I seldom copy and paste the code it writes - I rewrite it myself to fit into the context of the codebase. But it saves me a lot of time when I am unsure about how to do something. Other times I don't like the suggestion at all, but that's useful as well, as it often clarifies the problem space in my head.
- DonHopkins 3y agoIt's a hell of an articulate rubber duck! https://en.wikipedia.org/wiki/Rubber_duck_debugging https://en.wikipedia.org/wiki/Rubber_duck_debugging
- bryancoxwell 3y agoI’ve also found the act of describing my problem to GPT4 is sometimes just a helpful as the answer itself. It’s almost like enhanced rubber duck debugging.
- dkjaudyeqooe 3y agoWe need an inverse GPT4-style LLM that doesn't provide answers but instead asks relevant questions.
- awestroke 3y agoGPT4 can do that too. Just show it something (code or text) and ask it to ask coaching questions about it.
- seanhunter 3y agoI have tried adding prompts like this and it works really well. "Rather than giving me the answer, guide me using questions in the Socratic method".
- airstrike 3y agoSo true. I've written entire prompts with several lines worth of explanation, only to realize what my issue was and never hit the "send" button. Guess I should do that more often in life, in general
- pydry 3y agoBCG : We know layoffs are in fashion and we'd just like you to know that if you need industrial grade ass covering excuses from a legitimate-ish sounding authority to justify what you were planning to do anyway, our 23 year old consultants and their PowerPoint presentations have got you covered.
- _jplc 3y agoPipe /dev/random, transform to decimal, and you just got an amazing increase in performance for calculating decimals of Pi. Nobody said precision was important anyway.
- segfaltnh 3y agoHonestly if you don't care about precision, /dev/zero is going to give you more throughput. Plus, I personally guarantee it's correct to within an error margin of 4.0. You can't offer the same with /dev/random!
- iudqnolq 3y agoReminds me of the study that found a massive change in GPT's proficiency at identifying primes. It switched from always guessing composite to always guessing prime. Much less accurate.
- ShamelessC 3y agoWhat do you mean?
- DonHopkins 3y agoI always wanted the minor number of the device /dev/zero uses to select the byte you get, so if you go "mknod /dev/seven c 1 7" that would make an infinite source of beeps!
- seanhunter 3y agoWe're not trying to hit a comet with a rocket here. 1 significant figure is more than sufficient for an initial consultation. Any additional accuracy required would be billable follow-on work.
- leoff 3y agoThis is a good thing, since increased perfomance means that the clients will have less billed hours, right? Right?
- ellyagg 3y agoNo, it increases the load one can successfully manage in a day. There isn't this tiny discrete amount of work that people need to handle. We gave that up when we left the campfires. We're trying to grow.
- belter 3y agoWithing 3 months, companies will be applying to Y Combinator, where both Founder and CEO are ML Models... :-)
- MrThoughtful 3y agoWhat LLMs do for me is that they make me a pro in every programming language. "How do I do x in language y" always gives me the knowledge I need. Within seconds, I can continue coding. After more than 10 years of coding fulltime, I know some languages very well, like PHP and Javascript. But even in those, LLMs often come up with a better solution than what I wrote. Because they know every fricking thing about those languages.
- hubraumhugo 3y agoWell, this sounds like perfect tasks for GPT: "Participants responded to a total of 18 tasks (or as many as they could within the given time frame). These tasks spanned various domains. Specifically, they can be categorized into four types: creativity (e.g., “Propose at least 10 ideas for a new shoe targeting an underserved market or sport.”), analytical thinking (e.g., “Segment the footwear industry market based on users.”), writing proficiency (e.g., “Draft a press release marketing copy for your product.”), and persuasiveness (e.g., “Pen an inspirational memo to employees detailing why your product would outshine competitors.”)." Here is the GPT response to the first task: https://chat.openai.com/share/db7556f7-6036-4b3d-a61a-9cd253f094fd https://chat.openai.com/share/db7556f7-6036-4b3d-a61a-9cd253... A confident GPT hallucination is almost indistinguishable from typical management consulting material...
- smeej 3y agoSeveral of those aren't even new.
- highwaylights 3y ago1) Your ideas are bad. 2) Spreadsheets exist. 3) No-one cares about your marketing copy. 4) No-one finds your c-suite babble inspirational. This is almost perfect input to an LLM exactly because of how low value it is in the first place.
- djtango 3y agoHa it's fun to dunk on management consultants but I think the magic is they are like pop music producers. Somehow they're able to make the C suite hoover up LLM shovelware the same way top producers can take super obvious music and sell I V vi IV but when we try the same chords it's uninspired and no one wants to listen to it
- FrustratedMonky 3y agoOr, you have a break out hit like the Spice Girls. If it was so bad, then why do people listen? There is still a market. Does the market suck? Full of idiots? Your argument ends up being that successful things are bad, because humans are just idiots and thus if something is successful it is because it is just liked by idiots. As much as I might agree generally, it doesn't get you far.
- jakey_bakey 3y ago> Two distinct patterns of AI use emerged: “Centaurs,” who divided and delegated tasks between themselves and the AI, and “Cyborgs,” who integrated their workflow with the AI. It was nice of them to explain that the article was total horse dung before having to read the whole paper
- sebzim4500 3y agoThe article certainly didn't invent the term 'Centaur', but I haven't seen cyborg used in that way. It does seem a bit clickbaity.
- niels_bom 3y agoI like Cory Doctorow’s use in “Chickenized Reverse Centaur”.
- Jeff_Brown 3y agoPowerful stuff. If anyone wants a short explanation here's one: https://m.youtube.com/watch?v=6pieIoEi8Ds https://m.youtube.com/watch?v=6pieIoEi8Ds I love it when a complicated set of conditions can be losslessly encoded in a short, easily remembered label.
- qingcharles 3y agoThis was really profound, thank you.
- mensetmanusman 3y agoCentuar is a term of art in the chess community: https://historyofinformation.com/detail.php?entryid=4724 https://historyofinformation.com/detail.php?entryid=4724
- digitcatphd 3y agoIndeed, I had two double check the authors before finally quickly browsing over it in disappointment
- simonmesmith 3y agoHaving been a consultant, what strikes me about this is the next, to me seemingly obvious question: What if you just removed the consultants entirely and just had GPT-4 do the work directly for the client? If you’re a client and need a consultant to do something, you have to explain the requirement to them, review the work, give feedback, and so forth. There will likely be a few meetings in there. But if GPT-4 can make consultants so much better, I imagine it can also do their work for them. And if you combine this with the reduction in communications overhead that comes from not working with an outside group, why wouldn’t clients just accrue all the benefits to themselves, plus the benefit of not paying outside consultants or dealing with the overhead of managing them? This is especially the case when the client is already a domain expert but just needs some additional horsepower. For example, marketing brand managers may work with marketing consultants even though they know their products and marketing very well. They just need more resources, which can come in the form of consultants for reasons such as internal head-count restrictions. Anyway, I just wonder if BCG thought through the implications of participating in this study. To me it feels like a very short step from “helps consultants help their clients” to “helps clients directly and shows consultants aren’t really necessary.” Especially so if the client just hires an intern and gives them GPT-4.
- deleted 3y ago[deleted]
- chaosbolt 3y agoCompanies like BCG and McKinsey are mostly about liability, as a CEO you call them, pay them the big bucks, have them make up plans and strategies, if it works out you get the credit, if it doesn't then well "we worked tightly with experts from McKinsey, etc. so the blame isn't on me"
- fragmede 3y agoThe frustrating one is when you've been telling management something for months (if not years), and the consultant comes in, and their report says what you been saying, and only then does the company finally do what you've been saying all along! Coulda saved the company 5-figures just listening to me. sigh politics.
- anfelor 3y agoThe headline "GPT-4 increased BCG consultants’ performance by over 40%" is misleading since it implies that they became more productive in their actual work, when this is a carefully controlled study that separates tasks by an "AI frontier". Only inside the frontier did the "quality" of work increase by 40%, while they completed 12% more tasks on average.
- skepticATX 3y agoExactly, and right in the introduction they even say: > while AI can actually decrease performance when used for work outside of the frontier There is some value here but the authors can define "frontier" however they please to end up with whatever productivity increase they are looking for.
- croes 3y agoSo they don't check the results for the clients.
- skepticATX 3y agoTwo things mentioned in the abstract that are worth pointing out. > For each one of a set of 18 realistic consulting tasks within the frontier of AI capabilities They specifically picked tasks that GPT-4 was capable of doing. GPT-4 could not do many tasks, so when we say that performance was significantly increased this is only for tasks GPT-4 is well suited to. There is still value here but let's put these results into context. > Consultants across the skills distribution benefited significantly from having AI augmentation, with those below the average performance threshold increasing by 43% and those above increasing by 17% compared to their own scores Even when cherry-picking tasks that GPT-4 is particularly suited for, above average performers only increased performance by 17%. This increase is still impressive, were it to be seen across the board. But I do think that 17% is a lot less than some people are trying to sell.
- ellyagg 3y agoYou're underestimating because it compounds. Small gains in efficency lead to huge advantages in long term growth. 17% would be absolutely monumental improvement.
- visarga 3y agoyet insignificant compared to how full automation would look, not 1.17x but 1000x but I can't find any automated AI tasks of critical importance, they all need human support
- denton-scratch 3y agoHmmm. Perhaps below-average performers are more likely to take GPT output at face-value, being less competent to review and edit it. And above-average performers are more likely to hack the GPT output around, because they're confident in their own abilities. Therefore below-average types will produce finished output more quickly; and this was a time-constrained test, so velocity matters. ChatGPT is very good at waffling, and marketing-speak and inspirational messages are essentially waffle. IOW, the tasks were tailor-made for unaided ChatGPT, so high-performers were penalized.
- xbmcuser 3y agoThere is a lot of office work that will overtime be optimized over time using gpt like services. I was tech savvy enough to know that a lot of office work that I do is repeatable and can be done using scripts but not good enough to write those scripts myself. Using Chat gpt allowed me to write those scripts it took me I think 15-20hrs to get the scripts working perfectly. I knew just a little bit of python scripting did not know anything about python pandas or xls writer etc but was able to create something that saves me I would estimate 20-25 hours a week. In my opinion a lot of people here on hackernews as they are themselves good at programing underestimate how services like chat gpt can open a new world to non programmers. They also probably make the non inquisitive learn less. Previously to learn how to stop multiple snapd services using a script I would have googled and then cobbled together something today I just ask chatgpt and get a working script in less than a min.
- dontupvoteme 3y agoCouldn't agree more. I've gone multiple times now from "I wonder if X is possible/how would you do X" to hacking out a crude proof of concept to a problem that I wouldn't even know how to google.
- helsinki 3y agoNo shit.
- benreesman 3y agoLLMs are stunningly good at language tasks: almost all of what us old-timers called NLP is just crushed these days. Summarization, Q&A, sentiment, the list goes on and on. Truly remarkable stuff. And where there isn’t a bright line around “fact”, and where it doesn’t need to come together like a Pynchon novel, the generative stuff is smoking hot: short-form fiction, opinion pieces, product copy? Massive productivity booster, you can prototype 20 ideas in one minute. But that’s about where we are: lift natural language into a latent space with some clear notion of separability, do some affine (ish) transformations, lower back down. Fucking impressive for a computer. But if it can really carry water for an expensive Penn grad? You’re paying for something other than blindingly insightful product strategy.
- Jeff_Brown 3y agoI wonder how long it takes AI to get good at law. Right now the verbal tasks it excels at are similar to the artistic ones: namely, solving problems with enormous solution spaces that are robust to small perturbations. That is, change a good picture of an angry tree man slightly and it's still probably a good picture of an angry tree man.
- Dr4kn 3y agoDepends what you would classify as good, but IBM Watson is already used in law firms [today](https://www.ibm.com/case-studies/legalmation https://www.ibm.com/case-studies/legalmation) LLMs iare most often best at helping humans do their tasks more effectively, not replacing them completely
- qingcharles 3y agoI've tried using a lot for writing motions. It can actually do a pretty decent job of writing motions, and it can come up with some arguments that you might not have thought of. You just have to ignore all its citations and look everything up yourself, otherwise this: https://www.reuters.com/legal/new-york-lawyers-sanctioned-using-fake-chatgpt-cases-legal-brief-2023-06-22/ https://www.reuters.com/legal/new-york-lawyers-sanctioned-us...
- 3y ago
- olalonde 3y agoHN is so bad at predictions. Just a few months ago HN was awash with comments that confidently claimed LLMs were no more than stochastic parrots and unlikely to amount to anything. > I can't help but think the next AI winter is around the corner. [0] Yeah, right. [0] https://news.ycombinator.com/item?id=23886325 https://news.ycombinator.com/item?id=23886325
- ResearchCode 3y agoWho claimed management consultants are not stochastic parrots?
- famouswaffles 3y agoBeing bad at predictions is ok. It's the absolute lack of re-calibration that does me in. If you make a hilariously bad prediction then that tells you your model about that thing is off and needs correcting. So if you do nothing to that model and still make predictions...
- Xcelerate 3y agoWe’re going to have legit AGI that can outperform humans in every way and HN will still find something to complain about. I love the tech news on here, but the constant cynicism on everything is exhausting.
- maxdoop 3y ago“It’s AGI, but does it really understand anything? And I can’t even load the AGI info my Linux mainframe — how useless . Just another crypto wave.”
- ResearchCode 3y agoHave "AGI" outperform truck drivers first. They said autonomous trucks would replace all truck drivers by 2018.
- refurb 3y agoI'm not sure this paper is proof of much? Regurgitating press releases is sort of a stochastic parrot task.
- user_named 3y agoReminder to not hire people who worked at MBB.
- AdamCraven 3y agoWell, they buried the lede with this one. Using LLMs were better for some tasks and actually made it worse for others. The first task was a generalist task ("inside the frontier" as they refer to it), which I'm not surprised has improved performance, as it purposely made to fall into an LLM's areas of strength: research into well-defined areas where you might not have strong domain knowledge. This also is the mainstay of early consultants' work, in which they are generalists in their early careers – usually as business analysts or similar – until they become more valuable and specialise later on. LLMs are strong in this area of general research because they have generalised a lot of information. But this generalisation is also its weakness. A good way to think about it is it's like a journalist of research. If you've ever read a newspaper, you often think you're getting a lot of insight. However, as soon as you read an article on an area of your specialisation, you realise they've made many flaws with the analysis; they don't understand your subject anywhere near the level you would. The second task (outside the frontier) required analysis of a spreadsheet, interviews and a more deeply analytical take with evidence to back it up. These are all tasks that LLMs aren't strong at currently. Unsurprisingly, the non-LLM group scored 84.5%, and between 60% and 70.6% for LLM users. The takeaway should be that LLMs are great for generalised research but less good for specialist analytical tasks.
- genman 3y agoComparing LLM to journalists is good insight.
- doitLP 3y agoI was thinking about this last night. It’s a new version of Gell-Mann amnesia. I call it LLm-man amnesia. When I ask a programming question, chat GPT hallucinates something about 20% of the time and I can only tell because I’m skilled enough to see it. For all the other domains I ask it questions if I should assume at least as much hallucination and incorrect information.
- utsuro 3y agoI see this as for drill-down thinking from a broad -> specific concept AI seems to be helpful when supplementing specialist work. However like you both mentioned: when needing more focused and integrated answers AI tends hinders performance. However as the paper noted, when working within AIs areas of strength it improved not only efficiency but the quality of the work as well (accounting for the hallucinations). As you mentioned: > When I ask a programming question, chat GPT hallucinates something about 20% of the time and I can only tell because I’m skilled enough to see it This matches their Centaur approach, delineating between AI and one’s own skills for a task which—with generalized work—seems to fair better than not using AI at all.
- nopinsight 3y agoMore details in this blog post by a Wharton professor: https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the-jagged https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the... My questions to naysayers: * Do you or anyone you know use GPT-4 (not the free GPT-3.5) to do productive tasks like coding and found it to help in many cases? * If you insist it’s useless, why do millions of people pay $20 a month to access GPT-4 and plugins?
- syntaxing 3y agoYes, GPT-4 is great for doing “boring work” and allows me to focus on the “fun work”. You still need to know what you’re doing though, you can’t blindly copy and paste. And for the second one, although I am paying for it too, this idea is more or less flawed nowadays. Utilization is a very hand wavy thing when it comes to this stuff. Like a purse, millions would pay money for it, some even pay thousands. But I have no use for it and wouldn’t even pay a $1 for one.
- nopinsight 3y ago> You still need to know what you’re doing though, you can’t blindly copy and paste. Agreed. > Like a purse, millions would pay money for it, some even pay thousands. Expensive purses have intangible value for some. They are often bought to signal social status. I'm pretty sure a significant portion of ChatGPT Plus subscribers are paying because it can help them with information or cognitive work that some people value.
- simonw 3y agoConsumer behavior around monthly subscription services that can be cancelled at any time looks very different from behavior around one-time luxury purchases.
- footy 3y agoI have free access to copilot because I do some open source work. I haven't been impressed by what it can do and I wouldn't pay even $3/month to use it. The second question doesn't make sense to me. There are tons of things I think are useless (or worse) that people pay for anyway. Meal kit boxes come to mind, and at least you can eat those at the end of the day.
- z991 3y agoThe actual research article: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4573321 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4573321 Summary: https://pdf2gpt.com/?summary=84ff84d4b98b4f0c985a17d07db482c9 https://pdf2gpt.com/?summary=84ff84d4b98b4f0c985a17d07db482c...
- FrustratedMonky 3y agoCan confirm. I popped the 20 bucks for GPT4, and have been using it more and more, every day for 3 weeks. Not sure how I can get by without it now. It's just so easy to have normal conversation and get answers. Like having an expert friend across the hall you can just shoutout questions, and ask for simple reminders, recommendations. Who cares if it gets things wrong sometimes, you would double check your co-workers answers also. And there are times when I insist I am correct, and GPT will argue back and eventually I find I was wrong.
- charbull 3y agoslideware professionals got better at making slides with LLMs
- digitcatphd 3y agoIf this is how so called consultants use AI… they should be very concerned. A moderately skilled intern with GPT Enterprise connected to data will make them quickly obsolete. Maybe they have some potential building their own fine tuned model but surely they will screw that up
- doubtfuluser 3y agoDidn’t have many interactions which BCG so far but in both we had, I was surprised at how much money they get for reshuffling information from what is all common knowledge and available in the net. I can see that this is something LLMs can do really well. It’s exactly the kind of “creativity” LLMs can do: “apply concept X to market / niche Y and give ideas on monetizing”. I don’t blaim BCG for doing this, they are giving an outside view and political uninfluenced (except for the party that pays the tap) view.
- ralfcheung 3y agoIn other words, companies can replace consultants with GPT-4.
- LightBug1 3y agoThey might as well. All they do is repeat what a decent manager has been telling them, verbatim usually, get paid a shit tonne of cash, and then walk. Absolutely zero add value in experience. The only add-value is the consultant overcoming the hearing deficiency of the Director involved. ConsulatancyGPT: Feed all internal opinions of a company into an LLM. Ask for the a recommendation. Done. /rant.
- m3kw9 3y agoWhere ChatGPT could excel is early education learning where the ideas are simple and universally agreed and written online. As you go higher level the chance of hallucinations becomes higher and you could be taught the wrong thing without knowing the risks
- NBJack 3y agoMy prediction? In about 6 months, every test, task, or use of a LLM for anything that requires a modicum of creativity is going to find that it only has a fixed set of "ideas" before it starts regurgitating them. [0] I can easily imagine this in their hypothetical shoe pitch question, and many models going for more factual answers have been rapidly showing this bias by design. [0] https://www.marktechpost.com/2023/06/16/this-paper-tests-chatgpts-sense-of-humor-over-90-of-chatgpt-generated-jokes-were-the-same-25-jokes/ https://www.marktechpost.com/2023/06/16/this-paper-tests-cha...
- simonw 3y agoI'm very unimpressed by that study. Look at how they generated the jokes - they fed it a prompt that was a slight variation on "please tell me a joke" and then wrote about how the jokes weren't varied enough. https://github.com/DLR-SC/JokeGPT-WASSA23/blob/main/01_joke_generation.py https://github.com/DLR-SC/JokeGPT-WASSA23/blob/main/01_joke_... That's a bad way to use an LLM for joke generation. Try "tell me a joke about a sea lion" - then replace sea lion with any other animal. Or "tell me ten jokes about a lawyer on the moon" - combine concepts like that and you get an infinite variety of jokes. Some of them might even be funny!
- ekianjo 3y agoSo useless BCG consultants were faster in delivery bullshit with ChatGPT? That's impressive.
- User23 3y agoI bet early search engines had similar or even better figures under similar conditions. I suppose this because I recall how much search improved my productivity over flipping through books and I know how for certain tasks ChatGPT is a better source of knowledge on how to do it than search. While often the GPT output isn’t entirely correct, more often than not it suffices to make the correct solution obvious thus saving a lot of time.
- chevman 3y agoDoes this increased efficiency mean my SOW estimates are going to start coming down????? Oh right, it's not that type of efficiency :)
- tbm57 3y agodoes 'paradox mindset' measure my ability to 'please accept the mystery'?
- emmender1 3y agoThe output of many professions is bag-of-words emotional persuation. eg. politicians, consultants, sociologists, psychologists, writers, economists, tv talking heads, media in general. A characteristic of these professions is that there is no accountability for output they produce. It is not like a profession that builds an engine for a car. They can bullshit with confidence and get away with it. chatGPT will replace all of them - as chatGPT itself can bullshit with the best of them.
- Hippocrates 3y agoThis is hilarious. As impressive as GPT-3/4 has been at writing, what's more shocking is just how bullshity-y human writing is.. And a "business consultant" is the epitome of a role requiring bullshit writing. Chat GPT could certainly out business-consultant the very best business consultants. Sometimes to be taken seriously at work, you need to take some concise idea or data and fluff it up into a multiple pages or a slide deck JUST so that others can immediately see how much work you put in. The ideal role for chatgpt at this moment is probably to take concise writings and to expand it into something way larger and full of filler. On the receiving end, people will endure your long-winded document or slide deck, recognize you "put in the work", and then feed it back into chatGPT to get the original key points summarized.
- klabb3 3y ago> As impressive as GPT-3/4 has been at writing, what's more shocking is just how bullshity-y human writing is.. Yeah. Most people have focused on what LLMs can do, but I think it’s equally if not more interesting what can they not do, and why? When we say LLMs can generate text we’re painting brush strokes as broad as a 10-lane highway. Apparently we have quite limited vocabulary about what writing actually is, and specifically what categories and levels exist. For instance, it’s fun (and in my view completely expected) to see that courteous emails, LinkedIn inspirational spam, corp-speech etc, GPT outperforms humans with flying colors, on the first attempt too! Whereas if you’re asking for the next book of Game of Thrones or any well-written literature it falls flat – incredibly boring, generic, full of platitudes and empty arcs and characters. We have to start mapping the field of writing to a better conceptual space. Currently it seems like we can’t even differentiate between the equivalent of arithmetic and abstract algebra.
- lambdaba 3y agoTo me it looks very analogous to AI-generated "art", it's very easy to generate some generally esthetically pleasing visuals, but the depth of the art stays in proportion with the input effort... Which is often not much. All of this shouldn't be very surprising really, and there's still a lot of usefulness to it, if only for depreciating the low-quality copy-paste productions and making the really unique and novel ones even more valuable.
- yafbum 3y ago... for a set of tasks selected to be answerable by AI Also access to AI significantly increased (!) incorrect answers in the case where the tasks were outside of AI capabilities.
- matt3D 3y agoFunnily enough, as a business consultant I use GPT to create executive summaries and sell people on the idea that my reports are as short as they possibly can be without information loss.
- bilsbie 3y agoDumb Offtopic question. Is there any way to ask gpt4 to summarize an article online? I tried giving it the url and it was a disaster. Is there a plug-in?
- xyst 3y agoThis only really confirms what we already know. Business consultants are useless.
- Animats 3y ago"The study introduces the concept of a “jagged technological frontier,” where AI excels in some tasks but falls short in others." D'oh.
- Avlin67 3y agoIt is quite efficient to generate unit tests using specific libraries
- Waterluvian 3y agoI always wondered if some of the biggest fear mongers against GPT are those who worry they’ll be outed as frauds. If your job is to generate nonsense… well…
- bigmattystyles 3y agoI was sort of wondering this with the latest (I think now resolved) writer's strike. The union wanted reassurance that they wouldn't be replaced by AI; however, if I was the studios, I would have said `sounds good` - knowing full well that the union members will likely be turning to it. Unless the union polices its members, the appeal to use it is just too high.
- MilStdJunkie 3y agoNonsense career threatened by nonsense generator. Beautiful.
- makach 3y agoGuilty! GPT is the best colleague I ever had, but boy does it speak. You can't just copy paste, but if you consider its responses as input I find myself less dependent on other senior consultants sharing their insights. It also makes me more confident in my assessments and deliveries. Purpose of technology is to enhance our performance, GPT is very much doing so - but with great powers comes great responsibility.
- ltr_ 3y agoshocking, bullshit tech for bullshit people improves bullshit.
- WirelessGigabit 3y agoTake some text and dump it into ChatGPT, and ask it to make it more formal. Sounds like the same text in the average deck...