8 ms·

The first empirical study of the real-world economic effects of new AI systems

115 points by SirLJ 3y ago

sanxiyn 3y ago
This is an interesting empirical study indeed. Direct link to the study: https://www.nber.org/papers/w31161 https://www.nber.org/papers/w31161
- deleted 3y ago
  [deleted]
JieJie 3y ago
This wasn't even GPT-4. Terrific news. It would be super cool if the people who were currently fretting about paperclips could focus a little more on bias, security, data, privacy, openness, and all the stuff we actually are going to have to deal with over the next five years.
- UnFleshedOne 3y ago
  Why don't you focus on that instead?
- p-e-w 3y ago
  > It would be super cool if the people who were currently fretting about paperclips [...] That's textbook trivializing of an issue that has been extensively analyzed by experts and found to have civilization-ending consequences with near certainty, unless extensive countermeasures are taken well in advance. Your comment is the equivalent of climate change denial.
  - EamonnMR 3y ago
    Have any countermeasures actually been implemented besides blog posts and podcast appearances? Climate activists regularly disrupt traffic, glue themselves to things, and protest pipelines. By contrast folks who believe AI will end the world have a propensity to start AI companies.
    - ben_w 3y ago
      At the moment, we're still trying to turn the question from an easy and obvious natural language form ("give me what I meant to ask for") into a form that is amenable to being solved in the world of weights and biases.[0] So several of the AI labs are published work on this topic, but it looks like e.g. "interpretable models" or philosophy of ethics etc. So today we're doing today with AI alignment research is the equivalent of asking Ada Lovelace how to prevent the Therac-25 deaths and the best response is "Why would you even do that, just don't write the wrong commands on the punched cards". [0] Some, myself included, are optimistic that early AI can help us do that. Yudkowsky appears to be dismissive of all options, but I don't see why any near-term AI would care to insert errors into later types of AI, so we've only got out own blind spots to look out for and we had that already… …but all the AI doomers I hang out with seem to consider me a terrible optimist, so YMMV.
- ben_w 3y ago
  While I reckon there's at least 5 (and probably 10+) years before it's even possible for a sufficiently independent AI to start literally paperclipping us (mostly because of the speed/power needs of good image processing AI), the people who are trying to solve that don't know if we can actually do so in even double that time frame. But the paperclip scenario was only ever an illustration that the AI doesn't need to hate us to destroy us, merely to want to use our atoms for something incompatible with our own desires. For disasters to happen, there are many options which may come sooner, varying from not paying attention to what the AI is doing when it makes mistakes — the default, as we can imagine even today someone putting a Boston Dynamics Spot to work in a virology lab and it walking out with a sample because it doesn't know any better — to the malicious, for example one idiot running a more competent version of ChaosGPT: https://youtu.be/g7YJIpkk7KM https://youtu.be/g7YJIpkk7KM Those other, lesser, disasters are sometimes called "fire alarms for AI".
  - jiggawatts 3y ago
    The cynic in me says that the #1 AI-related risk by far is that corporations will immediately apply it to the task of listening in on everything we say and write. This all reminds me of "A Fire Upon the Deep" by Vernor Vinge, where the Slavers enslave humans by essentially constantly monitoring them for misbehaviour. If I remember correctly, a plot element is that they use yet more slaves to keep the bulk of the slaves in line. With ChatGPT, you no longer need to do "sci fi things" like turning squishy meat brains into cyborg slavedrivers. You can simply ask the AI to do it, and it will. It will do it 24/7, without pity, or remorse. It will do vigilantly and faithfully. It will be a checkbox in Office 365 that any psychotic manager can turn on. And I guarantee they will turn it on. All of them.
    - ben_w 3y ago
      I've not yet had time to read/listen to A Fire Upon the Deep yet, only its sequel A Deepness in the Sky, but that certainly sounds like one of the many bad endings we might get. That said: > With ChatGPT, you no longer need to do "sci fi things" like turning squishy meat brains into cyborg slavedrivers. You can simply ask the AI to do it, and it will. It will do it 24/7, without pity, or remorse. It will do vigilantly and faithfully. ChatGPT has all the vigilance and faithfulness of my late mother's mid-stage Alzheimer's, so if someone tried this today it would rapidly result in everyone getting million-dollar-no-work-required contracts the company isn't allowed to break. That LLMs can be socially engineered just by asking nicely is a pleasant surprise twist in AI alignment, but unfortunately I think it illustrates how primitive is our current state of the art AI safety and alignment.
- gwd 3y ago
  I really don't understand this antagonism between different branches of the "AI safety" community. In the 1960's, some people were concerned about preventing a nuclear apocalypse, and some people were concerned about lead emissions and CFCs and exploding cars. But did the nuclear apocalypse people tell the exploding cars people they should stop their work on exploding cars because nuclear apocalypse was more important long-term? Did the exploding car people tell the nuclear apocalypse people to stop raving about nuclear apocalypse because there were more pressing problems affecting people right now, like exploding cars and lead poisoning and a hole in the ozone layer? I mean, maybe they did; but if they did then I think it was kind of silly. Both kinds of problems are important, then and now.
  - JieJie 3y ago
    I'm sorry it sounds antagonistic, but the negativity and "it's too late and we're all going to die no matter what we do" nihilism, is having the effect of sensationalizing the issue in ways that have been incredibly counterproductive. Here's what happened the last time technology was going to destroy us all: https://www.popularmechanics.com/science/a11217/what-stephen-hawking-really-said-about-destroying-the-universe-17192502/ https://www.popularmechanics.com/science/a11217/what-stephen... "The Higgs potential has the worrisome feature that it might become metastable at energies above 100 [billion] gigaelectronvolts (GeV). This could mean that the universe could undergo catastrophic vacuum decay, with a bubble of the true vacuum expanding at the speed of light. This could happen at any time and we wouldn't see it coming." —Stephen Hawking I'm not trying to make fun of Eliezer and co., but we have to have some proper perspective. We don't do that very well these days, because end-of-the-world scenarios are good for clicks.
    - srslack 3y ago
      Hawking's statement was made after the confirmation of the Higgs, and is still valid. A metastability vacuum event could be a problem (that's putting it lightly, hah) at high enough energies. It might have already happened, elsewhere. Eliezer and co. should be made fun of. They engage with journalists and lean into statements like this: https://www.lesswrong.com/posts/FKNtgZrGYwgsz3nHT/bankless-podcast-159-we-re-all-gonna-die-with-eliezer https://www.lesswrong.com/posts/FKNtgZrGYwgsz3nHT/bankless-p... They're absolutely hysterical. Convenient for them, and for the companies, the hysteria is to their benefit. More money to fund the MIRI for Yudowsky, and regulatory capture for the companies who believe they're ahead if you can convince some dementia-ridden Sentor to pass the bill regulating some vague notion of "AI safety." There's some people in these companies who understand that they have no moat, and that you can't make GPUs illegal, as demonstrated by the most recent story about the leaked google post, and that people can outperform their models in an hour with a LoRA on a workstation laptop for a given task, that the cat is out of the bag. But, with the hysteria of the media, where interns straight out of the shithole of liberal arts believe that a post on reddit or twitter where someone was slighted at the dinner table is breaking news, we're now in this situation where we have calls for regulation for something people simply don't understand, no thanks to the useless media.
miksumiksu 3y ago
It will be interesting to see how this company is doing 5 years. It seems that thanks to the AI system novice employees are able to cash out on skilled employees knowledge. But this system might also hinder the novices changes on gaining knowledge and becoming skilled employees. And those skilled emploees are essentialto maintain this system to respond to evolving needs. So the effect on job market might be the opposite than researcher wish: recruits need to be smarter than before and the gap between low-skilled and high-skilled employees will keep widening. So in that case introducing AI to this process would have the exactly same outcome as automation has had in manufacturing.
- deleted 3y ago
  [deleted]
- cosmodisk 3y ago
  I do agree with you. My feeling is that there will be a rise of so called super experienced users- those,who didn't just get all the answers from the prompt but learned instead and hordes of people who won't even know how to do basic stuff without the AI helper. The former group will be small, but extremely well rewarded,while the rest... Well, not so much..
  - somenameforme 3y ago
    It feels somewhat analogous to the calculator, except a million times more impactful because of the breadth of fields it can apply to. The times I go to a minimart type place and cashier laboriously, and slowly, whips out a calculator to determine how much change to give when paid 100 for a 32 cost item. Now imagine that analog generalized to basic knowledge and competence itself. It's kind of ironic because each one of these steps forward towards making knowledge egalitarian tends to do the exact opposite. Same thing with the internet which was going to democratize knowledge, information, education and more. And indeed it absolutely has, but 99% of people don't take advantage of it - so the 1% just pull that many orders of magnitude further ahead.
    - shswkna 3y ago
trojanalert 3y ago
I'd be A LOT more interested to see what happens to a whole set of workers who dabble in lower-skilled jobs and are now being rendered redundant with tools such as Chat GPT.
- iudqnolq 3y ago
  I'm not sure what you mean by dabble. Ignoring that that's exactly what this study is. They're looking at call center employees. The author calls employees who've worked there for more than six months experienced.
- shaky-carrousel 3y ago
  I guess that what will happen to them is more or less the same that happened to phone operators.
spaceman_2020 3y ago
It would be interesting to see how a general-purpose AI like GPT-4 trained on internal corporate knowledgebase would perform. Theoretically, such an AI agent would know your internal policies, biases, guidelines. It would also have more knowledge about the company than any individual worker, collating data from across divisions. In large corps, the silofication of knowledge is a real problem. But an AI agent trained across the company's functions would not have this issue.
- _glass 3y ago
  Especially with all of the meetings switching to virtual spaces with transcripts. This would be so powerful. When you can ask (or create documentation) based on all of those meetings. Then enriched by all of the chat messages, the company's code. Best thing, it doesn't even have to run in the cloud. As it looks like it can be an on-premise inference machine. The future is bright for surveillance.
  - shaky-carrousel 3y ago
    It sure looks like a beautiful future. I'll do my best to make it a reality as soon as possible.
  - spaceman_2020 3y ago
    Stretching that further, it is entirely possible to track every single interaction you have with your work computer. You can track your mouse movements, slack messages, work email, keystrokes... With this data, it is possible to create a digital version of you, the employee - a digital version who writes like you, moves his mouse like you, and even makes the same coding errors as you. I can totally see a near future where companies create digital avatars of star employees that stay with the company in perpetuity.
    - _glass 3y ago
      That sounds like an amazing sci-fi plot, and somehow doesn't seem so far off from the future. Only in Germany where I work I am sure this will be blocked by worker's regulations. But how long can they do that, if you have already synth workers from all around the world. We can start a recruiting agency for synths. The more I work with LLMs the more I am aware of the synthetic potential.
classified 3y ago
Isn't this a bit early, and probably distorted by hype fads?
Bukhmanizer 3y ago
I’ve always thought that in general, societies wants tend to expand to the limits of productivity. The real danger seems to be of leadership using AI as a cudgel to threaten employees into doing what they want.
- nullsense 3y ago
  Threatening is one of the tools humans use to align other humans, so why wouldn't this happen?
kriro 3y ago
The study is about an AI support system (a "recent version of GPT", sources link to GPT4, finetuned on internal company data) supporting customer support agents at a software company that provides business process software. It increses productivity (measured in number of chats resolved by employee) by 13.8%, the lower the performance before, the higher the impact. The performance increase is mostly based on being able to handle more chats in parallel and handling each chat quicker. For reference, an agent resolved about 2.12 chats/h before and this increaded by 0.47 chats/h (I was surprised that it's "only" 2.12 chats/h). Interestingly but also not surprising this increase in productivity at the lower end is atypical for tech innovation which tended to favour the higher end positions before. However, this type of customer support seems like a good candidate that will get automated away completely. It seems like the rollout can be in phases, it already increases productitivty as a support system. First, the productivity will be uplifted to a certain level for all workers where the skill difference matters less and less and finally the work will be done completely by AI.
- visarga 3y ago
  > First, the productivity will be uplifted to a certain level for all workers where the skill difference matters less and less and finally the work will be done completely by AI. First we will build a ladder, climb on it and be uplifted to a certain level until the distance matters less and less, and finally we will reach the moon! There is no single AI task that works on its own without needing human help at all. We are still on earth. The max speed of GPT is still the human reading speed. It's kind of a leap of faith to think we will close the gap, solve that last 1% which is exponentially harder than the first 99%. Where are the L5 cars after 14 years of research? Can we even predict when they will arrive? That's how hard it is to estimate the last 1%. Don't gloss over it. It will surely come some day, but we have no idea how. How do you go from 0 to 1? The mere fact that we are simultaneously stupefied by AI advancements and still not yet there - this shows how hard it is to estimate progress. We are glossing over future problems and imagining we're almost there. Slowly we are learning new things about AI limitations - hallucinations, regurgitated content, sycophancy, context size limits, complex planning. I bet there are many more unknown unknowns ahead.
  - 3y ago
deleted 3y ago
[deleted]
z3c0 3y ago
While I get that people are more concerned with productivity insofar as it pertains to their job, but I think the bigger economic concern is when competing companies begin using the same models for navigating the competitive landscape. I highly suspect that we will begin to see accidental trusts form as competitors sharing models begin to optimize around each other.
cgio 3y ago
The future is not that scary. Our path to it is, though.
iandanforth 3y ago
"Deflection" in the customer service world means when you prevent a customer from talking to a human. It's a critical metric for CSR orgs. Often you improve deflection rates by putting hurdles between a customer and a rep. If you're being generous those hurdles are helpful because they consist of "Did you try this? Have you read these related articles?" and can solve people's basic problems without taking up valuable human time. I am highly skeptical of this article because it doesn't mention deflection at all. There are other GPT powered B2B2C companies (e.g. https://yellow.ai/ https://yellow.ai/) which are entirely focused on deflection. It's not a huge logical leap to ask, if you can use a chatbot to turn an unskilled worker into a skilled worker, what exactly is the human adding? In many cases the answer is 'not much' or 'nothing.' You can completely replace a low skilled worker with a chatbot some economically viable percentage of the time. Because of this the claim that "the less experienced and less skilled workers [will] benefit the most" is suspect.
- huitzitziltzin 3y ago
  I’m a little confused by your skepticism. The underlying research is studying what happens given that an interaction reaches a human. Whatever deflection happens before that is irrelevant to the interaction with people afterwards. The paper studies the latter and the results are specific to that set of interactions, which are presumably more complex or ill-suited to deflection.
  - welshwelsh 3y ago
    Why does interaction need to reach a human though? I'm hoping that once the chatbots get good enough, we can give them all the powers of a human representative and get rid of the human.
  - iandanforth 3y ago
    I'm saying that's a false premise. Scenario 1: Hire 100 new people, give them AI -> they do better with AI Scenario 2: Hire 10 new people (because AI has reduced demand for humans), give them AI -> they do better with AI Both of these scenarios are in-line with the research, but are dramatically different when evaluating the proposition that AI helps low skill workers in general.
throwuwu 3y ago
“And what this system did was it took people with just two months of experience and had them performing at the level of people with six months of experience," Brynjolfsson says. "So it got them up the learning curve a lot faster — and that led to very positive benefits for the company." Literally the same story as every industrialization ever. Buckle up, we get to live through another industrial revolution.
deleted 3y ago
[deleted]
JoeAltmaier 3y ago
Doesn't take complex AI to benefit new workers (or even middle experience ones). A company I talked to did some simple sentiment analysis on emails, and reduced the closing time for a real estate office by 40%. Which means they can do that many more deals per month etc. Maybe the real change here, is people are beginning to consider using computer aids to do their job. Instead of insisting their experience is all that matters.

115 points by SirLJ 3y ago

This is an interesting empirical study indeed. Direct link to the study: https://www.nber.org/papers/w31161 https://www.nber.org/papers/w31161

deleted 3y ago
[deleted]

JieJie 3y ago

This wasn't even GPT-4. Terrific news. It would be super cool if the people who were currently fretting about paperclips could focus a little more on bias, security, data, privacy, openness, and all the stuff we actually are going to have to deal with over the next five years.

UnFleshedOne 3y ago
Why don't you focus on that instead?
p-e-w 3y ago
> It would be super cool if the people who were currently fretting about paperclips [...] That's textbook trivializing of an issue that has been extensively analyzed by experts and found to have civilization-ending consequences with near certainty, unless extensive countermeasures are taken well in advance. Your comment is the equivalent of climate change denial.
- EamonnMR 3y ago
  Have any countermeasures actually been implemented besides blog posts and podcast appearances? Climate activists regularly disrupt traffic, glue themselves to things, and protest pipelines. By contrast folks who believe AI will end the world have a propensity to start AI companies.
  - ben_w 3y ago
    At the moment, we're still trying to turn the question from an easy and obvious natural language form ("give me what I meant to ask for") into a form that is amenable to being solved in the world of weights and biases.[0] So several of the AI labs are published work on this topic, but it looks like e.g. "interpretable models" or philosophy of ethics etc. So today we're doing today with AI alignment research is the equivalent of asking Ada Lovelace how to prevent the Therac-25 deaths and the best response is "Why would you even do that, just don't write the wrong commands on the punched cards". [0] Some, myself included, are optimistic that early AI can help us do that. Yudkowsky appears to be dismissive of all options, but I don't see why any near-term AI would care to insert errors into later types of AI, so we've only got out own blind spots to look out for and we had that already… …but all the AI doomers I hang out with seem to consider me a terrible optimist, so YMMV.
ben_w 3y ago
While I reckon there's at least 5 (and probably 10+) years before it's even possible for a sufficiently independent AI to start literally paperclipping us (mostly because of the speed/power needs of good image processing AI), the people who are trying to solve that don't know if we can actually do so in even double that time frame. But the paperclip scenario was only ever an illustration that the AI doesn't need to hate us to destroy us, merely to want to use our atoms for something incompatible with our own desires. For disasters to happen, there are many options which may come sooner, varying from not paying attention to what the AI is doing when it makes mistakes — the default, as we can imagine even today someone putting a Boston Dynamics Spot to work in a virology lab and it walking out with a sample because it doesn't know any better — to the malicious, for example one idiot running a more competent version of ChaosGPT: https://youtu.be/g7YJIpkk7KM https://youtu.be/g7YJIpkk7KM Those other, lesser, disasters are sometimes called "fire alarms for AI".
- jiggawatts 3y ago
  The cynic in me says that the #1 AI-related risk by far is that corporations will immediately apply it to the task of listening in on everything we say and write. This all reminds me of "A Fire Upon the Deep" by Vernor Vinge, where the Slavers enslave humans by essentially constantly monitoring them for misbehaviour. If I remember correctly, a plot element is that they use yet more slaves to keep the bulk of the slaves in line. With ChatGPT, you no longer need to do "sci fi things" like turning squishy meat brains into cyborg slavedrivers. You can simply ask the AI to do it, and it will. It will do it 24/7, without pity, or remorse. It will do vigilantly and faithfully. It will be a checkbox in Office 365 that any psychotic manager can turn on. And I guarantee they will turn it on. All of them.
  - ben_w 3y ago
    I've not yet had time to read/listen to A Fire Upon the Deep yet, only its sequel A Deepness in the Sky, but that certainly sounds like one of the many bad endings we might get. That said: > With ChatGPT, you no longer need to do "sci fi things" like turning squishy meat brains into cyborg slavedrivers. You can simply ask the AI to do it, and it will. It will do it 24/7, without pity, or remorse. It will do vigilantly and faithfully. ChatGPT has all the vigilance and faithfulness of my late mother's mid-stage Alzheimer's, so if someone tried this today it would rapidly result in everyone getting million-dollar-no-work-required contracts the company isn't allowed to break. That LLMs can be socially engineered just by asking nicely is a pleasant surprise twist in AI alignment, but unfortunately I think it illustrates how primitive is our current state of the art AI safety and alignment.
gwd 3y ago
I really don't understand this antagonism between different branches of the "AI safety" community. In the 1960's, some people were concerned about preventing a nuclear apocalypse, and some people were concerned about lead emissions and CFCs and exploding cars. But did the nuclear apocalypse people tell the exploding cars people they should stop their work on exploding cars because nuclear apocalypse was more important long-term? Did the exploding car people tell the nuclear apocalypse people to stop raving about nuclear apocalypse because there were more pressing problems affecting people right now, like exploding cars and lead poisoning and a hole in the ozone layer? I mean, maybe they did; but if they did then I think it was kind of silly. Both kinds of problems are important, then and now.
- JieJie 3y ago
  I'm sorry it sounds antagonistic, but the negativity and "it's too late and we're all going to die no matter what we do" nihilism, is having the effect of sensationalizing the issue in ways that have been incredibly counterproductive. Here's what happened the last time technology was going to destroy us all: https://www.popularmechanics.com/science/a11217/what-stephen-hawking-really-said-about-destroying-the-universe-17192502/ https://www.popularmechanics.com/science/a11217/what-stephen... "The Higgs potential has the worrisome feature that it might become metastable at energies above 100 [billion] gigaelectronvolts (GeV). This could mean that the universe could undergo catastrophic vacuum decay, with a bubble of the true vacuum expanding at the speed of light. This could happen at any time and we wouldn't see it coming." —Stephen Hawking I'm not trying to make fun of Eliezer and co., but we have to have some proper perspective. We don't do that very well these days, because end-of-the-world scenarios are good for clicks.
  - srslack 3y ago
    Hawking's statement was made after the confirmation of the Higgs, and is still valid. A metastability vacuum event could be a problem (that's putting it lightly, hah) at high enough energies. It might have already happened, elsewhere. Eliezer and co. should be made fun of. They engage with journalists and lean into statements like this: https://www.lesswrong.com/posts/FKNtgZrGYwgsz3nHT/bankless-podcast-159-we-re-all-gonna-die-with-eliezer https://www.lesswrong.com/posts/FKNtgZrGYwgsz3nHT/bankless-p... They're absolutely hysterical. Convenient for them, and for the companies, the hysteria is to their benefit. More money to fund the MIRI for Yudowsky, and regulatory capture for the companies who believe they're ahead if you can convince some dementia-ridden Sentor to pass the bill regulating some vague notion of "AI safety." There's some people in these companies who understand that they have no moat, and that you can't make GPUs illegal, as demonstrated by the most recent story about the leaked google post, and that people can outperform their models in an hour with a LoRA on a workstation laptop for a given task, that the cat is out of the bag. But, with the hysteria of the media, where interns straight out of the shithole of liberal arts believe that a post on reddit or twitter where someone was slighted at the dinner table is breaking news, we're now in this situation where we have calls for regulation for something people simply don't understand, no thanks to the useless media.

miksumiksu 3y ago

It will be interesting to see how this company is doing 5 years. It seems that thanks to the AI system novice employees are able to cash out on skilled employees knowledge. But this system might also hinder the novices changes on gaining knowledge and becoming skilled employees. And those skilled emploees are essentialto maintain this system to respond to evolving needs. So the effect on job market might be the opposite than researcher wish: recruits need to be smarter than before and the gap between low-skilled and high-skilled employees will keep widening. So in that case introducing AI to this process would have the exactly same outcome as automation has had in manufacturing.

deleted 3y ago
[deleted]
cosmodisk 3y ago
I do agree with you. My feeling is that there will be a rise of so called super experienced users- those,who didn't just get all the answers from the prompt but learned instead and hordes of people who won't even know how to do basic stuff without the AI helper. The former group will be small, but extremely well rewarded,while the rest... Well, not so much..
- somenameforme 3y ago
  It feels somewhat analogous to the calculator, except a million times more impactful because of the breadth of fields it can apply to. The times I go to a minimart type place and cashier laboriously, and slowly, whips out a calculator to determine how much change to give when paid 100 for a 32 cost item. Now imagine that analog generalized to basic knowledge and competence itself. It's kind of ironic because each one of these steps forward towards making knowledge egalitarian tends to do the exact opposite. Same thing with the internet which was going to democratize knowledge, information, education and more. And indeed it absolutely has, but 99% of people don't take advantage of it - so the 1% just pull that many orders of magnitude further ahead.
  - shswkna 3y ago

trojanalert 3y ago

I'd be A LOT more interested to see what happens to a whole set of workers who dabble in lower-skilled jobs and are now being rendered redundant with tools such as Chat GPT.

iudqnolq 3y ago
I'm not sure what you mean by dabble. Ignoring that that's exactly what this study is. They're looking at call center employees. The author calls employees who've worked there for more than six months experienced.
shaky-carrousel 3y ago
I guess that what will happen to them is more or less the same that happened to phone operators.

spaceman_2020 3y ago

It would be interesting to see how a general-purpose AI like GPT-4 trained on internal corporate knowledgebase would perform. Theoretically, such an AI agent would know your internal policies, biases, guidelines. It would also have more knowledge about the company than any individual worker, collating data from across divisions. In large corps, the silofication of knowledge is a real problem. But an AI agent trained across the company's functions would not have this issue.

_glass 3y ago
Especially with all of the meetings switching to virtual spaces with transcripts. This would be so powerful. When you can ask (or create documentation) based on all of those meetings. Then enriched by all of the chat messages, the company's code. Best thing, it doesn't even have to run in the cloud. As it looks like it can be an on-premise inference machine. The future is bright for surveillance.
- shaky-carrousel 3y ago
  It sure looks like a beautiful future. I'll do my best to make it a reality as soon as possible.
- spaceman_2020 3y ago
  Stretching that further, it is entirely possible to track every single interaction you have with your work computer. You can track your mouse movements, slack messages, work email, keystrokes... With this data, it is possible to create a digital version of you, the employee - a digital version who writes like you, moves his mouse like you, and even makes the same coding errors as you. I can totally see a near future where companies create digital avatars of star employees that stay with the company in perpetuity.
  - _glass 3y ago
    That sounds like an amazing sci-fi plot, and somehow doesn't seem so far off from the future. Only in Germany where I work I am sure this will be blocked by worker's regulations. But how long can they do that, if you have already synth workers from all around the world. We can start a recruiting agency for synths. The more I work with LLMs the more I am aware of the synthetic potential.

classified 3y ago

Isn't this a bit early, and probably distorted by hype fads?

Bukhmanizer 3y ago

I’ve always thought that in general, societies wants tend to expand to the limits of productivity. The real danger seems to be of leadership using AI as a cudgel to threaten employees into doing what they want.

nullsense 3y ago
Threatening is one of the tools humans use to align other humans, so why wouldn't this happen?

kriro 3y ago

The study is about an AI support system (a "recent version of GPT", sources link to GPT4, finetuned on internal company data) supporting customer support agents at a software company that provides business process software. It increses productivity (measured in number of chats resolved by employee) by 13.8%, the lower the performance before, the higher the impact. The performance increase is mostly based on being able to handle more chats in parallel and handling each chat quicker. For reference, an agent resolved about 2.12 chats/h before and this increaded by 0.47 chats/h (I was surprised that it's "only" 2.12 chats/h). Interestingly but also not surprising this increase in productivity at the lower end is atypical for tech innovation which tended to favour the higher end positions before. However, this type of customer support seems like a good candidate that will get automated away completely. It seems like the rollout can be in phases, it already increases productitivty as a support system. First, the productivity will be uplifted to a certain level for all workers where the skill difference matters less and less and finally the work will be done completely by AI.

visarga 3y ago
> First, the productivity will be uplifted to a certain level for all workers where the skill difference matters less and less and finally the work will be done completely by AI. First we will build a ladder, climb on it and be uplifted to a certain level until the distance matters less and less, and finally we will reach the moon! There is no single AI task that works on its own without needing human help at all. We are still on earth. The max speed of GPT is still the human reading speed. It's kind of a leap of faith to think we will close the gap, solve that last 1% which is exponentially harder than the first 99%. Where are the L5 cars after 14 years of research? Can we even predict when they will arrive? That's how hard it is to estimate the last 1%. Don't gloss over it. It will surely come some day, but we have no idea how. How do you go from 0 to 1? The mere fact that we are simultaneously stupefied by AI advancements and still not yet there - this shows how hard it is to estimate progress. We are glossing over future problems and imagining we're almost there. Slowly we are learning new things about AI limitations - hallucinations, regurgitated content, sycophancy, context size limits, complex planning. I bet there are many more unknown unknowns ahead.
- 3y ago

deleted 3y ago

[deleted]

z3c0 3y ago

While I get that people are more concerned with productivity insofar as it pertains to their job, but I think the bigger economic concern is when competing companies begin using the same models for navigating the competitive landscape. I highly suspect that we will begin to see accidental trusts form as competitors sharing models begin to optimize around each other.

cgio 3y ago

The future is not that scary. Our path to it is, though.

iandanforth 3y ago

"Deflection" in the customer service world means when you prevent a customer from talking to a human. It's a critical metric for CSR orgs. Often you improve deflection rates by putting hurdles between a customer and a rep. If you're being generous those hurdles are helpful because they consist of "Did you try this? Have you read these related articles?" and can solve people's basic problems without taking up valuable human time. I am highly skeptical of this article because it doesn't mention deflection at all. There are other GPT powered B2B2C companies (e.g. https://yellow.ai/ https://yellow.ai/) which are entirely focused on deflection. It's not a huge logical leap to ask, if you can use a chatbot to turn an unskilled worker into a skilled worker, what exactly is the human adding? In many cases the answer is 'not much' or 'nothing.' You can completely replace a low skilled worker with a chatbot some economically viable percentage of the time. Because of this the claim that "the less experienced and less skilled workers [will] benefit the most" is suspect.

huitzitziltzin 3y ago
I’m a little confused by your skepticism. The underlying research is studying what happens given that an interaction reaches a human. Whatever deflection happens before that is irrelevant to the interaction with people afterwards. The paper studies the latter and the results are specific to that set of interactions, which are presumably more complex or ill-suited to deflection.
- welshwelsh 3y ago
  Why does interaction need to reach a human though? I'm hoping that once the chatbots get good enough, we can give them all the powers of a human representative and get rid of the human.
- iandanforth 3y ago
  I'm saying that's a false premise. Scenario 1: Hire 100 new people, give them AI -> they do better with AI Scenario 2: Hire 10 new people (because AI has reduced demand for humans), give them AI -> they do better with AI Both of these scenarios are in-line with the research, but are dramatically different when evaluating the proposition that AI helps low skill workers in general.

throwuwu 3y ago

“And what this system did was it took people with just two months of experience and had them performing at the level of people with six months of experience," Brynjolfsson says. "So it got them up the learning curve a lot faster — and that led to very positive benefits for the company." Literally the same story as every industrialization ever. Buckle up, we get to live through another industrial revolution.

deleted 3y ago

[deleted]

JoeAltmaier 3y ago

Doesn't take complex AI to benefit new workers (or even middle experience ones). A company I talked to did some simple sentiment analysis on emails, and reduced the closing time for a real estate office by 40%. Which means they can do that many more deals per month etc. Maybe the real change here, is people are beginning to consider using computer aids to do their job. Instead of insisting their experience is all that matters.