4 ms·

what a dystopia. Was it ever predicted in any science fiction book? Maybe the economists are right - AI wont make us all unemployed. The only problem is, our on

what a dystopia. Was it ever predicted in any science fiction book? Maybe the economists are right - AI wont make us all unemployed. The only problem is, our only jobs will be such abhorent as these ones those poor people have, improving the next version of AI

visarga 4y ago
It is hard for me to compute this. AI engineers know very well the value of good training data. They all get to label some, they get close and personal with it. But when it comes to job titles and perks, it's like the labelling people are inferior. I have worked as a ML engineer close with internal labelling teams and let me tell you, not everyone could do it. It is a hard job, and often requires deep thinking to solve. Many ML engineers themselves are actually bad at labelling. And the contribution of the labelling team can be equal sized or more important than that of the ML engineers. In normal conditions you need at least one labelling person / engineer, all projects need labelling, there's never a gap, labelling is never finished. Finding a good labeller is also hard - not everyone on the street could do it, contrary to public expectations.
- lifeisstillgood 4y ago
  Like a really good tester ?
- layer8 4y ago
  How do you assess labelling quality? What I’m wondering is, how would the feedback loop work, in the longer term, between quality of AI output and paying/selecting labellers?
  - BitJockey 4y ago
    > How do you assess labelling quality? You have multiple people to label the same piece of data. This increases accuracy of labeling and help to spot bad labelers.
    - layer8 4y ago
      It only spots labellers that are not in the middle of the curve of those who you employ. My point was, if factors like salary affect labelling quality, how do you steer that? You don’t know whether if you’d pay double, you’d get much better fact-checkers, for example.
      - kelseyfrog 4y ago
        Interater and intrarater reliability are two general approaches. You could even regress reliability with salary and make a case that reliability would improve by a certain amount(within a prediction interval), but that's a business decision to act on, not a statistics problem.
startupsfail 4y ago
There are big upsides as well. Ii the moment you’ve got ChatGPT interacting with millions of people and in general exhibiting better than median human reasoning, kindness, rationality, positivity, listening and understanding skills. These interactions to some degree are equivalent to having access to a good engaging teacher. Think Sezame Channel, only adult level one. The results might be very positive.
- kevviiinn 4y ago
  A good engaging teacher... that gives you incorrect information?
  - junofan 4y ago
    I haven’t had this problem in my own use. But I don’t try to trip it up.
    - kevviiinn 4y ago
      Every time I ask one of these models any sort of semi detailed question about a science/bio topic it either gets some info blatantly wrong, misuses terms, or a combination of both
      - startupsfail 4y ago
        Certainly. But the observation that I was making is that it is likely above median. You are likely far above the median yourself and an expert in the particular area.
        kevviiinn 4y ago
        I think the difference between chatgpt and a good teacher is that a good teacher knows when to say "I don't know" and not feed their student BS that makes no sense. That's where the real damage happens
mortenjorck 4y ago
I don’t know if any serious authors have, but my finish-it-one-of-these-days novel is premised on a perpetually almost-there state of the art in AI that is constantly hand-held by an invisible underclass. Although I’m starting to think I ought to finish it soon just in case this current pace of advancement in LLMs doesn't stall out for once.
- kevviiinn 4y ago
  Just have chatgpt write it for you
sonofhans 4y ago
Not as literal a take as you might like, but Fritz Lang’s _Metropolis_, of course — an invisible, downtrodden underclass breaks their backs on the machines so that a mediocre elite can live in the clouds. That was nearly 100 years ago.
huevosabio 4y ago
I never understand this type of hate-porn. These jobs are not horrible, not for humanity's historical standards. They can be done from home, without commute, without physical and demanding labor. A medieval peasant, a 1800s factory worker and probably many of the 1900s low skilled laborers would take this over the alternatives. Moreover, these people have every right and option to not do it, in fact right now the labor market is such that they have valuable alternatives! We keep thinking about how this or that job is not acceptable for _our personal standards_, but until we get Fully Automated Luxury Communism we have to be honest that progress looks this: slightly less shitty jobs in exchange for higher quality of life.
- AndrewKemendo 4y ago
  They aren't less shitty they are shittier in different ways, but the key point is that they are progressively more alienated from the value they provide and the community structure.
  - catiopatio 4y ago
    The value they individually provide is minuscule; it only matters in aggregate, and when leveraged by others who bring a great deal more to the table. Anyone of average IQ and education could do this work, and there’s no shortage of people willing to do it.
    - antifa 4y ago
      I'm more concerned with how this will trend downward over the next decade(s) than how high up one side of the titanic is.
rvba 4y ago
The main protagonist of the novel "Nineteen Eighty-Four" by Orewell worked in the Ministry of Truth and his job was to rewrite historical records to conform to the state's ever-changing version of history. Basically he was a censor, who would remove inconvenient people or facts from newspapers or books. (I dont think Internet existed in the situation described in the book). 1984 was a big critic of communism, where facts dont mater and history can be rewritten. Are we really far from that? * Those poorly paid raters label things - and a lot of things depends on quality of their work. Do those companies even cross-check their work, when the idea is to get the lowest bidder? (on a side note, why FAANG companies dont understand the "garbage in, garbage out" concept? They can pay 300k usd to some programmer, yet they skimp on getting classifiers from USA or Europe who know the cultural context) * Then there are probably censors, who can censor things based on their own ideas. Probably censorship can happen at two levels: first by using labels, then by deciding what to show based on those labels. * Some websites have "fact checkers", who are necessary (with the amount of lies written in troll-farms and soon lies written by AI), but you cant really appeal about their process in any way - and those are some faceless people from third world countries. *Often you cannot ever contact with a real person, Google and Facebook are the worst here. Microsoft is a bit better, because if you pay them money for support, you can get human support. * Wikipedia is a constant war between vandals, but also actors that are much smarter (seem state sponsored even). When we speak about Wikipedia it is incredibly sad that it never introduced any systems to review the work of their admins (I saw a situation where some articles are "guarded" by 2-3 admins who share their own >version of facts< and nobody can do anything about that). And this could be solved by setting up panels of judges that judge edits that are anonymous. Or setting a panel of judges who judge actions of admins (yes, there is that "Arbitration Committee" that barely works). Of course then there is the question "who watches the watchmen", but still - Wikipedia Foundation has millions of dollars that they waste on various things, instead of spending it to improve their core product. On a side note, I always thought about starting such a data labeling business; I think a lot of work done by current labelers is still low quality. I saw some start-ups that were checking things (e.g. medical data) and they would pay peanuts to the people who labeled their data. Then they would get bad labels.. and build wrong conclusions on that with their classifiers.
- antifa 4y ago
AndrewKemendo 4y ago
"The march toward perfect alienation and exploitation will make any job unthinkable at any pay rate." [1] [1] https://kemendo.com/Myth.pdf https://kemendo.com/Myth.pdf

visarga 4y ago

It is hard for me to compute this. AI engineers know very well the value of good training data. They all get to label some, they get close and personal with it. But when it comes to job titles and perks, it's like the labelling people are inferior. I have worked as a ML engineer close with internal labelling teams and let me tell you, not everyone could do it. It is a hard job, and often requires deep thinking to solve. Many ML engineers themselves are actually bad at labelling. And the contribution of the labelling team can be equal sized or more important than that of the ML engineers. In normal conditions you need at least one labelling person / engineer, all projects need labelling, there's never a gap, labelling is never finished. Finding a good labeller is also hard - not everyone on the street could do it, contrary to public expectations.

lifeisstillgood 4y ago
Like a really good tester ?
layer8 4y ago
How do you assess labelling quality? What I’m wondering is, how would the feedback loop work, in the longer term, between quality of AI output and paying/selecting labellers?
- BitJockey 4y ago
  > How do you assess labelling quality? You have multiple people to label the same piece of data. This increases accuracy of labeling and help to spot bad labelers.
  - layer8 4y ago
    It only spots labellers that are not in the middle of the curve of those who you employ. My point was, if factors like salary affect labelling quality, how do you steer that? You don’t know whether if you’d pay double, you’d get much better fact-checkers, for example.
    - kelseyfrog 4y ago
      Interater and intrarater reliability are two general approaches. You could even regress reliability with salary and make a case that reliability would improve by a certain amount(within a prediction interval), but that's a business decision to act on, not a statistics problem.

startupsfail 4y ago

There are big upsides as well. Ii the moment you’ve got ChatGPT interacting with millions of people and in general exhibiting better than median human reasoning, kindness, rationality, positivity, listening and understanding skills. These interactions to some degree are equivalent to having access to a good engaging teacher. Think Sezame Channel, only adult level one. The results might be very positive.

kevviiinn 4y ago
A good engaging teacher... that gives you incorrect information?
- junofan 4y ago
  I haven’t had this problem in my own use. But I don’t try to trip it up.
  - kevviiinn 4y ago
    Every time I ask one of these models any sort of semi detailed question about a science/bio topic it either gets some info blatantly wrong, misuses terms, or a combination of both
    - startupsfail 4y ago
      Certainly. But the observation that I was making is that it is likely above median. You are likely far above the median yourself and an expert in the particular area.
      - kevviiinn 4y ago
        I think the difference between chatgpt and a good teacher is that a good teacher knows when to say "I don't know" and not feed their student BS that makes no sense. That's where the real damage happens

mortenjorck 4y ago

I don’t know if any serious authors have, but my finish-it-one-of-these-days novel is premised on a perpetually almost-there state of the art in AI that is constantly hand-held by an invisible underclass. Although I’m starting to think I ought to finish it soon just in case this current pace of advancement in LLMs doesn't stall out for once.

kevviiinn 4y ago
Just have chatgpt write it for you

sonofhans 4y ago

Not as literal a take as you might like, but Fritz Lang’s _Metropolis_, of course — an invisible, downtrodden underclass breaks their backs on the machines so that a mediocre elite can live in the clouds. That was nearly 100 years ago.

huevosabio 4y ago

I never understand this type of hate-porn. These jobs are not horrible, not for humanity's historical standards. They can be done from home, without commute, without physical and demanding labor. A medieval peasant, a 1800s factory worker and probably many of the 1900s low skilled laborers would take this over the alternatives. Moreover, these people have every right and option to not do it, in fact right now the labor market is such that they have valuable alternatives! We keep thinking about how this or that job is not acceptable for _our personal standards_, but until we get Fully Automated Luxury Communism we have to be honest that progress looks this: slightly less shitty jobs in exchange for higher quality of life.

AndrewKemendo 4y ago
They aren't less shitty they are shittier in different ways, but the key point is that they are progressively more alienated from the value they provide and the community structure.
- catiopatio 4y ago
  The value they individually provide is minuscule; it only matters in aggregate, and when leveraged by others who bring a great deal more to the table. Anyone of average IQ and education could do this work, and there’s no shortage of people willing to do it.
  - antifa 4y ago
    I'm more concerned with how this will trend downward over the next decade(s) than how high up one side of the titanic is.

rvba 4y ago

The main protagonist of the novel "Nineteen Eighty-Four" by Orewell worked in the Ministry of Truth and his job was to rewrite historical records to conform to the state's ever-changing version of history. Basically he was a censor, who would remove inconvenient people or facts from newspapers or books. (I dont think Internet existed in the situation described in the book). 1984 was a big critic of communism, where facts dont mater and history can be rewritten. Are we really far from that? * Those poorly paid raters label things - and a lot of things depends on quality of their work. Do those companies even cross-check their work, when the idea is to get the lowest bidder? (on a side note, why FAANG companies dont understand the "garbage in, garbage out" concept? They can pay 300k usd to some programmer, yet they skimp on getting classifiers from USA or Europe who know the cultural context) * Then there are probably censors, who can censor things based on their own ideas. Probably censorship can happen at two levels: first by using labels, then by deciding what to show based on those labels. * Some websites have "fact checkers", who are necessary (with the amount of lies written in troll-farms and soon lies written by AI), but you cant really appeal about their process in any way - and those are some faceless people from third world countries. *Often you cannot ever contact with a real person, Google and Facebook are the worst here. Microsoft is a bit better, because if you pay them money for support, you can get human support. * Wikipedia is a constant war between vandals, but also actors that are much smarter (seem state sponsored even). When we speak about Wikipedia it is incredibly sad that it never introduced any systems to review the work of their admins (I saw a situation where some articles are "guarded" by 2-3 admins who share their own >version of facts< and nobody can do anything about that). And this could be solved by setting up panels of judges that judge edits that are anonymous. Or setting a panel of judges who judge actions of admins (yes, there is that "Arbitration Committee" that barely works). Of course then there is the question "who watches the watchmen", but still - Wikipedia Foundation has millions of dollars that they waste on various things, instead of spending it to improve their core product. On a side note, I always thought about starting such a data labeling business; I think a lot of work done by current labelers is still low quality. I saw some start-ups that were checking things (e.g. medical data) and they would pay peanuts to the people who labeled their data. Then they would get bad labels.. and build wrong conclusions on that with their classifiers.

antifa 4y ago

AndrewKemendo 4y ago

"The march toward perfect alienation and exploitation will make any job unthinkable at any pay rate." [1] [1] https://kemendo.com/Myth.pdf https://kemendo.com/Myth.pdf