6 ms·

I am personally of the opinion that ML will end up being 'normal technology', albeit incredibly transformative. I think you can combine 'Incanters' and 'Proces

by ej88 6mo ago

I am personally of the opinion that ML will end up being 'normal technology', albeit incredibly transformative. I think you can combine 'Incanters' and 'Process Engineers' into one - 'Users'. Jobs that encompass a role that requires accountability will be directing, providing context, and verifying the output of agents, almost like how millions of workers know basic computer skills and Microsoft Office. In my opinion, how at-risk a job is in the LLM era comes down to: 1: How easy is it to construct RL loops to hillclimb on performance? 2: How easy is it to construct a LLM harness to perform the tasks? 3: How much of the job is a structured set of tasks vs. taking accountability? What's the consequence of a mistake? How much of it comes down to human relationships? Hence why I've been quite bullish on software engineering (but not coding). You can easy set up 1) and 2) on contrived or sandboxed coding tasks but then 3) expands and dominates the rest of the role. On Model Trainers -- I'm not so convinced that RLHF puts the professional experts out of work, for a few reasons. Firstly, nearly all human data companies produce data that is somewhat contrived, by definition of having people grade outputs on a contracting platform; plus there's a seemingly unlimited bound on how much data we can harvest in the world. Secondly, as I mentioned before, the bottleneck is both accountability and the ability for the model to find fresh context without error.

netcan 6mo ago
In some sense, technology is "not normal" regardless. If we think of the digitization tech revolution... the changes it made to the economy are hard to describe well, even now. In the early days, it was going to turn banks from billion dollar businesses to million dollar ones. Universities would be able to eliminate most of their admin. Accounting and finances would be trivialized. Etc. Earlier tech revolution s were unpredictable too... But at lest retrospectively they made sense. It's not that clear what the core activities of our economy even are. It's clear at micro level, but as you zoom out it gets blurry. Why is accountability needed? It's clearly needed in its context... but it's hard to understand how it aggregates.
- bobthepanda 6mo ago
  Accountability is really a way to address liability. So long as people can sue and companies can pay out, or individuals can go to jail, there is always going to be a question of liability; and historically the courts have not looked kindly at those who throw their hands up in the air and say “I was just following orders from a human/entity”
  - pixl97 6mo ago
    >nd historically the courts have not looked This is dependent on having a court system uncaptured by corruption. We're already seeing that large corporations in the "too big to fail" categories fall outside of government control. And in countries with bribing/lobbying legalized or ignored they have the funds to capture the courts.
    - bobthepanda 6mo ago
      While this is true, this is somewhat mitigated by the fact that few sectors are truly monopolized and large corporations also sue each other.
  - cucumber3732842 6mo ago
    A huge component of compulsory (either by statute or de-facto as a result of adjacent statute, like mandatory insurance + requirements thereof) professional licensure is that if you follow the rules set by (some entity deputized by) government the government will in return never leave you holding the bag. The government gains partial control and the people under it's control get partial protection. "oh I'm sorry your hospital burned down mr plantiff but the electrician was following his professional rules so his liability is capped at <small number> you'll just have to eat this one" I would wager that a solid half if not more of the economy exists under some sort of arrangement like that.
    - bobthepanda 6mo ago
      Right, but usually that also involves verifying that the electrician actually followed the professional rules, and if not, they have liability
xienze 6mo ago
> Hence why I've been quite bullish on software engineering (but not coding). You can easy set up 1) and 2) on contrived or sandboxed coding tasks but then 3) expands and dominates the rest of the role. Why can't LLMs and agents progress further to do this software engineering job better than an actual software engineer? I've never seen anyone give a satisfactory answer to this. Especially the part about making mistakes. A lot of the defense of LLM shortcomings (i.e., generating crappy code) comes down to "well humans write bad code too." OK? Well, humans make mistakes too. Theoretically, an LLM software engineer will make far fewer than a human. So why should I prefer keeping you in the loop? It's why I just can't understand the mindset of software engineers who are giddy about the direction things are going. There really is nothing special about your expertise that an LLM can't achieve, theoretically. We're always so enamored by new and exciting technology that we fail to realize the people in charge are more than happy to completely bury us with it.
- guzfip 6mo ago
  > It's why I just can't understand the mindset of software engineers who are giddy about this brave new world. There really is nothing special about your expertise that an LLM can't achieve, theoretically. They’re stupid or they’re already set up for success. The general ideas seems to be generalists are screwed, domain experts will be fine.
  - xienze 6mo ago
    > domain experts will be fine But I don't see how this holds up to even the slightest amount of scrutiny. We're literally training LLMs to BE domain experts.
    - bwestergard 6mo ago
      I think these arguments tend to reach impasse because one gravitates to one of two views: 1) My experiences with LLMs are so impressive that I consider their output to generally be better than what the typical developer would produce. People who can't see this have not gotten enough experience with the models I find so impressive, or are in denial about the devaluation of their skills. 2) My experiences with LLMs have been mundane. People who see them as transformative lack the expertise required to distinguish between mediocre and excellent code, leading them to deny there is a difference.
aphyr 6mo ago
> I think you can combine 'Incanters' and 'Process Engineers' into one - 'Users' I wanted to talk about this more but couldn't quite figure out how to phrase it, so I cut a fair bit: with "incanters" I'm trying to point at a sort of ... intuitive, more informal practitioner knowledge / metis, and contrast it with a more statistically rigorous approach in "statistical/process engineers". I expect a lot of people will fuse the two, but I'm trying to stake out some tentpoles here. Users integrate a continuum of approaches, including individual intuition, folklore, formal and informal texts, scientific papers, and rigorously designed harnesses & in-house experiments. Like farming--there's deep, intuitive knowledge of local climate and landraces, but also big industrial practice, and also research plots, and those different approaches inform (and override) each other in complex ways.
dyauspitr 6mo ago
Throw on a wizard hat and robe at some of the voice only vibe coders you see on YouTube and it’s essentially “incanters”. Hilarious.
g9yuayon 6mo ago
> How much of the job is a structured set of tasks vs. taking accountability? More accurately, how many jobs are probabilistically mechanical. That is, how many jobs are really the execution of a serious Bayesian decisions with a strong prior. LLMs are really great at displacing such jobs.

by ej88 6mo ago

netcan 6mo ago

In some sense, technology is "not normal" regardless. If we think of the digitization tech revolution... the changes it made to the economy are hard to describe well, even now. In the early days, it was going to turn banks from billion dollar businesses to million dollar ones. Universities would be able to eliminate most of their admin. Accounting and finances would be trivialized. Etc. Earlier tech revolution s were unpredictable too... But at lest retrospectively they made sense. It's not that clear what the core activities of our economy even are. It's clear at micro level, but as you zoom out it gets blurry. Why is accountability needed? It's clearly needed in its context... but it's hard to understand how it aggregates.

bobthepanda 6mo ago
Accountability is really a way to address liability. So long as people can sue and companies can pay out, or individuals can go to jail, there is always going to be a question of liability; and historically the courts have not looked kindly at those who throw their hands up in the air and say “I was just following orders from a human/entity”
- pixl97 6mo ago
  >nd historically the courts have not looked This is dependent on having a court system uncaptured by corruption. We're already seeing that large corporations in the "too big to fail" categories fall outside of government control. And in countries with bribing/lobbying legalized or ignored they have the funds to capture the courts.
  - bobthepanda 6mo ago
    While this is true, this is somewhat mitigated by the fact that few sectors are truly monopolized and large corporations also sue each other.
- cucumber3732842 6mo ago
  A huge component of compulsory (either by statute or de-facto as a result of adjacent statute, like mandatory insurance + requirements thereof) professional licensure is that if you follow the rules set by (some entity deputized by) government the government will in return never leave you holding the bag. The government gains partial control and the people under it's control get partial protection. "oh I'm sorry your hospital burned down mr plantiff but the electrician was following his professional rules so his liability is capped at <small number> you'll just have to eat this one" I would wager that a solid half if not more of the economy exists under some sort of arrangement like that.
  - bobthepanda 6mo ago
    Right, but usually that also involves verifying that the electrician actually followed the professional rules, and if not, they have liability

xienze 6mo ago

> Hence why I've been quite bullish on software engineering (but not coding). You can easy set up 1) and 2) on contrived or sandboxed coding tasks but then 3) expands and dominates the rest of the role. Why can't LLMs and agents progress further to do this software engineering job better than an actual software engineer? I've never seen anyone give a satisfactory answer to this. Especially the part about making mistakes. A lot of the defense of LLM shortcomings (i.e., generating crappy code) comes down to "well humans write bad code too." OK? Well, humans make mistakes too. Theoretically, an LLM software engineer will make far fewer than a human. So why should I prefer keeping you in the loop? It's why I just can't understand the mindset of software engineers who are giddy about the direction things are going. There really is nothing special about your expertise that an LLM can't achieve, theoretically. We're always so enamored by new and exciting technology that we fail to realize the people in charge are more than happy to completely bury us with it.

guzfip 6mo ago
> It's why I just can't understand the mindset of software engineers who are giddy about this brave new world. There really is nothing special about your expertise that an LLM can't achieve, theoretically. They’re stupid or they’re already set up for success. The general ideas seems to be generalists are screwed, domain experts will be fine.
- xienze 6mo ago
  > domain experts will be fine But I don't see how this holds up to even the slightest amount of scrutiny. We're literally training LLMs to BE domain experts.
  - bwestergard 6mo ago
    I think these arguments tend to reach impasse because one gravitates to one of two views: 1) My experiences with LLMs are so impressive that I consider their output to generally be better than what the typical developer would produce. People who can't see this have not gotten enough experience with the models I find so impressive, or are in denial about the devaluation of their skills. 2) My experiences with LLMs have been mundane. People who see them as transformative lack the expertise required to distinguish between mediocre and excellent code, leading them to deny there is a difference.

aphyr 6mo ago

> I think you can combine 'Incanters' and 'Process Engineers' into one - 'Users' I wanted to talk about this more but couldn't quite figure out how to phrase it, so I cut a fair bit: with "incanters" I'm trying to point at a sort of ... intuitive, more informal practitioner knowledge / metis, and contrast it with a more statistically rigorous approach in "statistical/process engineers". I expect a lot of people will fuse the two, but I'm trying to stake out some tentpoles here. Users integrate a continuum of approaches, including individual intuition, folklore, formal and informal texts, scientific papers, and rigorously designed harnesses & in-house experiments. Like farming--there's deep, intuitive knowledge of local climate and landraces, but also big industrial practice, and also research plots, and those different approaches inform (and override) each other in complex ways.

dyauspitr 6mo ago

Throw on a wizard hat and robe at some of the voice only vibe coders you see on YouTube and it’s essentially “incanters”. Hilarious.

g9yuayon 6mo ago

> How much of the job is a structured set of tasks vs. taking accountability? More accurately, how many jobs are probabilistically mechanical. That is, how many jobs are really the execution of a serious Bayesian decisions with a strong prior. LLMs are really great at displacing such jobs.