4 ms·

In (language model-backed) natural language interfaces the loss of precision can be made up via iteration. If it were about getting it right on the first try th

In (language model-backed) natural language interfaces the loss of precision can be made up via iteration. If it were about getting it right on the first try this would be a dead end but there's no need for that restriction.

roflyear 4y ago
Yeah maybe! Just like if you have enough monkeys iterating on enough keyboards...
- westoncb 4y ago
  Just a few monkeys should be fine ;) If this weren't the case then it wouldn't be possible for (e.g.) the software industry to exist as it does: non-technical folks using natural language are able to converse with engineers who take informal descriptions and turn them into code, often leaning heavily on iteration the bring code and spec into conformance.
  - roflyear 4y ago
    "should be fine" - how do you determine this? There have been many, many cases where I was not able to get GPT-4 to "understand" my problem. No matter how much I tried (until I hit the rate limit for those hours, anyway). People are throwing these absolutes around, and it's just not totally true. Much of an engineer's job is to try and implement the correct solution for imperfect requirements, then to go back and quickly fix things to match the real requirements.
btown 4y ago
And specifically, if the agent implementing the requested actions has a mental model of the domain that is similar to the mental model held by the imprecise specification writer, then the specification doesn't need to be precise to have a high probability of being implemented right the first time! The miracle here is that LLMs don't even need fine-tuning to get to that level - and that's unprecedented for non-human agents.
ZephyrBlu 4y ago
Legibility is generally a requirement though. At least is currently is.
SanderNL 4y ago
What makes us think that encoding all functionality using natural language is somehow more compact than using code? Sure, describing a website with a button is easy, but I don’t quite see how you would describe Kubernetes or a device driver without writing a book in “legalese”. Or try formulating a math proof with natural language.
- flangola7 4y ago
  Who said you need to need to describe Kubernetes? "Make my programs work at any scale". Boom
  - SanderNL 4y ago
    That just.. doesn’t work that way. Edit: besides, if it could work you lose the competitive edge. I could describe a much faster more cost effective system which the machine can implement. And we are off to the races again..
    - flangola7 4y ago
      If/when it is a generally capable intelligence it very much does work that way. That's how it worked with humans. We saw that stuff needed to work at scale and created something to achieve that.
      - SanderNL 4y ago
        I guess you are right about AGI. I think (hope) that is still off in the distance. We’d have other problems than scaling crud apps, I think.
        flangola7 4y ago
        >I think (hope) that is still off in the distance. It's being rumored that OpenAI is currently training GPT-5 which will be ready in December, and that many people in the company think it will be a human or better level AGI. Even if it isn't the consistently supersonic jumps they are making every generation suggests we don't have long until human brains are outmoded legacy hardware. >We’d have other problems than scaling crud apps, I think. Ever since I first interacted with the original GPT-3 in 2020 I've had the realization that our future was going to be curtailed and distorted into an inconceivable Escher piece. It seems that future is nearly upon us.

roflyear 4y ago

Yeah maybe! Just like if you have enough monkeys iterating on enough keyboards...

westoncb 4y ago
Just a few monkeys should be fine ;) If this weren't the case then it wouldn't be possible for (e.g.) the software industry to exist as it does: non-technical folks using natural language are able to converse with engineers who take informal descriptions and turn them into code, often leaning heavily on iteration the bring code and spec into conformance.
- roflyear 4y ago
  "should be fine" - how do you determine this? There have been many, many cases where I was not able to get GPT-4 to "understand" my problem. No matter how much I tried (until I hit the rate limit for those hours, anyway). People are throwing these absolutes around, and it's just not totally true. Much of an engineer's job is to try and implement the correct solution for imperfect requirements, then to go back and quickly fix things to match the real requirements.

btown 4y ago

And specifically, if the agent implementing the requested actions has a mental model of the domain that is similar to the mental model held by the imprecise specification writer, then the specification doesn't need to be precise to have a high probability of being implemented right the first time! The miracle here is that LLMs don't even need fine-tuning to get to that level - and that's unprecedented for non-human agents.

ZephyrBlu 4y ago

Legibility is generally a requirement though. At least is currently is.

SanderNL 4y ago

What makes us think that encoding all functionality using natural language is somehow more compact than using code? Sure, describing a website with a button is easy, but I don’t quite see how you would describe Kubernetes or a device driver without writing a book in “legalese”. Or try formulating a math proof with natural language.

flangola7 4y ago
Who said you need to need to describe Kubernetes? "Make my programs work at any scale". Boom
- SanderNL 4y ago
  That just.. doesn’t work that way. Edit: besides, if it could work you lose the competitive edge. I could describe a much faster more cost effective system which the machine can implement. And we are off to the races again..
  - flangola7 4y ago
    If/when it is a generally capable intelligence it very much does work that way. That's how it worked with humans. We saw that stuff needed to work at scale and created something to achieve that.
    - SanderNL 4y ago
      I guess you are right about AGI. I think (hope) that is still off in the distance. We’d have other problems than scaling crud apps, I think.
      - flangola7 4y ago
        >I think (hope) that is still off in the distance. It's being rumored that OpenAI is currently training GPT-5 which will be ready in December, and that many people in the company think it will be a human or better level AGI. Even if it isn't the consistently supersonic jumps they are making every generation suggests we don't have long until human brains are outmoded legacy hardware. >We’d have other problems than scaling crud apps, I think. Ever since I first interacted with the original GPT-3 in 2020 I've had the realization that our future was going to be curtailed and distorted into an inconceivable Escher piece. It seems that future is nearly upon us.