6 ms·

The opposite might be true, and here’s why - 1) by using English as spec, the barrier of entry has gone lower, 2) LLMs can also write prompts and self introspec

The opposite might be true, and here’s why - 1) by using English as spec, the barrier of entry has gone lower, 2) LLMs can also write prompts and self introspect to debug.

notacoward 3y ago
> LLMs can also write prompts and self introspect to debug. Why should we assume that won't lead to a rabbit hole of misunderstanding or outright hallucination? If it doesn't know what "correct" really is, even infinite levels of supervision and reinforcement might still be toward an incorrect goal.
- ben_w 3y ago
  To which the normal response[0] is: that's just like humans. Of course, it's still bad that humans do it; but despite the scientific method etc., even successful humans often work towards an incorrect goal. [0] I am cultured, you're quoting memes, that AI is just a stochastic parrot: https://en.wikipedia.org/wiki/Emotive_conjugation https://en.wikipedia.org/wiki/Emotive_conjugation
  - notacoward 3y ago
    But it's not just like humans. For one thing it's built differently, with a different relationship between training and execution. It doesn't learn from its mistakes until it gets the equivalent of a brain transplant, and in fact extant AIs are notorious for doubling down instead of accepting correction. Even more importantly, the AI doesn't have real-world context, which is often helpful to notice when "correct" (to the spec) behavior is not useful, acceptable, or even safe in practice. This is why the idea of an AI controlling a physical system is so terrifying. Whatever requirement the prompter forgot to include will not be recognized by the AI either, whereas a human who knows about physical properties like mass or velocity or rigidity will intuitively honor requirements related to those. Adding layers is as likely to magnify errors as to correct them.
    - ben_w 3y ago
      > But it's not just like humans. For one thing it's built differently I'm referring to the behaviour, not the inner nature. > in fact extant AIs are notorious for doubling down instead of accepting correction. My experience suggests ChatGPT is better than, say, humans on Twitter. I've had the misfortune of several IRL humans who were also much, much worse; but the problem was much rarer outside social media. > Even more importantly, the AI doesn't have real-world context, which is often helpful to notice when "correct" (to the spec) behavior is not useful, acceptable, or even safe in practice. Absolutely a problem. Not only for AI, though. When I was a kid, my mum had a kneeling stool she couldn't use, because the woodworker she'd asked to reinforce it didn't understand it and put a rod where your legs should go. I've made the mistake of trying to use RegEx for what I thought was a limited-by-the-server subset of HTML, despite the infamous StackOverflow post, because I incorrectly thought it didn't apply to the situation. There's an ongoing two-way "real-world context" miss-match between those who want the state to be able to pierce encryption and those who consider that to be an existential threat to all digital services. > a human who knows about physical properties like mass or velocity or rigidity will intuitively honor requirements related to those Yeah, kinda, but also no. We can intuit within the range of our experience, but we had to invent counter-intuitive maths to make most of our modern technological wonders. -- All that said, with this: > It doesn't learn from its mistakes until it gets the equivalent of a brain transplant You've boosted my optimism that an ASI probably won't succeed if it decided it preferred our atoms to be rearranged to our detriments.
      - notacoward 3y ago
        > I'm referring to the behaviour, not the inner nature. Since the inner nature does affect behavior, that's a non sequitur. > we had to invent counter-intuitive maths to make most of our modern technological wonders. Indeed, and that's worth considering, but we shouldn't pretend it's the common case. In the common case, the machine's lack of real-world context is a disadvantage. Ditto for the absence of any actual understanding beyond "word X often follows word Y" which would allow it to predict consequences it hasn't seen yet. Because of these deficits, any "intuitive leaps" the AI might make are less likely to yield useful results than the same in a human. The ability to form a coherent - even if novel - theory and an experiment to test it is key to that kind of progress, and it's something these models are fundamentally incapable of doing.
- ModernMech 3y ago
  It’s like when you continually refine a Midjourney image. At first refining it gets better results, but if you keep going the pictures start coming out…really weird. It’s up to the human to figure out when to stop using some sort of external measure of aesthetics.
mooreds 3y ago
> by using English as spec, the barrier of entry has gone lower, I'm not sure that is true. The level of back and forth and refinements needed indicate to me that the "English" used is not the normal language I use when talking to people. It's almost like a refined version of cucumber with syntax that is slightly more forgiving. Maybe I'm being a codger, but LLMs seem (at least for now) far better for summarizing and giving high level overviews of concepts rather than nailing precise code requirements.
- ben_w 3y ago
  > It's almost like a refined version of cucumber with syntax that is slightly more forgiving. I don't know if "cucumber" is autocorrupt or an actual non-vegetable thing; can you clarify?
  - omnicognate 3y ago
    https://cucumber.io/ https://cucumber.io/ That "did they actually mean that or was it autowrong?" feeling is going to get worse I fear.
  - mjr00 3y ago
    Not a typo.[0] In the 00s/early 10s, software went through a fad phase where people earnestly thought that by implementing Gherkin frameworks like Cucumber, you'd be able to hand off writing tests to "business people" in "plain English." It went about as well as you'd expect. [0] https://cucumber.io/docs/gherkin/ https://cucumber.io/docs/gherkin/
    - ben_w 3y ago
      Thanks! Despite that period being when I finished my Software Engineering degree, got my first job, and then attempted self-employment, I'd never heard of it before. Looking at the book titles — "Cucumber Recipes" in particular — even if I had encountered it, I might have assumed the whole thing was a joke.
    - Tainnor 3y ago
ZephyrBlu 3y ago
I think English as a spec actually makes the barrier of entry higher, not lower. Code itself is far easier to understand than an English description of the code. To understand an English description of code you already have to have a deeper understanding of what the code is doing. For code itself you can reference the syntax to understand what's going on. The prompt in this case is using very technical language that a beginner will have no idea about. But if you gave them the code they could at least struggle along and figure it out by looking things up.
- deleted 3y ago
  [deleted]
- lcnPylGDnU4H9OF 3y ago
  This reminds me of rubber ducking[0] in how it necessitates a certain understanding. If one is able to explain it in plain English it's because it is understood. [0] https://en.wikipedia.org/wiki/Rubber_duck_debugging https://en.wikipedia.org/wiki/Rubber_duck_debugging
- nextworddev 3y ago
  Yes but LLMs can also be used by laypeople to explain the issue in plain English too. That’s the problem. Not that LLMs would need human to guide the debugging process anyways (at least in a few years)
  - ZephyrBlu 3y ago
    You still have the same problem... You cannot describe a technical field with plain English. If you did so the semantics would be incorrect. There is a reason jargon exists. The first two paragraphs alone are absolutely chock with terms that would not be easily explained to a layperson: "The current system is an online whiteboard system. Tech stack: typescript, react, redux, konvajs and react-konva. And vitest, react testing library for model, view model and related hooks, cypress component tests for view. All codes should be written in the tech stack mentioned above. Requirements should be implemented as react components in the MVVM architecture pattern." What is every library in that list? What is a model? What is a view model? What is a hook, component test, view, MVVM, etc? If a layperson could understand explanations for all these things then they would not be a layperson.
gumballindie 3y ago
I mean sure if the world were to run on basic code. Perhaps wordpress developers may feel slightly threatened by even that is well above all examples of a"i" code i've seen.
agentultra 3y ago
But you can't determine if a statement is true by simply reading more words. It's also not efficient for doing higher level work. There was a time before we had algebra where people were still expressing the same ideas but the notation wasn't there. Mathematics was expressed in "plain language." It's extremely difficult to read for us. For mathematician's of the time there was no other way to explain algorithms or expressions. For simple programs I have no doubt that these tools enable more people to generate code. However it's not going to be helpful for people working on hypervisors, networking stacks, operating systems, distributed databases, cryptography, and the like yet. For that you need a more precise language and an LLM that can reason about semantics and generate understandable proofs: not boilerplate proofs either -- they have to be elegant so that a human reading them can understand the problem as well. We're still a ways from being able to do that.
- nextworddev 3y ago
  Arguably reading code can’t lead to definitive conclusions about its bug-free-ness
  - agentultra 3y ago
    Precisely! And neither can generating a handful of unit tests. As EWD would say, they only prove the existence of one error. Not that there are no errors. If we want more programs that are correct with respect to their specifications we need to write better, precise specifications… not wave our hands around. However for a lot of line-of-business tasks we’re generally fine with ambiguous, informal specifications. We’re not certain our programs are correct with respect to the specifications, if we had written them out formally, but it’s good enough. I think most businesses that are writing software that needs to be reliable and precise are not going to benefit from these kinds of tools for some time.
    - tjr 3y ago
      This is true in aerospace software. Lots of process, lots of specification, lots of verification. I wouldn't want to say that GPT-seque tools would be useless here, but I really don't see them offering the same kind of magic leverage that they might offer on some other projects. And vice-versa! Most software projects do not benefit from the rigor used in aerospace, because it's just not needed, and would be a waste of time. I am definitely seeing ways that GPT tools could speed up some aerospace work, but we need to be really really sure that things are being done correctly... not just mostly correct, or seemingly correct.
have_faith 3y ago
English as a spec is incredibly "fuzzy", there are many valid interpretations of intent. I don't think that can be avoided?
- sarchertech 3y ago
  It can't. Legalese is an attempt to do so, and it's impenetrable by non experts and still frequently ambiguous.
bartimus 3y ago
But there's still going to have to be a human who has the ability to form a mental model of the thing that's needing to be implemented. Functionally and technically. The results of the LLM will vary depending on the level of know-how the human instructor has.

The opposite might be true, and here’s why - 1) by using English as spec, the barrier of entry has gone lower, 2) LLMs can also write prompts and self introspect to debug.

notacoward 3y ago

> LLMs can also write prompts and self introspect to debug. Why should we assume that won't lead to a rabbit hole of misunderstanding or outright hallucination? If it doesn't know what "correct" really is, even infinite levels of supervision and reinforcement might still be toward an incorrect goal.

ben_w 3y ago
To which the normal response[0] is: that's just like humans. Of course, it's still bad that humans do it; but despite the scientific method etc., even successful humans often work towards an incorrect goal. [0] I am cultured, you're quoting memes, that AI is just a stochastic parrot: https://en.wikipedia.org/wiki/Emotive_conjugation https://en.wikipedia.org/wiki/Emotive_conjugation
- notacoward 3y ago
  But it's not just like humans. For one thing it's built differently, with a different relationship between training and execution. It doesn't learn from its mistakes until it gets the equivalent of a brain transplant, and in fact extant AIs are notorious for doubling down instead of accepting correction. Even more importantly, the AI doesn't have real-world context, which is often helpful to notice when "correct" (to the spec) behavior is not useful, acceptable, or even safe in practice. This is why the idea of an AI controlling a physical system is so terrifying. Whatever requirement the prompter forgot to include will not be recognized by the AI either, whereas a human who knows about physical properties like mass or velocity or rigidity will intuitively honor requirements related to those. Adding layers is as likely to magnify errors as to correct them.
  - ben_w 3y ago
    > But it's not just like humans. For one thing it's built differently I'm referring to the behaviour, not the inner nature. > in fact extant AIs are notorious for doubling down instead of accepting correction. My experience suggests ChatGPT is better than, say, humans on Twitter. I've had the misfortune of several IRL humans who were also much, much worse; but the problem was much rarer outside social media. > Even more importantly, the AI doesn't have real-world context, which is often helpful to notice when "correct" (to the spec) behavior is not useful, acceptable, or even safe in practice. Absolutely a problem. Not only for AI, though. When I was a kid, my mum had a kneeling stool she couldn't use, because the woodworker she'd asked to reinforce it didn't understand it and put a rod where your legs should go. I've made the mistake of trying to use RegEx for what I thought was a limited-by-the-server subset of HTML, despite the infamous StackOverflow post, because I incorrectly thought it didn't apply to the situation. There's an ongoing two-way "real-world context" miss-match between those who want the state to be able to pierce encryption and those who consider that to be an existential threat to all digital services. > a human who knows about physical properties like mass or velocity or rigidity will intuitively honor requirements related to those Yeah, kinda, but also no. We can intuit within the range of our experience, but we had to invent counter-intuitive maths to make most of our modern technological wonders. -- All that said, with this: > It doesn't learn from its mistakes until it gets the equivalent of a brain transplant You've boosted my optimism that an ASI probably won't succeed if it decided it preferred our atoms to be rearranged to our detriments.
    - notacoward 3y ago
      > I'm referring to the behaviour, not the inner nature. Since the inner nature does affect behavior, that's a non sequitur. > we had to invent counter-intuitive maths to make most of our modern technological wonders. Indeed, and that's worth considering, but we shouldn't pretend it's the common case. In the common case, the machine's lack of real-world context is a disadvantage. Ditto for the absence of any actual understanding beyond "word X often follows word Y" which would allow it to predict consequences it hasn't seen yet. Because of these deficits, any "intuitive leaps" the AI might make are less likely to yield useful results than the same in a human. The ability to form a coherent - even if novel - theory and an experiment to test it is key to that kind of progress, and it's something these models are fundamentally incapable of doing.
ModernMech 3y ago
It’s like when you continually refine a Midjourney image. At first refining it gets better results, but if you keep going the pictures start coming out…really weird. It’s up to the human to figure out when to stop using some sort of external measure of aesthetics.

mooreds 3y ago

> by using English as spec, the barrier of entry has gone lower, I'm not sure that is true. The level of back and forth and refinements needed indicate to me that the "English" used is not the normal language I use when talking to people. It's almost like a refined version of cucumber with syntax that is slightly more forgiving. Maybe I'm being a codger, but LLMs seem (at least for now) far better for summarizing and giving high level overviews of concepts rather than nailing precise code requirements.

ben_w 3y ago
> It's almost like a refined version of cucumber with syntax that is slightly more forgiving. I don't know if "cucumber" is autocorrupt or an actual non-vegetable thing; can you clarify?
- omnicognate 3y ago
  https://cucumber.io/ https://cucumber.io/ That "did they actually mean that or was it autowrong?" feeling is going to get worse I fear.
- mjr00 3y ago
  Not a typo.[0] In the 00s/early 10s, software went through a fad phase where people earnestly thought that by implementing Gherkin frameworks like Cucumber, you'd be able to hand off writing tests to "business people" in "plain English." It went about as well as you'd expect. [0] https://cucumber.io/docs/gherkin/ https://cucumber.io/docs/gherkin/
  - ben_w 3y ago
    Thanks! Despite that period being when I finished my Software Engineering degree, got my first job, and then attempted self-employment, I'd never heard of it before. Looking at the book titles — "Cucumber Recipes" in particular — even if I had encountered it, I might have assumed the whole thing was a joke.
  - Tainnor 3y ago

ZephyrBlu 3y ago

I think English as a spec actually makes the barrier of entry higher, not lower. Code itself is far easier to understand than an English description of the code. To understand an English description of code you already have to have a deeper understanding of what the code is doing. For code itself you can reference the syntax to understand what's going on. The prompt in this case is using very technical language that a beginner will have no idea about. But if you gave them the code they could at least struggle along and figure it out by looking things up.

deleted 3y ago
[deleted]
lcnPylGDnU4H9OF 3y ago
This reminds me of rubber ducking[0] in how it necessitates a certain understanding. If one is able to explain it in plain English it's because it is understood. [0] https://en.wikipedia.org/wiki/Rubber_duck_debugging https://en.wikipedia.org/wiki/Rubber_duck_debugging
nextworddev 3y ago
Yes but LLMs can also be used by laypeople to explain the issue in plain English too. That’s the problem. Not that LLMs would need human to guide the debugging process anyways (at least in a few years)
- ZephyrBlu 3y ago
  You still have the same problem... You cannot describe a technical field with plain English. If you did so the semantics would be incorrect. There is a reason jargon exists. The first two paragraphs alone are absolutely chock with terms that would not be easily explained to a layperson: "The current system is an online whiteboard system. Tech stack: typescript, react, redux, konvajs and react-konva. And vitest, react testing library for model, view model and related hooks, cypress component tests for view. All codes should be written in the tech stack mentioned above. Requirements should be implemented as react components in the MVVM architecture pattern." What is every library in that list? What is a model? What is a view model? What is a hook, component test, view, MVVM, etc? If a layperson could understand explanations for all these things then they would not be a layperson.

gumballindie 3y ago

I mean sure if the world were to run on basic code. Perhaps wordpress developers may feel slightly threatened by even that is well above all examples of a"i" code i've seen.

agentultra 3y ago

But you can't determine if a statement is true by simply reading more words. It's also not efficient for doing higher level work. There was a time before we had algebra where people were still expressing the same ideas but the notation wasn't there. Mathematics was expressed in "plain language." It's extremely difficult to read for us. For mathematician's of the time there was no other way to explain algorithms or expressions. For simple programs I have no doubt that these tools enable more people to generate code. However it's not going to be helpful for people working on hypervisors, networking stacks, operating systems, distributed databases, cryptography, and the like yet. For that you need a more precise language and an LLM that can reason about semantics and generate understandable proofs: not boilerplate proofs either -- they have to be elegant so that a human reading them can understand the problem as well. We're still a ways from being able to do that.

nextworddev 3y ago
Arguably reading code can’t lead to definitive conclusions about its bug-free-ness
- agentultra 3y ago
  Precisely! And neither can generating a handful of unit tests. As EWD would say, they only prove the existence of one error. Not that there are no errors. If we want more programs that are correct with respect to their specifications we need to write better, precise specifications… not wave our hands around. However for a lot of line-of-business tasks we’re generally fine with ambiguous, informal specifications. We’re not certain our programs are correct with respect to the specifications, if we had written them out formally, but it’s good enough. I think most businesses that are writing software that needs to be reliable and precise are not going to benefit from these kinds of tools for some time.
  - tjr 3y ago
    This is true in aerospace software. Lots of process, lots of specification, lots of verification. I wouldn't want to say that GPT-seque tools would be useless here, but I really don't see them offering the same kind of magic leverage that they might offer on some other projects. And vice-versa! Most software projects do not benefit from the rigor used in aerospace, because it's just not needed, and would be a waste of time. I am definitely seeing ways that GPT tools could speed up some aerospace work, but we need to be really really sure that things are being done correctly... not just mostly correct, or seemingly correct.

have_faith 3y ago

English as a spec is incredibly "fuzzy", there are many valid interpretations of intent. I don't think that can be avoided?

sarchertech 3y ago
It can't. Legalese is an attempt to do so, and it's impenetrable by non experts and still frequently ambiguous.

bartimus 3y ago

But there's still going to have to be a human who has the ability to form a mental model of the thing that's needing to be implemented. Functionally and technically. The results of the LLM will vary depending on the level of know-how the human instructor has.