5 ms·

I would argue that poorly specified requirements will be solved by an Auto-coder that can produce the end-result in a very short time. Allowing the PM, to see h

I would argue that poorly specified requirements will be solved by an Auto-coder that can produce the end-result in a very short time. Allowing the PM, to see how bad their requirements were and make it more elaborate.

SoftTalker 4y ago
When your requiremets are so detailed and comprehensive that the generated program is correct -- haven't you basically written the program?
- philipkglass 4y ago
  It really depends on the program. Some problems derive most of their complexity from trying to correctly model complex real world rules, like calculating pension payments for employees that were hired at different times, for different positions, under different union agreements, having worked varying numbers of years. Some problems have short, clear definitions yet remain challenging. An algorithm for solving them may have been discovered decades (or even centuries) ago. But trying to produce more efficient solutions can be a life's work and more. One of the most obvious examples is integer factorization, which had a basic algorithm published 800 years ago yet is still a research problem today: https://en.wikipedia.org/wiki/Trial_division https://en.wikipedia.org/wiki/Trial_division https://en.wikipedia.org/wiki/Integer_factorization#Factoring_algorithms https://en.wikipedia.org/wiki/Integer_factorization#Factorin...
- aerovistae 4y ago
  This tangentially reminds me of a very interesting idea I think I encountered in a Borges story - for a map to be useful, it must be inaccurate. That is to say, a perfectly accurate map would just be a replica of the the terrain it is mapping - anything less would technically be inaccurate. Only by leaving things out and "summarizing" an area does a map become a useful representation of the terrain.
  - mrexroad 4y ago
    “The map is not the territory”… until the map is the territory?
- berniedurfee 4y ago
  The was (still is?) one of the major issues with outsourcing development work. External contractors tend to need excruciatingly detailed requirements to produce the expected results, because they typically lack the domain knowledge to guess correctly when faced with ambiguity. Very often this means getting into long cycles of iteration. If time zones are distant, those cycles can be very long and painful. Perhaps outsourcing to AI will work much better, as those iterative cycles can be done in near real-time. Maybe there needs to be many different models, each with some industry/domain specific training to increase the odds it’ll make the right guess when it needs to.
- krageon 4y ago
  With almost all software you're likely to write professionally, yes. There are edge cases (as there are with all things), but those are not things AI is good at to begin with :)
jvm___ 4y ago
The Bad Requirement -> Bad Implementation -> Better Requirement -> Better Implementation cycle is going to be improved. https://en.wikipedia.org/wiki/OODA_loop https://en.wikipedia.org/wiki/OODA_loop The US Army calls it observe–orient–decide–act. "The approach explains how agility can overcome raw power in dealing with human opponents. It is especially applicable to cyber security and cyberwarfare. According to Boyd, decision-making occurs in a recurring cycle of observe–orient–decide–act. An entity (whether an individual or an organization) that can process this cycle quickly, observing and reacting to unfolding events more rapidly than an opponent, can thereby "get inside" the opponent's decision cycle and gain the advantage." Humans using AI will be well "inside the loop" of non-AI humans. I don't think AI alone will eliminate jobs, but jobs that use AI to get inside the loop of other companies will quickly eliminate the competition. Why would you pay and wait days when you can pay and wait minutes or hours for a quicker Ask-Show-Ask-Show loop that quickly narrows down on what the intent of your Ask was (even if Ask #1 was poorly thought out or worded)
- enkid 4y ago
  To be nitpicky, the OODA loop is an Air Force thing, as Boyd was a fighter pilot.
michaelmrose 4y ago
So we are talking about redefining programming to be managing, redefining, bug fixing, troubleshooting the output of ML potentially increasing output but making the job more complicated not allowing the non coding pm to replace more skilled workers.
- visarga 4y ago
  Yes, but in a few years we have to reevaluate this, probably language models are going to be good enough to allow a PM to replace devs. AIs will code at superhuman level because it is possible to test code, it will learn like AlphaGo learned from self-play.
  - svachalek 4y ago
    Maybe. The way this usually goes is tools get better, everyone says we don't need devs anymore, devs use the tools way better than PMs, standards rise, PM code is junk again.
  - throw827474737 4y ago
    I claim pipe dream, otherwise AIs can program themselves and we will have the singularity. Will it be good or evil?
  - michaelmrose 4y ago
    If you were picking someone to ride herd on an AI generating code are you sure - You will never need to understand a single bit of what the AI has generated for legal, moral, or strategic reasons? - You will never need to manually rework a single thing or fix anything ever. Logically neither statement is true furthermore who do you imagine is writing the tests that ensure correctness? Logically the person you would want to do this task isn't a yokel who understands interpersonal relationships its someone who understands what is being generated so your company wont murder people or get sued out of existence. Of course you will probably want more than one person doing this task and there ought to be some interface between him and his fellows and between the lot of them and management to ensure good communication and company standards... Maybe a PM? I find it endlessly fascinating that those who can't do dream about the obsolescence of the useful class deeming the irreplaceable human part of the endeavor is the petty manipulation of each other and ego massaging we all do by necessity to keep human endeavors tied together with bailing wire and barely plodding on. It doesn't speak well for our species. Beyond narrow parameters self improving arbitrary representation and manipulation of symbolic data is basically AI complete. The toy you would hope to enable you to ask a computer instead of a person for a particular improvement to your website would upon completion represent a short step from the obsolescence not of coders but of humans wrought in vain hope that we can actually control tools smarter than us. It's probably neither "a few years" away nor that fantastic of an idea but do carry on.
A4ET8a8uTh0 4y ago
I am not sure that can be done that way. There are issues that are culture or specific business need related. Bad requirement is the hard part it seems. I think we all have been in situations, where 'common sense' is not common ( and does not translate to the same thing to various layers for people who pass it; and that does not even account for translation ). This is usually how we end up with weird situations like currency coded as text so it can't be sorted by amount by end user and so on. I agree that specs make or break the project, but at what point is it ok to assume "x should do y".
- throw827474737 4y ago
  Yeah, on top of that add wading through legacy code, figuring out why wtf, this is crap, just to figure out most times it is actually crap, but then at least equal amount of times that there was a very good reason to do it that way, either technical or by business logic... cannot imagine how any AI system could have success there without getting towards the true AI singularity. This kind of code that I'd believe makes up 99% of all code out there that is worked on, is not available to the public for training. What to do about that? Rewrite all?
  - aiphex 4y ago
    I think this may be the case, the AI will figure out what the legacy code does and then completely re-writes it using whatever modern standards and ability to host it all in the cloud.
    - majormajor 4y ago
      The stochasticness probably gets you into trouble here. If you get different results each time you try to "figure out and rewrite" the legacy program... you're gonna need to be able to right some extremely thorough, perfect test cases. And how much of that are you willing to put to chance?
    - goatlover 4y ago
      If it's not trained on the legacy code, I have my doubts it will be able to re-write it well enough to replace a human baby sitting it, particularly when we're talking about large projects. As impressive as ChatGPT is at spitting out code examples, what gives you confidence that and LLM could rewrite a million lines of legacy code it has never seen before? That seems to be well beyond the ability of any AI that's been developed so far.
api 4y ago
Doesn't that make the PM just a programmer then, but with a more powerful tool? We used to code in ASM. Then we coded in high level languages. Soon we'll code by giving prompts to auto code generators.
- SQueeeeeL 4y ago
  That's kinda missing the level of control. Inherently both ASM and programming produce the same output, "programming" just traded the generality of ASM for verbose and diverse functions. The auto code generators fundamentally don't understand the problems they work on, so any solutions they create will be incidental.
  - fragmede 4y ago
    What is "understanding"? Compilers aren't sentient and can't fundamentally understand anything, either though.
    - iterateofen 4y ago
      I would argue compilers have a perfect “understanding” as the input:output is generally 1:1. One input almost always produces the same output. It’s just that the input is extremely limited by a set of predefined rules. The same can’t be said about code generators like this that are producing millions of possibilites and using heuristics to guess which program will have the most success.
      - fragmede 4y ago
        Reproducibility is a big deal. With the Stable Diffusion model as a publicly available test model, using the exact same inputs, especially the seed value for the random number generator, should get you the same picture on subsequent runs or on a different machine. Whether or not ChatGPT is evolving with our input is a secret to make that not the same for their model is known only to OpenAI.
  - majormajor 4y ago

SoftTalker 4y ago

When your requiremets are so detailed and comprehensive that the generated program is correct -- haven't you basically written the program?

philipkglass 4y ago
It really depends on the program. Some problems derive most of their complexity from trying to correctly model complex real world rules, like calculating pension payments for employees that were hired at different times, for different positions, under different union agreements, having worked varying numbers of years. Some problems have short, clear definitions yet remain challenging. An algorithm for solving them may have been discovered decades (or even centuries) ago. But trying to produce more efficient solutions can be a life's work and more. One of the most obvious examples is integer factorization, which had a basic algorithm published 800 years ago yet is still a research problem today: https://en.wikipedia.org/wiki/Trial_division https://en.wikipedia.org/wiki/Trial_division https://en.wikipedia.org/wiki/Integer_factorization#Factoring_algorithms https://en.wikipedia.org/wiki/Integer_factorization#Factorin...
aerovistae 4y ago
This tangentially reminds me of a very interesting idea I think I encountered in a Borges story - for a map to be useful, it must be inaccurate. That is to say, a perfectly accurate map would just be a replica of the the terrain it is mapping - anything less would technically be inaccurate. Only by leaving things out and "summarizing" an area does a map become a useful representation of the terrain.
- mrexroad 4y ago
  “The map is not the territory”… until the map is the territory?
berniedurfee 4y ago
The was (still is?) one of the major issues with outsourcing development work. External contractors tend to need excruciatingly detailed requirements to produce the expected results, because they typically lack the domain knowledge to guess correctly when faced with ambiguity. Very often this means getting into long cycles of iteration. If time zones are distant, those cycles can be very long and painful. Perhaps outsourcing to AI will work much better, as those iterative cycles can be done in near real-time. Maybe there needs to be many different models, each with some industry/domain specific training to increase the odds it’ll make the right guess when it needs to.
krageon 4y ago
With almost all software you're likely to write professionally, yes. There are edge cases (as there are with all things), but those are not things AI is good at to begin with :)

jvm___ 4y ago

The Bad Requirement -> Bad Implementation -> Better Requirement -> Better Implementation cycle is going to be improved. https://en.wikipedia.org/wiki/OODA_loop https://en.wikipedia.org/wiki/OODA_loop The US Army calls it observe–orient–decide–act. "The approach explains how agility can overcome raw power in dealing with human opponents. It is especially applicable to cyber security and cyberwarfare. According to Boyd, decision-making occurs in a recurring cycle of observe–orient–decide–act. An entity (whether an individual or an organization) that can process this cycle quickly, observing and reacting to unfolding events more rapidly than an opponent, can thereby "get inside" the opponent's decision cycle and gain the advantage." Humans using AI will be well "inside the loop" of non-AI humans. I don't think AI alone will eliminate jobs, but jobs that use AI to get inside the loop of other companies will quickly eliminate the competition. Why would you pay and wait days when you can pay and wait minutes or hours for a quicker Ask-Show-Ask-Show loop that quickly narrows down on what the intent of your Ask was (even if Ask #1 was poorly thought out or worded)

enkid 4y ago
To be nitpicky, the OODA loop is an Air Force thing, as Boyd was a fighter pilot.

michaelmrose 4y ago

So we are talking about redefining programming to be managing, redefining, bug fixing, troubleshooting the output of ML potentially increasing output but making the job more complicated not allowing the non coding pm to replace more skilled workers.

visarga 4y ago
Yes, but in a few years we have to reevaluate this, probably language models are going to be good enough to allow a PM to replace devs. AIs will code at superhuman level because it is possible to test code, it will learn like AlphaGo learned from self-play.
- svachalek 4y ago
  Maybe. The way this usually goes is tools get better, everyone says we don't need devs anymore, devs use the tools way better than PMs, standards rise, PM code is junk again.
- throw827474737 4y ago
  I claim pipe dream, otherwise AIs can program themselves and we will have the singularity. Will it be good or evil?
- michaelmrose 4y ago
  If you were picking someone to ride herd on an AI generating code are you sure - You will never need to understand a single bit of what the AI has generated for legal, moral, or strategic reasons? - You will never need to manually rework a single thing or fix anything ever. Logically neither statement is true furthermore who do you imagine is writing the tests that ensure correctness? Logically the person you would want to do this task isn't a yokel who understands interpersonal relationships its someone who understands what is being generated so your company wont murder people or get sued out of existence. Of course you will probably want more than one person doing this task and there ought to be some interface between him and his fellows and between the lot of them and management to ensure good communication and company standards... Maybe a PM? I find it endlessly fascinating that those who can't do dream about the obsolescence of the useful class deeming the irreplaceable human part of the endeavor is the petty manipulation of each other and ego massaging we all do by necessity to keep human endeavors tied together with bailing wire and barely plodding on. It doesn't speak well for our species. Beyond narrow parameters self improving arbitrary representation and manipulation of symbolic data is basically AI complete. The toy you would hope to enable you to ask a computer instead of a person for a particular improvement to your website would upon completion represent a short step from the obsolescence not of coders but of humans wrought in vain hope that we can actually control tools smarter than us. It's probably neither "a few years" away nor that fantastic of an idea but do carry on.

A4ET8a8uTh0 4y ago

I am not sure that can be done that way. There are issues that are culture or specific business need related. Bad requirement is the hard part it seems. I think we all have been in situations, where 'common sense' is not common ( and does not translate to the same thing to various layers for people who pass it; and that does not even account for translation ). This is usually how we end up with weird situations like currency coded as text so it can't be sorted by amount by end user and so on. I agree that specs make or break the project, but at what point is it ok to assume "x should do y".

throw827474737 4y ago
Yeah, on top of that add wading through legacy code, figuring out why wtf, this is crap, just to figure out most times it is actually crap, but then at least equal amount of times that there was a very good reason to do it that way, either technical or by business logic... cannot imagine how any AI system could have success there without getting towards the true AI singularity. This kind of code that I'd believe makes up 99% of all code out there that is worked on, is not available to the public for training. What to do about that? Rewrite all?
- aiphex 4y ago
  I think this may be the case, the AI will figure out what the legacy code does and then completely re-writes it using whatever modern standards and ability to host it all in the cloud.
  - majormajor 4y ago
    The stochasticness probably gets you into trouble here. If you get different results each time you try to "figure out and rewrite" the legacy program... you're gonna need to be able to right some extremely thorough, perfect test cases. And how much of that are you willing to put to chance?
  - goatlover 4y ago
    If it's not trained on the legacy code, I have my doubts it will be able to re-write it well enough to replace a human baby sitting it, particularly when we're talking about large projects. As impressive as ChatGPT is at spitting out code examples, what gives you confidence that and LLM could rewrite a million lines of legacy code it has never seen before? That seems to be well beyond the ability of any AI that's been developed so far.

api 4y ago

Doesn't that make the PM just a programmer then, but with a more powerful tool? We used to code in ASM. Then we coded in high level languages. Soon we'll code by giving prompts to auto code generators.

SQueeeeeL 4y ago
That's kinda missing the level of control. Inherently both ASM and programming produce the same output, "programming" just traded the generality of ASM for verbose and diverse functions. The auto code generators fundamentally don't understand the problems they work on, so any solutions they create will be incidental.
- fragmede 4y ago
  What is "understanding"? Compilers aren't sentient and can't fundamentally understand anything, either though.
  - iterateofen 4y ago
    I would argue compilers have a perfect “understanding” as the input:output is generally 1:1. One input almost always produces the same output. It’s just that the input is extremely limited by a set of predefined rules. The same can’t be said about code generators like this that are producing millions of possibilites and using heuristics to guess which program will have the most success.
    - fragmede 4y ago
      Reproducibility is a big deal. With the Stable Diffusion model as a publicly available test model, using the exact same inputs, especially the seed value for the random number generator, should get you the same picture on subsequent runs or on a different machine. Whether or not ChatGPT is evolving with our input is a secret to make that not the same for their model is known only to OpenAI.
- majormajor 4y ago