4 ms·

I agree with the sentiment that they only augment our capabilities, but then we end up with another problem, which is that the people who write the software are

I agree with the sentiment that they only augment our capabilities, but then we end up with another problem, which is that the people who write the software are not fully aware of how it all works. Relevant to my argument is Jonathan Blow's `Preventing The collapse of Civilization` talk in which he discusses the disappearance of knowledge as generations pass: https://www.youtube.com/watch?v=pW-SOdj4Kkk&t=3407s https://www.youtube.com/watch?v=pW-SOdj4Kkk&t=3407s

wilburTheDog 6y ago
I think that's unavoidable. There is always a level at which your understanding of your tools gets fuzzy and incomplete. Do you understand everything about what your compiler is doing? Your OS? Your could provider? The hardware any of them are running on? The majority of us are standing on the shoulders of the few that do understand those things and provide them to us to use. And if you mean that we would not understand what the software we write is doing to produce the result it does, aren't we already there with machine learning?
- commandlinefan 6y ago
  On the other hand, the better you understand the compiler/OS/hardware, the better software you'll be able to write. Just like medical doctors could theoretically do their job without a deep knowledge of, say, organic chemistry, I can imagine a near future where software development means tweaking the inputs to GPT-3 (or some other AI) based on a deep knowledge of the layers beneath it: sort of a "computer doctor".
MattGaiser 6y ago
> the people who write the software are not fully aware of how it all works. Is this much different from now? I have no idea how most of the libraries I use are implemented.
- mrkeen 6y ago
  You don't know how other peoples' libraries work, and that's fine. I think in this case the argument is that the author of the library also doesn't know how it works. Which means they can't fix it.
  - chosenbreed37 6y ago
    In addition to this, if the source code is available you could potentially take a peek and generally understand what's going on. But I can imagine that we could have a generation of developers who know little about nuts and bolts that underpin how their software works. Perhaps this is now already the case in certain domains. E.g. you could be working on a Jupyter notebook and be effective without being aware of what's happening behind the scenes. I think is qualitatively different as in this example you could be working at such a high level of abstraction that the nuts and bolts are not something you'd even be aware of. Whereas if you're writing a Java program and you bring in some third party libraries you could potentially look up that library. But more importantly you're still relatively close to the metal.
- jefe_ 6y ago
  I've found that I typically don't need to know how it works if my use case is common and the library is well documented, but when either of those doesn't hold it can be very helpful to read through the implementation in the source to understand how best to implement. But I think the black box would be less of an issue when interfacing with documented third party libraries, than with internally developed services and libraries, particularly in smaller orgs. Team A 'generated' Service A and Team B needs to integrate it in their 'generated' Service B, it seems that could get messy and would be tough to test or troubleshoot. Possibly an additional AI tool specifically for compatibility and integration could solve that problem.
fredliu 6y ago
Cargo-Cult programming is already happening at a not-insignificant scale due to the vast amount of sample code/tutorial/Q&A online. There could be lots of nuances in that snippet you just copied from a SO comment, which "just works". GPT-3 based tools could help you generate a working (or barely working) sample fairly easily. But the decision to just ship that code as is, or to understand it and tune it for your specific needs still mostly depends on the developer.

wilburTheDog 6y ago

I think that's unavoidable. There is always a level at which your understanding of your tools gets fuzzy and incomplete. Do you understand everything about what your compiler is doing? Your OS? Your could provider? The hardware any of them are running on? The majority of us are standing on the shoulders of the few that do understand those things and provide them to us to use. And if you mean that we would not understand what the software we write is doing to produce the result it does, aren't we already there with machine learning?

commandlinefan 6y ago
On the other hand, the better you understand the compiler/OS/hardware, the better software you'll be able to write. Just like medical doctors could theoretically do their job without a deep knowledge of, say, organic chemistry, I can imagine a near future where software development means tweaking the inputs to GPT-3 (or some other AI) based on a deep knowledge of the layers beneath it: sort of a "computer doctor".

MattGaiser 6y ago

> the people who write the software are not fully aware of how it all works. Is this much different from now? I have no idea how most of the libraries I use are implemented.

mrkeen 6y ago
You don't know how other peoples' libraries work, and that's fine. I think in this case the argument is that the author of the library also doesn't know how it works. Which means they can't fix it.
- chosenbreed37 6y ago
  In addition to this, if the source code is available you could potentially take a peek and generally understand what's going on. But I can imagine that we could have a generation of developers who know little about nuts and bolts that underpin how their software works. Perhaps this is now already the case in certain domains. E.g. you could be working on a Jupyter notebook and be effective without being aware of what's happening behind the scenes. I think is qualitatively different as in this example you could be working at such a high level of abstraction that the nuts and bolts are not something you'd even be aware of. Whereas if you're writing a Java program and you bring in some third party libraries you could potentially look up that library. But more importantly you're still relatively close to the metal.
jefe_ 6y ago
I've found that I typically don't need to know how it works if my use case is common and the library is well documented, but when either of those doesn't hold it can be very helpful to read through the implementation in the source to understand how best to implement. But I think the black box would be less of an issue when interfacing with documented third party libraries, than with internally developed services and libraries, particularly in smaller orgs. Team A 'generated' Service A and Team B needs to integrate it in their 'generated' Service B, it seems that could get messy and would be tough to test or troubleshoot. Possibly an additional AI tool specifically for compatibility and integration could solve that problem.

fredliu 6y ago

Cargo-Cult programming is already happening at a not-insignificant scale due to the vast amount of sample code/tutorial/Q&A online. There could be lots of nuances in that snippet you just copied from a SO comment, which "just works". GPT-3 based tools could help you generate a working (or barely working) sample fairly easily. But the decision to just ship that code as is, or to understand it and tune it for your specific needs still mostly depends on the developer.