3 ms·

I am always surprised by the amount of skepticism towards deep learning on HN. When I joined the field around 10 years ago, image classification was considered

by blt 5y ago

I am always surprised by the amount of skepticism towards deep learning on HN. When I joined the field around 10 years ago, image classification was considered a grand challenge problem (e.g. https://xkcd.com/1425/ https://xkcd.com/1425/). 5 years ago, only singularity enthusiast types were envisioning things like GPT-3 and Copilot in the short term. I think many people are uncomfortable with the idea that their own "intelligent" behavior is not that different from pattern recognition. I do not enjoy running deep learning experiments. Doing resource-hungry empirical work is not why I got into CS. But I still believe it is very powerful.

qayxc 5y ago
This scepticism shouldn't surprise you. Not being sceptical is just an indicator that you've not been in the field for long enough. 30 years ago, the end of programming was prophesised, because 5th generation languages (5GL) and visual programming would enable everybody to design and build software. 20 years ago, low-code and application builders were said to revolutionise the industry and allow people in business roles to build their applications using just a few clicks. End-to-end model-driven design and development (e.g. using Rational Rose and friends) were to put an end to bugs and maintenance problems. 10 years ago it was new programming languages (e.g. Rust, Go, Swift, ...) and a shift to functional programming that was advertised as being "the future". Today it's back to "no code", e.g. tool-(AI-)driven development that's all the rage. It's not so much being "uncomfortable" or clinging to the exceptionalism of the human mind. It's just experience. Every decade saw its great big hype and technological breakthrough, but the lofty promises didn't hold water. Note that this doesn't mean nothing changed - model driven development still has its niche, visual programming is widely used in video production, rendering and game development. Features of functional programming have been added to many "legacy" languages and many of the newly introduced programming languages have become mainstream. The same will happen with AI generated software. There a large portion of the "mechanical" process of programming will be done by AI. Large and complex software systems with changing requirements, however, will still be designed and implemented primarily by people. Programming is a conversation between humans and machines. AI will in many cases shift the conversation closer to the human side, but fundamentally it'll still be the same thing. I like to think of it as the difference between writing your program in assembly and writing it in Haskell; different approaches, same basic activity.
- hnfong 5y ago
  I think you and GP are talking about different things. You're saying a lot of so-called technological breakthrough is more hype than substance. The GP is saying that people tend to dismiss actual breakthroughs as mundane stuff. Once $method is published that solves $hardproblem, people comment as if $hardproblem was never hard in the first place, and moves the goalposts a bit saying "if $harderproblem can be solved, then that would be profound". I think the truth is (obviously) somewhere in between. Btw, I dare you go back to a 1980s programming environment and tell me that the programming paradigm shifts are just hype :D My one-liner python scripts can probably do much more than an average coder writing assembly... and given modern hardware my code runs faster too!
  - qayxc 5y ago
    > Btw, I dare you go back to a 1980s programming environment and tell me that the programming paradigm shifts are just hype Been there, done that. I did consulting for a huge company a few years back. They ran their entire business on IBM mainframes running an ancient VSE-based OS. I had the pleasure of maintaining IBM HLASM (high level assembly) programs with change logs dating back to 1982. Working with those programs (they were excellently documented) using ICCF wasn't much different from using vim really and the language itself is by far the best assembly dialect I've ever worked with (especially the powerful macro system). Sure, productivity is much higher in higher level languages if only because you need to write less code. Your Python one-liner, however, can still be as wrong as 100-lines of assembly or 20 lines of C if you make the wrong assumptions. That's the part that just doesn't change, no matter the underlying technology: garbage in - garbage out. Someone has to write the problem specification and more often than not, that's the part where things start to go sideways. It's also one of the reasons model-driven development didn't really catch on: MDD only works of you know your problem domain to a T beforehand, because iterating models is a pain; that's rarely the case, though as usually code and understanding of the problem evolve side-by-side. Explaining a problem precisely, concisely, and correctly so that an AI can synthesise software that hopefully implements it correctly is not as great as a leap forward as you might think. I'd really suggest taking a look a Rational Rose and similar platforms to get a glimpse at what automated code generation looked like 25 years ago - even back then you rarely had to write actual code (provided the problem domain was well-known and well-specified), even without AI.

by blt 5y ago

qayxc 5y ago

This scepticism shouldn't surprise you. Not being sceptical is just an indicator that you've not been in the field for long enough. 30 years ago, the end of programming was prophesised, because 5th generation languages (5GL) and visual programming would enable everybody to design and build software. 20 years ago, low-code and application builders were said to revolutionise the industry and allow people in business roles to build their applications using just a few clicks. End-to-end model-driven design and development (e.g. using Rational Rose and friends) were to put an end to bugs and maintenance problems. 10 years ago it was new programming languages (e.g. Rust, Go, Swift, ...) and a shift to functional programming that was advertised as being "the future". Today it's back to "no code", e.g. tool-(AI-)driven development that's all the rage. It's not so much being "uncomfortable" or clinging to the exceptionalism of the human mind. It's just experience. Every decade saw its great big hype and technological breakthrough, but the lofty promises didn't hold water. Note that this doesn't mean nothing changed - model driven development still has its niche, visual programming is widely used in video production, rendering and game development. Features of functional programming have been added to many "legacy" languages and many of the newly introduced programming languages have become mainstream. The same will happen with AI generated software. There a large portion of the "mechanical" process of programming will be done by AI. Large and complex software systems with changing requirements, however, will still be designed and implemented primarily by people. Programming is a conversation between humans and machines. AI will in many cases shift the conversation closer to the human side, but fundamentally it'll still be the same thing. I like to think of it as the difference between writing your program in assembly and writing it in Haskell; different approaches, same basic activity.

hnfong 5y ago
I think you and GP are talking about different things. You're saying a lot of so-called technological breakthrough is more hype than substance. The GP is saying that people tend to dismiss actual breakthroughs as mundane stuff. Once $method is published that solves $hardproblem, people comment as if $hardproblem was never hard in the first place, and moves the goalposts a bit saying "if $harderproblem can be solved, then that would be profound". I think the truth is (obviously) somewhere in between. Btw, I dare you go back to a 1980s programming environment and tell me that the programming paradigm shifts are just hype :D My one-liner python scripts can probably do much more than an average coder writing assembly... and given modern hardware my code runs faster too!
- qayxc 5y ago
  > Btw, I dare you go back to a 1980s programming environment and tell me that the programming paradigm shifts are just hype Been there, done that. I did consulting for a huge company a few years back. They ran their entire business on IBM mainframes running an ancient VSE-based OS. I had the pleasure of maintaining IBM HLASM (high level assembly) programs with change logs dating back to 1982. Working with those programs (they were excellently documented) using ICCF wasn't much different from using vim really and the language itself is by far the best assembly dialect I've ever worked with (especially the powerful macro system). Sure, productivity is much higher in higher level languages if only because you need to write less code. Your Python one-liner, however, can still be as wrong as 100-lines of assembly or 20 lines of C if you make the wrong assumptions. That's the part that just doesn't change, no matter the underlying technology: garbage in - garbage out. Someone has to write the problem specification and more often than not, that's the part where things start to go sideways. It's also one of the reasons model-driven development didn't really catch on: MDD only works of you know your problem domain to a T beforehand, because iterating models is a pain; that's rarely the case, though as usually code and understanding of the problem evolve side-by-side. Explaining a problem precisely, concisely, and correctly so that an AI can synthesise software that hopefully implements it correctly is not as great as a leap forward as you might think. I'd really suggest taking a look a Rational Rose and similar platforms to get a glimpse at what automated code generation looked like 25 years ago - even back then you rarely had to write actual code (provided the problem domain was well-known and well-specified), even without AI.