19 ms·

Why Programming Is Hard to Fundamentally Improve (2017)

197 points by mpweiher 6y ago

throwaway201103 6y ago
I expected it to boil down to the thesis that programming is hard to improve because the complexity that makes it hard is essential, not accidental. https://web.archive.org/web/20160910002130/http://worrydream.com/refs/Brooks-NoSilverBullet.pdf https://web.archive.org/web/20160910002130/http://worrydream... Surprised that the piece was all focused on the accidental complexity.
- dwohnitmok 6y ago
  I'm think (?) the author's point is that an immense amount of value is inextricably tied up with what appears to be "accidental complexity." From the article: > The legacy is the value, it can’t be thrown out. I agree the author presents this in a bit of a confusing fashion, since the immediately preceding sentence is > the ideas are what hold the real value.
  - munchbunny 6y ago
    From experience, I think this is absolutely true. One of the reasons for the 80/20 rule, specifically that it takes 20% of the time to make 80% of the product and the remaining 80% of the time to make the remaining 20% of the product, is that we perceive most of the accidental complexity to be in that final 20%: technical debt, localization edge cases, bugs, misuses that turn our products into spying tools, etc. I think that accidental complexity isn't a software paradigm limitation. It's a human thought limitation. It takes time for human organizations to learn how to handle the inevitable yet unanticipated issues and exceptions to the rule, and it takes time to turn them into processes and then to automate them.
- stevedonovan 6y ago
  The ecosystem delivers a lot of pre-packaged essential complexity - e.g. only now Rust is starting to get the mature libraries to handle difficult domain-specific things. It is hard to program from scratch (no pun intended)
izuchukwu 6y ago
The article’s position comes down to “no fundamentally new way to program would make sense for today’s programmers to switch to it”, and gives examples like the platforms of the no-code movement. From previous generational leaps, we’ve learned that the users post-leap don’t look like the pre-leap users at all. The iPod’s introduction brought about a generation of new digital music users that didn’t look like the Limewire generation, and the iPhone’s average user didn’t look like the average user of the BlackBerry before it. Modern programming is at the core of HN, and of most of SV, sure. That said, we should still be the first to realize that a successful, fundamentally new way to program would target a new generation and idea of software maker, one that won’t look like the modern developer at all.
- andrewjl 6y ago
  > That said, we should still be the first to realize that a successful, fundamentally new way to program would target a new generation and idea of software maker, one that won’t look like the modern developer at all. Channeling Gibson[1], do you see any potential successors already out there? [1] “The future is already here – it's just not evenly distributed."
  - izuchukwu 6y ago
    I’d keep an eye on the no-code movement. Real leaps can be distinguished from hype by where the passion is coming from. The fact that the movement’s passion is coming from actual, paying users and not just no-code platform makers is key here. It’s rapidly creating a new generation of software creators that could not create software before, and it’s improving very, very fast.
- KMag 6y ago
  Former Limewire developer here. I definitely had an iPod mini prior to Limewire's peak years, counted by number of monthly peers reachable via crawling the Gnutella network.
  - echelon 6y ago
    What's your take on how everything works these days? Do you miss p2p? Do you think we could ever get back to it?
akkartik 6y ago
I buy the thesis as stated; there's a common attitude among researchers of, "why don't we have this already?" that seems counter-productive. However, there's an adjacent thesis that's also worth stating: fundamentally improving programming is hard, but still worth attempting. There's just too much risk with the current status quo. We've built up our world atop a brittle, insecure, unstable tower of software, and it doesn't feel unreasonable to ask if it might lose capability over time[1]. The good news: you don't have to give it all up all at once and return to the stone age to try something new, as OP says. There's nothing stopping us from using the present as we like to build the future. The key, it seems to me, is to prevent the prospective future from inter-operating with the present. You won't get lots of users this way, but I think jettisoning the accumulated compatibility baggage of 50+ years of mainstream software might free us up to try new things. Even if the world doesn't switch to it en masse, it seems useful to diversify our eggs into multiple baskets. Here's what I work on: https://github.com/akkartik/mu https://github.com/akkartik/mu. It's a computer built up from machine code, designed to be unportable and run on a single processor family. It uses C early on, but it tries to escape it as quickly as possible. Mu programs can bootstrap up from C, but they can also build entirely without C, generating identical binaries either way[2]. (They do still need Linux at the moment. Just a kernel, nothing more like libc.) I call this ethos "barbarian programming", inspired by Veblen's use of the term[3]. I rely on artifacts (my computer, my build tools, my browser, my community support system, the list is long) from the surrounding "settled" mainstream, but I try to not limit myself to its norms (compatibility, social discouragement of forking, etc.). I'm researching a new, better, more anti-fragile way to collaborate with others. Here's a 2-minute video I just recorded, experimenting with a new kind of live-updating, purely-text-mode shell built atop the Mu computer: https://archive.org/details/akkartik-2min-2020-12-06 https://archive.org/details/akkartik-2min-2020-12-06. The shell is implemented in a memory-safe programming language that is translated almost 1:1 to 32-bit x86 machine code. Without any intervening libraries or dependencies except (a handful of syscalls from) a Linux kernel. [1] https://www.youtube.com/watch?v=pW-SOdj4Kkk https://www.youtube.com/watch?v=pW-SOdj4Kkk [2] http://akkartik.name/akkartik-convivial-20200607.pdf http://akkartik.name/akkartik-convivial-20200607.pdf [3] https://www.ribbonfarm.com/2011/03/10/the-return-of-the-barbarian https://www.ribbonfarm.com/2011/03/10/the-return-of-the-barb...
im3w1l 6y ago
They say you can narrow down from all of humanity to a single person with just 33 yes/no questions. I think a similar principle could be true for programs, if we just knew the right questions to ask! I think this is what makes programming hard to improve. We have a poor understanding of what kinds of programs people want to write.
- jmnicolas 6y ago
  > They say you can narrow down to a single person with just 33 yes/no questions. I resent those questionnaires since it's never a firm yes or no but something in between (or sometimes something completely different). You can't reduce the complexity of one human life to 33 questions.
- yetihehe 6y ago
  > I think a similar principle could be true for programs, if we just knew the right questions to ask! Those right questions to ask are VERY hard to find, so we programmers have to write many many very accurate answers about how our programs should behave. That process is called programming.
  - dwohnitmok 6y ago
    This is a dilemma that often rears its head (exemplified by the classic "42"). Intuitively difficult situations usually end up "conserving their difficulty." An easy solution means a difficult problem (hard to formulate the problem or the formalization of the problem is usually inapplicable). An easy problem usually means a difficult solution.
  - im3w1l 6y ago
    So many replies, I'll reply to you but it's really for everyone. The thought experiment was just meant to illustrate how much easier things could be if we knew better what programmers wanted to write. It was not a serious suggestion for how programming should work. What I do think should happen, and will happen, is that we will keep getting a better understanding of what behavior people want most of the time -- including the common variations, and make it ever easier to achieve it.
preommr 6y ago
Those generalizations about why change is hard can be applied to any change in every field. Ideas are always decades if not centuries ahead of their implementations. Not even ideas, but proof of concepts too. So lets talk specifically about software. The biggest problem that prevents moving forward are platforms and companies. Yes, businesses and organizations are usually the biggest factor in dampening progress because they have so much mass it influences everything around them. But you would think that with the low barrier to entry in software, anyone could just spin up some OSS project and it would quickly trickle it's way through. Even slow organizations are often quick to adopt new tech in certain conditions. There were fintech companies that adopted Docker really early on. But it's usually something that's isolated. It's platforms that hold us back. The reason we don't see huge jumps in things like functional programming is because that stuff is great for parallelism, and we don't see jumps in parallelism because the OS, or game engine, or drivers, or chips have poor support. We could have better functional languages if browsers decided to support more functional principles in ecmascript, or even some other scripting language. We don't have all the cool programming stuff because it's piles of abstractions with groups that don't want to coordinate. Sure there are other factors - there are cultural issues that lead to programmers making bad decisions and writing bad code. Java for example was a good language that got ruined by horrible enterprise culture. It's not like people didn't point out the flaws early on, but they got ignored. Lots of people took microservices are a good idea to mean "everything must be a microservice". But even then these problems are perpetuated by companies that mishandle their platform. If companies made user guides clearer we wouldn't have these issues. Not only is the communication bad, but it's often in arcane places. To figure out how to use something you have to read through github discussions or follow mailing lists to get useful information. If rungs to your ladder are broken, you're not going to be able to climb very high.
- esc_colon_q 6y ago
  Java was a good syntax. The language itself had way too many limitations back when Sun ran it, for really dumb reasons proffered by too smart for their own good people. I was sadly paying attention to the wretched arguments about how there was literally no way to reasonably make first class functions work, or closures, or value types, or escape analysis, or any of the million things that C# just said yes to from day one which worked out fine. It was a lot of edge case kvetching that really needed a dictator to cut through, but Sun were terrible stewards and let the worrywarts run the show. The enterprise culture could I guess be blamed, but Sun was a shite enterprise company when it came to financials, you'd think MS would be much worse in terms of command/control but C# ended up as a damn good version of Java, and once Oracle grabbed the ball hairs Java got pretty great. So it's not the whole problem.
bawolff 6y ago
I feel like as an explanation this fails: why do these arguments apply only to programming and not other things, why do these arguments apply to the present and not the past? Or are we also arguing all things are hard to improve, and that programming hasn't effectively improved since the beggining.
AmericanChopper 6y ago
Programming is hard to improve because it's not very simple to know what programming ideas are good and what ideas are bad. The feedback loop between decision and consequence is so long and opaque that if something works or if it doesn't work, it's not possible to easily point out why. To make things even more complicated, most good programming ideas are only good in certain situations, and terrible in others. On top of that, we're not even very good at knowing what our software is supposed to do. Building a race car is a lot more complicated than writing a simple node web app, but it's way easier to tell if you have a good race car than it is to tell you have a good node web app. If we could agree on what improved programming was, it might be a lot easier to improve. But everybody has their own opinion about what ideas are good, and in what context those ideas are good. We can't even agree about what's important to optimize. Which leads to situations where you can find highly experienced experts passionately disagreeing with each other about absolutely any give topic, and also have experts and charlatans in complete agreement, with no easy way to tell them apart. How would we even be able to tell if it has improved?
- Lichtso 6y ago
  I also think that the biggest problem we are facing is, that we still haven't developed any useful metrics or measurements of quality in our field. There are some easy things like performance or resource consumption, but: Others like security, robustness, maintainability or plasticity are equally important, yet different approaches can not be compared numerically only by argument.
MaxBarraclough 6y ago
> If you build something that’s too difficult to learn, but very productive you’ll turn a lot of people off. Counterexample: C++. To a lesser degree, perhaps also Rust? (Disclaimer: I don't know Rust.) > The problem is, within a few weeks of using any paradigm developers usually have built a repository of habits that keep them from making mistakes. No, developers constantly make mistakes. Even after all these decades, C programmers are still writing code with buffer overflows. The way to solve that is to have better tools: use a safe language, or formally verify the C code. At this point, we know that it's not enough to hire brilliant C programmers and have them exercise care. > Things have been getting better. The growth has just been in the ecosystems and not the paradigms themselves. I don't think there's a bright line between these two. Good progress is being made in making formal verification more approachable, for instance. This is no small thing, it takes hard work in computer science, but doing so lowers the barrier to the paradigm, making it more practical for more developers and for more problems. Merely thinking up the paradigm, it would be neat to have tooling to verify that our code matches our formal model, is only the first step. In a sense the paradigm only really exists after the hard work has been done. In a similar vein, recent advances in low-pause concurrent garbage-collection might broaden the scope of the problems that can be solved with garbage-collected languages. > The legacy is the value, it can’t be thrown out. I broadly agree. See the classic blog post from Spolsky, Things You Should Never Do, Part I, on how throwing out working code is very often a mistake: https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-...
- WJW 6y ago
  > Counterexample: C++. To a lesser degree, perhaps also Rust? Are these actually counterexamples? While in absolute terms there may be a fair number of people using them, in relative terms the amount of people who know C++ and/or Rust is absolutely dwarfed by the amount of people who know "easier" languages like JS, Python and PHP. In this case "easier" refers to the amount of effort needed to pick it up and get some result on screen, not to the amount of effort to be a guru. I think for a lot of beginners productivity and even (memory) safety is completely irrelevant. They are not very productive anyway and their code crashes all the time because by definition they make beginner mistakes. What is more important for beginners is to keep up momentum and get through the initial hurdle of "I don't understand this" to the point where you start thinking "this is fun, I can make cool stuff". Rust and C++ solve professional problems, not beginner problems.
mpweiher 6y ago
Found this blog via the wonderful futureofcoding podcasts, episode Building for Developers: https://futureofcoding.org/episodes/021 https://futureofcoding.org/episodes/021 What should be noted is that the author already did the low-code/no-code thing as a startup that is still active and thriving: https://www.dropsource.com https://www.dropsource.com So what he's doing and describing now is informed by that experience, which he talks about at length in the podcast episode. I find it much more insightful and less superficial than a lot of the writing on the topic.
FartyMcFarter 6y ago
> new programming languages including some flow & graph based paradigms and no-code visual builders. Does dressing up Turing-complete programming as visual graphs change anything, fundamentally? Does it make software easier to understand and maintain? In my experience, when people not trained as programmers (or perhaps even programmers) start using visual programming tools in real-world projects, what results is often unmaintainable spaghetti. For example, see https://blueprintsfromhell.tumblr.com/ https://blueprintsfromhell.tumblr.com/ But perhaps I misunderstood what the author meant with the above blurb.
- alkonaut 6y ago
  For some contexts it’s definitely helpful such as graphs in game engines, 3D modeling and synths. There is always a code-graph duality, the question is which representation is closest to what’s being modeled. Describing what is fundamentally mostly a DAG as a graph rather than as code really is helpful. That it also works for “non-programmers” is just a bonus.
- Viliam1234 6y ago
  Five lines of code, or five rectangles connected by arrows, both are simple. When people promote visual programming, they make it sound like the five rectangles will replace thousand lines of code. But if that is true, why not simply implement a library with functions equivalent to the individual rectangles, and replace the code with five function calls? Instead, when doing anything nontrivial, we get pictures with thousand rectangles connected by arrows, for pretty much the same reason we previously had thousand lines of code. It's because we want to implement functionality that is not trivially expressed by the provided primitives, duh.
  - jerf 6y ago
    One of the things I've taken to doing on the semi-annual "Hey HN I've Fixed All Programming Problems Forever with $THIS" posts is encouraging people to head straight at these hard problems. I've also been encouraging the commenters writing the writing the "dear lord it's so obvious that we just need to drop everything and use $THIS" to be thinking more about the hard problems. I believe I've now issued my challenge to the Visual Language of the Day developers to three different people to not show me how to add 1 over a list, but to implement Quicksort (or an equivalently complex algorithm of your choice) in your new graphical language and show me how it's at least as easy to understand as the current paradigm. I chose that one on purpose on the grounds that it's actually not that understandable in normal notation, so I didn't choose that to be difficult exactly, I chose it as something that has some space for improvement, and, like I said, I'll accept any other non-trivial algorithm that you can make a decent case for being easier. I think only one of them actually drew it out, and it wasn't terribly nice; the others have ignored me. Bret Victor's "live views of the variables updating" stuff looks amazeballs in a crafted demo where only a couple of entities have their X+Y positions displayed in real time, with real-time modification. But it's actually not as new as some people think; I've used debuggers with that capability for a while now. It gets a lot less "ooo.... aaaa..." when you've got hundreds of live-updating values on the screen, and when you've got structured values being manipulated, like, something is adding values to a list as you go, or adding entries to a hash or something. It's actually still really useful and perhaps underused by some people (debuggers, people, I'm down with claims you shouldn't live in them but you need to learn how to use them when you need them!), but it's been around for a long time actually and in practice it isn't quite as game-changing as it looks, because it doesn't solve the problem of finding the important data. Figuring out which parts of a log are important and figuring out which of the hundreds of changing, complicated variables are important aren't that different in terms of effort. In a nutshell, it's scale. A lot of these alternate "Solutions to Everything" don't scale. For all the faults of "text based programming" (instead of a richer manipulation system, or a visual system, or whatever) and our documentation schemes and the way we debug and how we write our code guarantees and so on and so on... they work. We build large systems with them at scale, routinely. The large systems are perhaps ugly, prone to failures, coughing smoke out of many orifices, and often built of complete garbage, but they work. These "Solutions to Everything" tend to break down at scales multiple orders of magnitude smaller than the current-practice competition. By no means does that mean nobody should be working on new ideas, be they revolutionary rewrites of the entire status quo, or small improvements to make a lot of people's code a little better. My point is more that especially if you want to rewrite the status quo, you have got to focus above all else on how your solution is going to scale. Your cute little demo that fits on a screen is still going to be necessary to introduce people to your idea, but show me how your system looks with several thousand of whatever it is you are doing, at the very least. I also tell you that my standard isn't "does that look good", because honestly, at that scale nothing looks good because it no longer fits in human cognition; my standard is that you need to have something better than the current standard, which also doesn't fit into human cognition. You don't need to propose something that we can rewrite operating systems or office suites in, there's plenty of other niches, but you do need to propose something that can go beyond a single screen. The main function programming languages face is not serving up bite-size bits of functionality in comprehensible ways, it's providing ways to slice things into bite-sized bits that fit into my pathetically limited human cognition in the first place. I don't need a better way to understand that this bit of code adds one to everything in the list, I need better ways of breaking problems down into smaller pieces without incurring disadvantages in system organization, performance, composability, etc.
deleted 6y ago
[deleted]
pron 6y ago
While the author speaks about difficulties due to momentum, habit, and baggage, I think the fundamental problem is much, much harder. Many have described the problem in different ways, from Fred Brooks's essential vs. accidental complexity to results on the intrinsic "hardness" of programming we've learned from computational complexity theory in the past two decades, but my favourite is the succinct description Gilles Dowek gave at a talk at NASA [1]: > Computers have been invented to surprise us... If we knew what computers do, we would not use them, and we would not have built any. In other words, not only is writing the programs we want hard because of some "reasonable" Brooks-like argument or some rigorous computational-complexity argument, but we write programs in the first place because it is hard. In practice, not only has Brooks's prediction about no 10x improvement in programming productivity due to a single development -- which was rejected by many at the time as too pessimistic -- been vindicated, but the reality is worse: we have not seen a 10x boost due to all developments combined and in over 30 years! And the boosts we have seen are mostly due to the availability of open-source software, and widespread knowledge sharing on the internet. [1]: https://shemesh.larc.nasa.gov/NFM2018/talks/Dowek.pdf https://shemesh.larc.nasa.gov/NFM2018/talks/Dowek.pdf
- AlexSW 6y ago
  Seems like a calculator is a good example of a program which we would write even though it is entirely unsurprising; even if we needed nothing else on a computer we might invent it still? It seems to me that we create computer programs not to surprise us but typically to do things very quickly or conveniently. Even a complex beast like Photoshop is reasonably unsurprising.
  - stult 6y ago
    The point is that the outputs of the calculator are fundamentally surprising, in the sense that if any individual programmer could easily know all the expected outputs in advance, they wouldn't need the calculator in the first place. Photoshop surprises us by calculating the precise location of millions of pixels based on relatively simple inputs like a handful of clicks on a screen. This logic does not apply to every single imaginable program, but it does apply to the non-trivial ones.
javier10e6 6y ago
Let's ponder on one example where programming was fundamentally improved in the last 10 years. Git Some developers adopted immediately. Others were dragged into it kicking and screaming. Some ignored it (managers). In my humble opinion improvements on a given tool latch on the esoteric definition of what is considered "better". Better for me? Better for the company? Better for the machine? In the case of Git, the case is base on the "me" (Linus Torvalds). In the dev-world we had the joke that goes: If you don't like the tool/language go ahead and write your own. My take away is that improving programming languages and tools, can happen if we take the joke above at heart.
hhas01 6y ago
“In my experience developers are very rational beings.” Yeah, no. Stopped there. Vanity is rationalizing the belief that you’re any better at being rational than the rest of us inchoate apes. There are many technical, logistical, and political arguments as to why improvement (in any sphere) is hard, but if you’re not going to start with “It’s Us, The People” then where usefully are you going to start?
- justin_oaks 6y ago
  Perhaps it's not that developers are more rational, but that they are better at expressing logical-sounding reasons that they don't want to change. It's also possible to see developers as more willing to change (more "rational") than others when something new can help them get their job done better. If nothing else, developers seem to like trying out new languages and tools for novelty's sake, even if it's not because they're actively trying to improve.
wruza 6y ago
Because (my personal opinion) it is not programming that has to improve. A, B, then C if D is okay and humane. It is weak base libraries that expose and require you to control the irrelevant junk have to improve/vanish. Your basic app, be it web, desktop or mobile, is full of bs and boilerplate of methodologies (not even of just code) to the level that one has to learn to create cruds and component designs. You create and secure endpoints, validate dtos, etc over and over again. For what? Why not have an orm/db system that easily spans server, client and b2b? Why not have a default global message bus that connects directly to user? And so on, depending on business requirements. Instead we have to spend a week or more to learn and remember how e.g. to make “webpush” work and how to serialize an xhr form. Meanwhile our libraries brag about yet another cross-platform #flatten() crap that helps no one but those who dig into this bs deeper than their peers to call themselves a senior. Of course normal people consider that hard, as we wallow in this madness and try to look smart.
- marcodave 6y ago
  well, the idea is that there are already "do-it-all" frameworks, although they are linked to specific programming languages or paradigm (ruby on rails comes to mind)
  - wruza 6y ago
    Sadly, rails, django and other “opinionated orm & http” are two or three steps below than what I meant. Some commercial things may get close in functionality (outsystems, salesforce, 1c enterprise to name a few, but not exactly), but not in ease of learn and use, and usually are too focused rather than being general purpose.
carapace 6y ago
I think programming is easy for lots of people, we (devs/techies) just don't talk about it because it happens in spreadsheets. Excel is the most successful IDE. I've come to suspect that syntax is a MacGuffin and programming languages are a dead end. We're complexity junkies and I suspect we know darn well (some of us consciously but most of us only unconsciously) that if we created a truly usable-by-the-masses IDE it would put us out of work, we wouldn't get paid to mainline our drug of choice.
- demosito666 6y ago
  You do realize that this is just like saying that "those pesky scientists out there don't cure cancer because that would put them out of the job"? It takes just one scientist to break this worldwide cartel, and he immediately will become billionaire and the one the most influential and famous persons of the century. Can you spot where your predictions contradict game theory? Besides, if there is one thing that programmers like to do, it's developing programming tools. Personally, I'm astonished by the amount of great tools available for free to everyone and the sheer amount of labor that people put there in their free time. Linux [1] alone is mind-boggling, and it's just a tiny fraction of the free codebase available to build your software on top of. Given that, I fail to see how it's possible that millions of developers in the industry somehow participate in anti-productivity cartel. Especially that it's potentially a very lucrative market, as you could cell this IDE in millions and not worry about the future, whatever it would be. [1] I know that most contributions to Linux are made by professionals on payroll, it doesn't change much overall.
  - carapace 6y ago
    > You do realize that this is just like saying that "those pesky scientists out there don't cure cancer because that would put them out of the job"? It takes just one scientist to break this worldwide cartel, and he immediately will become billionaire and the one the most influential and famous persons of the century. Can you spot where your predictions contradict game theory? I think in that situation game theory predicts that that scientist is found dead in a burned out car with a bullet hole or two in the skull, eh? But I'm not positing a conspiracy of mustache-twirling villains. It's obvious that the vast majority of programmers are well-intentioned. But it's also obvious that we ignore our own history (and we're slaves to fashion, but that's a tangent.) Engelbart "cured cancer" in 1968[1], Alan Kay and co. "cured cancer" in the 70's[2], and Niklaus Wirth "cured cancer" in the 80's[3]. Things like Alice Pascal were around in the 80's.[4] > Besides, if there is one thing that programmers like to do, it's developing programming tools. Personally, I'm astonished by the amount of great tools available for free to everyone and the sheer amount of labor that people put there in their free time. Linux [1] alone is mind-boggling, and it's just a tiny fraction of the free codebase available to build your software on top of. That's a symptom of the problem, not an intrinsic benefit, most of that software is little more than video games for devs from the POV of human productivity. It's a sickening thought but BASIC has done more for the average person than Lisp. Compare and contrast Red lang to Rust, or Elm lang to the whole of the JS ecosystem. It's a bitter pill, but it's clear that we're playing by different rules than what we tell ourselves and others. > Given that, I fail to see how it's possible that millions of developers in the industry somehow participate in anti-productivity cartel. Arguably we are all doing that all the time to a ridiculous extent. Agriculture is an "anti-productivity cartel" (to the extent that it destroys topsoil and fertility over time), our entire housing system is an "anti-productivity cartel"[5], compared to possibilities!.[6] How is it possible that millions of refrigerators have doors rather than drawers? Every time you open the fridge you spill cold air on the floor and warm air replaces it and you have to pay for the electricity to cool that air off. It's an anti-productivity cartel! > Especially that it's potentially a very lucrative market, as you could cell this IDE in millions and not worry about the future, whatever it would be. It's not like folks aren't trying.[7] [1] https://en.wikipedia.org/wiki/The_Mother_of_All_Demos https://en.wikipedia.org/wiki/The_Mother_of_All_Demos [2] https://en.wikipedia.org/wiki/Smalltalk https://en.wikipedia.org/wiki/Smalltalk [3] https://en.wikipedia.org/wiki/Oberon_(operating_system) https://en.wikipedia.org/wiki/Oberon_(operating_system) [4] https://www.templetons.com/brad/alice.html https://www.templetons.com/brad/alice.html https://www.atarimagazines.com/v6n2/Alice.html https://www.atarimagazines.com/v6n2/Alice.html [5] https://www.berkeleyside.com/2018/08/02/prefab-housing-complex-for-uc-berkeley-students-built-in-four-days https://www.berkeleyside.com/2018/08/02/prefab-housing-compl... https://news.ycombinator.com/item?id=17739805 https://news.ycombinator.com/item?id=17739805 [6] https://en.wikipedia.org/wiki/Dymaxion_house https://en.wikipedia.org/wiki/Dymaxion_house https://en.wikipedia.org/wiki/Old_Man_River%27s_City_project https://en.wikipedia.org/wiki/Old_Man_River%27s_City_project https://en.wikipedia.org/wiki/Arcology https://en.wikipedia.org/wiki/Arcology [7] E.g. https://darklang.com/ https://darklang.com/
sova 6y ago
Not all programming languages are equal.
- pklausler 6y ago
  So much this. We have had pure well-typed functional programming languages for decades, and they are great at forcing clean designs and catching bugs at build time, and yet they're still niche tools for the most part.
giantg2 6y ago
I just wish I was good at something and had a promising future.
Ericson2314 6y ago
These are silly arguments. As someone that works on tooling a lot, we could be doing massively better. But there isn't enough demand. - Slow overall economic growth - Stick to shit like JS and there is now a glut of entry-level programmers - QE and VC model is ridiculously supply-side already, could hardly sustain more. - Very hard to capture value of new tooling when alternative is free. Asking why we don't have the programming tooling we should have is not unlike asking why we don't have the subways we should have. ------- If someone wants to through more money at tooling anyways, do let me know! :D
lebuffon 6y ago
I am wondering if something like GTP-3 (or GPT-3+x) which has the beginnings of writing code from human language descriptions will be a useful thing. Creating working code to expand upon can be a significant part of the process of new software creation.
BoiledCabbage 6y ago
The author nails the fundamentally hard problem in software. If there is a new tech in bridge building, then every bridge going forward can use it, but software is successful because of its layers of abstraction and network effect. You can't introduce a new superior tech unless the huge amounts of existing tech and tools can first be made compatible with it. And that's harder to do because of all the degrees of freedom available in software rather than the relatively few in physics. And even if a method could be discovered to make them compatible there is no way for anyone to snap their finger and update them all - it takes tons of work. And who is going to convince all of these other people to stop what their doing to invest in supporting this better tech? Even if this better tech is there, how long will it take them to really understand the correct mental model to apply it to their tool. Functional programming is now still slowly gaining adoption because it's a different way to think. It takes us forever to learn and adopt a better method. And even if people could be made willing to adopt a better tech and do all of the above, how can we show a tech is better? We don't even have an objective method of evaluating what better is. It's all hearsay and opinion. So everyone needs to evaluate it on their own. But even if people were willing to do that, we don't even have an objective definition of what "better" is in software tooling! The problem is, in evaluating software, we are pre-scientific method. The single largest break through any one person could have, the only "singularity tech" in software development would be for someone to design a model to represent an objective human evaluation of software language tool, method or tool. Something that allows us to objective compare languages and automate the search space of fixing better tooling. Without that it will continue to take 40 years to adopt clearly better ideas. Until we can even say the equivalent of basic physical comparisons of "objective x is heavier than object y", or "object a is faster/stronger than object b" using an agreed upon standard progress will be fits and spurts. Imagine Civil engineering over the centuries and people can look at two bridges and be unable to tell which is stronger or wider or longer or costs more to build. Think how worse off our tech would be.
barrenko 6y ago
That Brett Victor talk is wildly overrated.
yters 6y ago
I'd say the progression from machine code to Python is a fundamental improvement. For simple cases, I think a visual/textual code interface is possible, but for any complicated case the visual interface will become inscrutable. Maybe an optimal tradeoff is like how game engines allow scripting languages to live on top. A number of 3D engines use this paradigm with a visual language in place of scripting language.
6gvONxR4sf7o 6y ago
As programmers, I think we're way too stuck to the idea that programming is a keyboard and mouse exercise when we are trying to envision the future. Like many other people, I've been struggling with carpal tunnel during the pandemic and I've switched to dictation based coding. Since making the switch, I've realized how much of how I code is specifically due to the keyboard being my way to tell the computer things. And graphical programming paradigms that I've seen are really just switching from keyboard to keyboard + mouse. Right now, dictation is still at the point where it's a royal pain in the ass to have it as the primary input method. As soon as that changes, I expect that we will see a flurry of new programming paradigms. Code is best represented as text when it is input (and modified!) as text. My current dictation set up includes lots of it asking me "did you mean X,Y, or Z" and as natural language processing improves, those questions are going to get a lot more interesting. Even if NLP weren't going to improve, dictation+keyboard+mouse(+touchscreen+eyetracking ...) based programming will probably be a very different experience from keyboard+mouse based programming. Until then, I really only expect incremental change.
- deleted 6y ago
  [deleted]
aceBacker 6y ago
I wonder how the author would feel about the low code solutions that are catching on in enterprises. For example outsystems seems to be changing how programming works while letting people use languages they are familiar with.
tabtab 6y ago
Re: The same things that make Scratch easy to learn also make it an unproductive environment for more serious programmers. For a professional, programming with drag and drop is way slower than keying in code — that’s just a fact. I would say the best tools allow one to switch back and forth between visual/drag-and-drop and code as needed. Web-based UI's never seemed to get that to work well. The stateless nature of the web is a productivity kicker in multiple ways. The same number of features could be created and managed by fewer programmer hours in IDE's from the 90's than modern web stacks in my observation. Web stacks are so bloated that orgs need layer specialists to manage it all. "Separation of concerns" (SOC) is to fit the shape of dev staff, not the shape of productivity nor the domain. It just creates busy-work interfaces between all those layers; a time drain. The older tools were closer to the domain such that it was less code to do the same thing, making SOC not worth it. SOC is symptom of bloat, not a solution. Concerns naturally interweave; the forced walls are just to serve Conway's Law. The counter argument is often, "yes, but we have choice". Sure, but you pay dearly for it. I believe a stateful GUI markup standard could help bring rank-and-file office-oriented CRUD productivity back to 90's levels. One could then spend time on domain issues instead of gluing bloated buggy stack layers together. I can't say it will help all domains, but one-browser-fits-all is failing CRUD, if productivity matters.
gampleman 6y ago
One issue that's been on my mind is how to actually fund a new programming paradigm. As far as I can tell there are the following options: 1. Start a business and seek investment (examples: Unison, Dark). This is tricky since programming environments are hard to monetise (all the competition is free), and the investment is quite large (you need a great team working for many years to maybe make something good enough you can actually attempt to charge for it). 2. Do it as a hobby project (examples: many of the small PLs nobody uses in production). This is hard because it takes a LOT of work to just get a good programming language, and then you need to also produce great tooling, a decent number of good libraries, amazing learning materials, etc. 3. Academia (examples: Haskell, Idris, Elm, ...). This can work, but generally the focus will need to be on some academically interesting problem. But a lot of what it takes for a language to succeed is a lot of down to earth engineering, which academic positions aren't generally great at (i.e. it took Haskell about 20 years to start being somewhat successful outside academia). 4. Corporate sponsorship (examples: Elm, C#, Swift, ...). This generally needs alignment between the language and the corporation's goals. This typically limits the amount of radical inventions you can make, as well as how freely you can pivot to new ideas. Fundamentally none of the above are readily useable for radical reinventions, but more for incremental improvements.
worik 6y ago
Is Swift really the second most popular language since 2000? Golly. I have been using Swift for some months now. It is more modern than C/C++, sure, but it really suffers from lack of development. The ergonomics of it is poor. Especially compared to Rust. Some design choices are quite odd (the definition of a constant defined with `let` for exampe) It is the only easily available choice for developing on Apple, AFAICT. Hence it is so popular. As a fresh developer on Apple, but grizzled and grey, I must say teh development tools and ecosystem reminds me of Linux in 1997. Everything mostly works

197 points by mpweiher 6y ago

throwaway201103 6y ago

I expected it to boil down to the thesis that programming is hard to improve because the complexity that makes it hard is essential, not accidental. https://web.archive.org/web/20160910002130/http://worrydream.com/refs/Brooks-NoSilverBullet.pdf https://web.archive.org/web/20160910002130/http://worrydream... Surprised that the piece was all focused on the accidental complexity.

dwohnitmok 6y ago
I'm think (?) the author's point is that an immense amount of value is inextricably tied up with what appears to be "accidental complexity." From the article: > The legacy is the value, it can’t be thrown out. I agree the author presents this in a bit of a confusing fashion, since the immediately preceding sentence is > the ideas are what hold the real value.
- munchbunny 6y ago
  From experience, I think this is absolutely true. One of the reasons for the 80/20 rule, specifically that it takes 20% of the time to make 80% of the product and the remaining 80% of the time to make the remaining 20% of the product, is that we perceive most of the accidental complexity to be in that final 20%: technical debt, localization edge cases, bugs, misuses that turn our products into spying tools, etc. I think that accidental complexity isn't a software paradigm limitation. It's a human thought limitation. It takes time for human organizations to learn how to handle the inevitable yet unanticipated issues and exceptions to the rule, and it takes time to turn them into processes and then to automate them.
stevedonovan 6y ago
The ecosystem delivers a lot of pre-packaged essential complexity - e.g. only now Rust is starting to get the mature libraries to handle difficult domain-specific things. It is hard to program from scratch (no pun intended)

izuchukwu 6y ago

The article’s position comes down to “no fundamentally new way to program would make sense for today’s programmers to switch to it”, and gives examples like the platforms of the no-code movement. From previous generational leaps, we’ve learned that the users post-leap don’t look like the pre-leap users at all. The iPod’s introduction brought about a generation of new digital music users that didn’t look like the Limewire generation, and the iPhone’s average user didn’t look like the average user of the BlackBerry before it. Modern programming is at the core of HN, and of most of SV, sure. That said, we should still be the first to realize that a successful, fundamentally new way to program would target a new generation and idea of software maker, one that won’t look like the modern developer at all.

andrewjl 6y ago
> That said, we should still be the first to realize that a successful, fundamentally new way to program would target a new generation and idea of software maker, one that won’t look like the modern developer at all. Channeling Gibson[1], do you see any potential successors already out there? [1] “The future is already here – it's just not evenly distributed."
- izuchukwu 6y ago
  I’d keep an eye on the no-code movement. Real leaps can be distinguished from hype by where the passion is coming from. The fact that the movement’s passion is coming from actual, paying users and not just no-code platform makers is key here. It’s rapidly creating a new generation of software creators that could not create software before, and it’s improving very, very fast.
KMag 6y ago
Former Limewire developer here. I definitely had an iPod mini prior to Limewire's peak years, counted by number of monthly peers reachable via crawling the Gnutella network.
- echelon 6y ago
  What's your take on how everything works these days? Do you miss p2p? Do you think we could ever get back to it?

akkartik 6y ago

I buy the thesis as stated; there's a common attitude among researchers of, "why don't we have this already?" that seems counter-productive. However, there's an adjacent thesis that's also worth stating: fundamentally improving programming is hard, but still worth attempting. There's just too much risk with the current status quo. We've built up our world atop a brittle, insecure, unstable tower of software, and it doesn't feel unreasonable to ask if it might lose capability over time[1]. The good news: you don't have to give it all up all at once and return to the stone age to try something new, as OP says. There's nothing stopping us from using the present as we like to build the future. The key, it seems to me, is to prevent the prospective future from inter-operating with the present. You won't get lots of users this way, but I think jettisoning the accumulated compatibility baggage of 50+ years of mainstream software might free us up to try new things. Even if the world doesn't switch to it en masse, it seems useful to diversify our eggs into multiple baskets. Here's what I work on: https://github.com/akkartik/mu https://github.com/akkartik/mu. It's a computer built up from machine code, designed to be unportable and run on a single processor family. It uses C early on, but it tries to escape it as quickly as possible. Mu programs can bootstrap up from C, but they can also build entirely without C, generating identical binaries either way[2]. (They do still need Linux at the moment. Just a kernel, nothing more like libc.) I call this ethos "barbarian programming", inspired by Veblen's use of the term[3]. I rely on artifacts (my computer, my build tools, my browser, my community support system, the list is long) from the surrounding "settled" mainstream, but I try to not limit myself to its norms (compatibility, social discouragement of forking, etc.). I'm researching a new, better, more anti-fragile way to collaborate with others. Here's a 2-minute video I just recorded, experimenting with a new kind of live-updating, purely-text-mode shell built atop the Mu computer: https://archive.org/details/akkartik-2min-2020-12-06 https://archive.org/details/akkartik-2min-2020-12-06. The shell is implemented in a memory-safe programming language that is translated almost 1:1 to 32-bit x86 machine code. Without any intervening libraries or dependencies except (a handful of syscalls from) a Linux kernel. [1] https://www.youtube.com/watch?v=pW-SOdj4Kkk https://www.youtube.com/watch?v=pW-SOdj4Kkk [2] http://akkartik.name/akkartik-convivial-20200607.pdf http://akkartik.name/akkartik-convivial-20200607.pdf [3] https://www.ribbonfarm.com/2011/03/10/the-return-of-the-barbarian https://www.ribbonfarm.com/2011/03/10/the-return-of-the-barb...

im3w1l 6y ago

They say you can narrow down from all of humanity to a single person with just 33 yes/no questions. I think a similar principle could be true for programs, if we just knew the right questions to ask! I think this is what makes programming hard to improve. We have a poor understanding of what kinds of programs people want to write.

jmnicolas 6y ago
> They say you can narrow down to a single person with just 33 yes/no questions. I resent those questionnaires since it's never a firm yes or no but something in between (or sometimes something completely different). You can't reduce the complexity of one human life to 33 questions.
yetihehe 6y ago
> I think a similar principle could be true for programs, if we just knew the right questions to ask! Those right questions to ask are VERY hard to find, so we programmers have to write many many very accurate answers about how our programs should behave. That process is called programming.
- dwohnitmok 6y ago
  This is a dilemma that often rears its head (exemplified by the classic "42"). Intuitively difficult situations usually end up "conserving their difficulty." An easy solution means a difficult problem (hard to formulate the problem or the formalization of the problem is usually inapplicable). An easy problem usually means a difficult solution.
- im3w1l 6y ago
  So many replies, I'll reply to you but it's really for everyone. The thought experiment was just meant to illustrate how much easier things could be if we knew better what programmers wanted to write. It was not a serious suggestion for how programming should work. What I do think should happen, and will happen, is that we will keep getting a better understanding of what behavior people want most of the time -- including the common variations, and make it ever easier to achieve it.

preommr 6y ago

Those generalizations about why change is hard can be applied to any change in every field. Ideas are always decades if not centuries ahead of their implementations. Not even ideas, but proof of concepts too. So lets talk specifically about software. The biggest problem that prevents moving forward are platforms and companies. Yes, businesses and organizations are usually the biggest factor in dampening progress because they have so much mass it influences everything around them. But you would think that with the low barrier to entry in software, anyone could just spin up some OSS project and it would quickly trickle it's way through. Even slow organizations are often quick to adopt new tech in certain conditions. There were fintech companies that adopted Docker really early on. But it's usually something that's isolated. It's platforms that hold us back. The reason we don't see huge jumps in things like functional programming is because that stuff is great for parallelism, and we don't see jumps in parallelism because the OS, or game engine, or drivers, or chips have poor support. We could have better functional languages if browsers decided to support more functional principles in ecmascript, or even some other scripting language. We don't have all the cool programming stuff because it's piles of abstractions with groups that don't want to coordinate. Sure there are other factors - there are cultural issues that lead to programmers making bad decisions and writing bad code. Java for example was a good language that got ruined by horrible enterprise culture. It's not like people didn't point out the flaws early on, but they got ignored. Lots of people took microservices are a good idea to mean "everything must be a microservice". But even then these problems are perpetuated by companies that mishandle their platform. If companies made user guides clearer we wouldn't have these issues. Not only is the communication bad, but it's often in arcane places. To figure out how to use something you have to read through github discussions or follow mailing lists to get useful information. If rungs to your ladder are broken, you're not going to be able to climb very high.

esc_colon_q 6y ago
Java was a good syntax. The language itself had way too many limitations back when Sun ran it, for really dumb reasons proffered by too smart for their own good people. I was sadly paying attention to the wretched arguments about how there was literally no way to reasonably make first class functions work, or closures, or value types, or escape analysis, or any of the million things that C# just said yes to from day one which worked out fine. It was a lot of edge case kvetching that really needed a dictator to cut through, but Sun were terrible stewards and let the worrywarts run the show. The enterprise culture could I guess be blamed, but Sun was a shite enterprise company when it came to financials, you'd think MS would be much worse in terms of command/control but C# ended up as a damn good version of Java, and once Oracle grabbed the ball hairs Java got pretty great. So it's not the whole problem.

bawolff 6y ago

I feel like as an explanation this fails: why do these arguments apply only to programming and not other things, why do these arguments apply to the present and not the past? Or are we also arguing all things are hard to improve, and that programming hasn't effectively improved since the beggining.

AmericanChopper 6y ago

Programming is hard to improve because it's not very simple to know what programming ideas are good and what ideas are bad. The feedback loop between decision and consequence is so long and opaque that if something works or if it doesn't work, it's not possible to easily point out why. To make things even more complicated, most good programming ideas are only good in certain situations, and terrible in others. On top of that, we're not even very good at knowing what our software is supposed to do. Building a race car is a lot more complicated than writing a simple node web app, but it's way easier to tell if you have a good race car than it is to tell you have a good node web app. If we could agree on what improved programming was, it might be a lot easier to improve. But everybody has their own opinion about what ideas are good, and in what context those ideas are good. We can't even agree about what's important to optimize. Which leads to situations where you can find highly experienced experts passionately disagreeing with each other about absolutely any give topic, and also have experts and charlatans in complete agreement, with no easy way to tell them apart. How would we even be able to tell if it has improved?

Lichtso 6y ago
I also think that the biggest problem we are facing is, that we still haven't developed any useful metrics or measurements of quality in our field. There are some easy things like performance or resource consumption, but: Others like security, robustness, maintainability or plasticity are equally important, yet different approaches can not be compared numerically only by argument.

MaxBarraclough 6y ago

> If you build something that’s too difficult to learn, but very productive you’ll turn a lot of people off. Counterexample: C++. To a lesser degree, perhaps also Rust? (Disclaimer: I don't know Rust.) > The problem is, within a few weeks of using any paradigm developers usually have built a repository of habits that keep them from making mistakes. No, developers constantly make mistakes. Even after all these decades, C programmers are still writing code with buffer overflows. The way to solve that is to have better tools: use a safe language, or formally verify the C code. At this point, we know that it's not enough to hire brilliant C programmers and have them exercise care. > Things have been getting better. The growth has just been in the ecosystems and not the paradigms themselves. I don't think there's a bright line between these two. Good progress is being made in making formal verification more approachable, for instance. This is no small thing, it takes hard work in computer science, but doing so lowers the barrier to the paradigm, making it more practical for more developers and for more problems. Merely thinking up the paradigm, it would be neat to have tooling to verify that our code matches our formal model, is only the first step. In a sense the paradigm only really exists after the hard work has been done. In a similar vein, recent advances in low-pause concurrent garbage-collection might broaden the scope of the problems that can be solved with garbage-collected languages. > The legacy is the value, it can’t be thrown out. I broadly agree. See the classic blog post from Spolsky, Things You Should Never Do, Part I, on how throwing out working code is very often a mistake: https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-...

WJW 6y ago
> Counterexample: C++. To a lesser degree, perhaps also Rust? Are these actually counterexamples? While in absolute terms there may be a fair number of people using them, in relative terms the amount of people who know C++ and/or Rust is absolutely dwarfed by the amount of people who know "easier" languages like JS, Python and PHP. In this case "easier" refers to the amount of effort needed to pick it up and get some result on screen, not to the amount of effort to be a guru. I think for a lot of beginners productivity and even (memory) safety is completely irrelevant. They are not very productive anyway and their code crashes all the time because by definition they make beginner mistakes. What is more important for beginners is to keep up momentum and get through the initial hurdle of "I don't understand this" to the point where you start thinking "this is fun, I can make cool stuff". Rust and C++ solve professional problems, not beginner problems.

mpweiher 6y ago

Found this blog via the wonderful futureofcoding podcasts, episode Building for Developers: https://futureofcoding.org/episodes/021 https://futureofcoding.org/episodes/021 What should be noted is that the author already did the low-code/no-code thing as a startup that is still active and thriving: https://www.dropsource.com https://www.dropsource.com So what he's doing and describing now is informed by that experience, which he talks about at length in the podcast episode. I find it much more insightful and less superficial than a lot of the writing on the topic.

FartyMcFarter 6y ago

> new programming languages including some flow & graph based paradigms and no-code visual builders. Does dressing up Turing-complete programming as visual graphs change anything, fundamentally? Does it make software easier to understand and maintain? In my experience, when people not trained as programmers (or perhaps even programmers) start using visual programming tools in real-world projects, what results is often unmaintainable spaghetti. For example, see https://blueprintsfromhell.tumblr.com/ https://blueprintsfromhell.tumblr.com/ But perhaps I misunderstood what the author meant with the above blurb.

alkonaut 6y ago
For some contexts it’s definitely helpful such as graphs in game engines, 3D modeling and synths. There is always a code-graph duality, the question is which representation is closest to what’s being modeled. Describing what is fundamentally mostly a DAG as a graph rather than as code really is helpful. That it also works for “non-programmers” is just a bonus.
Viliam1234 6y ago
Five lines of code, or five rectangles connected by arrows, both are simple. When people promote visual programming, they make it sound like the five rectangles will replace thousand lines of code. But if that is true, why not simply implement a library with functions equivalent to the individual rectangles, and replace the code with five function calls? Instead, when doing anything nontrivial, we get pictures with thousand rectangles connected by arrows, for pretty much the same reason we previously had thousand lines of code. It's because we want to implement functionality that is not trivially expressed by the provided primitives, duh.
- jerf 6y ago
  One of the things I've taken to doing on the semi-annual "Hey HN I've Fixed All Programming Problems Forever with $THIS" posts is encouraging people to head straight at these hard problems. I've also been encouraging the commenters writing the writing the "dear lord it's so obvious that we just need to drop everything and use $THIS" to be thinking more about the hard problems. I believe I've now issued my challenge to the Visual Language of the Day developers to three different people to not show me how to add 1 over a list, but to implement Quicksort (or an equivalently complex algorithm of your choice) in your new graphical language and show me how it's at least as easy to understand as the current paradigm. I chose that one on purpose on the grounds that it's actually not that understandable in normal notation, so I didn't choose that to be difficult exactly, I chose it as something that has some space for improvement, and, like I said, I'll accept any other non-trivial algorithm that you can make a decent case for being easier. I think only one of them actually drew it out, and it wasn't terribly nice; the others have ignored me. Bret Victor's "live views of the variables updating" stuff looks amazeballs in a crafted demo where only a couple of entities have their X+Y positions displayed in real time, with real-time modification. But it's actually not as new as some people think; I've used debuggers with that capability for a while now. It gets a lot less "ooo.... aaaa..." when you've got hundreds of live-updating values on the screen, and when you've got structured values being manipulated, like, something is adding values to a list as you go, or adding entries to a hash or something. It's actually still really useful and perhaps underused by some people (debuggers, people, I'm down with claims you shouldn't live in them but you need to learn how to use them when you need them!), but it's been around for a long time actually and in practice it isn't quite as game-changing as it looks, because it doesn't solve the problem of finding the important data. Figuring out which parts of a log are important and figuring out which of the hundreds of changing, complicated variables are important aren't that different in terms of effort. In a nutshell, it's scale. A lot of these alternate "Solutions to Everything" don't scale. For all the faults of "text based programming" (instead of a richer manipulation system, or a visual system, or whatever) and our documentation schemes and the way we debug and how we write our code guarantees and so on and so on... they work. We build large systems with them at scale, routinely. The large systems are perhaps ugly, prone to failures, coughing smoke out of many orifices, and often built of complete garbage, but they work. These "Solutions to Everything" tend to break down at scales multiple orders of magnitude smaller than the current-practice competition. By no means does that mean nobody should be working on new ideas, be they revolutionary rewrites of the entire status quo, or small improvements to make a lot of people's code a little better. My point is more that especially if you want to rewrite the status quo, you have got to focus above all else on how your solution is going to scale. Your cute little demo that fits on a screen is still going to be necessary to introduce people to your idea, but show me how your system looks with several thousand of whatever it is you are doing, at the very least. I also tell you that my standard isn't "does that look good", because honestly, at that scale nothing looks good because it no longer fits in human cognition; my standard is that you need to have something better than the current standard, which also doesn't fit into human cognition. You don't need to propose something that we can rewrite operating systems or office suites in, there's plenty of other niches, but you do need to propose something that can go beyond a single screen. The main function programming languages face is not serving up bite-size bits of functionality in comprehensible ways, it's providing ways to slice things into bite-sized bits that fit into my pathetically limited human cognition in the first place. I don't need a better way to understand that this bit of code adds one to everything in the list, I need better ways of breaking problems down into smaller pieces without incurring disadvantages in system organization, performance, composability, etc.

deleted 6y ago

[deleted]

pron 6y ago

While the author speaks about difficulties due to momentum, habit, and baggage, I think the fundamental problem is much, much harder. Many have described the problem in different ways, from Fred Brooks's essential vs. accidental complexity to results on the intrinsic "hardness" of programming we've learned from computational complexity theory in the past two decades, but my favourite is the succinct description Gilles Dowek gave at a talk at NASA [1]: > Computers have been invented to surprise us... If we knew what computers do, we would not use them, and we would not have built any. In other words, not only is writing the programs we want hard because of some "reasonable" Brooks-like argument or some rigorous computational-complexity argument, but we write programs in the first place because it is hard. In practice, not only has Brooks's prediction about no 10x improvement in programming productivity due to a single development -- which was rejected by many at the time as too pessimistic -- been vindicated, but the reality is worse: we have not seen a 10x boost due to all developments combined and in over 30 years! And the boosts we have seen are mostly due to the availability of open-source software, and widespread knowledge sharing on the internet. [1]: https://shemesh.larc.nasa.gov/NFM2018/talks/Dowek.pdf https://shemesh.larc.nasa.gov/NFM2018/talks/Dowek.pdf

AlexSW 6y ago
Seems like a calculator is a good example of a program which we would write even though it is entirely unsurprising; even if we needed nothing else on a computer we might invent it still? It seems to me that we create computer programs not to surprise us but typically to do things very quickly or conveniently. Even a complex beast like Photoshop is reasonably unsurprising.
- stult 6y ago
  The point is that the outputs of the calculator are fundamentally surprising, in the sense that if any individual programmer could easily know all the expected outputs in advance, they wouldn't need the calculator in the first place. Photoshop surprises us by calculating the precise location of millions of pixels based on relatively simple inputs like a handful of clicks on a screen. This logic does not apply to every single imaginable program, but it does apply to the non-trivial ones.

javier10e6 6y ago

Let's ponder on one example where programming was fundamentally improved in the last 10 years. Git Some developers adopted immediately. Others were dragged into it kicking and screaming. Some ignored it (managers). In my humble opinion improvements on a given tool latch on the esoteric definition of what is considered "better". Better for me? Better for the company? Better for the machine? In the case of Git, the case is base on the "me" (Linus Torvalds). In the dev-world we had the joke that goes: If you don't like the tool/language go ahead and write your own. My take away is that improving programming languages and tools, can happen if we take the joke above at heart.

hhas01 6y ago

“In my experience developers are very rational beings.” Yeah, no. Stopped there. Vanity is rationalizing the belief that you’re any better at being rational than the rest of us inchoate apes. There are many technical, logistical, and political arguments as to why improvement (in any sphere) is hard, but if you’re not going to start with “It’s Us, The People” then where usefully are you going to start?

justin_oaks 6y ago
Perhaps it's not that developers are more rational, but that they are better at expressing logical-sounding reasons that they don't want to change. It's also possible to see developers as more willing to change (more "rational") than others when something new can help them get their job done better. If nothing else, developers seem to like trying out new languages and tools for novelty's sake, even if it's not because they're actively trying to improve.

wruza 6y ago

Because (my personal opinion) it is not programming that has to improve. A, B, then C if D is okay and humane. It is weak base libraries that expose and require you to control the irrelevant junk have to improve/vanish. Your basic app, be it web, desktop or mobile, is full of bs and boilerplate of methodologies (not even of just code) to the level that one has to learn to create cruds and component designs. You create and secure endpoints, validate dtos, etc over and over again. For what? Why not have an orm/db system that easily spans server, client and b2b? Why not have a default global message bus that connects directly to user? And so on, depending on business requirements. Instead we have to spend a week or more to learn and remember how e.g. to make “webpush” work and how to serialize an xhr form. Meanwhile our libraries brag about yet another cross-platform #flatten() crap that helps no one but those who dig into this bs deeper than their peers to call themselves a senior. Of course normal people consider that hard, as we wallow in this madness and try to look smart.

marcodave 6y ago
well, the idea is that there are already "do-it-all" frameworks, although they are linked to specific programming languages or paradigm (ruby on rails comes to mind)
- wruza 6y ago
  Sadly, rails, django and other “opinionated orm & http” are two or three steps below than what I meant. Some commercial things may get close in functionality (outsystems, salesforce, 1c enterprise to name a few, but not exactly), but not in ease of learn and use, and usually are too focused rather than being general purpose.

carapace 6y ago

I think programming is easy for lots of people, we (devs/techies) just don't talk about it because it happens in spreadsheets. Excel is the most successful IDE. I've come to suspect that syntax is a MacGuffin and programming languages are a dead end. We're complexity junkies and I suspect we know darn well (some of us consciously but most of us only unconsciously) that if we created a truly usable-by-the-masses IDE it would put us out of work, we wouldn't get paid to mainline our drug of choice.

demosito666 6y ago
You do realize that this is just like saying that "those pesky scientists out there don't cure cancer because that would put them out of the job"? It takes just one scientist to break this worldwide cartel, and he immediately will become billionaire and the one the most influential and famous persons of the century. Can you spot where your predictions contradict game theory? Besides, if there is one thing that programmers like to do, it's developing programming tools. Personally, I'm astonished by the amount of great tools available for free to everyone and the sheer amount of labor that people put there in their free time. Linux [1] alone is mind-boggling, and it's just a tiny fraction of the free codebase available to build your software on top of. Given that, I fail to see how it's possible that millions of developers in the industry somehow participate in anti-productivity cartel. Especially that it's potentially a very lucrative market, as you could cell this IDE in millions and not worry about the future, whatever it would be. [1] I know that most contributions to Linux are made by professionals on payroll, it doesn't change much overall.
- carapace 6y ago
  > You do realize that this is just like saying that "those pesky scientists out there don't cure cancer because that would put them out of the job"? It takes just one scientist to break this worldwide cartel, and he immediately will become billionaire and the one the most influential and famous persons of the century. Can you spot where your predictions contradict game theory? I think in that situation game theory predicts that that scientist is found dead in a burned out car with a bullet hole or two in the skull, eh? But I'm not positing a conspiracy of mustache-twirling villains. It's obvious that the vast majority of programmers are well-intentioned. But it's also obvious that we ignore our own history (and we're slaves to fashion, but that's a tangent.) Engelbart "cured cancer" in 1968[1], Alan Kay and co. "cured cancer" in the 70's[2], and Niklaus Wirth "cured cancer" in the 80's[3]. Things like Alice Pascal were around in the 80's.[4] > Besides, if there is one thing that programmers like to do, it's developing programming tools. Personally, I'm astonished by the amount of great tools available for free to everyone and the sheer amount of labor that people put there in their free time. Linux [1] alone is mind-boggling, and it's just a tiny fraction of the free codebase available to build your software on top of. That's a symptom of the problem, not an intrinsic benefit, most of that software is little more than video games for devs from the POV of human productivity. It's a sickening thought but BASIC has done more for the average person than Lisp. Compare and contrast Red lang to Rust, or Elm lang to the whole of the JS ecosystem. It's a bitter pill, but it's clear that we're playing by different rules than what we tell ourselves and others. > Given that, I fail to see how it's possible that millions of developers in the industry somehow participate in anti-productivity cartel. Arguably we are all doing that all the time to a ridiculous extent. Agriculture is an "anti-productivity cartel" (to the extent that it destroys topsoil and fertility over time), our entire housing system is an "anti-productivity cartel"[5], compared to possibilities!.[6] How is it possible that millions of refrigerators have doors rather than drawers? Every time you open the fridge you spill cold air on the floor and warm air replaces it and you have to pay for the electricity to cool that air off. It's an anti-productivity cartel! > Especially that it's potentially a very lucrative market, as you could cell this IDE in millions and not worry about the future, whatever it would be. It's not like folks aren't trying.[7] [1] https://en.wikipedia.org/wiki/The_Mother_of_All_Demos https://en.wikipedia.org/wiki/The_Mother_of_All_Demos [2] https://en.wikipedia.org/wiki/Smalltalk https://en.wikipedia.org/wiki/Smalltalk [3] https://en.wikipedia.org/wiki/Oberon_(operating_system) https://en.wikipedia.org/wiki/Oberon_(operating_system) [4] https://www.templetons.com/brad/alice.html https://www.templetons.com/brad/alice.html https://www.atarimagazines.com/v6n2/Alice.html https://www.atarimagazines.com/v6n2/Alice.html [5] https://www.berkeleyside.com/2018/08/02/prefab-housing-complex-for-uc-berkeley-students-built-in-four-days https://www.berkeleyside.com/2018/08/02/prefab-housing-compl... https://news.ycombinator.com/item?id=17739805 https://news.ycombinator.com/item?id=17739805 [6] https://en.wikipedia.org/wiki/Dymaxion_house https://en.wikipedia.org/wiki/Dymaxion_house https://en.wikipedia.org/wiki/Old_Man_River%27s_City_project https://en.wikipedia.org/wiki/Old_Man_River%27s_City_project https://en.wikipedia.org/wiki/Arcology https://en.wikipedia.org/wiki/Arcology [7] E.g. https://darklang.com/ https://darklang.com/

sova 6y ago

Not all programming languages are equal.

pklausler 6y ago
So much this. We have had pure well-typed functional programming languages for decades, and they are great at forcing clean designs and catching bugs at build time, and yet they're still niche tools for the most part.

giantg2 6y ago

I just wish I was good at something and had a promising future.

Ericson2314 6y ago

These are silly arguments. As someone that works on tooling a lot, we could be doing massively better. But there isn't enough demand. - Slow overall economic growth - Stick to shit like JS and there is now a glut of entry-level programmers - QE and VC model is ridiculously supply-side already, could hardly sustain more. - Very hard to capture value of new tooling when alternative is free. Asking why we don't have the programming tooling we should have is not unlike asking why we don't have the subways we should have. ------- If someone wants to through more money at tooling anyways, do let me know! :D

lebuffon 6y ago

I am wondering if something like GTP-3 (or GPT-3+x) which has the beginnings of writing code from human language descriptions will be a useful thing. Creating working code to expand upon can be a significant part of the process of new software creation.

BoiledCabbage 6y ago

The author nails the fundamentally hard problem in software. If there is a new tech in bridge building, then every bridge going forward can use it, but software is successful because of its layers of abstraction and network effect. You can't introduce a new superior tech unless the huge amounts of existing tech and tools can first be made compatible with it. And that's harder to do because of all the degrees of freedom available in software rather than the relatively few in physics. And even if a method could be discovered to make them compatible there is no way for anyone to snap their finger and update them all - it takes tons of work. And who is going to convince all of these other people to stop what their doing to invest in supporting this better tech? Even if this better tech is there, how long will it take them to really understand the correct mental model to apply it to their tool. Functional programming is now still slowly gaining adoption because it's a different way to think. It takes us forever to learn and adopt a better method. And even if people could be made willing to adopt a better tech and do all of the above, how can we show a tech is better? We don't even have an objective method of evaluating what better is. It's all hearsay and opinion. So everyone needs to evaluate it on their own. But even if people were willing to do that, we don't even have an objective definition of what "better" is in software tooling! The problem is, in evaluating software, we are pre-scientific method. The single largest break through any one person could have, the only "singularity tech" in software development would be for someone to design a model to represent an objective human evaluation of software language tool, method or tool. Something that allows us to objective compare languages and automate the search space of fixing better tooling. Without that it will continue to take 40 years to adopt clearly better ideas. Until we can even say the equivalent of basic physical comparisons of "objective x is heavier than object y", or "object a is faster/stronger than object b" using an agreed upon standard progress will be fits and spurts. Imagine Civil engineering over the centuries and people can look at two bridges and be unable to tell which is stronger or wider or longer or costs more to build. Think how worse off our tech would be.

barrenko 6y ago

That Brett Victor talk is wildly overrated.

yters 6y ago

I'd say the progression from machine code to Python is a fundamental improvement. For simple cases, I think a visual/textual code interface is possible, but for any complicated case the visual interface will become inscrutable. Maybe an optimal tradeoff is like how game engines allow scripting languages to live on top. A number of 3D engines use this paradigm with a visual language in place of scripting language.

6gvONxR4sf7o 6y ago

As programmers, I think we're way too stuck to the idea that programming is a keyboard and mouse exercise when we are trying to envision the future. Like many other people, I've been struggling with carpal tunnel during the pandemic and I've switched to dictation based coding. Since making the switch, I've realized how much of how I code is specifically due to the keyboard being my way to tell the computer things. And graphical programming paradigms that I've seen are really just switching from keyboard to keyboard + mouse. Right now, dictation is still at the point where it's a royal pain in the ass to have it as the primary input method. As soon as that changes, I expect that we will see a flurry of new programming paradigms. Code is best represented as text when it is input (and modified!) as text. My current dictation set up includes lots of it asking me "did you mean X,Y, or Z" and as natural language processing improves, those questions are going to get a lot more interesting. Even if NLP weren't going to improve, dictation+keyboard+mouse(+touchscreen+eyetracking ...) based programming will probably be a very different experience from keyboard+mouse based programming. Until then, I really only expect incremental change.

deleted 6y ago
[deleted]

aceBacker 6y ago

I wonder how the author would feel about the low code solutions that are catching on in enterprises. For example outsystems seems to be changing how programming works while letting people use languages they are familiar with.

tabtab 6y ago

Re: The same things that make Scratch easy to learn also make it an unproductive environment for more serious programmers. For a professional, programming with drag and drop is way slower than keying in code — that’s just a fact. I would say the best tools allow one to switch back and forth between visual/drag-and-drop and code as needed. Web-based UI's never seemed to get that to work well. The stateless nature of the web is a productivity kicker in multiple ways. The same number of features could be created and managed by fewer programmer hours in IDE's from the 90's than modern web stacks in my observation. Web stacks are so bloated that orgs need layer specialists to manage it all. "Separation of concerns" (SOC) is to fit the shape of dev staff, not the shape of productivity nor the domain. It just creates busy-work interfaces between all those layers; a time drain. The older tools were closer to the domain such that it was less code to do the same thing, making SOC not worth it. SOC is symptom of bloat, not a solution. Concerns naturally interweave; the forced walls are just to serve Conway's Law. The counter argument is often, "yes, but we have choice". Sure, but you pay dearly for it. I believe a stateful GUI markup standard could help bring rank-and-file office-oriented CRUD productivity back to 90's levels. One could then spend time on domain issues instead of gluing bloated buggy stack layers together. I can't say it will help all domains, but one-browser-fits-all is failing CRUD, if productivity matters.

gampleman 6y ago

One issue that's been on my mind is how to actually fund a new programming paradigm. As far as I can tell there are the following options: 1. Start a business and seek investment (examples: Unison, Dark). This is tricky since programming environments are hard to monetise (all the competition is free), and the investment is quite large (you need a great team working for many years to maybe make something good enough you can actually attempt to charge for it). 2. Do it as a hobby project (examples: many of the small PLs nobody uses in production). This is hard because it takes a LOT of work to just get a good programming language, and then you need to also produce great tooling, a decent number of good libraries, amazing learning materials, etc. 3. Academia (examples: Haskell, Idris, Elm, ...). This can work, but generally the focus will need to be on some academically interesting problem. But a lot of what it takes for a language to succeed is a lot of down to earth engineering, which academic positions aren't generally great at (i.e. it took Haskell about 20 years to start being somewhat successful outside academia). 4. Corporate sponsorship (examples: Elm, C#, Swift, ...). This generally needs alignment between the language and the corporation's goals. This typically limits the amount of radical inventions you can make, as well as how freely you can pivot to new ideas. Fundamentally none of the above are readily useable for radical reinventions, but more for incremental improvements.

worik 6y ago

Is Swift really the second most popular language since 2000? Golly. I have been using Swift for some months now. It is more modern than C/C++, sure, but it really suffers from lack of development. The ergonomics of it is poor. Especially compared to Rust. Some design choices are quite odd (the definition of a constant defined with `let` for exampe) It is the only easily available choice for developing on Apple, AFAICT. Hence it is so popular. As a fresh developer on Apple, but grizzled and grey, I must say teh development tools and ecosystem reminds me of Linux in 1997. Everything mostly works