5 ms·

This list, together with already mainstream features, turns a programmer’s learning curve into an uncompromising cliff. It’s clear that right now big software h

by wruza 5y ago

This list, together with already mainstream features, turns a programmer’s learning curve into an uncompromising cliff. It’s clear that right now big software has to be formal to live longer and stay healthy, but for me it feels like we’re losing something important on this way. Computers ought to be our helpers which could understand our plans and implement them properly. Instead we are burying ourselves and those who will come later in what is essentially a deep math. And our software doesn’t even get faster or better, it just fits whatever performance:annoyance ratio is acceptable at the end of the day. I think, and this is more of an intuitive rather than an informed guess, that the current practical languages are still too low level and that may be one of big reasons why software development is so complex. You probably feel it too, when you have to either A) go find, copy and fine-tune yet another boilerplate or a snippet, making it again too hairy to use as is next time, or B) use a standard solution which almost implemented another programming language on top of its configuration and isn’t really much easier to use than if it were just a from-scratch code. Shovels get better, but we are still digging careers with them. And we had predecessors of a theoretical language which could be high-level enough to make programming great again. E.g. FoxPro with all its DOS restrictions was a platform where one could read, write, iterate, update data, never thinking how to map classes to tables or forms, or how to make it run. Or Delphi, in which most students could write client-server apps by dragging tables, queries, databases and forms right from a toolbar. The entry complexity skyrocketed since then or, more precisely, returned to “serious” levels of heavy C++ setups which only seasoned programmers by trade could manage without pulling hairs. I don’t think that human[ity] will become any significantly smarter in the near future, and that, together with our all time high software demands, means that we will experience a disabling shortage of software developers until we fix our internalized tolerance to the increasing complexity.

duped 5y ago
Which parts? The points on modules and effect systems are totally in line with creating a shallower learning curve. My reading of his point on formal specifications was that it was important to design software that works as it behaves (like compilers that compile the languages they say they do, paraphrasing). That's not a burden on users of the language directly, more on the creators.
- matu3ba 5y ago
  An effect system is redundant to code and is not free (compilation time cost) and neither helps when you dont know your future program spec or behavior. https://en.wikipedia.org/wiki/Effect_system https://en.wikipedia.org/wiki/Effect_system It only prevents some misuse classes of APIs. I think what you want more dearly is to express atomiticity of functions and (automatic) rollback on error to a "known good state".
  - grumpyprole 5y ago
    What do you mean "an effect system is redundant to code"? An effect system may or not come with additional static typing. I would argue that static effect types are certainly not redundant, having programmed Haskell professionally for many years. But that's just the usual static types debate. Unrelated to typing, algebraic effect handlers probably have the most ergonomic implementation of async I have ever seen (looking at the OCaml multicore previews).
    - matu3ba 5y ago
      The point I was trying to make is that you can get comparable guarantees, if you structure the code with atomticity + rollback and trace generalizations. Most people do not code like this due to missing language support for good error handling. (explicitly encoding errors as integers sucks and more so that you need to do it globally) Yes. That is compiler provided functionality on top, which looks excellent.
    - Twisol 5y ago
      > algebraic effect handlers probably have the most ergonomic implementation of async I have ever seen (looking at the OCaml multicore previews) Just dropping in to say strong agree. The biggest stumbling block for me has been that the effect handlers are captured as part of the continuation; I was expecting them to need to be manually re-established every time the continuation is resumed. The way it works makes sense when you always `continue` within the body of an effect handler, but it feel a little unfortunate that there's no way to change the handlers for future resumptions. Other than that, the possibilities for userland coroutines, dynamically-scoped variables, and centralization of state (like `ref` but provided by a scoped allocator) are incredibly exciting. I've wanted a system like this for a long time, and for some use-cases (scheduling of concurrent simulation tasks) I had to abuse threads and thread-locals to get the separation I wanted.
grumpyprole 5y ago
> This list, together with already mainstream features, turns a programmer’s learning curve into an uncompromising cliff. This is because many insist on having one giant programming language to do everything both low-level and high-level together. Rust just follows C++ along this same path. The resultant language will always be a complex compromise that is suboptimal for almost everything. For example, both Rust and C++ have too many high-level abstractions to easily reason about where allocations will happen and their manual memory management makes lambdas and async painful. I personally would favour the use of multiple languages. Such an approach is commonly emerging anyway due to practicalities, e.g. Python/C/Fortran use in machine learning.
- einpoklum 5y ago
  > a complex compromise that is suboptimal for almost everything Whenever you have a programming language with multiple goals (or, let's say, more than two distinct goals) - it's almost certainly going to be suboptimal for any individual one of them. The questions are: * Whether choice of goals is worthy/reasonable; and * Whether a good balance and compromise has been struck by that language's specifiers and implementers. Also remember that languages evolve; and if one of the language goals is backwards compatibility (which is the case for C++, less so for Rust) - then part of the compromise is that they have to support multiple older and less-recommended idioms and facilities while offering newer, hopefully better ones.
  - grumpyprole 5y ago
    I hope it was clear from my earlier post that I am arguing "No" for your questions above, especially regarding C++ and Rust. This doesn't necessarily mean I don't think great software cannot be built with these languages, but rather I do not see them as the future.
- ncmncm 5y ago
  > [because languages] do everything both low-level and high-level together This is very far from true. Like, not even the tiniest bit true. It is easy to program in C++ or Rust and pay no attention at all to where allocations happen. Almost all application-level coding in those languages is written exactly that way. Even in programs where such attention is needed, the overwhelming bulk of the program, such as all the initialization and setup code, gets away with completely ignoring allocation. Certain libraries whose value proposition includes having been heavily optimized need to pay attention to memory placement, but probably more than half of libraries would only ever be used in places where performance doesn't matter at all, and the author knew it. Anyplace where you do need to pay attention to details of allocation, or of atomic event ordering, or of interrupt context, you are not obliged to use every last bit of abstraction your language enables. It is totally allowed to drop to a C level of specification. (In Rust you might even put some of it in "unsafe" blocks.) Rust and C++ do impose inconveniences for various reasons -- C++ mainly because of backward compatibility, Rust mainly for extra static enforcement -- but the inconveniences are completely unrelated to the occasional need to pay attention to memory placement. Where existing languages do still demand low-level attention is in obliging the programmer to be aware of control flow, and what thread is executing which bit of code. They are only starting to loosen that. It used to be that you didn't care because there was only one thread. Then, we made threads, but not too many, because of numerous costs. Now, we would mostly like our programs to use as many threads, scheduled any which way, as would be useful.
verdagon 5y ago
Well said, and I think every new language should take its learning curve seriously. A language should get one of one's way, so they're thinking about their project's inherent complexity, not about the language's complexities. I might be biased, but I'm excited for Zig and Cone, which I think will make it easier for newcomers to the low-level space.
- erik_seaberg 5y ago
  We should optimize for those who have mastered the language, because that is where each of us will spend most of his career. Helping newcomers is nice if it doesn’t come at the expense of the majority. The top of the learning curve is the point where the language no longer has any help to offer, and that should be as high as feasible.
scythe 5y ago
The list includes, from my reading, modules, errors, asynchronicity, side-effect types, fancy linting, and formalization. Elements 1, 2, and 5 exist in some sense in almost every area of engineering, while 3 and 6 are easily ignored if you don't need them. That leaves side-effects, the old bugbear of all functional programming. The difficulty with Rust is that it's "batteries not included", which is a term we use in software for the equivalent of a car sold with no wheels, body panels or seats. Sure there's a package manager, there's also an auto parts catalog.
pansa2 5y ago
> big software has to be formal to live longer and stay healthy Can’t disagree, but where are the languages for little software? For example, I write lots of small (100-1000 LoC) command-line tools. I’m not a full-time programmer, so I don’t have time to learn a complex language - what I ideally want is executable pseudocode. Are there languages that fit that description? Python used to, but even it has now become very complex - because its designers are all employees of big companies working on big software.
- lifthrasiir 5y ago
  I suppose that'd be the original intended use of "scripting" languages. And we came to realize that things get start small but can grow significantly larger and we can no longer use those languages. Many languages that got initially advertised as scripting languages became much more complex partly in order to handle larger softwares. As such I'm not confident that your tools have zero potential to grow larger.
- Mikeb85 5y ago
  Try Ruby. It still fits the bill. And of course there's Perl 6 aka. Raku.
  - bbkane 5y ago
    Isn't Raku considered an extremely complex language?
    - lizmat 5y ago
      I wouldn't say that Raku is a complex language. Raku is a typical example of a programming language in which the *developers* of the language are tortured to make the *use* of the language much easier. A carefully balanced quantity of DWIM (Do What I Mean) without too much WAT, while having awesome error messages to help you get going and continue to be productive.
- robbedpeter 5y ago
  Lua is phenomenal. AutoIt if you're on windows - it's underappreciated for sheer utility.
crate_barre 5y ago
I think it’s fine. Software usually Darwins out a language winner. There’s a reason a lot of things got built with Perl, python, js, Java, c++, ruby, php, and there will be a reason why things won’t be written in those languages anymore (or aren’t anymore). There’s no amount of will that can force the hand of god and make Rust or Typescript ubiquitous or not-ubiquitous in the long run. We learn our lessons slowly. The winning language always surfaces itself in the form of useful software. If it’s true that these new language constructs are useful, pain-free, then we will surely see the ubiquity of those languages and the corresponding software with it. If it’s not useful, then nothing useful will come of it. It will be just another exercise in technical virtuosity, a master violinist playing a technically profound tune that no one likes to listen to. You’ll just have to enjoy it for the feat that it is, but it’s not something people will want to sit through regularly. Cheers, design away, I’ll see you on the other side of the cacophony.
akavel 5y ago
<enso-fanboi mode="on" disclaimer="&employee-soon;"> Have you seen https://enso.org https://enso.org? Personally, I'm strongly hoping for it to become a modern contender in such space (of making programming easier and faster, for non- and programmers alike)
fho 5y ago
> I don’t think that human[ity] will become any significantly smarter in the near future Anecdotal side note: I recently visited a archaeological museum on greek civilization. Some of the artifacts were 6000 years old and I was really surprised how ... delicate and "modern" for the lack of a better word ... the earthenware looked. That really made me think that humans back then were just as intelligent as we are today. I think that may be a fact that may be supported by biological evidence? Brain size comparisons of skulls maybe? Either way, if humans have not been less intelligent back then, there is only technical progress left.
tmaly 5y ago
On the flip side, supply and demand will keep wages high. But, it will also bring more people into the field that will likely increase the amount of poorly written code that needs to be maintained. What happened to all the talk around making the compiler do all of the work?
6gvONxR4sf7o 5y ago
I for one can’t wait to have to climb that cliff. Unfortunately, our industry has a relatively shallow onramp, followed by a ton of difficulty using our building blocks to make stuff that does what we want in reliable, maintainable, performant ways. If we can have more difficult to learn, but easier to use tools to get us where we’re trying to go, I’m all for it. I guess, instead of seeing it as an increase in complexity, I see it as a tradeoff of increased learning curve for decreased complexity. You’re spot on about the labor market side though. More learning curve for the best tools has tons of labor market implications. I’d bet the end game is more computing being in the general education tracks to deal with the overhead, as well as a wider (hopefully continuous) spectrum of computing tools for people ranging from those using computing as a small part of their work to those using computing for all of it. Like gradual typing. You don’t need to add types to python, but you can add mypy. Gradual depth, maybe.

by wruza 5y ago

duped 5y ago

Which parts? The points on modules and effect systems are totally in line with creating a shallower learning curve. My reading of his point on formal specifications was that it was important to design software that works as it behaves (like compilers that compile the languages they say they do, paraphrasing). That's not a burden on users of the language directly, more on the creators.

matu3ba 5y ago
An effect system is redundant to code and is not free (compilation time cost) and neither helps when you dont know your future program spec or behavior. https://en.wikipedia.org/wiki/Effect_system https://en.wikipedia.org/wiki/Effect_system It only prevents some misuse classes of APIs. I think what you want more dearly is to express atomiticity of functions and (automatic) rollback on error to a "known good state".
- grumpyprole 5y ago
  What do you mean "an effect system is redundant to code"? An effect system may or not come with additional static typing. I would argue that static effect types are certainly not redundant, having programmed Haskell professionally for many years. But that's just the usual static types debate. Unrelated to typing, algebraic effect handlers probably have the most ergonomic implementation of async I have ever seen (looking at the OCaml multicore previews).
  - matu3ba 5y ago
    The point I was trying to make is that you can get comparable guarantees, if you structure the code with atomticity + rollback and trace generalizations. Most people do not code like this due to missing language support for good error handling. (explicitly encoding errors as integers sucks and more so that you need to do it globally) Yes. That is compiler provided functionality on top, which looks excellent.
  - Twisol 5y ago
    > algebraic effect handlers probably have the most ergonomic implementation of async I have ever seen (looking at the OCaml multicore previews) Just dropping in to say strong agree. The biggest stumbling block for me has been that the effect handlers are captured as part of the continuation; I was expecting them to need to be manually re-established every time the continuation is resumed. The way it works makes sense when you always `continue` within the body of an effect handler, but it feel a little unfortunate that there's no way to change the handlers for future resumptions. Other than that, the possibilities for userland coroutines, dynamically-scoped variables, and centralization of state (like `ref` but provided by a scoped allocator) are incredibly exciting. I've wanted a system like this for a long time, and for some use-cases (scheduling of concurrent simulation tasks) I had to abuse threads and thread-locals to get the separation I wanted.

grumpyprole 5y ago

> This list, together with already mainstream features, turns a programmer’s learning curve into an uncompromising cliff. This is because many insist on having one giant programming language to do everything both low-level and high-level together. Rust just follows C++ along this same path. The resultant language will always be a complex compromise that is suboptimal for almost everything. For example, both Rust and C++ have too many high-level abstractions to easily reason about where allocations will happen and their manual memory management makes lambdas and async painful. I personally would favour the use of multiple languages. Such an approach is commonly emerging anyway due to practicalities, e.g. Python/C/Fortran use in machine learning.

einpoklum 5y ago
> a complex compromise that is suboptimal for almost everything Whenever you have a programming language with multiple goals (or, let's say, more than two distinct goals) - it's almost certainly going to be suboptimal for any individual one of them. The questions are: * Whether choice of goals is worthy/reasonable; and * Whether a good balance and compromise has been struck by that language's specifiers and implementers. Also remember that languages evolve; and if one of the language goals is backwards compatibility (which is the case for C++, less so for Rust) - then part of the compromise is that they have to support multiple older and less-recommended idioms and facilities while offering newer, hopefully better ones.
- grumpyprole 5y ago
  I hope it was clear from my earlier post that I am arguing "No" for your questions above, especially regarding C++ and Rust. This doesn't necessarily mean I don't think great software cannot be built with these languages, but rather I do not see them as the future.
ncmncm 5y ago
> [because languages] do everything both low-level and high-level together This is very far from true. Like, not even the tiniest bit true. It is easy to program in C++ or Rust and pay no attention at all to where allocations happen. Almost all application-level coding in those languages is written exactly that way. Even in programs where such attention is needed, the overwhelming bulk of the program, such as all the initialization and setup code, gets away with completely ignoring allocation. Certain libraries whose value proposition includes having been heavily optimized need to pay attention to memory placement, but probably more than half of libraries would only ever be used in places where performance doesn't matter at all, and the author knew it. Anyplace where you do need to pay attention to details of allocation, or of atomic event ordering, or of interrupt context, you are not obliged to use every last bit of abstraction your language enables. It is totally allowed to drop to a C level of specification. (In Rust you might even put some of it in "unsafe" blocks.) Rust and C++ do impose inconveniences for various reasons -- C++ mainly because of backward compatibility, Rust mainly for extra static enforcement -- but the inconveniences are completely unrelated to the occasional need to pay attention to memory placement. Where existing languages do still demand low-level attention is in obliging the programmer to be aware of control flow, and what thread is executing which bit of code. They are only starting to loosen that. It used to be that you didn't care because there was only one thread. Then, we made threads, but not too many, because of numerous costs. Now, we would mostly like our programs to use as many threads, scheduled any which way, as would be useful.

verdagon 5y ago

Well said, and I think every new language should take its learning curve seriously. A language should get one of one's way, so they're thinking about their project's inherent complexity, not about the language's complexities. I might be biased, but I'm excited for Zig and Cone, which I think will make it easier for newcomers to the low-level space.

erik_seaberg 5y ago
We should optimize for those who have mastered the language, because that is where each of us will spend most of his career. Helping newcomers is nice if it doesn’t come at the expense of the majority. The top of the learning curve is the point where the language no longer has any help to offer, and that should be as high as feasible.

scythe 5y ago

The list includes, from my reading, modules, errors, asynchronicity, side-effect types, fancy linting, and formalization. Elements 1, 2, and 5 exist in some sense in almost every area of engineering, while 3 and 6 are easily ignored if you don't need them. That leaves side-effects, the old bugbear of all functional programming. The difficulty with Rust is that it's "batteries not included", which is a term we use in software for the equivalent of a car sold with no wheels, body panels or seats. Sure there's a package manager, there's also an auto parts catalog.

pansa2 5y ago

> big software has to be formal to live longer and stay healthy Can’t disagree, but where are the languages for little software? For example, I write lots of small (100-1000 LoC) command-line tools. I’m not a full-time programmer, so I don’t have time to learn a complex language - what I ideally want is executable pseudocode. Are there languages that fit that description? Python used to, but even it has now become very complex - because its designers are all employees of big companies working on big software.

lifthrasiir 5y ago
I suppose that'd be the original intended use of "scripting" languages. And we came to realize that things get start small but can grow significantly larger and we can no longer use those languages. Many languages that got initially advertised as scripting languages became much more complex partly in order to handle larger softwares. As such I'm not confident that your tools have zero potential to grow larger.
Mikeb85 5y ago
Try Ruby. It still fits the bill. And of course there's Perl 6 aka. Raku.
- bbkane 5y ago
  Isn't Raku considered an extremely complex language?
  - lizmat 5y ago
    I wouldn't say that Raku is a complex language. Raku is a typical example of a programming language in which the *developers* of the language are tortured to make the *use* of the language much easier. A carefully balanced quantity of DWIM (Do What I Mean) without too much WAT, while having awesome error messages to help you get going and continue to be productive.
robbedpeter 5y ago
Lua is phenomenal. AutoIt if you're on windows - it's underappreciated for sheer utility.

crate_barre 5y ago

I think it’s fine. Software usually Darwins out a language winner. There’s a reason a lot of things got built with Perl, python, js, Java, c++, ruby, php, and there will be a reason why things won’t be written in those languages anymore (or aren’t anymore). There’s no amount of will that can force the hand of god and make Rust or Typescript ubiquitous or not-ubiquitous in the long run. We learn our lessons slowly. The winning language always surfaces itself in the form of useful software. If it’s true that these new language constructs are useful, pain-free, then we will surely see the ubiquity of those languages and the corresponding software with it. If it’s not useful, then nothing useful will come of it. It will be just another exercise in technical virtuosity, a master violinist playing a technically profound tune that no one likes to listen to. You’ll just have to enjoy it for the feat that it is, but it’s not something people will want to sit through regularly. Cheers, design away, I’ll see you on the other side of the cacophony.

akavel 5y ago

<enso-fanboi mode="on" disclaimer="&employee-soon;"> Have you seen https://enso.org https://enso.org? Personally, I'm strongly hoping for it to become a modern contender in such space (of making programming easier and faster, for non- and programmers alike)

fho 5y ago

> I don’t think that human[ity] will become any significantly smarter in the near future Anecdotal side note: I recently visited a archaeological museum on greek civilization. Some of the artifacts were 6000 years old and I was really surprised how ... delicate and "modern" for the lack of a better word ... the earthenware looked. That really made me think that humans back then were just as intelligent as we are today. I think that may be a fact that may be supported by biological evidence? Brain size comparisons of skulls maybe? Either way, if humans have not been less intelligent back then, there is only technical progress left.

tmaly 5y ago

On the flip side, supply and demand will keep wages high. But, it will also bring more people into the field that will likely increase the amount of poorly written code that needs to be maintained. What happened to all the talk around making the compiler do all of the work?

6gvONxR4sf7o 5y ago

I for one can’t wait to have to climb that cliff. Unfortunately, our industry has a relatively shallow onramp, followed by a ton of difficulty using our building blocks to make stuff that does what we want in reliable, maintainable, performant ways. If we can have more difficult to learn, but easier to use tools to get us where we’re trying to go, I’m all for it. I guess, instead of seeing it as an increase in complexity, I see it as a tradeoff of increased learning curve for decreased complexity. You’re spot on about the labor market side though. More learning curve for the best tools has tons of labor market implications. I’d bet the end game is more computing being in the general education tracks to deal with the overhead, as well as a wider (hopefully continuous) spectrum of computing tools for people ranging from those using computing as a small part of their work to those using computing for all of it. Like gradual typing. You don’t need to add types to python, but you can add mypy. Gradual depth, maybe.