8 ms·
Why are systems languages always overly complex?
I do most of my programming in Node and TypeScript. It's nice and simple.
I always wonder why we don't just use this syntax to write low-level system code. The module system is nice; functions, objects, and arrays are nice to work with; and the syntax is simple and familiar.
A systems language essentially just adds access to pointers.
In most of these systems languages, the raw memory management is often abstracted away into libraries, and we are just writing basic logic in a new convoluted syntax that a single dev happened to like. Think Rust, Zig, etc.
It's quite clear that TypeScript is going to be around forever. Why don't we just use this syntax for a systems language, and add a few constraints to make it low-level.
Imagine the same syntax/tooling for embedded, kernel, system (browsers, interpreters), backend, and frontend.
I just don't know why there is not more interest in this.
- bayindirh 4y agoThe thing is, this convoluted logic and raw access to pointers translate directly to CPU instructions very well. C++'s atomic::read_and_increment_one() (name may be wrong) is a single CPU instruction. Similarly the logic operations and other pointer operations have direct assembly equivalents. This direct translatability allows a lot of the things we do at the low level. All the abstractions added by JS, python and other languages incur great performance penalties that we don't notice at application level. However, some functions are called a couple of million times per second, and you don't notice that, or your CPU runs them pretty effectively with no discernible load. Embedded is something entirely different, where you have 10x-50x less resources and insane time constraints to do things at real time. There are even standards to further limit C programming to several guidelines/patterns/ways to make sure the code is deterministic, down to the CPU cycle counts (since boot-up) in some cases. For some personal opinion, I don't consider C & C++ complex languages. They're easy. They just needs the programmer to know what he/she's doing. Otherwise it just goes bang, and it's a good thing in most cases.
- OhNoNotAgain_99 4y agoAnd Python is a great macro language for c / c++, brings back the point why is the use of higher levels language not more common, afterall who still does assembler these days (and c++ is heavily background optimized for asmblr as well).
- deltasevennine 4y ago> The thing is, this convoluted logic and raw access to pointers translate directly to CPU instructions very well. The OP isn't talking about this. The OP is saying at a basic level python/typescript is simply C++ with no pointers. So the OP is asking can't we design a python-like language with pointers? >For some personal opinion, I don't consider C & C++ complex languages. They're easy. This is not an opinion shared by many. It's like someone saying they enjoy licking the ground. C++ is known for it's over complexity. It is complex mostly not because it's low level but because it's carrying around decades of design baggage and poor design decisions while being modern and backwards compatible. >Otherwise it just goes bang, and it's a good thing in most cases. Hard disagree. Low level programs can go bang at locations far away from the error and become incredibly hard to find. Additionally memory leaks are really challenging to find sometimes. These errors happen despite programmers knowing what they are doing as they are "mistakes" or bugs. These problems happen much less in higher level languages. C++ and C go bang at all the wrong times, while something like unsafe python goes bang at all the right times.
- mirker 4y agoMemory leaks still happen if you have a misunderstanding of memory management, regardless of language. Keeping references to dead objects is one straightforward way to do it. C++ also has smart pointers so it’s usually a case of someone writing code that is smarter than they are. One example: since some people in Python don’t think about memory they are prone to writing loops that grow memory proportionally to the loop counter in machine learning code (e.g., implicit graph references across loops).
- deltasevennine 4y ago>Memory leaks still happen if you have a misunderstanding of memory management, regardless of language. Apologies. What does this have to do with what I'm talking about? Not sure I said anything about memory leaks not happening in languages like python. I believe I said these issues happen "much less" in higher level languages so I'm not sure what you're trying to clarify here.
- frou_dh 4y agoThe key word in your post is "familiar". You want it to be familiar. There's no need to overstep that and make vague judgements of simplicity and complexity.
- deltasevennine 4y agoNo. Disagree. His post has a good point, and nowhere in it does he mention "familiarity." He obviously likes Typescript, but beyond that his post has a point.
- frou_dh 4y agoDon't know what to tell you. Two-thirds of the post are about subjective comfy syntax. And they did use the word familiar, which is why I quoted it.
- grumblingdev 4y agoIt's the most familiar to the most amount of programmers. Despite maybe Python, but that doesn't target the web so it's not ideal. I just don't see a new language being created that can be a systems language, and is also the language of choice for web apps, etc. TS already targets everything, except native (although there is NativeScript and AssemblyScript and a TypeScript LLVM compiler laying around). What I want is one syntax to target every platform, and when I click run, I can step through the code and it doesn't matter if it's running native or JIT.
- SamReidHughes 4y agoC++ is overly complex because of its backwards compatibility with C, and then its backwards compatibility with C++. And a few design decisions whose consequences are visible in hindsight. Zig and Rust are not overly complex. > A systems language essentially just adds access to pointers. They remove the garbage collector, and are designed to facilitate manually managing memory. That's why they're more complex than you expect. If you just want to add pointers, you get something like Go.
- shrubble 4y agoHave you examined the Wirth family of languages used to implement, with just a few programmer-years, the Lilith and Ceres OS's for workstations?
- deltasevennine 4y ago>It's quite clear that TypeScript is going to be around forever. Why don't we just use this syntax for a systems language, and add a few constraints to make it low-level. This is the essential point of your post, and I'm not completely sure why this can't/can be done. I can certainly name some reasons why it hasn't been done, but the actuality of the concept is separate from this. Nobody in this thread so far has answered your question. I will say one thing though, if it is done it won't just a "few" constraints. It will be a huge amount of constraints and language feature changes. In the end my prediction is you will get something that is both significantly more challenging to deal with then typescript but simpler then C++.
- jmillikin 4y agoThere's different things people mean by "systems language", but a common theme is extremely precise control over the machine code emitted by the compiler. If you take TypeScript and add pointers, you'd end up with something like Java, or maybe Go. This is not necessarily a bad outcome (Java and Go are both very popular), but those languages don't provide the sort of control you need when writing a kernel, video codec, embedded device firmware, or the runtime for a higher-level language. Some folks might disagree, but I wouldn't personally consider Go a "systems language" in the same way that C or Rust are. The complexity you see in languages like Rust reflects the programmer's desire/need for controlled compiler output, plus a healthy dose of innate complexity of the problem domain. TypeScript has one type of string, Rust has dozens, because String and OsString and CString each have different semantics that will cause bugs if they get mixed up. We know this because older systems languages did mix them up (C++'s std::string, C *char vs *void vs *uint_8), and there were bugs, and late nights crying into a GDB prompt looking for a missing NUL terminator. Also, I suspect from your focus on syntax that you're new to the field of programming. Once you've been writing non-web code professionally for 10 years or so, you'll pay a lot less attention to the notation. The most popular languages (C/C++/C#/Java/JavaScript/Rust/Swift/Go) all seem very similar syntactically compared to Haskell, Erlang, Lisp, or Forth.
- deltasevennine 4y ago>If you take TypeScript and add pointers, you'd end up with something like Java, or maybe Go. He's not just talking about pointers. He's implicitly talking about allocation and free. If you want facilities similar to just pure pointers without memory management then you have this already with all high level languages like JS or python. In fact the garbage collectors in these languages work by counting the references (aka pointers) pointing to a primitive. Thus from this reasoning he's talking about more than just pointers.
- jmillikin 4y agoYou're talking about references. The OP and I are talking about pointers. TypeScript does not have pointers. Python technically does (`ctypes`), but 99.99% of Python code will never touch them. There are important differences between pointers and references, for example the ability to perform arithmetic on them.
- skribanto 4y agoWhat? Rust and typescript have basically the same syntax?
- pornel 4y agoTypeScript abstracts away details that systems programming usually wants/needs to precisely control. For example, in TypeScript you don't need to care whether a closure captures any variables or not, and whether these variables are primitive copyable values (like integers) or objects. It works all the same in TypeScript, but at the low level these are different cases, which can be implemented in different, incompatible ways with varying performance. When a closure doesn't capture anything, it can be simplified to a plain function pointer. If it captures some variables, it needs to carry the data context, and that requires a (pointer, data) pair that has a different representation in memory. If the data it carries can't be trivially copied, then it also needs extra code inserted to keep this data allocated and freed at appropriate time. TypeScript makes all these 3 cases invisible to you, so it needs no syntax for each of these, but it comes at a run-time cost (a closure may end up represented in its fattest, costliest representation, and you never get the faster/leaner representation). There's a trope of a Sufficiently Smart Compiler that could in theory figure all these details out for you, and in this case see if your closures never use context, and use the cheaper pointer-only representation instead of always using a costlier object representation. The problem with this is that nobody has written a Sufficiently Smart Compiler yet. These smarts end up needing to analyze the whole program, which at very least is pretty slow to do, and in some cases is even provably impossible to do when program's flow depends on run-time/unknown values. So the best solution we have is to offer more syntax to the programmer to say exactly what they want.
- grumblingdev 4y agoAnd the syntax could be backwards compatible TypeScript is what I am saying, instead of an entirely new syntax, package manager, etc.
- pornel 4y agoTypeScript's syntax as it exists today wouldn't be usable. TS's type system isn't sound — a valid TS program may still have type errors. This is not a big deal for a dynamic JS VM, but in natively-compiled code it can cause crashes and security vulnerabilities. TS doesn't have syntax for value vs reference types or memory ownership/lifetimes. It relies on a garbage collector to "guess" the information missing from the syntax, but systems programming languages almost by definition do not want to do depend on this. TS's syntax for immutability is almost non-existent. There's no syntax for compile-time vs run-time evaluation distinction. You may be severely overestimating how much it would help you to know syntax of a system's programming language without knowing systems programming (very little, because syntax is the easy part). At best TS's syntax could used as a starting point for a new syntax of a new language, not compatible with TypeScript in any useful sense, with changes and extensions for all the features that low-level programming needs. But if you kept the syntax similar to TS, then new features may end up looking bolted on and forced to use less convenient syntax, since all the most convenient syntax is already taken by TS. This is why Objective-C looks so bizarre — it tried to be a superset of C, but C features already took all the nice syntax. C++ also suffered a lot by trying to be a syntax superset of C and older versions of itself. Building on a wrong syntax as a base leads to poor defaults, confusing recycling of old keywords, and traps caused by having convenient syntax for features you're not supposed to be using any more.
- SAI_Peregrinus 4y agoSystems languages (those usable in a "freestanding" environment with no OS or other runtime) are usually only about as complex as the underlying system. I'm thinking of languages like C (without the bits of libc that allocate), Rust (in `#![no_std]` mode), FORTH, and assembly. TypeScript is far more complex but that complexity is hidden from the user. A garbage collector is an extremely powerful abstraction, and makes a language much easier to use for tasks that don't need to deal with memory management in a precise manner. The problem is that systems languages (as opposed to applications languages like Java, Go, or Typescript) are used precisely where you need to deal with memory management in a precise manner.
- grumblingdev 4y ago> systems languages (as opposed to applications languages We shouldn't need such a wide separation between what is system vs application. The difference is essentially a few additional keywords and a compiler. But the simplicity comes from a high-level language, so you always need the high-level language. You only need the low-level language when you need to optimize something. So naturally, the high-level language should be the basis for a language that can do both. TypeScript is now well-established enough that its syntax should be used as this basis, and it natively targets the web which nothing else does.
- Sharlin 4y agoI'm not sure what measure of complexity you're thinking of. Are you familiar of the concepts of accidental and essential complexity? C is a very simple and conservative language; so simple that its simplicity has lead to all sorts of trouble. TypeScript, on the other hand, is a surprisingly complex language! Its type system is very advanced, and Javascript, and thus TypeScript too, has a lot of complexity these days, both accidental and essential. Indeed TypeScript would be a much simpler, cleaner language if it didn't have to be a source-compatible strict superset of JS. JS used to be a simple language, back in the late 90s! Zig tries to be C with minimal extra complexity added in order to make it a modern language that's nicer to program in. Rust tries to do that and in addition shield the programmer from the sort of mistakes that have cost the industry billions, possibly trillions of dollars, and that unfortunately entails adding some amount of essential complexity in the form of the ownership and lifetime system. One could argue that Rust's syntax is ugly, being a sort of chimera mixing C and ML heritage, but honestly, syntax is the least interesting, least important aspect of a programming language. C++, well, C++ actually is a hideously complicated beast. The reasons are manifold, but include path dependence, the need to maintain C compatibility, design by committee, and the fact that C++ users in the industry are a very heterogeneous bunch, and everybody wants to get their pet feature in.
- colangelod 4y agoSorry in advance if this post come across a bit harsh but I think there is a fair bit of misconception in your post. First and foremost I think it's safe to say that things like TypeScript and JavaScript would not be possible without the lower level "system" languages. In all fairness they were there first, by a matter of decades in most cases. Ultimately the V8 engine, Node and other similar stuff is largely built with C++ and C anyway. Fundamentally what you are asking about CAN be done, there is a larger, fairly academic discussion of "should" it be done or "why hasn't" it been done. "It's quite clear that TypeScript is going to be around forever" Really? Why? Type script is 9 years old, C is 50 this year. Sure C is still here but lots of things have come and gone in that time, small talk, lisp, fortran, cobal... (as long as you dont look TOO closely in some basement mainframes). Ive been at this for a while and the way I see it typescript is still in the late "fad" stage, it has yet to truly prove its staying power and there is nothing indicating with will "be around forever" just yet. Languages dont have real staying power until they last through the the subsequent fad, if people are still using it after the next cool thing comes out then its likely here to stay. "It's nice and simple." as others have noted, it's nice and "familiar" there are plenty of us out there who love C/C++ syntax and coding styles. I consider C to be one of the simplest languages out there. "A systems language essentially just adds access to pointers." Sure, but it also does LOTS of other things. If you want to make a vague comparison systems languages are closer to running "on the metal" than interpreted languages and for some use cases this this is critical. If your flying a jet at 35,000 feet you don't want your altitude tape to all of a sudden read NaN... In many cases, especially mission critical systems, programing close to the metal is the only way to achieve the sort of optimization you really need to run a system, abstraction is NOT always a good thing "Imagine the same syntax/tooling for embedded, kernel, system (browsers, interpreters), backend, and frontend." -> You could define a systems language that had similar syntax to JS but the main point of JS is the broad abstraction of some things that you very much want access to when you working on a system closer to the metal. Sure its nice when all numbers are of type "Number" but there are lots of cases where you need far stricter control over that. Another piece of historical context here is that the web era brought around the notion of a "full stack" developer, someone who could work on "front end" and "back end" code. In my experience the best dev's are good at one of the two and have a cursory understanding of the one they are not good at for the sake of knowing whats going on most of the time. Realistically these people are not tinkering with the "low level" aspects of the system since its all provided by other things. "I just don't know why there is not more interest in this." Perhaps because there is no real reason to sink the engineering effort into adding a new syntax to something that works and there are plenty of competent, educated engineers out there working on the stack already. There is lots of historical context here as well, JavaScript more or less became popular in the web era to manipulate front ends, Node made a viable server side platform so it was "one language up and down the stack" which made engineers easier to train (supposedly, I have yet to see this in practice). This was all great but thats a web centered view of the whole situation. In reality there are computers in EVERYTHING these days and many of those computers/microcontrollers/microprocessors simply can't run javascript (for what ever reason) and systems languages are quite relevant. To sum it all up I think your largely ignoring: 1) The huge historical contexts to why some languages are the way they are 2) Why some syntax/language designs lend to certain use cases 3) Why the "web" era made JS popular for better or worse 4) Why OOP is not a silver bullet for all computing use cases It took me the better part of the first decade of my career to understand that the purest developers/comp sci's fundamentally know that the language does not matter but instead understanding WHY a language/framework/library/system is good for the use case. Roller Coaster Tycoon was mostly written in assembly anyway...
- frazbin 4y agoNote to self, see how this post has aged in 5, 10, and 20 years
- swatcoder 4y agoWith TypeScript, I can mostly control the algorithmic complexity of my code, but I have almost no control over the actual code produced, what core runs it, the arrangement and alignment of memory, and generally can’t know the state of the host system I’m programming because all that stuff is abstracted away to make it well suited to application development. Some “higher level” languages can let you break through to those details, but it usually means losing the simplicity you find appealing. In contrast, it’s actually very simple and natural to work with all those concepts in C, C++, Rust, etc Different tools suit different tasks.
- unrealhoang 4y agoYou might want to check out assembly script: https://www.assemblyscript.org/ https://www.assemblyscript.org/ But apparently not enough people care for it to be popular, maybe because people prefer languages that are designed specifically for system programming than retrofitting typescript to write it.
- camgunz 4y agoGenerally the problem is that dealing w/ memory is hard. Either it's explicit like C and you've got a lot of memory management code, or it's implicit like Rust and you've got a lot of constraints on what you can do, or you're using GC and you're not a systems language. Integrating this idea, other ways of phrasing what you're asking is: - Why don't we add pointers, malloc, and free to TypeScript? - Why don't we add a borrow checker to TypeScript? It might seem simple to just add "access to pointers", but the totality of what this means is you have to define semantics for allocation. That means things like .map, .filter, .foreach, the spread operator, and so on. From the TypeScript point of view, that would gum up the works a lot. Or you could add a borrow checker, but that would invalidate... maybe every existing non-trivial TypeScript program. --- Secondly, you're asking for an entirely different backend infrastructure. TypeScript compiles to JavaScript, systems languages compile to object code. There would be significant complexity converting all the features of TypeScript into object code, generators and async/await come to mind. --- TL;DR you really can't separate a language's features from the rest of its design. It influences everything from semantics to syntax, and designers carefully balance all this based on the problems they assume their users are trying to solve.
- grumblingdev 4y agoI think AssemblyScript is the best example. Adding the borrow checker is quite invasive though. This guy is trying https://github.com/alshdavid/BorrowScript https://github.com/alshdavid/BorrowScript. I think it's a kind of fun constraint that experienced and bored devs like to challenge themselves with - the borrow checker. The latest obsession. You absolutely don't need a borrow checker, just like you didn't need everything to be functional programming, but it's intellectually stimulating.
- AnimalMuppet 4y agoRust syntax is a bit more refined than just "a new convoluted syntax that a single dev happened to like." You can't just add pointers to TypeScript to create a systems language. You also have to remove garbage collection. (Because how are you going to do hard real time, and write custom memory allocators, with a garbage collector in the way?) And then you have to do that in a consistent way, and preferably in a way that doesn't invite the programmer to make mistakes. (Introducing pointers introduces a lot of new ways to make mistakes. A good systems language should help the programmer not make them.) This leaves you needing to do a lot more modifications to the language than just introducing pointers.
- bediger4000 4y agoI'm not sure this is generally true. The company Symbolics used Lisp as their systems language, for example.
- mamcx 4y ago> Complex is better than complicated (Zen of Python) Complexity, on itself, is not bad. Something complex CAN be understood with adequate application of study and practique, ie: A intelectual matter, that is in fact easier for humans. "Complicated" instead is a mastery of moral fortitude: Patience, dedication, patience, hard work, resilience to many failed attempts, patience... and that is much harder for humans! --- Exist "accidental complexity" that is unfortunate (most/some of C++ is like that!) and "intentional complexity" that is ok (the Rust borrow checker is that). If the complexity is designed, then is easier to teach and must have a solid reason to be, so, is easier to accept (that is part of why Rust is loved: Yes, is hard, but we know why is hard and is clear why is an acceptable trade-off). --- In the other hand, Rust/C/C++/Zig have a theme: Are part of a, lets say, "design school" that are prone to do amazing local optimization but not very well on the whole simplicity, "the C school of things"(?) Pascal (and few others) are actual system languages with overall simplicity as goal, without a better name, of the "design school of Nicolas". So, you CAN get simpler yet performant/system languages but the industry is dominated by a "design" camp that in part just copy each other and don't see when are introducing, yet again, another accidental complexity. Luckily some of this forgotten "design school" is being rediscovered, because too much complexity have proven to be too much! so in the case of late entries (Rust, Zig, Go...) you actually see some push toward simplicity, so is not like nobody is pushing for it, is that not everybody (aka: Meaning the who design them) push for simplicity as a major goal. To make a example, in Rust the use of structs/algebraic enums/traits IS a show of simplicity.
- Jtsummers 4y agoThe Pascal line, which includes Oberon (a systems language with garbage collection included), is usually termed "Wirthian".
- grumblingdev 4y agoI just wish the creator of Rust had said "I'm going to add a native compilation target and the minimal new syntax to support borrow checking to TypeScript and make interop extremely easy". Imagine that! We would have one syntax that can potentially be used for everything. A Rust-like syntax will never be used to write web apps. Then you have simplicity with opt-in designed complexity when needed. It's like the separation of beginner vs expert features in design.
- yupis 4y agoAll "high abstractions" programing languages comes with the costs of computing such abstractions which comes from either: compiler complexity and/or CPU usage. All programing languages will output the compiled code to CPU instructions. And the CPU is indeed "imperative" in nature. CPU works using simple logic gates. At the moment with the current knowledge the best performance we can get is with low level programing languages that can directly talk to the CPU. That's why such low level programming languages have been created this way.
- grumblingdev 4y agoI just find I write code in a syntax that could probably be transpiled directly to C/C++ if I followed a few rules, and that it could also run on the web or in a JIT. A dev shouldn't have to switch between syntaxes to write native code.
- valbaca 4y agoTS compiles to JS, JS runs on a runtime (V8, Deno, Bun) Systems languages need to be able to run on systems that literally cannot afford any overhead of an additional runtime. They need to run on systems that do not even have an operating system or more than a few kb of RAM. > It's quite clear that TypeScript is going to be around forever. Wow. "Forever" is quite the claim. > add a few constraints to make it low-level. This is...inaccurate.
- grumblingdev 4y agoTS would compile to assembly. AssemblyScript already compiles to web assembly as an example.
- vivegi 4y agoA systems programming language operates much closer to the machine level than other languages. One could compile source files into .o (object files), call functions implemented in any object file using its calling convention and link them using a linker. This ability to use an established toolchain is useful when components that one writes has to communicate with components that others have written which occurs quite often in the systems programming area. Sometimes, you need to peel away the abstractions and work with the raw memory layout or when working in real-time/embedded systems you cannot rely on garbage collectors because your code needs to provide runtime guarantees.
- pestatije 4y agoNot true. Node and TypeScript are not nice and simple. As a rough comparison the GNU C Reference manual ticks at 366Kb whereas the TypeScript Handbook does at 663Kb. https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.html https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.html https://www.typescriptlang.org/assets/typescript-handbook.pdf https://www.typescriptlang.org/assets/typescript-handbook.pd...
- Rochus 4y agoWell, Oberon is in fact a system language and definitely much simpler than TypeScript. Just have a look at the specification size (16 pages vs 180 pages).