3 ms·

>> Many sub-languages to learn, many with different syntaxes and semantics. For example: >> >> the expression language >> unsafe runtime language >

>> Many sub-languages to learn, many with different syntaxes and semantics. For example: >> >> the expression language >> unsafe runtime language >> safe runtime language >> compile time language >> the type language >> the trait language >> the macro language >> the attribute language I've been wanting to ask Rust experts about this..... to what extent is this really necessary? Is it possible to create a language with all the memory and thread safety of Rust, based on a single language with no sub languages?

atoav 4y ago
I am not a expert, but I wrote things I am happy with and I think it is a little over the top. What is called the type language, the trait language and the attribute language here are just one thing for example. Those are not necessarily sub languages, they are different parts of the same thing. The hard thing is not the syntax or the way you write it, it is the concepts behind it. The hard things about Rust are hard, because they are novel concepts that solve existing, hard problems in often very intelligent ways, while still upholding the guarantuees the language gives you. Granted, in C there would be less language to learn (although getting the details about value conversion right can be hard) — but using the language at a level where you'd be able to utilize it safely at a level like that would take a few decades as well. Because now you have to uphold the guarantees. C has no concept of ownership and lifetimes for example, but if you use your pointers without having a mental concept of ownership and lifetimes you are going to introduce the next exploitable zero day. That being said: You can also happily program without ever touching half of that list and you would still end up with programs that are fast, maintainable and compile fine. In fact I'd argue unless you are building some hardcore library where you have to do things in optimal ways while still getting the ergonomics you'd be totally fine with e.g. avoiding unsafe, macros etc. The language is flexible in that way, you can go extremely deep or stay on the surface (or even do both in different parts of your program at the same time). The type system is something that you need to get used to if you come from dynamic languages, but once you grasp it, the guarantuees it provides really simplify a lot of things in your head. E.g. typically when a compile fails I would know right away where to look and how to fix it, even in big bespoke programs. It prevents a whole class of bugs that every programmer makes once in a while. But type systems are old programming concepts. Sure there is a certain way it is written and used in Rust, but the hard thing again is the concepts underneath (e.g. how a type system makes your variables map to memory and what cool tricks you can do with that).
- iopq 4y ago
  The macro language is legitimately ugly and hard to read. I can spit out a macro definition, but later I will have difficulty understanding it
nindalf 4y ago
Maybe it is. But each of these perform a role that can’t be easily removed. - unsafe. You could make a language that didn’t allow third parties to write unsafe code. Except there are a few cases where it’s necessary. Me personally, I’ve never written any unsafe but I have relied on well written, vetted implementations written by others (like the standard library Vec). - compile time language. A language either needs run time reflection or it needs compile time meta programming. Rust chooses the bare minimum for a runtime so there are no reflection APIs. That said, there probably is space for more ergonomic meta programming APIs. It still wouldn’t look like regular Rust, but it could be easier to read. I don’t know what they mean by expression language. And I thought the type and trait language is the same? Calling these “languages” seems excessive to me, but sure. I think it’s almost certainly possible to improve on Rust, either in place or by creating a new language that makes different trade-offs. Maybe that one would provide run time reflection, for instance.
- ngrilly 4y ago
  I guess that by “expressions language” the author means the language used for expressions and statements (the latter also being expressions in Rust).
  - nindalf 4y ago
    Yeah, I would just call that Rust I guess.
bjz_ 4y ago
I think it's more a reflection of how Rust evolved, and the techniques and approaches known and understood at the time and the strangeness budget they were (understandably) willing to take on at the time as opposed to something inherent. And also sometimes having separate, complicated features for similar things (as opposed to simple, generalised features that compose powerfully) can be useful pedagogically as well. At any rate, this is something I'm personally interested in based on my experience working with Rust over the last decade, and so that's why it appears so high up on my list. Often you really do want sub-languages for different purposes, but managing how they interact and work together, what is the same and what is different, and how that impacts usability is interesting (and difficult) part. I feel like it should be possible to do this, but it's going to take some work and there's still lots of unknowns. In technical terms, I'm interested in dependently typed module systems, multistage programming[1], graded modal type theory[2], elaborator reflection, and two level type theory[3]. These all sound pretty intimidating, but you can actually see glimmers of some of this stuff in how Zig handles type parameters and modules, for example, something that most programmers really like the first time they see it! I do feel like there is the core of a simple, flexible, powerful systems language out there... but finding it, and making it approachable while maintaining a solid footing in the theory and being sensitive to the practical demands of systems programming is a nontrivial task, and many people will be understandably skeptical that this is even a good direction to pursue. Thankfully the barrier to entry for programming language designers to implementing languages in this style has reduced significantly in just the last number of years[4], so I have hope that we might see some interesting stuff in the coming decade or so. In the meantime we have Rust as well, which is still an excellent language. I'm just one of those people who's never content with the status quo, always wishing we can push the state of the art further. This is why I got excited by Rust in the first place! :) [1]: https://github.com/metaocaml/metaocaml-bibliography https://github.com/metaocaml/metaocaml-bibliography [2]: https://granule-project.github.io/ https://granule-project.github.io/ [3]: https://github.com/AndrasKovacs/staged https://github.com/AndrasKovacs/staged [4]: https://github.com/AndrasKovacs/elaboration-zoo/ https://github.com/AndrasKovacs/elaboration-zoo/
- throwaway17_17 4y ago
  I think you share a lot of my interests on the theory side of things. I also share your belief that those areas probably can allow for the development of the core of systems level programming language. Of course, I tend to find that graded, or the more general contextual modal type theory subsumes multi-staged, two level and grade modal theories. I also think a broad spectrum dependently typed language (with modal enforcement of computational irrelevance) can take care of dependent modules. I think you are also spot on about Rust having a strangeness budget and that could be responsible for the syntactic state of the language as it exists today. I have a much higher tolerance for non-conventional syntax so almost all of my type theory implementation and PL work has been outside the normal syntactic bounds for the last 2 years. I doubt I ever produce a language that is public, let alone a language that is used by any portion of the software engineering field. But my belief is that this arena is fertile ground for a more fundamental core, like you mentioned. As an aside, I happened to stumble upon Andras’ video presentations on YouTube on Saturday and flagged them to watch and bookmarked the repo for them earlier today. So bravo on linking what look to be really nice resources for this area of work.
api 4y ago
Of all those the only one that bugs me is macros. They are legitimately alien to the Rust language and remind me of Perl (a.k.a. Herl). I do understand why macros are a sub-language because they're doing something totally unlike what the main language does, but could the syntax have been less fugly?
pornel 4y ago
The micro-syntax in attributes could have been avoided. There isn't any technical reason why: #[cfg(all(feature = "a", target = "b"))] couldn't look more like the rest of Rust: #{ has_feature("a") && is_target("b") } I'm not sure if declarative macros would have been better without a custom syntax. Procedural macros are just regular Rust, and they're verbose and hard to follow. In this case languages that aren't LISP just have to accept some trade-offs. In case of safe Rust vs unsafe Rust vs compile-time Rust, I think they're fine. They're still using the same language with the same syntax. The specifics of what is allowed in safe/unsafe/const are different, but that's the point of these features.

atoav 4y ago

I am not a expert, but I wrote things I am happy with and I think it is a little over the top. What is called the type language, the trait language and the attribute language here are just one thing for example. Those are not necessarily sub languages, they are different parts of the same thing. The hard thing is not the syntax or the way you write it, it is the concepts behind it. The hard things about Rust are hard, because they are novel concepts that solve existing, hard problems in often very intelligent ways, while still upholding the guarantuees the language gives you. Granted, in C there would be less language to learn (although getting the details about value conversion right can be hard) — but using the language at a level where you'd be able to utilize it safely at a level like that would take a few decades as well. Because now you have to uphold the guarantees. C has no concept of ownership and lifetimes for example, but if you use your pointers without having a mental concept of ownership and lifetimes you are going to introduce the next exploitable zero day. That being said: You can also happily program without ever touching half of that list and you would still end up with programs that are fast, maintainable and compile fine. In fact I'd argue unless you are building some hardcore library where you have to do things in optimal ways while still getting the ergonomics you'd be totally fine with e.g. avoiding unsafe, macros etc. The language is flexible in that way, you can go extremely deep or stay on the surface (or even do both in different parts of your program at the same time). The type system is something that you need to get used to if you come from dynamic languages, but once you grasp it, the guarantuees it provides really simplify a lot of things in your head. E.g. typically when a compile fails I would know right away where to look and how to fix it, even in big bespoke programs. It prevents a whole class of bugs that every programmer makes once in a while. But type systems are old programming concepts. Sure there is a certain way it is written and used in Rust, but the hard thing again is the concepts underneath (e.g. how a type system makes your variables map to memory and what cool tricks you can do with that).

iopq 4y ago
The macro language is legitimately ugly and hard to read. I can spit out a macro definition, but later I will have difficulty understanding it

nindalf 4y ago

Maybe it is. But each of these perform a role that can’t be easily removed. - unsafe. You could make a language that didn’t allow third parties to write unsafe code. Except there are a few cases where it’s necessary. Me personally, I’ve never written any unsafe but I have relied on well written, vetted implementations written by others (like the standard library Vec). - compile time language. A language either needs run time reflection or it needs compile time meta programming. Rust chooses the bare minimum for a runtime so there are no reflection APIs. That said, there probably is space for more ergonomic meta programming APIs. It still wouldn’t look like regular Rust, but it could be easier to read. I don’t know what they mean by expression language. And I thought the type and trait language is the same? Calling these “languages” seems excessive to me, but sure. I think it’s almost certainly possible to improve on Rust, either in place or by creating a new language that makes different trade-offs. Maybe that one would provide run time reflection, for instance.

ngrilly 4y ago
I guess that by “expressions language” the author means the language used for expressions and statements (the latter also being expressions in Rust).
- nindalf 4y ago
  Yeah, I would just call that Rust I guess.

bjz_ 4y ago

I think it's more a reflection of how Rust evolved, and the techniques and approaches known and understood at the time and the strangeness budget they were (understandably) willing to take on at the time as opposed to something inherent. And also sometimes having separate, complicated features for similar things (as opposed to simple, generalised features that compose powerfully) can be useful pedagogically as well. At any rate, this is something I'm personally interested in based on my experience working with Rust over the last decade, and so that's why it appears so high up on my list. Often you really do want sub-languages for different purposes, but managing how they interact and work together, what is the same and what is different, and how that impacts usability is interesting (and difficult) part. I feel like it should be possible to do this, but it's going to take some work and there's still lots of unknowns. In technical terms, I'm interested in dependently typed module systems, multistage programming[1], graded modal type theory[2], elaborator reflection, and two level type theory[3]. These all sound pretty intimidating, but you can actually see glimmers of some of this stuff in how Zig handles type parameters and modules, for example, something that most programmers really like the first time they see it! I do feel like there is the core of a simple, flexible, powerful systems language out there... but finding it, and making it approachable while maintaining a solid footing in the theory and being sensitive to the practical demands of systems programming is a nontrivial task, and many people will be understandably skeptical that this is even a good direction to pursue. Thankfully the barrier to entry for programming language designers to implementing languages in this style has reduced significantly in just the last number of years[4], so I have hope that we might see some interesting stuff in the coming decade or so. In the meantime we have Rust as well, which is still an excellent language. I'm just one of those people who's never content with the status quo, always wishing we can push the state of the art further. This is why I got excited by Rust in the first place! :) [1]: https://github.com/metaocaml/metaocaml-bibliography https://github.com/metaocaml/metaocaml-bibliography [2]: https://granule-project.github.io/ https://granule-project.github.io/ [3]: https://github.com/AndrasKovacs/staged https://github.com/AndrasKovacs/staged [4]: https://github.com/AndrasKovacs/elaboration-zoo/ https://github.com/AndrasKovacs/elaboration-zoo/

throwaway17_17 4y ago
I think you share a lot of my interests on the theory side of things. I also share your belief that those areas probably can allow for the development of the core of systems level programming language. Of course, I tend to find that graded, or the more general contextual modal type theory subsumes multi-staged, two level and grade modal theories. I also think a broad spectrum dependently typed language (with modal enforcement of computational irrelevance) can take care of dependent modules. I think you are also spot on about Rust having a strangeness budget and that could be responsible for the syntactic state of the language as it exists today. I have a much higher tolerance for non-conventional syntax so almost all of my type theory implementation and PL work has been outside the normal syntactic bounds for the last 2 years. I doubt I ever produce a language that is public, let alone a language that is used by any portion of the software engineering field. But my belief is that this arena is fertile ground for a more fundamental core, like you mentioned. As an aside, I happened to stumble upon Andras’ video presentations on YouTube on Saturday and flagged them to watch and bookmarked the repo for them earlier today. So bravo on linking what look to be really nice resources for this area of work.

api 4y ago

Of all those the only one that bugs me is macros. They are legitimately alien to the Rust language and remind me of Perl (a.k.a. Herl). I do understand why macros are a sub-language because they're doing something totally unlike what the main language does, but could the syntax have been less fugly?

pornel 4y ago

The micro-syntax in attributes could have been avoided. There isn't any technical reason why: #[cfg(all(feature = "a", target = "b"))] couldn't look more like the rest of Rust: #{ has_feature("a") && is_target("b") } I'm not sure if declarative macros would have been better without a custom syntax. Procedural macros are just regular Rust, and they're verbose and hard to follow. In this case languages that aren't LISP just have to accept some trade-offs. In case of safe Rust vs unsafe Rust vs compile-time Rust, I think they're fine. They're still using the same language with the same syntax. The specifics of what is allowed in safe/unsafe/const are different, but that's the point of these features.