5 ms·

> Nor is this all you can do with macros; it's just one region in a space of program-manipulating techniques that even now is far from fully explored. I like m

> Nor is this all you can do with macros; it's just one region in a space of program-manipulating techniques that even now is far from fully explored. I like macros. I find myself, not infrequently, thinking about where they would be convenient despite using a language without them. Nevertheless none of the languages that include them (that I’m aware of) seem very appealing. Part of this is “program-manipulating techniques”. Less powerful languages are easier for programs to reason statically about, and so usually have great editor refactoring / navigation / static analysis already available. This makes the 99.5% of programming that is mundane more pleasant, and it’s hard to imagine special programs that don’t involve at least a good chunk of this. The alternative to macros is not never doing metaprogramming, you can do code generation, parse and process the AST, or have program behavior be data-directed at runtime up to including an embedded interpreter. These things are surely less convenient than macros, but it’s a trade off against making all the other parts of the program less convenient to write. Another issue is most languages with macros are Lisp/Scheme types, and having linked-lists-of-boxed-objects be the default representation for everything often isn’t great for domains that are performance-sensitive (this probably wouldn’t apply to some languages like Nim). Of course there are the other downsides of using uncommon languages, like often less mature runtimes / library ecosystems. It seems technically feasible for a language with macros to address all these issues (and maybe Rust does if you’re willing to give up GC). If I was writing a new language I would probably start by compiling to either the JVM/.NET/Go to get a runtime and libraries, and focus on the problem of incremental language extension while maintaining good editor and static analysis support (I think there are some promising approaches for that but this comment is already pretty long).

brabel 5y ago
> Another issue is most languages with macros are Lisp/Scheme types, Rust, Julia and Zig (if you consider comptime to be a specific type of macros) don't look Lispy to me but all have macros. In Rust I would consider them essential to the language (don't know the others well enough to say), without them Rust would be terribly verbose. > having linked-lists-of-boxed-objects be the default representation for everything often isn’t great for domains that are performance-sensitive Disagree, there are extremely performant Lisps out there, including Common Lisp and Chez Scheme. They run as fast as, or faster than, any language that's not a systems-language (C, C++, Rust, Zig).
- wnoise 5y ago
  Julia is a Dylan is a Lisp.
- G3rn0ti 5y ago
  > there are extremely performant Lisps out there Yes, that’s because CommonLisp also provides native high performance data structures like vectors, arrays and hash tables that complement basic linked lists. Furthermore, most Lisp compilers are pretty mature and create well-optimized x86 code. You can annotate your code with type hints to help and guide the compiler in hot spots of your program.
- throwaway34241 5y ago
  > Rust, Julia and Zig […] don’t look Lispy to me Sure, I even mentioned both Nim and Rust (Zig’s comptime is interesting but I wouldn’t put it on the same level). It just seems like most language implementations with macros are Lisp variants (although which ones count as different languages is somewhat fuzzy). > there are extremely performant Lisps out there, including Common Lisp and Chez Scheme If you look at for example the benchmarks game [1], Lisp puts in an extremely impressive showing relative to Python/Ruby but often lags slightly Java/C#/Go. But more importantly, the Lisp code is often very non-idiomatic [2], with lots of special annotations and even turning safety features off. Maybe it’s acceptable for optimizing very narrow hot spots, but I wouldn’t want to write a lot of code in this style and wouldn’t expect most third party libraries to be written that way. I don’t think any of this is the fault of Lisp implementers, but more that the basically default (although not only) data type provides the semantics of dynamically typed linked lists, which are inherently (relatively) slow and tricky to optimize. And even if very clever optimizations exist, relying on them to get adequate performance is less than ideal, since it’s often not obvious where you fall off the fast path. Of course many applications aren’t performance sensitive enough for these sorts of things to matter much, I’m thinking about 3D games etc here where you usually want control of memory layout. [1] https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/nbody.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... [2] https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/mandelbrot-sbcl-1.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
mariusor 5y ago
Maybe you'd be interested in looking at the way Jonathan Blow implemented macros in his programming language. The video is quite long (and has a Q&A accompanying followup), but I think just the example of creating custom iterators with macros is really interesting. https://www.youtube.com/watch?v=QX46eLqq1ps https://www.youtube.com/watch?v=QX46eLqq1ps
- throwaway34241 5y ago
  Jai seems pretty interesting, I’m looking forward to trying it out once a public beta is available! Not the least because there’ll hopefully be a fair amount of batteries already included for 3D graphics / math stuff. In general I really like the approach of designing a language in conjunction with implementing a large project (other than the language itself).
mark_l_watson 5y ago
Sort of like how the Hy language (hylang) compiles a Lisp dialect into a Python AST?
- throwaway34241 5y ago
  Yes, like that! Or like Clojure. If I was targeting the JVM or CLR I’d probably use bytecode, or for Go I’d just generate ordinary source code to feed into the compiler. It sounds a little ungainly but parsing shouldn’t be a big percent of compile times, so the overhead shouldn’t be that big. And it addresses two of the biggest issues, libraries and (current and future) platform support, things that new language adopters should legitimately be concerned about. A lot of the most popular mobile games actually now use a similar technology. The game engine Unity converts the bytecode for game C# scripts to C++ which it can feed into the system toolchain. Even as a billion dollar company they decided it was more practical to piggyback on another language for platform support (in their case game consoles, which come with their own C++ compiler but may not be supported out of the box by other language toolchains).

brabel 5y ago

> Another issue is most languages with macros are Lisp/Scheme types, Rust, Julia and Zig (if you consider comptime to be a specific type of macros) don't look Lispy to me but all have macros. In Rust I would consider them essential to the language (don't know the others well enough to say), without them Rust would be terribly verbose. > having linked-lists-of-boxed-objects be the default representation for everything often isn’t great for domains that are performance-sensitive Disagree, there are extremely performant Lisps out there, including Common Lisp and Chez Scheme. They run as fast as, or faster than, any language that's not a systems-language (C, C++, Rust, Zig).

wnoise 5y ago
Julia is a Dylan is a Lisp.
G3rn0ti 5y ago
> there are extremely performant Lisps out there Yes, that’s because CommonLisp also provides native high performance data structures like vectors, arrays and hash tables that complement basic linked lists. Furthermore, most Lisp compilers are pretty mature and create well-optimized x86 code. You can annotate your code with type hints to help and guide the compiler in hot spots of your program.
throwaway34241 5y ago
> Rust, Julia and Zig […] don’t look Lispy to me Sure, I even mentioned both Nim and Rust (Zig’s comptime is interesting but I wouldn’t put it on the same level). It just seems like most language implementations with macros are Lisp variants (although which ones count as different languages is somewhat fuzzy). > there are extremely performant Lisps out there, including Common Lisp and Chez Scheme If you look at for example the benchmarks game [1], Lisp puts in an extremely impressive showing relative to Python/Ruby but often lags slightly Java/C#/Go. But more importantly, the Lisp code is often very non-idiomatic [2], with lots of special annotations and even turning safety features off. Maybe it’s acceptable for optimizing very narrow hot spots, but I wouldn’t want to write a lot of code in this style and wouldn’t expect most third party libraries to be written that way. I don’t think any of this is the fault of Lisp implementers, but more that the basically default (although not only) data type provides the semantics of dynamically typed linked lists, which are inherently (relatively) slow and tricky to optimize. And even if very clever optimizations exist, relying on them to get adequate performance is less than ideal, since it’s often not obvious where you fall off the fast path. Of course many applications aren’t performance sensitive enough for these sorts of things to matter much, I’m thinking about 3D games etc here where you usually want control of memory layout. [1] https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/nbody.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... [2] https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/mandelbrot-sbcl-1.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

mariusor 5y ago

Maybe you'd be interested in looking at the way Jonathan Blow implemented macros in his programming language. The video is quite long (and has a Q&A accompanying followup), but I think just the example of creating custom iterators with macros is really interesting. https://www.youtube.com/watch?v=QX46eLqq1ps https://www.youtube.com/watch?v=QX46eLqq1ps

throwaway34241 5y ago
Jai seems pretty interesting, I’m looking forward to trying it out once a public beta is available! Not the least because there’ll hopefully be a fair amount of batteries already included for 3D graphics / math stuff. In general I really like the approach of designing a language in conjunction with implementing a large project (other than the language itself).

mark_l_watson 5y ago

Sort of like how the Hy language (hylang) compiles a Lisp dialect into a Python AST?

throwaway34241 5y ago
Yes, like that! Or like Clojure. If I was targeting the JVM or CLR I’d probably use bytecode, or for Go I’d just generate ordinary source code to feed into the compiler. It sounds a little ungainly but parsing shouldn’t be a big percent of compile times, so the overhead shouldn’t be that big. And it addresses two of the biggest issues, libraries and (current and future) platform support, things that new language adopters should legitimately be concerned about. A lot of the most popular mobile games actually now use a similar technology. The game engine Unity converts the bytecode for game C# scripts to C++ which it can feed into the system toolchain. Even as a billion dollar company they decided it was more practical to piggyback on another language for platform support (in their case game consoles, which come with their own C++ compiler but may not be supported out of the box by other language toolchains).