5 ms·

Show HN: es6_maps, new Elixir syntax feature via runtime compiler hacking

71 points by kzemek 2y ago

Hi HN! I thought you might enjoy this even if you don't know much about Elixir. `es6_maps` is a small library that introduces a "shorthand" map creation syntax, similar to shorthand object construction in JavaScript/TypeScript. For those unfamiliar with Elixir, the most interesting aspect might be how it achieves this. `es6_maps` takes advantage of BEAM (Erlang's VM) hot-reload capabilities to amend the Elixir compiler bytecode at runtime, adding functions that expand the shorthand syntax for further compilation. I think it's a nice showcase of the power you can wield (with care) when writing in BEAM languages. Let me know what you think!

arrowsmith 2y ago
This is great! I've always thought that if I could change one thing about Elixir, it would be to add exactly this shorthand syntax that I'm used to from JS. I'm sick of having to type everything twice, like e.g. `%{foobar: foobar}`. I'm impressed to see it implemented. I didn't know it was possible to pass custom compilers to your Mix project definition - but now that I think about it, I can see how Elixir's metaprogramming features would lend themselves well to implementing custom syntax like this. Fantastic work, all I want to know is why Elixir doesn't have this syntax built-in already.
- dudul 2y ago
  > I'm sick of having to type everything twice, like e.g. `%{foobar: foobar}` I'm actually very confused by this use case. How often do you have to do that? Fill a map with kv tuples where keys and values have the exact same value?
  - cmoski 2y ago
    the key is an atom called foobar, and the value is being extracted to a variable called foobar. The variable's value is not shown in the example. You do this quite a bit in Elixir, unless you use the . syntax, e.g. somemap.foobar which is marginally slower.
    - dudul 2y ago
      Got it - I misunderstood the short code sample, I thought the 2nd half of the tuple was a literal instead of a variable. Makes sense.
  - gmassman 2y ago
    This pattern appears frequently in just about all programming languages. I’ve written a lot of JS and Python, and a moderate amount of Elixir, and this pattern crops up quite often. Usually it’s a side effect of complex scopes with several layers of function calls. Honestly I’m a little surprised you haven’t seen this pattern. The only times it wouldn’t be used are when people rename variables, which is frankly a practice I abhor.
    - leononame 2y ago
      JS and python are just about all programming languages? I'm not trying to be snarky, but e.g. I haven't seen it in Go, F# and Rust, some languages where I have at least seen _some_ code. I also don't remember a similar pattern in Java, C# and C nor PHP or Dart, but I will readily admit that it's I haven't coded in any of these in the last two years, so I might be a bit out of touch and I'm not super proficient in all those languages
      - boxed 2y ago
        Go, and F# do almost all positional arguments. I think Rust too. Java, C#, C, PHP are also positional only or 99% of the time. I assume Dart too. We are talking named/keyword/etc. Personally I find positional arguments to be an anti-pattern in all but a very specific set of circumstances.
        debugnik 2y ago
        C#, F# and PHP all support named arguments and they're quite common for optional parameters.
        leononame 2y ago
        If I understand correctly, we aren't talking about named/keyword arguments. This is about a shorthand syntax in ES6 objects, nothing to do with function arguments. In JS, when creating an object, instead of writing {name: name}, where "name" is both the key in the map as well as the variable you want it to assign to, you can do {name}, whereas e.g. in Go you would do map[string]any{"name": name}. Edit: Or, for completeness' sake, it's the same with structs in Go, where you would instantiate a struct like this: Person{name: name}.
  - cess11 2y ago
    The use of : in 'foobar: foobar' here is syntax sugar for :foobar, foobar where :foobar is a symbol that evaluates to itself and foobar evaluates to the value the variable points to.
cmoski 2y ago
Does it work for nested maps?
- kzemek 2y ago
  Yes it does!
jzimbel 2y ago
This was done previously with the `short_maps` library, which the author—a member of the Elixir core team—eventually retired and archived due to regrets from the undesirable impacts on clarity. https://github.com/whatyouhide/short_maps https://github.com/whatyouhide/short_maps https://andrealeopardi.com/posts/a-story-of-regret-and-retiring-a-library-from-hex/ https://andrealeopardi.com/posts/a-story-of-regret-and-retir...
- MrJohz 2y ago
  That's quite surprising to me. As someone who writes a lot of Javascript, I usually find it easiest to read with this syntax than without it, and I generally recommend to my team that they use the shorthand where they can. This is because the meaning is clear and well-known, and being concise (while still being clear) is very valuable. I wonder what specifically the original author found made it more difficult to read. Was it unfamiliarity? They allude to some confusion between sigils and strings, so I wonder if the issue was partly an Elixir-specific one.
  - igsomething 2y ago
    The map '%{username, age, first_name, last_name}' would be 1 character away from the tuple '{username, age, first_name, last_name}'. It is easy to miss the '%' character and you'll waste some time figuring out why your code is throwing match and/or function clause errors at runtime.
    - Dkuku 2y ago
      Using this logic [username, age first_name, last_name] is only 2 characters away. This example is easy to catch because tuple and map work completly different-it's not python.
- aarongraham 2y ago
  I really don’t understand why some people are so against this as a language feature. I get that the community tries to stay away from “magic” but where is the line between magic and convenient syntactic sugar. For example keyword lists as the last argument to a function don’t need brackets, and keyword lists are really lists of tuples where the first item is an atom. I feel like those two things are in some ways less obvious and more magical than this map shorthand.
rubyn00bie 2y ago
I personally don’t think this is a good idea, and I think the formatter plugin is going to give someone insane amounts of regret one day. It’s neat as “look what I did” but Elixir is all about removing magic. It (lack of magic) is one of the things I appreciated most, having originally come from Ruby and Rails. I’ve been using it professionally for nearly ten years, and I can say this isn’t a feature I’ve ever really wanted. My text editor can save the typing via shortcuts I type, and in doing so supports string keys, atom keys, has zero compile time dependencies, and no new syntax ;) I’d really encourage you to put a note at the top of the readme that this shouldn’t be used in production code.
- karmajunkie 2y ago
  i don’t like adding foundational syntax using libraries, but i do wish they would add this to the language. deconstruction by pattern is super common and the syntactic sugar would be most welcome.
- kzemek 2y ago
  > I think the formatter plugin is going to give someone insane amounts of regret one day The formatter plugin features a "reverse" flag exactly to prevent any kind of regret like that. You can reformat your code automatically to remove all shorthand maps and remove the dependency in a couple simple steps. > I’d really encourage you to put a note at the top of the readme that this shouldn’t be used in production code. Noted, although I do think it's production ready. I'm going to keep updating the library if or when a new version of the Elixir compiler breaks it, and it does feature an easy way out in case I disappear.
- sb8244 2y ago
  I don't agree that this is magic. It's only "magic" because the language doesn't do it today (thus requiring the compiler extension.) If this were in the language itself, it would probably be pretty popular. The argument against it feels like "because we've always done it this way."
  - oxidant 2y ago
    Agreed. It's just syntax, similar to implicit keywords (mentioned already) and & anonymous functions[0]. Pattern matching a large number of items off a map is verbose. Putting many new keys in a map can be verbose if they're existing variables. I'd rather have shorter code that was just as readable than many lines that add to cruft. [0] Reading `&%{foo: &1.bar}` as a new Elixir dev looks like character soup and is arguably more magical.
vessenes 2y ago
A good reminder that macros give you super powers! I don't know a lot of elixir, so I can't say if this is a great idea for inclusion into the standard or not, but it's undeniable that giving macro-based access to the AST adds a lot of expressivity and flexibility.
bryantwolf 2y ago
I've been using Destucture [1] for a few years. It's an interesting compromise that works for atom and string keys because it's explicit, kind of like the sigil approach. Though this seems way more streamlined, especially as someone who uses JS for front-end apps. [1]https://github.com/danielberkompas/destructure/ https://github.com/danielberkompas/destructure/
- kzemek 2y ago
  Oh nice, I didn't know about that one! It's interesting how many solutions are in this space; aside from Destructure[0] and shorter_maps[1] I also know about shorthand[2] and synex[3]. [0] https://github.com/danielberkompas/destructure https://github.com/danielberkompas/destructure [1] https://github.com/meyercm/shorter_maps https://github.com/meyercm/shorter_maps [2] https://hex.pm/packages/shorthand https://hex.pm/packages/shorthand [3] https://hex.pm/packages/synex https://hex.pm/packages/synex
- h0l0cube 2y ago
  I like this because it makes the sugaring distinct. Anyone unfamiliar with the sugar will know there's something different going on, while not really adding much noise.

71 points by kzemek 2y ago

arrowsmith 2y ago

This is great! I've always thought that if I could change one thing about Elixir, it would be to add exactly this shorthand syntax that I'm used to from JS. I'm sick of having to type everything twice, like e.g. `%{foobar: foobar}`. I'm impressed to see it implemented. I didn't know it was possible to pass custom compilers to your Mix project definition - but now that I think about it, I can see how Elixir's metaprogramming features would lend themselves well to implementing custom syntax like this. Fantastic work, all I want to know is why Elixir doesn't have this syntax built-in already.

dudul 2y ago
> I'm sick of having to type everything twice, like e.g. `%{foobar: foobar}` I'm actually very confused by this use case. How often do you have to do that? Fill a map with kv tuples where keys and values have the exact same value?
- cmoski 2y ago
  the key is an atom called foobar, and the value is being extracted to a variable called foobar. The variable's value is not shown in the example. You do this quite a bit in Elixir, unless you use the . syntax, e.g. somemap.foobar which is marginally slower.
  - dudul 2y ago
    Got it - I misunderstood the short code sample, I thought the 2nd half of the tuple was a literal instead of a variable. Makes sense.
- gmassman 2y ago
  This pattern appears frequently in just about all programming languages. I’ve written a lot of JS and Python, and a moderate amount of Elixir, and this pattern crops up quite often. Usually it’s a side effect of complex scopes with several layers of function calls. Honestly I’m a little surprised you haven’t seen this pattern. The only times it wouldn’t be used are when people rename variables, which is frankly a practice I abhor.
  - leononame 2y ago
    JS and python are just about all programming languages? I'm not trying to be snarky, but e.g. I haven't seen it in Go, F# and Rust, some languages where I have at least seen _some_ code. I also don't remember a similar pattern in Java, C# and C nor PHP or Dart, but I will readily admit that it's I haven't coded in any of these in the last two years, so I might be a bit out of touch and I'm not super proficient in all those languages
    - boxed 2y ago
      Go, and F# do almost all positional arguments. I think Rust too. Java, C#, C, PHP are also positional only or 99% of the time. I assume Dart too. We are talking named/keyword/etc. Personally I find positional arguments to be an anti-pattern in all but a very specific set of circumstances.
      - debugnik 2y ago
        C#, F# and PHP all support named arguments and they're quite common for optional parameters.
      - leononame 2y ago
        If I understand correctly, we aren't talking about named/keyword arguments. This is about a shorthand syntax in ES6 objects, nothing to do with function arguments. In JS, when creating an object, instead of writing {name: name}, where "name" is both the key in the map as well as the variable you want it to assign to, you can do {name}, whereas e.g. in Go you would do map[string]any{"name": name}. Edit: Or, for completeness' sake, it's the same with structs in Go, where you would instantiate a struct like this: Person{name: name}.
- cess11 2y ago
  The use of : in 'foobar: foobar' here is syntax sugar for :foobar, foobar where :foobar is a symbol that evaluates to itself and foobar evaluates to the value the variable points to.

cmoski 2y ago

Does it work for nested maps?

kzemek 2y ago
Yes it does!

jzimbel 2y ago

This was done previously with the `short_maps` library, which the author—a member of the Elixir core team—eventually retired and archived due to regrets from the undesirable impacts on clarity. https://github.com/whatyouhide/short_maps https://github.com/whatyouhide/short_maps https://andrealeopardi.com/posts/a-story-of-regret-and-retiring-a-library-from-hex/ https://andrealeopardi.com/posts/a-story-of-regret-and-retir...

MrJohz 2y ago
That's quite surprising to me. As someone who writes a lot of Javascript, I usually find it easiest to read with this syntax than without it, and I generally recommend to my team that they use the shorthand where they can. This is because the meaning is clear and well-known, and being concise (while still being clear) is very valuable. I wonder what specifically the original author found made it more difficult to read. Was it unfamiliarity? They allude to some confusion between sigils and strings, so I wonder if the issue was partly an Elixir-specific one.
- igsomething 2y ago
  The map '%{username, age, first_name, last_name}' would be 1 character away from the tuple '{username, age, first_name, last_name}'. It is easy to miss the '%' character and you'll waste some time figuring out why your code is throwing match and/or function clause errors at runtime.
  - Dkuku 2y ago
    Using this logic [username, age first_name, last_name] is only 2 characters away. This example is easy to catch because tuple and map work completly different-it's not python.
aarongraham 2y ago
I really don’t understand why some people are so against this as a language feature. I get that the community tries to stay away from “magic” but where is the line between magic and convenient syntactic sugar. For example keyword lists as the last argument to a function don’t need brackets, and keyword lists are really lists of tuples where the first item is an atom. I feel like those two things are in some ways less obvious and more magical than this map shorthand.

rubyn00bie 2y ago

I personally don’t think this is a good idea, and I think the formatter plugin is going to give someone insane amounts of regret one day. It’s neat as “look what I did” but Elixir is all about removing magic. It (lack of magic) is one of the things I appreciated most, having originally come from Ruby and Rails. I’ve been using it professionally for nearly ten years, and I can say this isn’t a feature I’ve ever really wanted. My text editor can save the typing via shortcuts I type, and in doing so supports string keys, atom keys, has zero compile time dependencies, and no new syntax ;) I’d really encourage you to put a note at the top of the readme that this shouldn’t be used in production code.

karmajunkie 2y ago
i don’t like adding foundational syntax using libraries, but i do wish they would add this to the language. deconstruction by pattern is super common and the syntactic sugar would be most welcome.
kzemek 2y ago
> I think the formatter plugin is going to give someone insane amounts of regret one day The formatter plugin features a "reverse" flag exactly to prevent any kind of regret like that. You can reformat your code automatically to remove all shorthand maps and remove the dependency in a couple simple steps. > I’d really encourage you to put a note at the top of the readme that this shouldn’t be used in production code. Noted, although I do think it's production ready. I'm going to keep updating the library if or when a new version of the Elixir compiler breaks it, and it does feature an easy way out in case I disappear.
sb8244 2y ago
I don't agree that this is magic. It's only "magic" because the language doesn't do it today (thus requiring the compiler extension.) If this were in the language itself, it would probably be pretty popular. The argument against it feels like "because we've always done it this way."
- oxidant 2y ago
  Agreed. It's just syntax, similar to implicit keywords (mentioned already) and & anonymous functions[0]. Pattern matching a large number of items off a map is verbose. Putting many new keys in a map can be verbose if they're existing variables. I'd rather have shorter code that was just as readable than many lines that add to cruft. [0] Reading `&%{foo: &1.bar}` as a new Elixir dev looks like character soup and is arguably more magical.

vessenes 2y ago

A good reminder that macros give you super powers! I don't know a lot of elixir, so I can't say if this is a great idea for inclusion into the standard or not, but it's undeniable that giving macro-based access to the AST adds a lot of expressivity and flexibility.

bryantwolf 2y ago

I've been using Destucture [1] for a few years. It's an interesting compromise that works for atom and string keys because it's explicit, kind of like the sigil approach. Though this seems way more streamlined, especially as someone who uses JS for front-end apps. [1]https://github.com/danielberkompas/destructure/ https://github.com/danielberkompas/destructure/

kzemek 2y ago
Oh nice, I didn't know about that one! It's interesting how many solutions are in this space; aside from Destructure[0] and shorter_maps[1] I also know about shorthand[2] and synex[3]. [0] https://github.com/danielberkompas/destructure https://github.com/danielberkompas/destructure [1] https://github.com/meyercm/shorter_maps https://github.com/meyercm/shorter_maps [2] https://hex.pm/packages/shorthand https://hex.pm/packages/shorthand [3] https://hex.pm/packages/synex https://hex.pm/packages/synex
h0l0cube 2y ago
I like this because it makes the sugaring distinct. Anyone unfamiliar with the sugar will know there's something different going on, while not really adding much noise.