4 ms·

I am on the portable shell team. It is hard to write for all shells, but totally worth it. If we all did it, reusing shell code would be easier. We never left

I am on the portable shell team. It is hard to write for all shells, but totally worth it. If we all did it, reusing shell code would be easier. We never left the hell caused by the war between shells to provide interactive features. They were all feature creeped for terminal usage and the scripting API was left to rot. I get the people that say we should use python or something else instead. It won't work though if that language is not capable of replacing the shell _as a build dependency_. As long as we need the shell to build stuff, we're stuck to it. Dockerfiles, GitHub Actions YAMLs, all these tools accept shell input. This is shiny new software that could have picked any language for their proprietary DSLs, but the shell was chosen. What the shell needs is what happened to JavaScript around the 2000s. Different interpreters started to align, standards were written, reusable libraries started to pop up everywhere, and now it runs in our brains. Before all of that, JavaScript was merely a cute toy to animate pages.

js2 4y ago
The one thing I really miss in portable shell code is arrays. There’s a place reserved in hell for whoever thought it was a good idea to design a build system around shell code in yaml files. I’m in the process of moving thousands of lines of such code into standalone scripts that then get called from the yaml files. Keep the yaml as minimal as possible. Then I can shellcheck the shell code or easily port bits to other languages as needed.
- alganet 4y ago
  You can reimplement arrays using pure shell! I saw it first here: https://github.com/shellfire-dev/core/blob/master/variable/array.functions https://github.com/shellfire-dev/core/blob/master/variable/a... For many cases, this is even better than bash native arrays. You can pass these as arguments and preserve the structure, while the bash ones will autoexpand if you try that. I ended up cooking my own array implementation later, but it is a bit more unorthodox (I create extra vars with numeric pointers to handle the values). Let me know if you want to see it, it's not complete but it works. I'm not very fond of YAML either, I feel your pain :(
  - js2 4y ago
    I knew I should have qualified my comment about arrays with: “without using a hack such as storing the values in a string with a custom delimiter.” Those aren’t really arrays. What happens if the delimiter appears among the values you want to store? This is a pretty big disclaimer: > Do not use arrays with file paths if there's any chance they could contain \r.
    - alganet 4y ago
      That's why I did my own implementation using pointers. var lorem = text_create "Hello" var foo = array_create array_push $foo $lorem `var`, `array_push` and `array_create` are shell functions working together. They create pointers and return just integers. You don't even have to quote the variable. The code above should create the following variables: _123="Hello" _124=1 _124_0=_123 foo=_124 lorem=_123 This way, I can even implement more complex data structures. The numbers start from zero and the program will stop if the global variable counter overflows. My current implementation also has a primitive GC (ARC-based), which could be used to avoid that (but I didn't, yet). Despite all of this runtime processing and jerry rigging, I have some evidence that it could be faster than typical shell scripts. One single subshell or path lookup is often more expensive than these fake pointer arrays. These strucutres have zero subshells and run on PATH='', just with builtins. The subshell only becomes faster if the volume of data is larger than a few kilobytes (when the cost of forking and exec'in is worth the payoff). I know this is far from ideal. I should try to put these ideas in a new interpreter, not hack them on existing ones (JS calls them polyfills, maybe I should do that). If I did another interpreter though, it would end up like the fish shell (nice, fast, doesn't compile the kernel and no one uses it). Maybe I should probably stick to bash, ksh and zsh that offers the most complete set of features. Sounds like "best viewed in internet explorer" though.
      - jrumbut 4y ago
        That's an impressive level of devotion to portable bash arrays.
    - kevin_thibedeau 4y ago
- deleted 4y ago
  [deleted]

js2 4y ago

The one thing I really miss in portable shell code is arrays. There’s a place reserved in hell for whoever thought it was a good idea to design a build system around shell code in yaml files. I’m in the process of moving thousands of lines of such code into standalone scripts that then get called from the yaml files. Keep the yaml as minimal as possible. Then I can shellcheck the shell code or easily port bits to other languages as needed.

alganet 4y ago
You can reimplement arrays using pure shell! I saw it first here: https://github.com/shellfire-dev/core/blob/master/variable/array.functions https://github.com/shellfire-dev/core/blob/master/variable/a... For many cases, this is even better than bash native arrays. You can pass these as arguments and preserve the structure, while the bash ones will autoexpand if you try that. I ended up cooking my own array implementation later, but it is a bit more unorthodox (I create extra vars with numeric pointers to handle the values). Let me know if you want to see it, it's not complete but it works. I'm not very fond of YAML either, I feel your pain :(
- js2 4y ago
  I knew I should have qualified my comment about arrays with: “without using a hack such as storing the values in a string with a custom delimiter.” Those aren’t really arrays. What happens if the delimiter appears among the values you want to store? This is a pretty big disclaimer: > Do not use arrays with file paths if there's any chance they could contain \r.
  - alganet 4y ago
    That's why I did my own implementation using pointers. var lorem = text_create "Hello" var foo = array_create array_push $foo $lorem `var`, `array_push` and `array_create` are shell functions working together. They create pointers and return just integers. You don't even have to quote the variable. The code above should create the following variables: _123="Hello" _124=1 _124_0=_123 foo=_124 lorem=_123 This way, I can even implement more complex data structures. The numbers start from zero and the program will stop if the global variable counter overflows. My current implementation also has a primitive GC (ARC-based), which could be used to avoid that (but I didn't, yet). Despite all of this runtime processing and jerry rigging, I have some evidence that it could be faster than typical shell scripts. One single subshell or path lookup is often more expensive than these fake pointer arrays. These strucutres have zero subshells and run on PATH='', just with builtins. The subshell only becomes faster if the volume of data is larger than a few kilobytes (when the cost of forking and exec'in is worth the payoff). I know this is far from ideal. I should try to put these ideas in a new interpreter, not hack them on existing ones (JS calls them polyfills, maybe I should do that). If I did another interpreter though, it would end up like the fish shell (nice, fast, doesn't compile the kernel and no one uses it). Maybe I should probably stick to bash, ksh and zsh that offers the most complete set of features. Sounds like "best viewed in internet explorer" though.
    - jrumbut 4y ago
      That's an impressive level of devotion to portable bash arrays.
  - kevin_thibedeau 4y ago
deleted 4y ago
[deleted]