3 ms·

Okay, so when I worked at Sony about 25 years ago, I got assigned this project to fix our order management system, which was extremely slow, and kept crashing.

Okay, so when I worked at Sony about 25 years ago, I got assigned this project to fix our order management system, which was extremely slow, and kept crashing. I jumped in and started digging around, and to my horror, the OMS was a giant set of shell scripts running on an AIX server, which evolved over a decade and was abandoned. It was over 50,000 lines of code! It was horrendous and shit kept timing out everywhere -- orders, payments, and other information were moved from server to server over FTP, parsed with complicated sed/awk, and inventory was tracked in text files (also FTPd around.) At the time, perl seemed like the most practical way for me to migrate the mess -- I rewrote all of the shell piece by piece, starting with the simplest peices and replaced them with small perl modules as part of a larger perl application, refactoring along the way. It took me 3 months and I moved the whole thing to about 5000 lines of perl, and it ran 10-100x faster with almost none of the failures in the original system. As terrible as it was, it's one of the most satisfying things I've ever done. :-)

shawn_w 2y ago
perl is still the most practical way to mitigate shell script abominations like that. Though tcl's a good option too.
- nextos 2y ago
  Or Ruby, which is essentially Smalltalk for Unix, plus lots of Perl-isms. Haskell (e.g. shh) and Clojure (Babashka) are also a nice for this usecase, but more niche options.
- chubot 2y ago
  Oils aims to be the absolute best way to migrate shell scripts! (I created the project, and the wiki page being discussed) https://www.oilshell.org/ https://www.oilshell.org/ OSH is the most bash-compatible shell in the world, and YSH is a new language ls | sort | uniq | wc -l # this is both OSH and YSH var mydict = {foo: 42, bar: ['a', 'b']} # this is new YSH stuff you can start using json write (mydict) The difference between OSH and YSH is exactly a set of "shopt" options [1], although YSH feels like a brand new language too! There is a smooth blend. I think it's worth it for 2 things alone - YSH checks all errors - you never lose an exit code - YSH has real arrays and doesn't mangle your variables with word splitting There's a lot more: modules with namespaces (use mymodule.ysh), buffered I/O that's not slow, etc. Gradually upgrading -https://github.com/oils-for-unix/oils/wiki/Gradually-Upgrading-Shell-to-YSH https://github.com/oils-for-unix/oils/wiki/Gradually-Upgradi... (people are writing new YSH, but not many people have gradually upgraded, so I'd definitely appreciate feedback from people with a big "shell script problem") --- There is a FAQ here about Perl: Are you reinventing Perl? - https://www.oilshell.org/blog/2021/01/why-a-new-shell.html#are-you-reinventing-perl https://www.oilshell.org/blog/2021/01/why-a-new-shell.html#a... Not to say that migrating to Perl is worse in any way, i.e. if you already know Perl or your team knows it. But objectively YSH is also a shell, so I think more of the code carries over, and there is a more direct upgrade path. --- [1] Unix Shell Should Evolve like Perl 5 - https://www.oilshell.org/blog/2020/07/blog-roadmap.html#the-unix-shell-should-evolve-like-perl-5 https://www.oilshell.org/blog/2020/07/blog-roadmap.html#the-... - i.e. with compatible upgrade options
  - RestartKernel 2y ago
    That looks great! I moved from Fish and NuShell to Zsh because of its reasonable compatibility with bash, so OSH seems right up my alley.
martin-t 2y ago
Just 3 months? That's deleting 800 lines a day, each day. Did you need to read through the original code, get a deep understanding and match its behavior exactly or did you throw away huge chunks and write new code as you thought it should behave? Was there a lot of boilerplate that could be replaced quickly?
- almostgotcaught 2y ago
  > That's deleting 800 lines a day, each day 50,000/90 = 555 ???
  - ciupicri 2y ago
    Not all days are working days.
- zeroxfe 2y ago
  There was tons of duplicate code, unnecessary code, dead code, etc. There was also a lot of code for which CPAN modules could entirely replace. (Also, I'm a workaholic who obsesses about a problem until it's fully solved.)
InitEnabler 2y ago
That's wild.

shawn_w 2y ago

perl is still the most practical way to mitigate shell script abominations like that. Though tcl's a good option too.

nextos 2y ago
Or Ruby, which is essentially Smalltalk for Unix, plus lots of Perl-isms. Haskell (e.g. shh) and Clojure (Babashka) are also a nice for this usecase, but more niche options.
chubot 2y ago
Oils aims to be the absolute best way to migrate shell scripts! (I created the project, and the wiki page being discussed) https://www.oilshell.org/ https://www.oilshell.org/ OSH is the most bash-compatible shell in the world, and YSH is a new language ls | sort | uniq | wc -l # this is both OSH and YSH var mydict = {foo: 42, bar: ['a', 'b']} # this is new YSH stuff you can start using json write (mydict) The difference between OSH and YSH is exactly a set of "shopt" options [1], although YSH feels like a brand new language too! There is a smooth blend. I think it's worth it for 2 things alone - YSH checks all errors - you never lose an exit code - YSH has real arrays and doesn't mangle your variables with word splitting There's a lot more: modules with namespaces (use mymodule.ysh), buffered I/O that's not slow, etc. Gradually upgrading -https://github.com/oils-for-unix/oils/wiki/Gradually-Upgrading-Shell-to-YSH https://github.com/oils-for-unix/oils/wiki/Gradually-Upgradi... (people are writing new YSH, but not many people have gradually upgraded, so I'd definitely appreciate feedback from people with a big "shell script problem") --- There is a FAQ here about Perl: Are you reinventing Perl? - https://www.oilshell.org/blog/2021/01/why-a-new-shell.html#are-you-reinventing-perl https://www.oilshell.org/blog/2021/01/why-a-new-shell.html#a... Not to say that migrating to Perl is worse in any way, i.e. if you already know Perl or your team knows it. But objectively YSH is also a shell, so I think more of the code carries over, and there is a more direct upgrade path. --- [1] Unix Shell Should Evolve like Perl 5 - https://www.oilshell.org/blog/2020/07/blog-roadmap.html#the-unix-shell-should-evolve-like-perl-5 https://www.oilshell.org/blog/2020/07/blog-roadmap.html#the-... - i.e. with compatible upgrade options
- RestartKernel 2y ago
  That looks great! I moved from Fish and NuShell to Zsh because of its reasonable compatibility with bash, so OSH seems right up my alley.

martin-t 2y ago

Just 3 months? That's deleting 800 lines a day, each day. Did you need to read through the original code, get a deep understanding and match its behavior exactly or did you throw away huge chunks and write new code as you thought it should behave? Was there a lot of boilerplate that could be replaced quickly?

almostgotcaught 2y ago
> That's deleting 800 lines a day, each day 50,000/90 = 555 ???
- ciupicri 2y ago
  Not all days are working days.
zeroxfe 2y ago
There was tons of duplicate code, unnecessary code, dead code, etc. There was also a lot of code for which CPAN modules could entirely replace. (Also, I'm a workaholic who obsesses about a problem until it's fully solved.)

InitEnabler 2y ago

That's wild.