15 ms·

Scripts should be written using the project main language

118 points by freitzzz 3y ago

rpigab 3y ago
I like the general idea, but it depends. Sometimes these bash scripts are used in lots of contexts, local dev, CI pipelines, Docker build steps... good luck running Java or Rust in the last two. Even if you get it to work, good luck debugging if there are any issues.
dusted 3y ago
I think it's a strong "it depends", if the script interface with data used/produced by the project, sure, seems natural.. If the script does something more system-near and the main language is not as convenient for it, then.. No. I'd hate to do with typescript what we're doing in scripts with bash.. just like I'd hate doing in bash what we do in typescript..
teknico 3y ago
Well, it depends. If your project uses Zig, you obviously use the Zig build system. If your project uses C or C++, you also use the Zig build system. Advocacy - Maintain it With Zig: https://kristoff.it/blog/maintain-it-with-zig/ https://kristoff.it/blog/maintain-it-with-zig/ HN discussion: https://news.ycombinator.com/item?id=35566791 https://news.ycombinator.com/item?id=35566791 Reference: https://ziglang.org/learn/build-system/ https://ziglang.org/learn/build-system/
simne 3y ago
I don't agree. From my exp really big problem, when you can't easy see borders between parts of project, and this is case with React and many other fullstack envs. From other view, yes, it is not good, when for example on front you use strict typed lang and on back dynamic, so need to constantly do conversions. So need some reasonable combination. Traditional Java+JS is looking reasonable. JS (front) +Python (back) is also reasonable. C backend from my view is weird, so should be some intermediate lang, for example Lua.
gwbas1c 3y ago
Where I work, most programming is C#. We use a lot of LINQPad scripts, which is a lightweight environment to run C# in.
- olvy0 3y ago
  There's also CS-Script https://github.com/oleg-shilo/cs-script https://github.com/oleg-shilo/cs-script
- 90sHackySack 3y ago
  My preferred lightweight environment to run c# in is a console app. I have never understood the appeal of LinqPad whatsoever.
  - gwbas1c 3y ago
    Next time you want to write a few lines of C# to verify a detail about syntax, count how many clicks you need to go through to create a new console app project / solution. It's much faster to just hit the + in LINQPad. It's quite useful when I'm reviewing someone else's code, or if I'm "in the zone" in a large change. I can verify some syntax in a few seconds, as opposed to the minutes it takes to make a throwaway console app project / solution.
  - neonsunset 3y ago
    It is kind of like csharprepl[0] but more powerful for the scenarios it targets. However, since it only supports Windows it's a non-starter personally so I never used it much. [0] https://github.com/waf/CSharpRepl https://github.com/waf/CSharpRepl
deleted 3y ago
[deleted]
jimbokun 3y ago
For Java projects, scripts can be written in a JVM language like Clojure or JRuby. Can still leverage the Java code in the rest of your project, while writing in a more dynamic, interactive language.
- hocuspocus 3y ago
  If you think JBang isn't dynamic enough for such scenario, check this out: https://scala-cli.virtuslab.org/scripting https://scala-cli.virtuslab.org/scripting Scala is a lot less exotic than Clojure or JRuby to most Java devs, as expressive as Groovy, yet fully type-checked.
elric 3y ago
> For example, writing scripts on JVM languages would require additional effort to build a toolchain that compiles and runs files on the fly, with a short start time. That hasn't been true since at least Java 11. You can execute any .java file using `java foo.java`. No compilation required. You can reference dependencies using the usual classpath options etc. Startup time is minimal. Been using such scripts in exactly the way the author suggests for years. Much more pleasant than messing around with maven or gradle plugins.
- Supermancho 3y ago
  > You can reference dependencies using the usual classpath options etc I don't know anyone who thinks that a java script (being the main project language) is going to surpass: #!/bin/sh dep-start.sh ./gradlew clean bootRun Maybe there are outliers with this "hot take". The result of years of projects (even changing hands), are a lot more instructive than someone posing theoretical value of reusability.
  - elric 3y ago
    That example doesn't even need a script, it could literally be a single gradle task. Some things are lot easier to do in a platform independent way in Java (or Groovy) than in shell scripts. And unlike the latter, the former can be tested just like any other part of the code base. It always bugged me that build scripts are hardly ever tested or engineered. They just grow into giant balls of mud.
P_I_Staker 3y ago
I'll admit I'm not reading the article, but this is a hard no for C development. Yeah, it can be done not so terribly, but just use a normal scripting language. In C#, I could see doing this.
jolt42 3y ago
Clojure -> Babashka, better than regular bash
bdcravens 3y ago
Using the same language allows for eventually placing those scripts in background jobs. In Ruby/Rails, the concept of ad-hoc script execution is baked in via rake.
mike_d 3y ago
Use the right tool for the job. The authors complaints seem to stem from not properly maintaining the supporting scripts in their project, which isn't at all a function of the language.
- mangodrunk 3y ago
  Assuming that a different language is better for a specific job, there is a cost associated to having another language.
  - CuriouslyC 3y ago
    There's no cost associated with bash unless you're hiring muppets.
    - snapcaster 3y ago
      I would have disagreed (obscure syntax thats hard to memorize if you write bash rarely) but LLMs have really removed all of that
      - dave4420 3y ago
        My boss introduced a bug into our build system because of misunderstanding bash semantics. LLMs really haven’t solved this. Bash makes everyone a muppet if you push it hard enough. (True of all tools, perhaps. But you don’t have to push very hard with bash.)
        wrs 3y ago
        With Bash quoting rules it’s not like you have to push, it’s more like you don’t pull hard enough to keep it from going over the edge.
        fiddlerwoaroof 3y ago
        Shellcheck basically solves this problem
electroly 3y ago
I have a C++ project where the documentation is generated using a script that I wrote in C++. Woof. I didn't want to add a compile-time dependency on another programming language, but C++ is rough as a scripting language. If my script needs were any more complex, I'd be thinking hard about how bad a compile-time dependency on Python really is.
- kyllo 3y ago
  I'm not sure I've even seen a recent, large C++ project that didn't depend on Python or some other external scripting language just to build it, so it's kind of hard to imagine using C++ itself to solve the problem that using C++ creates.
  - MathMonkeyMan 3y ago
    I worked with a guy in a C++ RPC team (think Envoy, but proprietary). He wrote the build tool that was used by our team, which maintained several fairly large C++ programs and libraries. He wrote it all in C++. He was of the opinion that most scripting tasks on the team could be accomplished with a small C++ program. It helped that we had a portable kitchen sink of libraries at our disposal, but he wasn't above using std::system to avoid the hassle.
- deckard1 3y ago
  someone added a script written in Rust to our project. They compiled it for ARM. So only people on Macbooks can run the damn thing. Nice one Rust bro.
JustLurking2022 3y ago
Hard pass - many languages simply don't prioritize the experience of executing shell commands.
- deleted 3y ago
  [deleted]
- williamdclt 3y ago
  Tbh bash/sh don’t prioritise the experience of _writing_ commands, which might be worse. I’m decent with Bash, but to this day I don’t know how to properly document and parse flags (long and short form) and positional arguments (with validation, while we’re at it) so that it all works as I would expect.
  - MathMonkeyMan 3y ago
    There's no best answer. I do in manually in a `while` with `case` and `shift`. For anything more, I use environment variables or Python.
senkora 3y ago
> If they can use the main language, awesome. If they can’t, a higher-level scripting language with native support (e.g., Python) should be adopted, since it provides the means to increase maintainability in the long run. I think this point is especially important for C++ projects. It is my gut feeling that C++ and Python cluster very closely in terms of developer familiarity. That is, a C++ developer very likely is also a passable Python developer. Given that it tends to take more time to write a C++ program than the equivalent Python program, the stable result is that many C++ projects 1) expose C++ to Python (via e.g. pybind11) and 2) write all scripts in Python. And you get almost all of the benefits that the article suggests, because almost all C++ developers are also Python developers.
dvh 3y ago
Scripts should be written in JavaScript, only if proven insufficient a different scripting language should be used. /Serious!
- hooverd 3y ago
  It does have Script in the name!
dragonwriter 3y ago
What it really seems like the issue is is: 1. Scripts should be maintained and tested, and 2. The language used for scripts necessarily is and should be treated as a project language, and the appropriateness of that choice should have all the factors that go into choosing a langauge for any other purpose, including the impact on complexity if it isn't the main language and fitness for purpose and any additional dev platform, tooling, etc., constraints it imposes, but... This doesn't imply scripts should be in the main project language, any more than it is generally the case that projects must be monolingual.
- Shawnj2 3y ago
  If you’re using a C++ project saying that all of your scripts related to that project must also be in C++ when Python would do fine is ridiculous. You should just pick a reasonable language
- l0b0 3y ago
  Much more nuanced, thank you! For one thing, you need a language to bootstrap your environment in dev envs and CI. You wouldn't want to write a Python script which has to support a bunch of minor versions (with vastly different capabilities) just to run `poetry install`.
tgv 3y ago
> The learning curve is minimal since you already know the corners of the language But that does not imply you know how to get the creation date of a file or how to zip a directory. > Internal language APIs can be leveraged, which drastically changes the mental model to write the script (for the better) That sounds like a rather empty statement. > Scripts feel more natural and eventually maintainability increases. Team members are familiarized with the language! Don't you think familiarity with the OS (or OSes) comes first? And that knowledge usually comes with the knowledge of a shell or batch language. > Development machines compatibility increases. Windows users can finally run all scripts. Script development time increases, too. And Windows has WSL nowadays.
kjksf 3y ago
I do that in my Go projects. In fact my "scripts" are actually part of the main executable. I use cmd-line args to invoke the needed functionality. For example, in the past I would have written a Python script to deploy my Go binary to a server, possibly using tools like Fabric that provide functionality to make it easier. Today I add `-deploy-hetzner` cmd-line to my Go binary and it does the work. It builds itself, copies the binary to the server, kills the old instances, configures caddy if needed, starts newly uploaded instance etc. For example my deploy.go is 409 lines of code, which is not that bad. You can see exactly how this works: https://github.com/kjk/edna/blob/main/server/deploy.go https://github.com/kjk/edna/blob/main/server/deploy.go I standardized on how I deploy things so deploy.go is mostly re-used among several projects. Writing this code isn't much more difficult that what I used to write in Python. This kind of code can be shorter because I don't have to handle errors, I just panic if something goes wrong. I like that I don't have to switch between different languages and that I have full control and understanding over what happens. Fabric used to be a bit of a black box. I even wrote an article about this idea: https://blog.kowalczyk.info/article/4b1f9201181340099b698246857ea98d/using-go-instead-of-bash-for-scripts.html https://blog.kowalczyk.info/article/4b1f9201181340099b698246...
- zer00eyz 3y ago
  Im a big fan of //go:build exclude ... and having separate CLI scripts in a project where I can. Candidly I think that it's much easier to do this in go (or rust), rather than say python/ruby/node as I can use complied binaries without needing a run time. Edit: //go:build ignore is idiomatic
  - bbkane 3y ago
    Do build-excluded files get tested with `go test`? (tbf, not sure it's practical to test most scripty tasks)
    - zer00eyz 3y ago
      Think of this as a way to have a file with package main, and a func main that does not interfere with your normal build process... The best example of this, and a decent util is this: https://go.dev/src/crypto/tls/generate_cert.go https://go.dev/src/crypto/tls/generate_cert.go All of the real testing happens elsewhere this just provides utility.
CuriouslyC 3y ago
No! General purpose scripts should almost always be written in bash. It's basically the best language for doing simple things with files, it's universally available and it makes almost no assumptions about the environment in which it executes. Have windows users use WSL (the VSCode integration is great!), and mac users should install GNU tools since the system tools are obnoxiously incompatible. The only time I've found that scripts should be in another language is: 1. You need to call libs that to do something fancy and it would be too troublesome to make a small Unix style executable to do the thing. 2. The developers on your team lack Unix/bash experience, and you don't trust them to learn in a timely manner (sad).
- sgarland 3y ago
  Sad that you're being downvoted for the truth. Unless you're doing some extremely niche work, Bash >= 3.2 (because Mac) is nearly always going to be available. Even if it _isn't_, there will still be sh or dash, and it's not _that_ hard to stick with pure POSIX for most small uses. The last time I (by which I mean my team) rewrote a script from Bash into Python was because it had gotten unwieldy over time, I was the sole maintainer, and very few other people at the company knew Bash well enough to understand some of it. The upside was testing frameworks in Python are way better than Bash.
  - MathMonkeyMan 3y ago
    First you write in shell without knowing the language, and blow your foot off. A few years go by. "I should use a _real_ language, I'm not an amateur anymore." So you write everything in Python. A few years go by. "I should learn shell, and use it only when appropriate." Maybe this is what Perl is for, but I never learned it.
- deleted 3y ago
  [deleted]
- wodenokoto 3y ago
  > Have windows users use WSL (the VSCode integration is great!), and mac users should install GNU tools since the system tools are obnoxiously incompatible. At that point you might as well target Python 3.6. Seems like the same hassle for the developer to install and you don't have to worry about wonky differences for users who haven't installed GNU tools, but still think they can run your script because it says `.sh`
graypegg 3y ago
I've always found it weird that the NPM ecosystem doesn't have something like Rake from the Ruby world to run tasks. Javascript things tend to be VERY task heavy, with dev servers, bundlers, testing, and coverage all being defined in the project normally. The package.json scripts "work", but it's quite clunky, and relying on shell scripts that run node.js scripts causes issues. (cross-env solving a problem that really shouldn't exist.)
- williamdclt 3y ago
  I’m of the opinion that the package.json scripts was a mistake (although not a huge deal in the grand scheme of things): it solves 80% of the problem, which removes enough pain that there’s not enough incentive to solve the last 20%. I’m sure there’s something like rake that exists for Node, but the community won’t standardise on it because it’s not enough a problem
- chuckadams 3y ago
  Gulp is still a thing. Or even Grunt if that’s your poison.
simonw 3y ago
I would have agreed more with this a couple of years ago when maintaining a script written in an alternative language required me to know that language. These days I'm much more comfortable both writing and maintaining code in languages that aren't my daily driver (like Bash or jq or AppleScript or even Go) because I can get an LLM to do most of the work for me, and help me understand the bits that don't make sense to me.
- dguest 3y ago
  Soon many projects will be written in LLM: a mix of whatever languages the models generated on given day + an LLM so that the maintainers can understand it.
- darby_eight 3y ago
  I still find it hard to believe that LLMs provide any real edge over the kind of googling/man-paging that's necessary for understanding scripts, especially given the false positives chatbots are known for. Typically language + feature + library is enough to look something up in under ten seconds for me. Granted, this probably takes a fair amount of experience to even know what you're looking at well enough to search for it.
  - simonw 3y ago
    The good ones (Claude 3 Opus, GPT-4) are really incredibly good at understanding scripts. It's very rare that they hallucinate anything, especially important details for the more widely used tools that I tend to stick to. I trust LLMs with tools like Bash and jq and ffmpeg which have been around for years. I wouldn't trust them with anything released within the past 12-24 months. An example from just the other day: https://til.simonwillison.net/go/installing-tools https://til.simonwillison.net/go/installing-tools I wanted to understand this: go install github.com/icholy/semgrepx@latest How does that @latest reference mention? There's no branch or tag on that repo called "latest". I tried and failed to find documentation. I gave up and asked GPT-4, which said: > @latest: This specifies the version of the package you want to install. In this case, latest means that the Go tool will install the latest version of the package available. The Go tool uses the versioning information from the repository's tags to determine the latest version. If the repository follows semantic versioning, the latest version is the one with the highest version number. If there are no version tags, latest will refer to the most recent commit on the default branch of the repository. Is that correct? I have no idea! But it still gave me more to go on than my failed attempts with the real documentation.
SamBorick 3y ago
I like to take this a step further and try to have all my most important tooling be written in my projects major languages. It's not a must have, but having a build system for a go project that is also go under the hood means I'm more comfortable diving into the source if needed. I don't think it's a requirement, but it's an advantage
arwhatever 3y ago
We should use YAML for all of our automation so that non-developers can build and maintain the automation, and then exclusively assign developers who are proficient in a general purpose programming language to build and maintain the YAML automation. </s>
- hooverd 3y ago
  Simply put the general purpose programming language inside the YAML automation.
quwert95 3y ago
For some languages this can make sense, but I think most languages aren't suited for installing things, manipulating the filesystem, etc. I think of scripts as the middleware between the operating system and the shipped code. The code is controlled by the operating system, so the operating system's tools should be used to manage it. In many cases this means bash or make. Plus, I don't want modern Javascript to do things on the filesystem that would require importing dozens of projects that I need to vet before using. Golang or Python perhaps, but the buildchain for modern Javascript is hell as-is; it doesn't need another layer of Javascript.
csours 3y ago
Scripts should be commented with intent. (I can figure out what it is doing, but I have no clue WHY) Scripts should be testable. Scripts should use functions with appropriately scoped variables. (This really helps with testability) Scripts should list assumptions. Scripts should CHECK assumptions. Scripts should call commands with --long-style-options instead of -L, especially uncommon options. --- As someone who migrated a couple hundred shell scripts over the past year, I'd rather have these done before I ask someone to write a script in C. edit: and for ${deity} sake, use shellcheck.
- shrimp_emoji 3y ago
  > Scripts should call commands with --long-style-options instead of -L, especially uncommon options. Too bad `getopts` only supports single-char options. :p
  - csours 3y ago
    I'm not a fan of getopts And to be more specific - I meant when the script calls someone else, not how it handles options.
- jjgreen 3y ago
  I think you mean "${deity}" in case of spaces ...
MichaelRo 3y ago
>> Scripts should be written using the project main language ... as long as that main language is a scripting language. Otherwise it's just dumb. Also, tunnel vision by the author: "Almost all projects I’ve worked on have scripts we wrote to automate a repetitive process. " Well almost all projects I’ve worked on have scripts we wrote to automate a ONE-TIME process. Like collect some data from the log to figure out a bug, fix it and forget about both the bug and the script. Automate it since can't manually process 30Gb of data and grep only can do so much. Sure as funk won't write the "script" in C++ but Python or Perl or something.
cjk 3y ago
Ehhhhhh. I get the sentiment, but I think this really depends on what the "main" language is and how amenable it is to scripting. Personally, I prefer writing shell scripts regardless of what the main language is in a given project. They're portable (more or less), and can e.g. detect missing dependencies and install them if necessary, which isn't possible with the main language if you're missing its compiler or interpreter.
mthoms 3y ago
Using Google’s zx I write my support scripts in JS and can use shell features like piping right in the same file.
taeric 3y ago
Disagreed from me. Though, I'm not sure how hard my disagreement truly is. Discrete programs are superior in many ways, as you do not immediately incur maintenance costs to write them. More, they typically force you to have a discrete API that they work with, and then you can lean on that. Yes, you can do all of this with modular programming techniques. Indeed, "unit tests" are easy to see as similar to what I'm advocating here. Such that I think my assertion is softer than many folks are probably seeing. If you are "scripting" something to add data to the system, it should emphatically not hit the database directly. I don't know where this lands me on the infrastructure as code (IAS) debate. I'm sympathetic to the desire. I start to think of it as navel gazing when I see some of the very engineered testing practices some people take those to.
wakawaka28 3y ago
Absolutely not. Use the right tool for the job. There's nothing particularly wrong with shell scripts, as long as the people who maintain them are reasonably good at not making them into a dangerous nightmare waiting to delete your home directory or something.
jes5199 3y ago
man, I work in python but I really don’t enjoy writing scripts in it. Everyone here acting like it’s a scripting language. Culture shock.
rottc0dd 3y ago
The performance concerns might creep up for scripts and tricky to figure out the issue if you are not internal abstractions. Example, https://github.com/berry-thawson/diff2html https://github.com/berry-thawson/diff2html
noobermin 3y ago
This really only applies to a subset of languages and projects with such languages. I wish developers had the courage to face the deeper problem (being unwilling to read code...older than a few weeks? Such a developer will face worse things in the furure, they should train their focus stamina)
hgs3 3y ago
The authors main bullet points are: > The learning curve is minimal since you already know the corners of the language. Learning a new language shouldn't be difficult. Programmers are expected to familiarize themselves with new tech. > Internal language APIs can be leveraged, which drastically changes the mental model to write the script (for the better). This is true. I myself have encountered situations where I needed to call into my C API's from a higher-level language, but since most languages can interface with C this hasn't been an issue for me. For example I've interop'd Go+C, Python+C, and Lua+C. > Scripts feel more natural and eventually maintainability increases. Team members are familiarized with the language! This sounds like a subjective rehash of the first point. > Development machines compatibility increases. Windows users can finally run all scripts. This is true if you're talking about shell scripting, but if you're scripting with a general purpose programming language then it shouldn't be an issue. What language (besides shell) isn't portable these days? And even then, you can install a *nix environment on Windows.
- psychoslave 3y ago
  I think Crystal is a popular one which doesn't yet support windows fully, at least last time I checked
- hyperhopper 3y ago
  > Learning a new language shouldn't be difficult. Programmers are expected to familiarize themselves with new tech. I wish any large company agreed with this. I've worked for a company that on boarded every single new engineer to a very niche language (F#) in a few days. Also, everybody I worked with there was amazing. Probably because of that kind of mindset. Meanwhile google tiptoes around teams adopting kotlin because "oh no, what if other teams touching the code might not be able to read it". Google is supposed to be hiring the brightest but internally is worried the brightest can't review slightly-different-java. It's shocking how everybody acts like senior engineers might need months to learn a new language. Sure, maybe for some esoteric edge cases, but 5 mins on https://learnxinyminutes.com/ https://learnxinyminutes.com/ should get you 80% of the way there, and an afternoon looking at big projects or guidelines/examples should you another 18% of the way.
nosefurhairdo 3y ago
Depends on the project, depends on the team, depends on the language, depends on the script. Pick a sensible tool for the job, write understandable code, don't lose sleep over silly generalizations.
__mharrison__ 3y ago
So ... Python
LispSporks22 3y ago
We have some Perl scripts that are freaking immortal. The rest of the code base started in Java, then Clojure, now it’s Go. The scripts are there still in their very-not-modern Perl style though. They have a self-evaluating behavior consisting of data blocks that are interpolated. To be honest, I’m not sure exactly how it works. Very discouraging for the casual passerby looking for some cleanup to do.
throwbadubadu 3y ago
Say this to your C++ embedded project (no lang rants now, it is what it is;)): ehrm, no, please no, ridiculous.
brunooliv 3y ago
I think while the basic idea of advocating for stack consistency between the main project and any support scripts is very nice, I always find it not worthy to pursue in practice for several reasons: a lot of the “environment” around the main codebase will be a weird mix of YAML, bash and a scripting language like Python or Ruby for things like gitlab, airflow, GitHub actions, etc. Given this heterogeneity of the project environment and additional complexity of “forcing” something to use a language it might not be very well suited for, makes this really a no brainer for me: use the most convenient tool for the job. Plus as a JVM bound developer I love me my occasional Python
- treyd 3y ago
  I agree with this sentiment. At work our main codebase is in Rust and early on we wrote some CI tooling using the cargo xtask workflow and it adds a lot to compile times when we need to rebuild it (which is often since it depends on several of our main codebase's crates). It really kills iteration times. This is a mess and added a lot of extra complexity to the build pipeline since now we had to manage an additional xtask container. Python is very well suited for CI scripting, xtask should only be used for things directly run by the developer, and even then Python may be a better choice for most things.
doktrin 3y ago
Given the resultant comment section I clicked into the article expecting the most sith-like of absolutist proclamations taking itself extremely seriously and actively fanning arbitrary ideological flames. Instead I found a concise, measured and sane opinion that I honestly can’t disagree with. Like, sure, if your language broadly supports the secondary use case (scripting) why not? If it doesn’t, then the juice almost certainly won’t be worth the squeeze.
pjmlp 3y ago
The clear example of only having an hammer and every problem is a nail.
t43562 3y ago
Scripts are usually no better than they have to be - they're not the main purpose of the system. Scripting languages handle all sorts of issues that take pages of code in other languages so they are the easiest way to do what you need and easy wins. IMO some languages like C/C++ cry out for an embedded scripting language so that you don't write the basic, one off, performance insensitive parts of your code in a "hard" language. This is taking it the other way around - suggesting that as much as possible of your "main project" should be in a scripting language so that you're not wasting "hard" development cycles on areas that don't need it.
- GoblinSlayer 3y ago
  >pages of code Code reuse. The benefits of hard language: 1) I now can do interesting things like text parsing, 2) my scripts are cross platform, 3) I don't need to figure out how to deploy python everywhere in advance or on demand, 4) if the user has python, I don't need to tell him I don't like his python version and he must install a different OS, 5) I don't have python as an extra dependency, 6) I can reuse my main code in my scripts, 7) scripts are written in a language with a decent type system.
  - t43562 3y ago
    One small counter I have is that most software development has dependencies anyhow - and with compiled languages this is quite a pain in the arse to manage. So installing tcl or lua or python or ruby or perl is just a normal problem amongst other problems. If you want type systems everywhere then I think that's another debate. That is really a total rejection of almost all scripting languages for all purposes.
willtemperley 3y ago
Keeping scripts in the main project language usually means the monolith gets larger and more like a final boss than something the team controls. Use the language that best fits the job. For me that's TypeScript on a serverless platform, SwiftUI front-end and Java for business logic. I used to feel like I was winning using only Java for UI, Web and business logic, but the complexity became immense. It's too easy to create yet-another-internal-API that gets forgotten until it needs to be refactored. Also learning a language gives you a new perspective on software engineering, a bit like learning a foreign language gives a new view on human culture.
stevage 3y ago
Agreed. My scripts got much better when I abandoned Bash, and switched to Zx, which is basically NodeJS.
amne 3y ago
I try very hard to do just the opposite. If I need a script to do something existing code can't ,then it is most likely exciting and I'm going to use that energy to learn something new, be it a language or tool. Here's a hotter take: your team colleagues are more likely to wear your clothes than to use your scripts.
rkerno 3y ago
Really interesting comments here. I haven't found an appealing option for scripts and CI ( I think I am allergic to yaml ), bash is just way too fragile. So I've decided to write my own scripting language that is written in Go, similar in many ways to Lua, but with first class support for executing other commands, running API tests, and manipulating data. Currently dogfooding with plans to share in a few months once I'm confident it's good to start getting feedback. So, one executable plus your scripts, cross platform, no yaml.
mike_hearn 3y ago
Experience report: I did this for my company that uses Kotlin as its main language and it worked out brilliantly. We extended Kotlin Scripting into "HShell" and it swiftly replaced basically all uses of bash or Python for our internal scripting needs. The docs are public here, although the product isn't open source. It gives you a flavor of how it works though: https://hshell.hydraulic.dev/14.0/ https://hshell.hydraulic.dev/14.0/ Advantages: • It's as concise as bash but far more readable, logical and less bug-prone thanks to the static type system with plenty of type inference. • IntelliJ can provide a lot of assistance. • You can easily import and use any JVM library, and there are lots of them. • Ditto for internal project modules if you work on the JVM. • If you need to, you can easily mix in and evaluate Python/Java/etc using Graal. It has an integrated disk cache that can be used for storing file trees that manages free disk space automatically, and there's a command to set up a Python virtualenv in the disk cache transparently. • We have a high level shell API that makes console, file and network operations as easy as in bash, and sometimes easier because the commands aren't afraid to deviate from POSIX when that would be more convenient. For example most operations are recursive by default. • A smart progress tracking framework is integrated into every operation including things like wget. • It's fully portable to Windows including edge cases like being able to set POSIX permissions on a file, add it to a tar, and the resulting tar will retain the correct permissions. • SSH is deeply integrated, and so commands "do the right thing" automatically. For example all the commands take Path objects as well as strings, and path objects track which machine they refer to, so if you open up an SSH sub-shell you can easily copy to/from the remote machine using regular copy/move commands. Other commands are "smart" for example the wget() function given a path that's on a remote machine will execute curl or wget remotely rather than download locally then reupload. Although this sounds like it was all a lot of work to build, in reality our main product is a kind of build system (it makes deploying desktop apps easy, see bio for link). So all that functionality is in actuality functionality we built for the product and just kept nicely factored out into modules. The work invested into the scripting-specific parts is probably a couple of weeks over a period of a couple of years, and it was well worth it given the number of scripts we have for things like QA, deployment, server management and so on.
theshrike79 3y ago
I usually go with the Google Shell Style Guide: https://google.github.io/styleguide/shellguide.html https://google.github.io/styleguide/shellguide.html * If performance matters, use something other than shell. * If you are writing a script that is more than 100 lines long, or that uses non-straightforward control flow logic, you should rewrite it in a more structured language now. Bear in mind that scripts grow. Rewrite your script early to avoid a more time-consuming rewrite at a later date. * When assessing the complexity of your code (e.g. to decide whether to switch languages) consider whether the code is easily maintainable by people other than its author.
jakupovic 3y ago
There is a reason shells and shell scripts exist. Reinventing the basic shell functionality seems pretty dull and better spent staring at a wall to decompress. I always start with a simple shell script which does most of the work as simple as possible, even simpler is using `make` then introducing shell scripts when needed.

118 points by freitzzz 3y ago

I like the general idea, but it depends. Sometimes these bash scripts are used in lots of contexts, local dev, CI pipelines, Docker build steps... good luck running Java or Rust in the last two. Even if you get it to work, good luck debugging if there are any issues.

dusted 3y ago

I think it's a strong "it depends", if the script interface with data used/produced by the project, sure, seems natural.. If the script does something more system-near and the main language is not as convenient for it, then.. No. I'd hate to do with typescript what we're doing in scripts with bash.. just like I'd hate doing in bash what we do in typescript..

teknico 3y ago

Well, it depends. If your project uses Zig, you obviously use the Zig build system. If your project uses C or C++, you also use the Zig build system. Advocacy - Maintain it With Zig: https://kristoff.it/blog/maintain-it-with-zig/ https://kristoff.it/blog/maintain-it-with-zig/ HN discussion: https://news.ycombinator.com/item?id=35566791 https://news.ycombinator.com/item?id=35566791 Reference: https://ziglang.org/learn/build-system/ https://ziglang.org/learn/build-system/

simne 3y ago

I don't agree. From my exp really big problem, when you can't easy see borders between parts of project, and this is case with React and many other fullstack envs. From other view, yes, it is not good, when for example on front you use strict typed lang and on back dynamic, so need to constantly do conversions. So need some reasonable combination. Traditional Java+JS is looking reasonable. JS (front) +Python (back) is also reasonable. C backend from my view is weird, so should be some intermediate lang, for example Lua.

gwbas1c 3y ago

Where I work, most programming is C#. We use a lot of LINQPad scripts, which is a lightweight environment to run C# in.

olvy0 3y ago
There's also CS-Script https://github.com/oleg-shilo/cs-script https://github.com/oleg-shilo/cs-script
90sHackySack 3y ago
My preferred lightweight environment to run c# in is a console app. I have never understood the appeal of LinqPad whatsoever.
- gwbas1c 3y ago
  Next time you want to write a few lines of C# to verify a detail about syntax, count how many clicks you need to go through to create a new console app project / solution. It's much faster to just hit the + in LINQPad. It's quite useful when I'm reviewing someone else's code, or if I'm "in the zone" in a large change. I can verify some syntax in a few seconds, as opposed to the minutes it takes to make a throwaway console app project / solution.
- neonsunset 3y ago
  It is kind of like csharprepl[0] but more powerful for the scenarios it targets. However, since it only supports Windows it's a non-starter personally so I never used it much. [0] https://github.com/waf/CSharpRepl https://github.com/waf/CSharpRepl

deleted 3y ago

[deleted]

jimbokun 3y ago

For Java projects, scripts can be written in a JVM language like Clojure or JRuby. Can still leverage the Java code in the rest of your project, while writing in a more dynamic, interactive language.

hocuspocus 3y ago
If you think JBang isn't dynamic enough for such scenario, check this out: https://scala-cli.virtuslab.org/scripting https://scala-cli.virtuslab.org/scripting Scala is a lot less exotic than Clojure or JRuby to most Java devs, as expressive as Groovy, yet fully type-checked.

elric 3y ago

> For example, writing scripts on JVM languages would require additional effort to build a toolchain that compiles and runs files on the fly, with a short start time. That hasn't been true since at least Java 11. You can execute any .java file using `java foo.java`. No compilation required. You can reference dependencies using the usual classpath options etc. Startup time is minimal. Been using such scripts in exactly the way the author suggests for years. Much more pleasant than messing around with maven or gradle plugins.

Supermancho 3y ago
> You can reference dependencies using the usual classpath options etc I don't know anyone who thinks that a java script (being the main project language) is going to surpass: #!/bin/sh dep-start.sh ./gradlew clean bootRun Maybe there are outliers with this "hot take". The result of years of projects (even changing hands), are a lot more instructive than someone posing theoretical value of reusability.
- elric 3y ago
  That example doesn't even need a script, it could literally be a single gradle task. Some things are lot easier to do in a platform independent way in Java (or Groovy) than in shell scripts. And unlike the latter, the former can be tested just like any other part of the code base. It always bugged me that build scripts are hardly ever tested or engineered. They just grow into giant balls of mud.

P_I_Staker 3y ago

I'll admit I'm not reading the article, but this is a hard no for C development. Yeah, it can be done not so terribly, but just use a normal scripting language. In C#, I could see doing this.

jolt42 3y ago

Clojure -> Babashka, better than regular bash

bdcravens 3y ago

Using the same language allows for eventually placing those scripts in background jobs. In Ruby/Rails, the concept of ad-hoc script execution is baked in via rake.

mike_d 3y ago

Use the right tool for the job. The authors complaints seem to stem from not properly maintaining the supporting scripts in their project, which isn't at all a function of the language.

mangodrunk 3y ago
Assuming that a different language is better for a specific job, there is a cost associated to having another language.
- CuriouslyC 3y ago
  There's no cost associated with bash unless you're hiring muppets.
  - snapcaster 3y ago
    I would have disagreed (obscure syntax thats hard to memorize if you write bash rarely) but LLMs have really removed all of that
    - dave4420 3y ago
      My boss introduced a bug into our build system because of misunderstanding bash semantics. LLMs really haven’t solved this. Bash makes everyone a muppet if you push it hard enough. (True of all tools, perhaps. But you don’t have to push very hard with bash.)
      - wrs 3y ago
        With Bash quoting rules it’s not like you have to push, it’s more like you don’t pull hard enough to keep it from going over the edge.
        fiddlerwoaroof 3y ago
        Shellcheck basically solves this problem

electroly 3y ago

I have a C++ project where the documentation is generated using a script that I wrote in C++. Woof. I didn't want to add a compile-time dependency on another programming language, but C++ is rough as a scripting language. If my script needs were any more complex, I'd be thinking hard about how bad a compile-time dependency on Python really is.

kyllo 3y ago
I'm not sure I've even seen a recent, large C++ project that didn't depend on Python or some other external scripting language just to build it, so it's kind of hard to imagine using C++ itself to solve the problem that using C++ creates.
- MathMonkeyMan 3y ago
  I worked with a guy in a C++ RPC team (think Envoy, but proprietary). He wrote the build tool that was used by our team, which maintained several fairly large C++ programs and libraries. He wrote it all in C++. He was of the opinion that most scripting tasks on the team could be accomplished with a small C++ program. It helped that we had a portable kitchen sink of libraries at our disposal, but he wasn't above using std::system to avoid the hassle.
deckard1 3y ago
someone added a script written in Rust to our project. They compiled it for ARM. So only people on Macbooks can run the damn thing. Nice one Rust bro.

JustLurking2022 3y ago

Hard pass - many languages simply don't prioritize the experience of executing shell commands.

deleted 3y ago
[deleted]
williamdclt 3y ago
Tbh bash/sh don’t prioritise the experience of _writing_ commands, which might be worse. I’m decent with Bash, but to this day I don’t know how to properly document and parse flags (long and short form) and positional arguments (with validation, while we’re at it) so that it all works as I would expect.
- MathMonkeyMan 3y ago
  There's no best answer. I do in manually in a `while` with `case` and `shift`. For anything more, I use environment variables or Python.

senkora 3y ago

> If they can use the main language, awesome. If they can’t, a higher-level scripting language with native support (e.g., Python) should be adopted, since it provides the means to increase maintainability in the long run. I think this point is especially important for C++ projects. It is my gut feeling that C++ and Python cluster very closely in terms of developer familiarity. That is, a C++ developer very likely is also a passable Python developer. Given that it tends to take more time to write a C++ program than the equivalent Python program, the stable result is that many C++ projects 1) expose C++ to Python (via e.g. pybind11) and 2) write all scripts in Python. And you get almost all of the benefits that the article suggests, because almost all C++ developers are also Python developers.

dvh 3y ago

Scripts should be written in JavaScript, only if proven insufficient a different scripting language should be used. /Serious!

hooverd 3y ago
It does have Script in the name!

dragonwriter 3y ago

What it really seems like the issue is is: 1. Scripts should be maintained and tested, and 2. The language used for scripts necessarily is and should be treated as a project language, and the appropriateness of that choice should have all the factors that go into choosing a langauge for any other purpose, including the impact on complexity if it isn't the main language and fitness for purpose and any additional dev platform, tooling, etc., constraints it imposes, but... This doesn't imply scripts should be in the main project language, any more than it is generally the case that projects must be monolingual.

Shawnj2 3y ago
If you’re using a C++ project saying that all of your scripts related to that project must also be in C++ when Python would do fine is ridiculous. You should just pick a reasonable language
l0b0 3y ago
Much more nuanced, thank you! For one thing, you need a language to bootstrap your environment in dev envs and CI. You wouldn't want to write a Python script which has to support a bunch of minor versions (with vastly different capabilities) just to run `poetry install`.

tgv 3y ago

> The learning curve is minimal since you already know the corners of the language But that does not imply you know how to get the creation date of a file or how to zip a directory. > Internal language APIs can be leveraged, which drastically changes the mental model to write the script (for the better) That sounds like a rather empty statement. > Scripts feel more natural and eventually maintainability increases. Team members are familiarized with the language! Don't you think familiarity with the OS (or OSes) comes first? And that knowledge usually comes with the knowledge of a shell or batch language. > Development machines compatibility increases. Windows users can finally run all scripts. Script development time increases, too. And Windows has WSL nowadays.

kjksf 3y ago

I do that in my Go projects. In fact my "scripts" are actually part of the main executable. I use cmd-line args to invoke the needed functionality. For example, in the past I would have written a Python script to deploy my Go binary to a server, possibly using tools like Fabric that provide functionality to make it easier. Today I add `-deploy-hetzner` cmd-line to my Go binary and it does the work. It builds itself, copies the binary to the server, kills the old instances, configures caddy if needed, starts newly uploaded instance etc. For example my deploy.go is 409 lines of code, which is not that bad. You can see exactly how this works: https://github.com/kjk/edna/blob/main/server/deploy.go https://github.com/kjk/edna/blob/main/server/deploy.go I standardized on how I deploy things so deploy.go is mostly re-used among several projects. Writing this code isn't much more difficult that what I used to write in Python. This kind of code can be shorter because I don't have to handle errors, I just panic if something goes wrong. I like that I don't have to switch between different languages and that I have full control and understanding over what happens. Fabric used to be a bit of a black box. I even wrote an article about this idea: https://blog.kowalczyk.info/article/4b1f9201181340099b698246857ea98d/using-go-instead-of-bash-for-scripts.html https://blog.kowalczyk.info/article/4b1f9201181340099b698246...

zer00eyz 3y ago
Im a big fan of //go:build exclude ... and having separate CLI scripts in a project where I can. Candidly I think that it's much easier to do this in go (or rust), rather than say python/ruby/node as I can use complied binaries without needing a run time. Edit: //go:build ignore is idiomatic
- bbkane 3y ago
  Do build-excluded files get tested with `go test`? (tbf, not sure it's practical to test most scripty tasks)
  - zer00eyz 3y ago
    Think of this as a way to have a file with package main, and a func main that does not interfere with your normal build process... The best example of this, and a decent util is this: https://go.dev/src/crypto/tls/generate_cert.go https://go.dev/src/crypto/tls/generate_cert.go All of the real testing happens elsewhere this just provides utility.

CuriouslyC 3y ago

No! General purpose scripts should almost always be written in bash. It's basically the best language for doing simple things with files, it's universally available and it makes almost no assumptions about the environment in which it executes. Have windows users use WSL (the VSCode integration is great!), and mac users should install GNU tools since the system tools are obnoxiously incompatible. The only time I've found that scripts should be in another language is: 1. You need to call libs that to do something fancy and it would be too troublesome to make a small Unix style executable to do the thing. 2. The developers on your team lack Unix/bash experience, and you don't trust them to learn in a timely manner (sad).

sgarland 3y ago
Sad that you're being downvoted for the truth. Unless you're doing some extremely niche work, Bash >= 3.2 (because Mac) is nearly always going to be available. Even if it _isn't_, there will still be sh or dash, and it's not _that_ hard to stick with pure POSIX for most small uses. The last time I (by which I mean my team) rewrote a script from Bash into Python was because it had gotten unwieldy over time, I was the sole maintainer, and very few other people at the company knew Bash well enough to understand some of it. The upside was testing frameworks in Python are way better than Bash.
- MathMonkeyMan 3y ago
  First you write in shell without knowing the language, and blow your foot off. A few years go by. "I should use a _real_ language, I'm not an amateur anymore." So you write everything in Python. A few years go by. "I should learn shell, and use it only when appropriate." Maybe this is what Perl is for, but I never learned it.
deleted 3y ago
[deleted]
wodenokoto 3y ago
> Have windows users use WSL (the VSCode integration is great!), and mac users should install GNU tools since the system tools are obnoxiously incompatible. At that point you might as well target Python 3.6. Seems like the same hassle for the developer to install and you don't have to worry about wonky differences for users who haven't installed GNU tools, but still think they can run your script because it says `.sh`

graypegg 3y ago

I've always found it weird that the NPM ecosystem doesn't have something like Rake from the Ruby world to run tasks. Javascript things tend to be VERY task heavy, with dev servers, bundlers, testing, and coverage all being defined in the project normally. The package.json scripts "work", but it's quite clunky, and relying on shell scripts that run node.js scripts causes issues. (cross-env solving a problem that really shouldn't exist.)

williamdclt 3y ago
I’m of the opinion that the package.json scripts was a mistake (although not a huge deal in the grand scheme of things): it solves 80% of the problem, which removes enough pain that there’s not enough incentive to solve the last 20%. I’m sure there’s something like rake that exists for Node, but the community won’t standardise on it because it’s not enough a problem
chuckadams 3y ago
Gulp is still a thing. Or even Grunt if that’s your poison.

simonw 3y ago

I would have agreed more with this a couple of years ago when maintaining a script written in an alternative language required me to know that language. These days I'm much more comfortable both writing and maintaining code in languages that aren't my daily driver (like Bash or jq or AppleScript or even Go) because I can get an LLM to do most of the work for me, and help me understand the bits that don't make sense to me.

dguest 3y ago
Soon many projects will be written in LLM: a mix of whatever languages the models generated on given day + an LLM so that the maintainers can understand it.
darby_eight 3y ago
I still find it hard to believe that LLMs provide any real edge over the kind of googling/man-paging that's necessary for understanding scripts, especially given the false positives chatbots are known for. Typically language + feature + library is enough to look something up in under ten seconds for me. Granted, this probably takes a fair amount of experience to even know what you're looking at well enough to search for it.
- simonw 3y ago
  The good ones (Claude 3 Opus, GPT-4) are really incredibly good at understanding scripts. It's very rare that they hallucinate anything, especially important details for the more widely used tools that I tend to stick to. I trust LLMs with tools like Bash and jq and ffmpeg which have been around for years. I wouldn't trust them with anything released within the past 12-24 months. An example from just the other day: https://til.simonwillison.net/go/installing-tools https://til.simonwillison.net/go/installing-tools I wanted to understand this: go install github.com/icholy/semgrepx@latest How does that @latest reference mention? There's no branch or tag on that repo called "latest". I tried and failed to find documentation. I gave up and asked GPT-4, which said: > @latest: This specifies the version of the package you want to install. In this case, latest means that the Go tool will install the latest version of the package available. The Go tool uses the versioning information from the repository's tags to determine the latest version. If the repository follows semantic versioning, the latest version is the one with the highest version number. If there are no version tags, latest will refer to the most recent commit on the default branch of the repository. Is that correct? I have no idea! But it still gave me more to go on than my failed attempts with the real documentation.

SamBorick 3y ago

I like to take this a step further and try to have all my most important tooling be written in my projects major languages. It's not a must have, but having a build system for a go project that is also go under the hood means I'm more comfortable diving into the source if needed. I don't think it's a requirement, but it's an advantage

arwhatever 3y ago

We should use YAML for all of our automation so that non-developers can build and maintain the automation, and then exclusively assign developers who are proficient in a general purpose programming language to build and maintain the YAML automation. </s>

hooverd 3y ago
Simply put the general purpose programming language inside the YAML automation.

quwert95 3y ago

For some languages this can make sense, but I think most languages aren't suited for installing things, manipulating the filesystem, etc. I think of scripts as the middleware between the operating system and the shipped code. The code is controlled by the operating system, so the operating system's tools should be used to manage it. In many cases this means bash or make. Plus, I don't want modern Javascript to do things on the filesystem that would require importing dozens of projects that I need to vet before using. Golang or Python perhaps, but the buildchain for modern Javascript is hell as-is; it doesn't need another layer of Javascript.

csours 3y ago

Scripts should be commented with intent. (I can figure out what it is doing, but I have no clue WHY) Scripts should be testable. Scripts should use functions with appropriately scoped variables. (This really helps with testability) Scripts should list assumptions. Scripts should CHECK assumptions. Scripts should call commands with --long-style-options instead of -L, especially uncommon options. --- As someone who migrated a couple hundred shell scripts over the past year, I'd rather have these done before I ask someone to write a script in C. edit: and for ${deity} sake, use shellcheck.

shrimp_emoji 3y ago
> Scripts should call commands with --long-style-options instead of -L, especially uncommon options. Too bad `getopts` only supports single-char options. :p
- csours 3y ago
  I'm not a fan of getopts And to be more specific - I meant when the script calls someone else, not how it handles options.
jjgreen 3y ago
I think you mean "${deity}" in case of spaces ...

MichaelRo 3y ago

>> Scripts should be written using the project main language ... as long as that main language is a scripting language. Otherwise it's just dumb. Also, tunnel vision by the author: "Almost all projects I’ve worked on have scripts we wrote to automate a repetitive process. " Well almost all projects I’ve worked on have scripts we wrote to automate a ONE-TIME process. Like collect some data from the log to figure out a bug, fix it and forget about both the bug and the script. Automate it since can't manually process 30Gb of data and grep only can do so much. Sure as funk won't write the "script" in C++ but Python or Perl or something.

cjk 3y ago

Ehhhhhh. I get the sentiment, but I think this really depends on what the "main" language is and how amenable it is to scripting. Personally, I prefer writing shell scripts regardless of what the main language is in a given project. They're portable (more or less), and can e.g. detect missing dependencies and install them if necessary, which isn't possible with the main language if you're missing its compiler or interpreter.

mthoms 3y ago

Using Google’s zx I write my support scripts in JS and can use shell features like piping right in the same file.

taeric 3y ago

Disagreed from me. Though, I'm not sure how hard my disagreement truly is. Discrete programs are superior in many ways, as you do not immediately incur maintenance costs to write them. More, they typically force you to have a discrete API that they work with, and then you can lean on that. Yes, you can do all of this with modular programming techniques. Indeed, "unit tests" are easy to see as similar to what I'm advocating here. Such that I think my assertion is softer than many folks are probably seeing. If you are "scripting" something to add data to the system, it should emphatically not hit the database directly. I don't know where this lands me on the infrastructure as code (IAS) debate. I'm sympathetic to the desire. I start to think of it as navel gazing when I see some of the very engineered testing practices some people take those to.

wakawaka28 3y ago

Absolutely not. Use the right tool for the job. There's nothing particularly wrong with shell scripts, as long as the people who maintain them are reasonably good at not making them into a dangerous nightmare waiting to delete your home directory or something.

jes5199 3y ago

man, I work in python but I really don’t enjoy writing scripts in it. Everyone here acting like it’s a scripting language. Culture shock.

rottc0dd 3y ago

The performance concerns might creep up for scripts and tricky to figure out the issue if you are not internal abstractions. Example, https://github.com/berry-thawson/diff2html https://github.com/berry-thawson/diff2html

noobermin 3y ago

This really only applies to a subset of languages and projects with such languages. I wish developers had the courage to face the deeper problem (being unwilling to read code...older than a few weeks? Such a developer will face worse things in the furure, they should train their focus stamina)

hgs3 3y ago

The authors main bullet points are: > The learning curve is minimal since you already know the corners of the language. Learning a new language shouldn't be difficult. Programmers are expected to familiarize themselves with new tech. > Internal language APIs can be leveraged, which drastically changes the mental model to write the script (for the better). This is true. I myself have encountered situations where I needed to call into my C API's from a higher-level language, but since most languages can interface with C this hasn't been an issue for me. For example I've interop'd Go+C, Python+C, and Lua+C. > Scripts feel more natural and eventually maintainability increases. Team members are familiarized with the language! This sounds like a subjective rehash of the first point. > Development machines compatibility increases. Windows users can finally run all scripts. This is true if you're talking about shell scripting, but if you're scripting with a general purpose programming language then it shouldn't be an issue. What language (besides shell) isn't portable these days? And even then, you can install a *nix environment on Windows.

psychoslave 3y ago
I think Crystal is a popular one which doesn't yet support windows fully, at least last time I checked
hyperhopper 3y ago
> Learning a new language shouldn't be difficult. Programmers are expected to familiarize themselves with new tech. I wish any large company agreed with this. I've worked for a company that on boarded every single new engineer to a very niche language (F#) in a few days. Also, everybody I worked with there was amazing. Probably because of that kind of mindset. Meanwhile google tiptoes around teams adopting kotlin because "oh no, what if other teams touching the code might not be able to read it". Google is supposed to be hiring the brightest but internally is worried the brightest can't review slightly-different-java. It's shocking how everybody acts like senior engineers might need months to learn a new language. Sure, maybe for some esoteric edge cases, but 5 mins on https://learnxinyminutes.com/ https://learnxinyminutes.com/ should get you 80% of the way there, and an afternoon looking at big projects or guidelines/examples should you another 18% of the way.

nosefurhairdo 3y ago

Depends on the project, depends on the team, depends on the language, depends on the script. Pick a sensible tool for the job, write understandable code, don't lose sleep over silly generalizations.

__mharrison__ 3y ago

So ... Python

LispSporks22 3y ago

We have some Perl scripts that are freaking immortal. The rest of the code base started in Java, then Clojure, now it’s Go. The scripts are there still in their very-not-modern Perl style though. They have a self-evaluating behavior consisting of data blocks that are interpolated. To be honest, I’m not sure exactly how it works. Very discouraging for the casual passerby looking for some cleanup to do.

throwbadubadu 3y ago

Say this to your C++ embedded project (no lang rants now, it is what it is;)): ehrm, no, please no, ridiculous.

brunooliv 3y ago

I think while the basic idea of advocating for stack consistency between the main project and any support scripts is very nice, I always find it not worthy to pursue in practice for several reasons: a lot of the “environment” around the main codebase will be a weird mix of YAML, bash and a scripting language like Python or Ruby for things like gitlab, airflow, GitHub actions, etc. Given this heterogeneity of the project environment and additional complexity of “forcing” something to use a language it might not be very well suited for, makes this really a no brainer for me: use the most convenient tool for the job. Plus as a JVM bound developer I love me my occasional Python

treyd 3y ago
I agree with this sentiment. At work our main codebase is in Rust and early on we wrote some CI tooling using the cargo xtask workflow and it adds a lot to compile times when we need to rebuild it (which is often since it depends on several of our main codebase's crates). It really kills iteration times. This is a mess and added a lot of extra complexity to the build pipeline since now we had to manage an additional xtask container. Python is very well suited for CI scripting, xtask should only be used for things directly run by the developer, and even then Python may be a better choice for most things.

doktrin 3y ago

Given the resultant comment section I clicked into the article expecting the most sith-like of absolutist proclamations taking itself extremely seriously and actively fanning arbitrary ideological flames. Instead I found a concise, measured and sane opinion that I honestly can’t disagree with. Like, sure, if your language broadly supports the secondary use case (scripting) why not? If it doesn’t, then the juice almost certainly won’t be worth the squeeze.

pjmlp 3y ago

The clear example of only having an hammer and every problem is a nail.

t43562 3y ago

Scripts are usually no better than they have to be - they're not the main purpose of the system. Scripting languages handle all sorts of issues that take pages of code in other languages so they are the easiest way to do what you need and easy wins. IMO some languages like C/C++ cry out for an embedded scripting language so that you don't write the basic, one off, performance insensitive parts of your code in a "hard" language. This is taking it the other way around - suggesting that as much as possible of your "main project" should be in a scripting language so that you're not wasting "hard" development cycles on areas that don't need it.

GoblinSlayer 3y ago
>pages of code Code reuse. The benefits of hard language: 1) I now can do interesting things like text parsing, 2) my scripts are cross platform, 3) I don't need to figure out how to deploy python everywhere in advance or on demand, 4) if the user has python, I don't need to tell him I don't like his python version and he must install a different OS, 5) I don't have python as an extra dependency, 6) I can reuse my main code in my scripts, 7) scripts are written in a language with a decent type system.
- t43562 3y ago
  One small counter I have is that most software development has dependencies anyhow - and with compiled languages this is quite a pain in the arse to manage. So installing tcl or lua or python or ruby or perl is just a normal problem amongst other problems. If you want type systems everywhere then I think that's another debate. That is really a total rejection of almost all scripting languages for all purposes.

willtemperley 3y ago

Keeping scripts in the main project language usually means the monolith gets larger and more like a final boss than something the team controls. Use the language that best fits the job. For me that's TypeScript on a serverless platform, SwiftUI front-end and Java for business logic. I used to feel like I was winning using only Java for UI, Web and business logic, but the complexity became immense. It's too easy to create yet-another-internal-API that gets forgotten until it needs to be refactored. Also learning a language gives you a new perspective on software engineering, a bit like learning a foreign language gives a new view on human culture.

stevage 3y ago

Agreed. My scripts got much better when I abandoned Bash, and switched to Zx, which is basically NodeJS.

amne 3y ago

I try very hard to do just the opposite. If I need a script to do something existing code can't ,then it is most likely exciting and I'm going to use that energy to learn something new, be it a language or tool. Here's a hotter take: your team colleagues are more likely to wear your clothes than to use your scripts.

rkerno 3y ago

Really interesting comments here. I haven't found an appealing option for scripts and CI ( I think I am allergic to yaml ), bash is just way too fragile. So I've decided to write my own scripting language that is written in Go, similar in many ways to Lua, but with first class support for executing other commands, running API tests, and manipulating data. Currently dogfooding with plans to share in a few months once I'm confident it's good to start getting feedback. So, one executable plus your scripts, cross platform, no yaml.

mike_hearn 3y ago

Experience report: I did this for my company that uses Kotlin as its main language and it worked out brilliantly. We extended Kotlin Scripting into "HShell" and it swiftly replaced basically all uses of bash or Python for our internal scripting needs. The docs are public here, although the product isn't open source. It gives you a flavor of how it works though: https://hshell.hydraulic.dev/14.0/ https://hshell.hydraulic.dev/14.0/ Advantages: • It's as concise as bash but far more readable, logical and less bug-prone thanks to the static type system with plenty of type inference. • IntelliJ can provide a lot of assistance. • You can easily import and use any JVM library, and there are lots of them. • Ditto for internal project modules if you work on the JVM. • If you need to, you can easily mix in and evaluate Python/Java/etc using Graal. It has an integrated disk cache that can be used for storing file trees that manages free disk space automatically, and there's a command to set up a Python virtualenv in the disk cache transparently. • We have a high level shell API that makes console, file and network operations as easy as in bash, and sometimes easier because the commands aren't afraid to deviate from POSIX when that would be more convenient. For example most operations are recursive by default. • A smart progress tracking framework is integrated into every operation including things like wget. • It's fully portable to Windows including edge cases like being able to set POSIX permissions on a file, add it to a tar, and the resulting tar will retain the correct permissions. • SSH is deeply integrated, and so commands "do the right thing" automatically. For example all the commands take Path objects as well as strings, and path objects track which machine they refer to, so if you open up an SSH sub-shell you can easily copy to/from the remote machine using regular copy/move commands. Other commands are "smart" for example the wget() function given a path that's on a remote machine will execute curl or wget remotely rather than download locally then reupload. Although this sounds like it was all a lot of work to build, in reality our main product is a kind of build system (it makes deploying desktop apps easy, see bio for link). So all that functionality is in actuality functionality we built for the product and just kept nicely factored out into modules. The work invested into the scripting-specific parts is probably a couple of weeks over a period of a couple of years, and it was well worth it given the number of scripts we have for things like QA, deployment, server management and so on.

theshrike79 3y ago

I usually go with the Google Shell Style Guide: https://google.github.io/styleguide/shellguide.html https://google.github.io/styleguide/shellguide.html * If performance matters, use something other than shell. * If you are writing a script that is more than 100 lines long, or that uses non-straightforward control flow logic, you should rewrite it in a more structured language now. Bear in mind that scripts grow. Rewrite your script early to avoid a more time-consuming rewrite at a later date. * When assessing the complexity of your code (e.g. to decide whether to switch languages) consider whether the code is easily maintainable by people other than its author.

jakupovic 3y ago

There is a reason shells and shell scripts exist. Reinventing the basic shell functionality seems pretty dull and better spent staring at a wall to decompress. I always start with a simple shell script which does most of the work as simple as possible, even simpler is using `make` then introducing shell scripts when needed.