11 ms·
OK, dumb question, because there have been a LOT of these types of stories lately: Why does it matter (beside to the implementor, or for pedagogy) what languag
by colanderman 8y ago
OK, dumb question, because there have been a LOT of these types of stories lately:
Why does it matter (beside to the implementor, or for pedagogy) what language a CLI tool is implemented in?
- cube2222 8y agoRust and Go are languages with ecosystems, in which writing everything in a cross-platform way is the default. They also compile to a single statically linked binary. You don't need an additional runtime + load of dependencies like node or python. They are lightweight and fast to start up as opposed to the jvm.
- shawn 8y agoPython is installed literally everywhere. If it’s a CLI tool, and it’s written in python, the chance that it won’t work is slim. Go has a massive number of problems. For example, I tried to run Keybase’s standard “go get” build instructions. It failed with 200 import errors. That was the end of my attempt to install keybase on my raspberry pi. Others had said that it works. Rust requires a massive amount of hard drive space and takes a long time to build. You also have to build it. That’s antithetical to rapid development. I can't wait until the pendulum swings back away from static typing and the next generation of programmers discover the benefits of literally ignoring everybody and doing your own thing. It’ll be painful, but at least it’ll be effective. And you won’t have to compile anything.
- cube2222 8y agoYou don't need to go get or install the rust sdk if you just download the small, statically linked binary.
- shawn 8y agoThere is no such binary for the raspberry pi. If it was written in python, it would have worked. It likely would have turned out smaller, too.
- burntsushi 8y agoxsv doesn't exist if it was written in Python. It would definitively be too slow. If you don't care about performance and would rather not wait a couple minutes to build the tool on your Pi, then go use csvkit, which is written in Python. The availability of software isn't a zero sum game.
- shawn 8y agoOhhh, you wanna throw down eh? Hmmm. This would be a fun weekend project to reimplement XSV in Python and prove this wrong. :) Now, to make this a fair comparison, are you excluding pypy? Or is that allowed for our game? How about LuaJIT or Node? Are those fair too?
- sanderjd 8y agoYou should definitely do this. Personally, I strongly suspect you wouldn't prove him wrong if you did attempt this in any of the languages you mentioned. But if you're right and we're wrong, I'd love it! It would be great and eye-opening to dig into your implementation(s) to see how you pulled it off.
- JoshTriplett 8y ago> This would be a fun weekend project to reimplement XSV in Python and prove this wrong. I don't know as much about the internals or performance characteristics of XSV (though it certainly touts performance as a feature), but if you can reimplement ripgrep in Python and get anywhere close to the same performance, I'd certainly be interested to see that. > Now, to make this a fair comparison, are you excluding pypy? Or is that allowed for our game? PyPy is not available everywhere, unlike the CPython runtime or the ability to run compiled binaries.
- burntsushi 8y agoI would definitely love to be proven wrong, because if I am, I am certain I would learn something new. I am pretty comfortable with Python, so I am pretty comfortable saying that you could not write a tool as fast as xsv in Python without writing some substantial portion of it in C. Using Python's standard library CSV parser is certainly fair game, but writing more C code on top of that which included the loop over records in a CSV file feels like a cheat. The problem here is that you've already lost because xsv's CSV parser is faster than Python's parser at the C level[1]. :-) Assuming I haven't made any silly gaffes in xsv itself, I don't see how you're going to make up the difference. Because of that, I am willing to extend a handicap: if you can get within the same order of magnitude as xsv while doing the same amount of work in a robust way, then I think I might even learn something there too. :-) I am less certain about PyPy, LuaJIT or Node, but I am certain I'd learn something if you proved me wrong. Note that a problematic part of this challenge is that your program would need to correctly handle CSV in a robust manner. It is very easy to write a very fast CSV parser that doesn't correctly handle all of the corner cases. Python's CSV parser certainly passes that test, but I don't know if Node's or Lua's CSV parser does because I've never used them. [1] - https://github.com/BurntSushi/rust-csv/blob/master/csv-core/src/reader.rs https://github.com/BurntSushi/rust-csv/blob/master/csv-core/...
- h1d 8y ago> Python is installed literally everywhere. I figured Ubuntu 18.04 didn't have Python pre installed. Besides, installing all the dependencies with pip is another step to do and gets annoying when deploying to many servers. For something that gets distributed, a single static binary is very welcomed.
- ptman 8y agoI think it has python 3, but not python 2
- mmirate 8y agoWhich Python? 2 or 3?
- GordonS 8y ago> Python is installed literally everywhere Nope, not on Windows.
- colanderman 8y agoEverything you said applies to many other languages, including C, which is probably the most common language used to write CLIs if you rank by hours used. Why not advertise the unique properties of this tool (fast, easy to install) than its implementation language? Does this ultimately boil down to "go install" (or whatever the Rust equivalent is) is easier to deal with than "./configure; make"? Maybe I'm just a curmudgeon and don't understand why it seems to be in vogue to ignore OS package managers, which hide all this complexity anyway.
- leshow 8y agoI think the tools speak for themselves. They get use and then they get included in repos, not just by virtue of the fact they were written in lang X.
- colanderman 8y agoThen why advertise that? Why not advertise their benefits instead (fast, cross-platform, easy install)? "Written in X" is only useful to advertise pedagogy or integration.
- mmirate 8y agoShorthand. Compare the lengths of these two strings: > fast, cross-platform, easy install > Written in Rust Also note how the second statement expresses a single concept; whereas the first statement is a list of concepts that must be memorized and reproduced by rote, and is therefore constantly at-risk of being stated incompletely or inaccurately. (And it is incomplete: there are many more reasons why "Written in Rust" might be important, such as correctness, security, and ease of maintenance/refactoring.) * * * * * Alternatively: why convey a set of vague descriptors when you can instead convey precisely why they are applicable, and let the audience's pre-existing memories fill in the details for them?
- colanderman 8y agoLike I said, "written in X" conveys pedagogy and integration. (Or, if we're being cynical, hip language clickbait.) It's not a very good shorthand.
- GyYZTfWBfQw 8y agoI have had more problems with pip, go, and cargo, than pacman and ./configure && make && make install. I don't know what you are on about.
- shawn 8y agoIt doesn’t. It’s one of the most annoying aspects of Rust that “Written in Rust” is considered an attractive quality. But, whatever. There’s no reason to be gratuitously negative. I’m just a bit salty that lesser known languages are often excluded on the basis that they’re lesser known.
- pests 8y agoIt does matter. Not only for the reasons my sibling comment states but others as well. Such as the safety/security guarantees Rust makes by default. I would trust any binary decoder that is written in Rust more than the equivalent written in C.
- shawn 8y agoI’d trust Tarsnap with my life, and it’s written in C. You have to evaluate the merits of an individual project, not blindly put your faith in their security claims. I say this as a former pentester.
- sullyj3 8y agohttps://www.mckinsey.com/business-functions/strategy-and-corporate-finance/our-insights/daniel-kahneman-beware-the-inside-view https://www.mckinsey.com/business-functions/strategy-and-cor... In general, when you write C, there are a multitude of ways to introduce memory related vulnerabilities if you don't know what you're doing, and historically and empirically speaking, even if you do (heartbleed comes to mind, but I'm sure 5 minutes of research would turn up hundreds of examples). Safe rust prevents you from double freeing, dereferencing null, leaking memory, and a whole bunch of memory related bugs/potential vulnerabilities. Now, programmers are human. We make mistakes. Why, a priori, should anyone trust the programmers of one particular project to be superhuman and never screw up? That seems much more like blind faith to me than trusting a language which has been specifically designed to eliminate these kinds of errors.
- shawn 8y ago
- pests 8y agoA lot of CLI tools are quick scripts or other programs that will start, execute, and then die quickly. If a tool is written in an interpreted language or has to load a large VM on each invocation than a large percentage of the total runtime of the CLI tool will be spent on language-specific tasks. With a non CLI tool such as a server or daemon the expectation is you start it once and it runs for awhile. In these cases startup time does not matter as much. Furthermore, for interpreted languages like JS/Python/PHP and languages that require a VM such as Java or C#, not everyone will have or want to install each languages runtime.
- colanderman 8y agoYes, but Rust is far from unique in this regard. Maybe I shouldn't be surprised that "written in language without onerous runtime and installation overhead" is apparently now rare enough to be a selling point.
- wybiral 8y agoLanguages like Rust and Go are just really easy to cross-compile compared to working with lower level OS-dependent APIs in C or something. There's nothing inherently unique, they're just a good combo of features.
- saagarjha 8y ago> Languages like Rust and Go are just really easy to cross-compile compared to working with lower level OS-dependent APIs in C or something. You make it seem like C has no standard library. C is really easy to cross compile: you just add a -arch flag to your compiler frontend and, provided you're staying inside POSIX (which you should!) it should "just work".
- hsivonen 8y agoC's standard library is completely broken for text handling (depends on external locale instead of caller-specified locale) and on Windows broken for file system access if your paths contain non-ASCII. (At least until the latest dev version of Windows 10.) Rust has the best file system abstraction for exposing Windows file paths in a way that allows you to write common file path handling application code for Windows and *nix.
- sullyj3 8y agoWell for starters you know it's much less likely to leak memory, segfault, or experience buffer overflows. Not necessarily relevant to xsv specifically, but speaking of rust tools more generally.
- littlestymaar 8y agoThe language itself doesn't have much importance, what matters is : - is it portable ? Or will the users have to install a specific runtime to run the tool ? For Linux it's not a big deal because it comes pre-installed in most distributions, but making Python work on Windows requires some work. It's not too much if you're going to use the tool often, but for merely testing it, the runtime installation burden can deter users. - is it fast ? For many application, speed doesn't matter, but for some it matters. - is it stable ? Or will it crash every now and then ? C or C++ software have memory bugs, that leads to crashes (segfault). The most mainstream one don't have too much of it, but as soon as you start using niche software, you'll face crashes more often than not. For these three reasons, the language matters, but there's not much difference if it's Rust, Go or Haskell.
- earenndil 8y ago* Python is not all that difficult to package into an exe. C/C++ are very cross-platform. * Python can be reasonably fast with pypy. * Go and rust maybe don't have segfaults, but they can still crash and burn, or have other kinds of bugs.
- TheCoelacanth 8y agoPython is not hard to package into an exe, but most Python programs aren't packaged like that. C and C++ are very cross-platform, if you only use cross-platform dependencies and if you are careful to write portable code. Most C and C++ applications don't do that.
- earenndil 8y agoHuh? What do you have to do to be careful? Just: * Don't include windows.h * Use a gui toolkit like gtk and qt (which you probably want to do anyway) Generally, making a software windows-only is a deliberate choice, not something that happens 'by accident'.
- realusername 8y ago
- nickm12 8y agoThe language tells you a great deal about whether the tool will fit in with your other tools in just a word or two. "command-line" here tells you a bunch as well, but there are command-line tools that require a runtime env that you might not want to deploy and maintain just for one tool.
- lilydjwg 8y agoBecause: * Some languages don't run on some platforms, e.g. C# doesn't work well on Linux. * Some languages are hard to build (for real projects), e.g. C or C++ projects, Python with C extensions on Windows. I don't have a good time to build Haskell projects (on Linux) either. * Some languages require a huge runtime (either install seperately or bundled), e.g. Java. They take a lot of time to download and install, and occupy a lot of disk space and memory.
- mwcampbell 8y ago> C# doesn't work well on Linux. Not true anymore. .NET Core, including ASP.NET Core, works just fine on Linux.
- k__ 8y agoI just read good about Mono.
- xnyan 8y agoMono is great. .NET core is also underrated to the detriment of all imo.
- PioNono 8y agoI’ll share a use case where Rust specifically has been a godsend. I write software for users who have strong engineering backgrounds but tend to know just enough about computers to be dangerous. They tend to know a smattering if Python, Perl, SQL, SAS, R, or Matlab, and will happily use a well-documented CLI tool, but cannot be relied upon to do basic sysadmin tasks effectively. They run a mix of windows and OSX, and some set up Linux VMs. They are hired for their expertise in their domain, not because they are great IMS technicians. They deal with large, heterogenous, difficult-to-schematize datasets and value speed, but it is a total waste of their time to troubleshoot environment/linking issues that frequently accompany building and using C/C++ utilities for many non-programmer users. I’ve found Rust to be hugely beneficial for writing programs to serve this user base (RF engineers in my case, but I’d imagine people who write software for ag/life science people, banking analysts, civil/environmental engineers and others have similar challenges). * it’s also nice that it’s hard to segfault in Rust. Many existing tools in my domain are notoriously fragile in this regard.
- bigger_cheese 8y agoWow that sums up my world precisely - also engineering. Especially the part about people knowing a smattering of Perl, Python, SAS and Matlab. Add in C++ (written in circa early 90's style i.e C with "new" and "delete") and Fortran (Fortan 77 style straight out of the classical Numerical Recipes text book) you could be describing my org. People here have so many issues managing things like library dependencies. This is especially a problem on Windows. I remember witnessing the guy next to me waste hours trying to build and link an image analysis library written in C++. I could feel his frustration from across the partition "This library needs this dependency, which has to build from scratch but before I can do that I have to build library 'zzz' but to do that I have to download something called "CMAKE". I've finally done all that but then I forgot to set a compiler flag correctly in some library 3 steps up the chain and now I have to start all over again. At the end of all that you better be damn sure you were paying attention and didn't accidentally mix some 32 / 64 bit code together which apparently is pretty easy to do when you are downloading random code from various websites." I think these were instructions he was trying to follow (https://docs.opencv.org/2.4/doc/tutorials/introduction/windows_install/windows_install.html https://docs.opencv.org/2.4/doc/tutorials/introduction/windo...) Stuff like configuring environment variables, editing make files etc are very difficult for a lot of otherwise smart people to understand. These people have no problem writing code to do quite complex analysis just building it and distributing it is quite challenging.
- groestl 8y agoBesides the ops-story and culture a language implies, if you rally for contributions to your free software'd tool, you can use the language as part of the marketing message to make sure your users also like your language (and therefore are more likely to contribute).
- sh4z 8y agoWell, my guess is it matters for the upvotes the post receives on HN - hence you're seeing a lot of these posts. Leave out "written in X" and you'll be missing out on upvotes from the X community.
- coldtea 8y ago(1) It matters to those who follow the language's progress, because it points to increased adoption. (2) It matters for tools that, like csvkit and ripgrep, make parts of their implementation available as libraries. (3) It matters because e.g. Rust is both fast as C/C++ and safe-by-default for more things than C/C++, so one can expect xsv to be faster than e.g. the Python based csvkit. (4) Startup costs, static binary, etc, things that are strongly influenced by the language used, matter. (5) It matters for those wanting to contribute to the tool, since it's also open source. (6) It matters because we are programmers here on HN, and we want to know about the internals of the programs we use, check their implementation, learn from their coding style, etc., not just to use them as mere consumers. Should I go on?
- _ph_ 8y agoFrom the perspective of the program: you might get a hint about the speed and robustness, knowing it is written in Rust. But what is more important: for all, who are interested in programming languages, there is no better showcase for a tool, in this case a programming language, than to have real-world examples of useful things created with it.