6 ms·
Babashka is a fast-starting scripting environment for Clojure
- timdeve 4y agoI've been using it for quick web scrapping scripts and it's really nice.
- nerpderp82 4y agoWhat libraries do you use? I do most of my scraping in Python using beautifulsoup.
- aeonik 4y agoNot OP but I use Reaver with good results. It supports all of JSoup's selectors, and makes it very clean to extract data from HTML. The documentation is a little lacking though, I had to look up other examples on GitHub to figure out how to use all the features. https://github.com/mischov/reaver https://github.com/mischov/reaver
- Borkdude 4y agoBabashka doesn't have a built-in HTML parsing library but it supports it through pods: https://github.com/babashka/pod-registry https://github.com/babashka/pod-registry Pods can be written in any language and they can expose functions to babashka by implementing a protocol. One pod exposing HTML parsing is: https://github.com/retrogradeorbit/bootleg https://github.com/retrogradeorbit/bootleg Here is an example of how to use that: https://github.com/babashka/pod-registry/blob/master/examples/bootleg.clj https://github.com/babashka/pod-registry/blob/master/example...
- noblepayne 4y agoAs mentioned by the one and only Borkdude, bootleg is a nice option for this. It includes the Hickory library: https://github.com/clj-commons/hickory https://github.com/clj-commons/hickory I'm a previous BeautifulSoup user and have found the combination of (1) having the scraped data presented in plain Clojure data structures, and (2) Hickory's built in selectors, to be a very nice experience. Happy scraping!
- nathell 4y agoI plan to port my scraping framework (Skyscraper, https://github.com/nathell/skyscraper https://github.com/nathell/skyscraper) to babashka one day. I’m not sure how easy it will be, though, since it uses core.async (which I believe bb has limited support for) and SQLite via clojure.java.jdbc.
- kot-behemoth 4y agoCore.async is listed as “built-in” in the Babashka Toolbox (https://babashka.org/toolbox https://babashka.org/toolbox). Might be worth checking if the compatibility has improved. And bb supports Honey SQL and SQL pods (https://github.com/babashka/babashka-sql-pods https://github.com/babashka/babashka-sql-pods) - so you might be already compatible!
- timdeve 4y agoAs other people have said Bootleg + Hickory. Here is an, admitedly not very clean, example that grabs stream urls from hltv.org: https://github.com/TimDeve/.dotfiles/blob/master/scripts/general/hltv https://github.com/TimDeve/.dotfiles/blob/master/scripts/gen... Also a basic RSS reader using the clojure XML lib: https://github.com/TimDeve/.dotfiles/blob/master/scripts/general/read-that https://github.com/TimDeve/.dotfiles/blob/master/scripts/gen...
- nerpderp82 4y agoThanks everyone, all great pointers. I'll do my next scrape in Babashka!
- bokchoi 4y agoBabashka is great! I've used it for doing some munging of csv and xml files.
- the-alchemist 4y agoIt's this really nice combination of bash, curl, and libraries for JSON, CSV, ZIP files, and almost everything else you could need. Full list at [0]. You can always spawn a shell too. It starts just as fast as Python too, if not faster, at least on my machine: time python3 -c 'import os.path; print(os.path.exists("README.md"))' time bb -e '(.exists (new java.io.File "README.md"))' Python averages around 40-50ms, and bb averages 30-40ms. [0]: https://book.babashka.org/#built-in-namespaces https://book.babashka.org/#built-in-namespaces
- capableweb 4y ago> Python averages around 40-50ms, and bb averages 30-40ms. Sounds like you have a slow harddrive here, rather than measuring the startup of the interpreters. With hyperfine: $ hyperfine --warmup=100 --runs=1000 'python3 -c "import os.path; print(os.path.exists('"'README.md'"'))"' 'bb -e '"'(.exists (new java.io.File "'"README.md"'"))'"'' Benchmark 1: python3 -c "import os.path; print(os.path.exists('README.md'))" Time (mean ± σ): 5.3 ms ± 0.7 ms [User: 4.9 ms, System: 1.2 ms] Range (min … max): 4.5 ms … 9.8 ms 1000 runs Benchmark 2: bb -e '(.exists (new java.io.File "README.md"))' Time (mean ± σ): 6.0 ms ± 2.5 ms [User: 2.5 ms, System: 4.3 ms] Range (min … max): 0.4 ms … 10.6 ms 1000 runs Summary 'python3 -c "import os.path; print(os.path.exists('README.md'))"' ran 1.12 ± 0.49 times faster than 'bb -e '(.exists (new java.io.File "README.md"))'' Running on the following "not absolutely shabby" specs: > Intel i7-1185G7, 6.0.11-arch, Toshiba NVMe SSD (KBG40ZNS256G) Seems Python has more consistent startup performance (`4.5 ms … 9.8 ms`) while Babashka can startup faster (`0.4 ms … 10.6 ms`) but on average, Python startups faster.
- __jem 4y agobb is 1.32 ± 0.08 times faster on my computer (m1 mbp).
- charesjrdan 4y agoI think this is one of my favourite pieces of tech in the past five years tbh. I still use bash for short <5 line scripts but everything else is bb (though I’ve started looking into nbb because you can use node libs like ink which seems pretty cool) And repl integration with neovim and conjure is great!
- musha68k 4y agoI personally stopped using it since both shellcheck and jq fill their own niches way too nicely “unfortunately”… None the less, a lovely tool and the couple of scripts I wrote are mostly still in use and well maintained for a reason ;) Clojure is just fantastic and so is the speed of the GraalVM.
- pixelmonkey 4y agoThis is a particularly joyful piece of open source software. After seeing Michiel Borkent -- its author and writer of OP -- present on it at a meetup, I wrote a tidy overview of babashka (bb) which you can find here: https://amontalenti.com/2020/07/11/babashka https://amontalenti.com/2020/07/11/babashka
- kopos 4y agoAll our cron jobs and scripts with non trivial logic are in babashka now. It has been a joyful experiences
- __MatrixMan__ 4y agoThe positivity in the comments here makes me want to learn Clojure. Apparently it makes you happy.
- capableweb 4y agoIt absolutely makes you happy. Be beware, finding a language that makes you too happy, can lead to you not wanting to program in anything else anymore. You're better off being in the middle of grumpy and happy, like a normal programmer.
- eyelidlessness 4y agoDon’t worry, folks! Clojure will make you happy, but even with fast perf you may find yourself wanting static types or even just dynamic runtime result/option types. Of course it’s a lisp with macros, and a general purpose language, so you can have all of that too. But there’s no shortage of green in other pastures ;)
- fud101 4y agothis is why i avoid common lisp now. for years i hated programming until i finally decided CL was dead to me. Now i happily write Python for a living and don't care it sucks relative to CL.
- lordgroff 4y agoDon't learn a Lisp. For one, you'll end up in endless arguments whether your Lisp is a proper Lisp, and two, everything else will look ludicrously constraining for no good reason.
- eamonnsullivan 4y agoI used this for my back up system: https://github.com/eamonnsullivan/backup-scripts https://github.com/eamonnsullivan/backup-scripts The server runs on a Raspberry Pi with a 1-2TB USB disk attached.
- user3939382 4y agoI've been looking for a good excuse to use Clojure, small shell scripts sounds like a great low-risk way to do that.
- lukaszkorecki 4y agoBB is awesome - it powers our RabbitMQ management scripts, we have written a sidecar process to scrape metrics from ECS and send them to vector.dev over UDP and a bunch of other things. For personal stuff I use it in a lot of scripts used by Alfred.app including quick-fill for Things with date parsing, emoji completion, etc
- whacked_new 4y agobabashka is so good it's mind-boggling. super-easy shell interop + instant nREPL = replaces any complex bash script, driven from emacs! what's more, you can namespace your scripts and import them. functionalizing code makes scripts easy to refactor, or even trivially move into a full Clojure backend or ClojureScript frontend. With squint-cljs you can even generate self-contained JavaScript directly from babashka code.
- ducktective 4y agoIs there something similar for Common Lisp?
- comma_at 4y agoI think SBCL starts up fast enough as-is, no? If you want a batteries included version with libraries, just load them and save a new base image with SAVE-LISP-AND-DIE
- vindarel 4y agoNo, we have to build a binary, which starts up super quickly. I began to put together a "distribution" of useful CL libraries for everyday tasks: https://github.com/ciel-lang/CIEL/ https://github.com/ciel-lang/CIEL/ It comes as: - a lisp core, which you can use in your editor setup instead of sbcl or ccl, the advantage is that it loads instantly with all these libraries built-in (instead of quickloading all of them when needed) - a readline REPL, with completion, syntax highlighting and some goodies (a shell passthrough, quick doc lookup…), albeit it needs more love - a lisp library which you can quickload like the rest, it just comes with a bunch of dependencies. - I want to add scripting capacity too. It ships libraries for: HTTP requests, file formats handling (JSON, XML, CSV, TOML…), shell utils, file utils, threads, a nice hash-table notation, DB access, plotting, strings, other language extensions etc. It is not finished but I am doog-fooding it for clients' projects.
- ducktective 4y agoThank you very much! So the scope of it is like Janet-lang, right?
- vindarel 4y agoHowdy, I just added scripting support. $ ciel myscript.lisp where we can use all the available libraries, it works instantly. It's brand new, not many options right now ;) > like janet-lang I don't think so, since Janet advertises itself as an embedable Lisp like Lua or Guile (and ECL). But yes, in the sense it comes with batteries included. Like Babashka, I guess. Thanks for your initial question! --- So, in CIEL scripts right now, we can make HTTP requests (dex:get or http:get), parse JSON, CSV etc, run shell commands (cmd:cmd, the "!" shortcut in the CIEL shell), or even (defalias ! #'cmd:cmd) (! "pwd") and more. https://github.com/ciel-lang/CIEL/ https://github.com/ciel-lang/CIEL/