5 ms·
An Overview of the Starlark Language
- jacknews 2y ago"Some people might say it’s a bit like Lua with Python syntax" ye gods
- benreesman 2y agoI can appreciate that it might not sound that appetizing but it’s really pretty good. I worked on the original Buck and kind of caught the “principled and performant polyglot build system” bug and I’ve been diving down that rabbit hole since. There are still usability and adoption challenges, but it’s the future and between buck2 and bazel7/bzlmod the gap to slow, non-deterministic alternatives is closing fast. I wouldn’t write it off.
- __float 2y agoWhy do you call out bzlmod? I haven't heard great things about it.
- ithkuil 2y agoI'm happy with bzlmod. What I hate about it is that not every bazel ruleset I use has yet been ported to it. But it's getting there. For example in my monorepo I'm only missing rule_distroless and I migrated all my deps to bzlmod. The modules lock file situation has been greatly improved since 7.2.0-rc1. The lock file contains much less digests in it and its less likely to cause spurious merge conflicts With bazelisk and .bazelversion we managed to have everybody use the same version of bazel and thus we could cheaply try out the rc1 and so far is working fine.
- benreesman 2y agoTo the extent there’s a real issue with ‘MODULE.bazel’ (aka bzlmod), it’s that Bazel has such a rich ecosystem even on the old, janky ‘WORKSPACE’-style that the slick new thing that always works is impossible to migrate in one go. The Bazel team flags things off by default for years, and then things on for years, because they’re engineers and try hard not to break people’s shit. Are feature flags and migration paths and interim solutions more work? Yeah, no doubt. But for all their recent fumbles in AI and steadily downward trajectory on search and all the things that happens when a big company is run by people who were never all that technical? Google still has a hard core of the baddest hackers you don’t want to mess with.
- deleted 2y ago[deleted]
- i_am_a_peasant 2y agoI never felt before that I am wasting brain cells as much as when I have to do something in bazel. The hermecicity thing it does relatively well but god does the documentation suck, at some point you can only get farther if you just read the java code implementing it...
- benreesman 2y agoCheat codes breaking Bazel 7’s sandbox: ‘cd -P’. sudo give me my files.
- danpalmer 2y agoI think bazel is only worthwhile when you are big enough to have a team that builds and maintains the bazel setup in a company/team. At Google it works fantastically, but I've also tried putting it into a project from scratch and it was too much.
- dilyevsky 2y agoIt’s been many years since last time i had to do anything in bazel’s java code. Most warts have been massively improved and only require occasional starlark hack or the right incantation of flags. The docs are honestly not too bad but the system is so massive and deals with so much inherent complexity that it always going to be missing something
- kajecounterhack 2y agoThis is my experience too. Just waiting for bzlmod to finally be supported by all the commonly used libraries. Looking at you, gRPC.
- jez 2y agoThe biggest feature I want out of Starlark the language (not Bazel the build system) is to allow optional type annotations. They don’t have to do anything, they don’t have to specified, it just needs to be syntactically valid to put type annotations somewhere. If there were syntax for type annotations that tools like Bazel ignored, it would be possible for some enterprising soul who’s forced to use Bazel at work to throw up an initial prototype of a type checker and a language server for Starlark in Bazel. The language is not that complicated that a type checker couldn’t be written by a hobbiest. My biggest frustration when using Bazel is not even Bazel—it’s the fact that when I’m looking at code like this[1], everything comes from a single, unannotated `rctx` variable that has no type annotation. So when I’m trying to read the code, it’s a matter of constantly grepping the repo, grepping the bazel docs, and grepping the bazel source code to get any new code written. Why can’t I just hover and see the docs? Why can’t I press `rctx.` and see all the attributes available? I frequently hear “why do you need types? the language is Turing complete just run it and see if it fails” but that misses the point that I have want to use types to guide what code to write in the first place. [1] https://github.com/bazel-contrib/toolchains_llvm/blob/master/toolchain/internal/configure.bzl#L59 https://github.com/bazel-contrib/toolchains_llvm/blob/master...
- kccqzy 2y agoA type system is a very good thing, but it directly contradicts one of the goals for this language, simplicity. As someone who has written type inference code for plain Hindley-Milner type systems and those with subtyping, I can tell you this is very much an open research problem. Inferred types, understandable type errors, expressive language: choose two out of three. If you sacrifice good type inference and require users to specify type annotations themselves they will not like it especially considering users' mental model of this language comes from Python; if you sacrifice understandable type errors, users will reject the whole system as soon as they encounter a type error; if you sacrifice expressiveness you are taking away features users want like subtyping. If you want something that's ignored by Bazel and only used by some other tools you already have it: docstrings. You can enforce docstrings to be of a certain format and contain type annotations. Be the solution you want to see: be that hobbyist that writes a type checker and language server.
- travisgriggs 2y agoI rtfa excited to learn about another uPython in the “embedded” space. Alas, they meant embedded in a different way. Wish we had better terms for different variants of embedded.
- ajayvk 2y agoI used go-starlark[1] to build a configurable reverse proxy and container manager for web app deployment [2]. Starlark is used to configure the proxy and it can be used to actually implement the backend apis. One challenge with Starlark was not having exceptions, which by itself is a good thing. But not having go-style multi-value returns in Starlark makes error handling verbose. Since the errors originate in plugin calls to go code, the solution I implemented was a thread-local to store the last error and handle that on the next plugin call if the error was not explicitly handled in the code [3]. Worked out pretty well in my test apps. For example, this app [4] implements a bookmark manager, with minimal error handling required in the code. Starlark is great for adding dynamic behavior for go applications. I was worried about performance when I started but perf has not been a concern in the context of a web app server. [1] https://github.com/google/starlark-go https://github.com/google/starlark-go [2] https://github.com/claceio/clace https://github.com/claceio/clace [3] https://clace.io/docs/plugins/overview/#automatic-error-handling https://clace.io/docs/plugins/overview/#automatic-error-hand... [4] https://github.com/claceio/apps/blob/main/utils/bookmarks/app.star https://github.com/claceio/apps/blob/main/utils/bookmarks/ap...
- abpavel 2y ago[flagged]
- yegle 2y agoI have a hobby project that forwards qBittorent's logs to Telegram in Go, and uses Starlark to configure the filtering rules without hard coding the expression in Go. Pretty trivial to implement using the Go implementation of Starlark.
- tripdout 2y agoHow do they guarantee both deterministic results and parallelism?
- rockwotj 2y agoDeterminism: via sandboxing builds, it’s a little leaky by default (but getting better). See https://bazel.build/basics/hermeticity https://bazel.build/basics/hermeticity Parallelism: Easier with sandboxing, but the build declares all inputs/outputs so it builds a merkle tree and strictly inforces a DAG. This way you can parallelize independent subgraphs and cache at a fine grained level.
- laurentlb 2y agoThe evaluation of a file happens in a single thread. At the end of the file evaluation, Starlark exports the values and functions defined in the file. All values are frozen, meaning they won't ever be mutated. Since no shared object can be modified, you can have any number of threads reading the object. So you can safely evaluate many files in parallel. Caveat: this assumes the interpreter doesn't expose functions like reading/writing on a disk, etc.
- wiz21c 2y agoWhat I miss in some of these cripting/embedded languages is coroutines where one coroutine is in the embedded language and the other is in the host language... That would be very helpful ! So are there languages that do that ?
- djoldman 2y ago> Deterministic evaluation. If you execute the same code twice, you will get the same results. In Python, the code won’t be deterministic if it relies on the output of functions like “id”, “hash”, if it iterates over a hashtable, if it has race conditions, or if it measures the execution time. > if it has race conditions Does this mean starlark cannot generate a race condition? If so, how does threading work?
- laurentlb 2y agoYes. See my other comment - https://news.ycombinator.com/item?id=40595125 https://news.ycombinator.com/item?id=40595125 In short: Evaluating a file has no visible side-effect. The evaluation returns a set of frozen values. These values that can accessed by other threads become immutable.
- vitiral 2y ago> how does threading work? Basic answer: there is no threading, at least not for the user. The build system runs a thread per BUILD file (in bazel). Since the BUILD files are written in starlark and starlark has no threads -- there is no problem.
- laurentlb 2y agoYou can define a value (let's say a dictionary) in a .bzl file and load it from multiple BUILD files. This is safe only because the dictionary becomes immutable before it is shared. Bazel is massively parallel. It has to evaluate a potentially large number of BUILD files; each file can have multiple load statements. You end up with a graph of dependencies and Bazel will evaluate as many files as possible in parallel.
- narodnik 2y agoStarlark needs metering so you can do fair polling when embedded in an application, then resume the paused processes.
- HelloNurse 2y agoPolling for events sounds like the polar opposite of hermetic execution.
- kccqzy 2y agoIt sounds like OP just wants the ability to suspend and resume Starlark execution. To first approximation they can just send SIGSTOP and SIGCONT signals to it, but they are probably asking for a more integrated way.
- vitiral 2y agoIt's really interesting that the original author compares it to Lua. I've never heard that comparison, but I've been recently considering building a similar build system. Mimicking starlark files would be easy: all you would have to do is use Lua's load[1] method with a highly constraint environment and you would have the deterministic environment that starlark has (you would have to inject deterministic iteration functions and importing/etc obviously). [1]: https://lua.org/manual/5.4/manual.html#pdf-load https://lua.org/manual/5.4/manual.html#pdf-load