11 ms·
The Starlark Programming Language
- scotty79 2y agoThis looks like a great language for a game about programming.
- laurentlb 2y agoIndeed, that's something I had in the back of my mind too. (if anyone has ideas in the area, you can reach out to me)
- Rhapso 2y agoStarklark isn't exactly "functional" but by design it is hermetic. You can't call any code that would have a side-effect (other than burning compute resources) or get data that wasn't part of your initial input. It could make a of sense as a "contract language", or as intended: part of a build system.
- kccqzy 2y agoI believe Starlark allows side effects the first time it evaluates a file but not subsequent times. Otherwise something as simple as calling .append on a list won't work because that returns None but has an input-mutating side effect. I think it's a good choice.
- etamponi 2y agoYour mileage may vary, but Starlark was one of the most complicated languages I've ever read. No types, just layers and layers of indirection, with the goal of making a very complex build rule look "simple". I don't like hiding non-accidental complexity, in particular when the abstraction leaks everywhere. Perhaps it was only due to the Starlark I was exposed to (I worked at Google, probably most of the Starlark code I've read has been written by SREs).
- kccqzy 2y agoWhat you describe is really all large-scale Python projects look like. No types, just layers and layers of indirection, with the goal of making the interface simple and Pythonic while hiding implementation complexity. I don't think this is necessarily a fault given that this language explicitly decides to look like Python (and was in fact simplified from Python). The worst Starlark code I've read has been written not by SREs, but by the Boq team as they have a fetish for accomplishing complicated configuration at build time. This was one of the reasons I've avoided Boq: an incomplete code base that's under development doesn't even begin to build, which is far worse than building something and seeing a real compiler error.
- yodsanklai 2y ago> No types It's pretty easy to add types to Python nowadays. I'd consider it bad practice not to do so in a large project.
- seanmceligot 2y agoIt would be nice if python would show types in the documentation. Not only do I need that all the time, it would show python was taking type safety serious. For example, knowing the return type of a function is Union[DataFrame,Series] rather than simply DataFrame would save a lot of bad errors.
- zellyn 2y agoStarlark, unfortunately, does not really support (Python style) types yet. Facebook's version has some kind of types, but ideally Starlark would just learn to do mypy types.
- kccqzy 2y agoThat's only true for greenfield projects where people start a new project with types in mind. It's absolutely a nightmare for old projects because, without the need to write types, people write all kinds of code that cannot fit within what's possible in python's type annotations.
- kevindamm 2y agoThis appears to be the same language that started as a custom build rule DSL for bazel (and evolving from lessons learned in blaze's Python extension), the page positions it as a much more general-purpose embedded language. I'm all for using the right tool for the job, and I'll acknowledge the benefits of hermeticity and parallelism in many contexts. But, I also think that you need to weigh the cost of adding another language to a project. I don't imagine myself using this anywhere outside of a build system, and even then I would first try to take the declarative rules of that build system as far as I can. It's probably better than shell scripts, though, at least most of the time? May depend on your team's prior familiarity with shell and/or love of Python. There are three implementations (Go, Rust, Java) which could be a good thing if the language doesn't change often. They would have to be kept in sync for any changes, as well as keep from drifting due to changes in the host language's semantics. I also think that the closeness in syntax with Python can be a disadvantage, especially if users are expecting more recent Python additions like the walrus operator or pattern matching to be available. I like to design languages, too, so these comments come from a place of love and understanding. I think it would help if there were some more specific justification (or examples) of why this is better than something more established (e.g. Lua or MicroPython) or more distinct (e.g. pure data in configs instead of embedded code). I do like that the language attempts to remain as simple as needed.
- foooorsyth 2y ago>There are three implementations (Go, Rust, Java) There are so many people at Google just goofing around, lol. People just doing hobby stuff for $350k/year. There is no justification to implement this custom dialect of Python for a build system that drives everyone crazy 3 times. Reminds me of when it was revealed that 400 people were working on Fuschia — a hobby OS that only shipped on a single smart home device.
- deleted 2y ago[deleted]
- ithkuil 2y agoIIRC initially blaze was literally using python as a config layer. That made it too easy to write build scripts that were too slow and too hard to optimize and that negatively affected the overall build experience. An increase in edit/build/run cycle efficiency of thousands of employees justifies an investment of a few engineers. They could have invented a new bespoke DSL. Instead they choose to stick to a well known and familiar language and just limit its expressiveness to a subset that would be easier to optimize. I think that's quite reasonable
- paulddraper 2y agoStarlark (originally Skylark) is the configuration/extension language of Google's build tool Bazel. It's a bespoke Python subset... functions but no recursion, dicts but no sets, etc. It's also been adopted by the latest version of Buck (Meta's Bazel analog). I use it daily. It's an option for lightweight embeddable scripting language, with implementations in Java and Go. If you want Python familiarity, consider it as an option.
- nrr 2y agoStarlark is Turing-incomplete, which makes it somewhat unique among embeddable languages. It's definitely a draw for me for something I'm working on.
- kevindamm 2y agoThe primitive-recursive property of Cue (https://cuelang.org https://cuelang.org) is a big draw for me, and may be an alternative worth checking out. The authors have spent a great amount of attention to the type system (they learned a lot of lessons from previous config language designs that did not take lattice theory and unification into account).
- verdverm 2y agoThe tl;dr is that inheritance is bad in config, whether it be from OOP or layering yaml files like Helm. The reason being that it is hard to understand where a value is coming from and where one must make an edit to correct it in high-stress SRE situations like downtime. Marcel worked on both major config languages at Google, and iirc Starlark is based on GCL ideas The Logic of CUE is a great read: https://cuelang.org/docs/concept/the-logic-of-cue/ https://cuelang.org/docs/concept/the-logic-of-cue/
- kccqzy 2y agoDhall is another configuration language that's deliberately Turing-incomplete. Though its Haskell-inspired syntax turns people off who aren't already Haskell programmers. It's based on calculus of constructions.
- anothername12 2y ago> Deterministic evaluation - Executing the same code twice will give the same results. What’s this all about? Don’t most languages?
- lann 2y agoOne specific example: many languages use randomized seeds in builtin dict/map types, leading to randomized iteration order.
- chubot 2y agoYeah, also Starlark is embedded like Lua, and doesn't come with batteries included like Python So that means you can control the APIs, and say opendir() closedir() in Unix returns filenames in different orders. Depending on what the data structure in the kernel is So many programs in other languages aren't deterministic just because they use APIs that aren't deterministic
- phyrex 2y agoYou don't get access to randomness or time functions or anything that could change the output of a function with the same input
- zellyn 2y agoIt can be very valuable to know that if you put the same files in, you get _exactly_ byte-for-byte identical artifacts out of your build system. Even letting your language access the actual current date/time can break that. (IIRC, I don't believe Bazel actually has fully deterministic builds yet, though.)
- laurentlb 2y agoTry this in Python: id("ab") # not deterministic hash("ab") # not deterministic def foo(): pass str(foo) # not deterministic Another sneaky one is the `is` operator in Python, where the Python documentation says: > Due to automatic garbage-collection, free lists, and the dynamic nature of descriptors, you may notice seemingly unusual behaviour in certain uses of the `is` operator Related to that is the `__del__` method: when exactly is it called? It's quite easy to get non-deterministic code in Python and in many languages. And of course, there are lots of non-deterministic functions in the standard library (Starlark doesn't provide them).
- lihaoyi 2y agoStarlark is definitely a mixed experience IMO, from my 7 years working with it in Bazel On one hand, having a "hermetic" subset of Python is nice. You can be sure your Bazel starlark codebase isn't going to be making network calls or reading files or shelling out to subprocesses and all that. The fact that it is hermetic does help make things reproducible and deterministic, and enables paralleization and caching and other things. Everyone already knows Python syntax, and its certainly nicer than the templated-bash-in-templated-yaml files common elsewhere in the build tooling/infra space On the other hand, a large Starlark codebase is a large Python codebase, and large Python codebases are imperative, untyped, and can get messy even without all the things mentioned above. Even though your Starlark is pure and deterministic, it still easily ends up a rats nest of sphagetti. Starlark goes the extra mile to be non-turing-complete, but that doesn't mean it's performant or easy to understand. And starlark's minimalism is also a curse as it lacks many features that help you manage large Python codebases such as PEP484 type annotations, which also means IDEs also cannot provide much help since they rely on types to understand the code For https://mill-build.org https://mill-build.org we went the opposite route: not enforcing purity, but using a language with strong types and a strong functional bent to it. So far it's been working out OK, but it remains to be seen how well it scales to ever larger and more complex build setups
- kstrauser 2y agoPython has always been strongly, dynamically typed. It isn’t untyped.
- jerf 2y agoThe sort of "untyped" that your last sentence is referring to is a dead term, though. The only "untyped" language still in common use is assembler, and that's not commonly written by hand anymore (and when it is, it's primarily running on numbers, not complex structs and complex values). There aren't any extant languages anymore that just accept numbers in RAM and just treat them as whatever. So increasingly, this objection is meaningless, because nobody is using "untyped" that way anymore. The way in which people do use the term, Python is only "optionally" typed, and a lot of real-world Python code is "untyped".
- mahmoudimus 2y agoI love Starlark. I was a major implementor of it at VGS (the repo is open: https://github.com/verygoodsecurity/starlarky https://github.com/verygoodsecurity/starlarky). It had unique distinct features that made it much easier to control and sandbox than many other languages out there. I even built a codemod library that does a very basic python -> starlark so that one can develop using python ecosystem libraries and just copy & paste into a secure execution environment. It was a huge success at my last company. I'm very thankful to Laurent Le-brun and Alan Donovan -- both of whom are exceptional engineers that I learned so much from. I thought I was skilled but both of those individuals are just on another level.
- adsharma 2y agoWhat is this codemod library called?
- mahmoudimus 2y agohttps://github.com/mahmoudimus/py2star https://github.com/mahmoudimus/py2star
- srmatto 2y agohttps://tilt.dev/ https://tilt.dev/ also uses Starlark.
- fabmilo 2y agoThis is interesting as I was evaluating starlark few days ago. The fact that has a customizable implementation in golang, and a python similar syntax makes it an interesting choice for agents generated code.
- jcmfernandes 2y agoTo those pointing out that it's dynamically typed, meta's rust implementation - that they use in buck2 - supports type annotations.
- xnacly 2y agoI feel like no other embedded scripting language will ever surpass lua. Neovim, roblox and all my projects that needed scripting support use lua, its my first choice.
- dang 2y agoRelated: Starlark Language - https://news.ycombinator.com/item?id=40700549 https://news.ycombinator.com/item?id=40700549 - June 2024 (49 comments) An Overview of the Starlark Language - https://news.ycombinator.com/item?id=40573689 https://news.ycombinator.com/item?id=40573689 - June 2024 (49 comments) (The) Starlark Language - https://news.ycombinator.com/item?id=39457410 https://news.ycombinator.com/item?id=39457410 - Feb 2024 (1 comment) RepoKitteh: Github workflow automation using Starlark - https://news.ycombinator.com/item?id=26674781 https://news.ycombinator.com/item?id=26674781 - April 2021 (7 comments)
- mdaniel 2y agoI would have thought for sure this was submitted due to the Bazel 8 release but this thread predates that one by quite a bit Anyway, I guess "see also:" https://news.ycombinator.com/item?id=42370744 https://news.ycombinator.com/item?id=42370744
- laurentlb 2y agoThe timing was a coincidence. It's unrelated to the release of Bazel 8.
- deleted 2y ago[deleted]
- ForHackernews 2y agoCan you use Starlark inside Python? We have a rules engine where we'd like the rules themselves to be maintained outside the regular codebase, but we'd rather not invent a new DSL for them.
- laurentlb 2y agoYes, you can try these bindings: - https://github.com/caketop/python-starlark-go https://github.com/caketop/python-starlark-go - https://github.com/inducer/starlark-pyo3 https://github.com/inducer/starlark-pyo3 (I haven't tried them myself)
- jmann99999 2y agoI ran into the language very differently than many posts here. The Tidbyt [0], a retro-style smart display designed to provide at-a-glance information on the weather, sports scores, transit schedules, etc., uses it to enable people to create their own custom displays. For those purposes, it is simple. The simplicity reminds me of Lua. It is a perfect choice for the device. [0] https://tidbyt.dev/docs/build/build-for-tidbyt https://tidbyt.dev/docs/build/build-for-tidbyt
- amadeusw 2y agoI have the opposite perception from you; I tried to create apps for Tidbyt, only to run into obscure differences between Starlark and Python, inability to parse data from API endpoint, inability to debug. I followed your link and saw that the platform somewhat matured so I'll give it one more try
- jmann99999 2y agoYeah, perhaps my "advantage" is that I am a Ruby guy. So, I had little real Python knowledge to trip me up. :-)
- Pet33231 2y ago[dead]