6 ms·

Starlark is definitely a mixed experience IMO, from my 7 years working with it in Bazel On one hand, having a "hermetic" subset of Python is nice. You can be s

Starlark is definitely a mixed experience IMO, from my 7 years working with it in Bazel On one hand, having a "hermetic" subset of Python is nice. You can be sure your Bazel starlark codebase isn't going to be making network calls or reading files or shelling out to subprocesses and all that. The fact that it is hermetic does help make things reproducible and deterministic, and enables paralleization and caching and other things. Everyone already knows Python syntax, and its certainly nicer than the templated-bash-in-templated-yaml files common elsewhere in the build tooling/infra space On the other hand, a large Starlark codebase is a large Python codebase, and large Python codebases are imperative, untyped, and can get messy even without all the things mentioned above. Even though your Starlark is pure and deterministic, it still easily ends up a rats nest of sphagetti. Starlark goes the extra mile to be non-turing-complete, but that doesn't mean it's performant or easy to understand. And starlark's minimalism is also a curse as it lacks many features that help you manage large Python codebases such as PEP484 type annotations, which also means IDEs also cannot provide much help since they rely on types to understand the code For https://mill-build.org https://mill-build.org we went the opposite route: not enforcing purity, but using a language with strong types and a strong functional bent to it. So far it's been working out OK, but it remains to be seen how well it scales to ever larger and more complex build setups

kstrauser 2y ago
Python has always been strongly, dynamically typed. It isn’t untyped.
- jerf 2y ago
  The sort of "untyped" that your last sentence is referring to is a dead term, though. The only "untyped" language still in common use is assembler, and that's not commonly written by hand anymore (and when it is, it's primarily running on numbers, not complex structs and complex values). There aren't any extant languages anymore that just accept numbers in RAM and just treat them as whatever. So increasingly, this objection is meaningless, because nobody is using "untyped" that way anymore. The way in which people do use the term, Python is only "optionally" typed, and a lot of real-world Python code is "untyped".
  - js2 2y ago
    I think the objection is to the conflation of strong/weak with dynamic/static and it being unclear exactly what typed/untyped means, since it can refer to either. Python has always been strongly typed at runtime (dynamic), vs say JavaScript which is relatively weakly typed at runtime. Obviously lihaoyi was referring to static/dynamic when they wrote untyped (as made clear by the reference to type annotations) but kstrauser is objecting to using the term "untyped" since that can be interpreted to mean weak typing as well, which Python is not. $0.02 anyway.
    - lolinder 2y ago
      Strong/weak is a meaningless dichotomy that could be replaced by nice/icky while conveying the same meaning. It just distinguishes whether I, personally, believe a given language has sufficient protections against dumb programmer errors. What counts as strong or weak depends entirely on who's talking. Some will say that everything from C on is strong, others draw the line at Java, still others aren't comfortable until you get to Haskell, and then there are some who want to go even further before it's truly "strong". Typed versus untyped is, on the other hand, a rigorously defined academic distinction, and one that very clearly places pre-type-hints Python in the untyped category. That's not a bad thing—untyped isn't inherently a derogatory term—but because untyped languages have fallen out of vogue there's a huge effort to rebrand them.
      - poincaredisk 2y ago
        ...but Python is obviously typed. It has types. In fact everything has a type, and even the types are of "type" type. It has type errors. Saying it's "untyped" invokes a wrong impression. Your usage is very non-standard in programmer circles. What's wrong with universally understood and well defined concepts of "statically" and "dynamically" typed languages?
        lolinder 2y ago
        As I said in another comment [0], it depends on what definition of types we're using. But if we're going to pedantically jump down someone's throat correcting their usage (in this case OP's usage of "untyped"), we should at least use the most pedantically correct definition, which is the one used by academics who study type systems and which pointedly excludes dynamic checks. I have no problem with people using the other terminology in casual usage—I do so myself more often than not. I do have a problem with people pedantically correcting usage that is actually more correct than their preferred usage. I dislike pedantry in general, but I especially dislike incorrect pedantry. [0] https://news.ycombinator.com/item?id=42367659 https://news.ycombinator.com/item?id=42367659
- lolinder 2y ago
  It depends on what definition of "type system" you're using. Colloquially many programmers use it to refer to any system that checks whether objects have specific shapes. Academics, on the other hand, have a very specific definition of a type system that excludes dynamic detect languages. From TAPL (one of the authoritative works on the subject): > A type system is a tractable syntactic method for proving the absence of certain program behaviors by classifying phrases according to the kinds of values they compute. And later on: > A type system can be regarded as calculating a kind of static approximation to the runtime behaviors of the terms in a program. ... Terms like "dynamically typed" are arguably misnomers and should probably be replaced by "dynamically checked," but the usage is standard. In other words, you're both correct in your definitions depending on who you're talking to, but if we're going to get pedantic (which you seem to be) OP is slightly more correct. Personally, it feels like dynamically typed language advocates have been getting more and more vocal about their language of choice being "typed" as static typing has grown in popularity in recent years. This seems like misdirected energy—static typing advocates know what they're advocating for and know that dynamically typed languages don't fill their need. You're not accomplishing much by trying to force them to use inclusive language. Rather than trying to push Python as a typed language it seems like it would be more effective to show why dynamic checks have value.
  - kstrauser 2y ago
    Here's a Stack Overflow question about it from 15 years ago: https://stackoverflow.com/questions/2025353/is-python-a-weakly-typed-language-as-variables-can-switch-types https://stackoverflow.com/questions/2025353/is-python-a-weak... It was an old discussion before then, even. It has nothing to do with advocacy and it's certainly not recent. It's about accuracy so that people stop hearing and then repeating the same incorrect ideas. There's no common definition of types by which Python is untyped, as though it doesn't have types at all when in fact every Python object has a type.
    - lolinder 2y ago
      > There's no common definition of types by which Python is untyped You mean besides the one used by every programming languages researcher and hobbyist? Sure, you can define "common" to exclude them, but I would give at least some credence to the definitions put forward by the teams of people who invent type theory. As I've said here and elsewhere, I have no problem with people casually using "dynamically typed" as a term—I do so as well. But there's no cause to correct someone for using the more correct terminology. If hearing it makes you feel defensive of python, that implies that you perceive "untyped" as a pejorative that needs defending against. In that case, your efforts would be better spent correcting the evolving consensus that (statically) typed is better than they would be spent trying to shout people down for using the academic definitions of typed and untyped.
- grumpyprole 2y ago
  This "strong typing" message from the Python community has always sounded like propaganda to me - designed to confuse management. Strong typing is about machine checked proofs of invariants, not whether you avoid a few daft built-in coercions.
  - ithkuil 2y ago
    There is a static vs dynamic distinction and strong vs weak typing There is also a semi humorously named "stringly typed" which means weakly typed in such a way that incompatible types are promoted to strings before being operated on. I'm not aware of any static weakly typed language, but it's logically possible to have one
ajayvk 2y ago
I have been building an internal tools development and deployment platform [1]. It is built in Go, using Starlark for configuration and API business logic. Starlark has been great to build with. You get the readability of having a simple subset of python, without python's dependency management challenges. It is easily extensible with plugin APIs. Concurrent API performance is great without the python async challenges. One challenge wrt using Starlark as an general purpose embedded scripting language is that it does not support usual error handling features. There are no exceptions and no multi value return for error values, all errors result in an abort. This works for a config language, where a fail-fast behavior is good. But for a general purpose script, you need more fine grained error handling. Since I am using Starlark for API logic only, I came up with a user definable error handling behavior. This uses thread locals to keep track of error state [2], which might not work for more general purpose scripting use cases. [1] https://github.com/claceio/clace https://github.com/claceio/clace [2] https://clace.io/docs/plugins/overview/#automatic-error-handling https://clace.io/docs/plugins/overview/#automatic-error-hand...
- jcmfernandes 2y ago
  This is also one of my major complaints at this point. I'm building a developer tool with starklark resting at its core and I had already came across your work. I wish the starklark team had addressed it at this point.
  - mahmoudimus 2y ago
    We solved this by introducing a Result library. load("@.../result", result=result) def throw(arg): return 1/0 if result.Result(throw).map(arg).is_ok: # proceed else: fail("...")
  - ajayvk 2y ago
    I wrote up the approach I used at https://clace.io/blog/errors/ https://clace.io/blog/errors/, started a discussion at https://news.ycombinator.com/item?id=42370488 https://news.ycombinator.com/item?id=42370488 since it could apply outside of Starlark also
- 2y ago
Rochus 2y ago
> a large Starlark codebase is a large Python codebase, and large Python codebases are imperative, untyped, and can get messy even without all the things mentioned above. Even though your Starlark is pure and deterministic, it still easily ends up a rats nest of sphagetti This brings it to the point. I'm still wondering why the achievements of software engineering of the past fifty years, like modularization and static type checking had apparently so little influence on build systems. I implemented https://github.com/rochus-keller/BUSY https://github.com/rochus-keller/BUSY for this reason, but it seems to require a greater cultural shift than most developers are already willing to make.
- klooney 2y ago
  The tyranny of the one-small-change use case having outsized importance. Usually the build system is no one's job, which means that all hurdles grow.
- kccqzy 2y ago
  I think it's a cultural thing. People like to think of a language for the build system as a little language that somehow doesn't "deserve" a type system. And even they do think a type system is necessary, they think such a language doesn't "deserve" a complicated type system (say Java-like with subtyping and generics) which makes that type system less useful. I'm curious, what kind of type system does BUSY use?
  - Rochus 2y ago
    It's a rather traditional type system; the specification is here: http://software.rochus-keller.ch/busy_spec.html http://software.rochus-keller.ch/busy_spec.html. The main advantage are the combination of modularization, types and formal declarations, so that if you make a change in a large build (such as my https://github.com/rochus-keller/LeanQt https://github.com/rochus-keller/LeanQt or https://github.com/rochus-keller/LeanCreator https://github.com/rochus-keller/LeanCreator systems) incompatibilites are immediately found by the compiler. Without these features you can never be sure whether all effects were checked.
marssaxman 2y ago
> a "hermetic" subset of Python That's funny: I've been using bazel (and previously blaze) for well over a decade, but it has never once occurred to me to think of starlark as having anything at all to do with Python! I can't see anything about it which is distinctively pythonic.
- IshKebab 2y ago
  Erm... the syntax?
  - marssaxman 2y ago
    I'm curious - what about it strikes you that way? To my eyes, starlark bears more resemblance to YAML, or TOML, or any other generic configuration language, than to Python.
    - laurentlb 2y ago
      You've probably looked only at the Bazel BUILD files. They are indeed quite declarative (as the syntax is restricted even more). If you open other Starlark files that have functions (in Bazel, that would be in .bzl files), you should recognize the Python syntax (e.g. `def` statements + space indentation).
    - IshKebab 2y ago
      Erm what? It's very very obviously based on Python. The docs even explicitly say that. This is the example they give: def fizz_buzz(n): """Print Fizz Buzz numbers from 1 to n.""" for i in range(1, n + 1): s = "" if i % 3 == 0: s += "Fizz" if i % 5 == 0: s += "Buzz" print(s if s else i) fizz_buzz(20) Does that look like YAML or TOML?
IshKebab 2y ago
The Rust version of Starlark used in Buck2 apparently supports type annotations. I've never used it though and I have no idea about IDE support.
- davidjfelix 2y ago
  Allegedly it has an LSP and vscode support but I also have never used either. https://github.com/facebook/buck2/tree/main/starlark-rust/vscode https://github.com/facebook/buck2/tree/main/starlark-rust/vs...

kstrauser 2y ago

Python has always been strongly, dynamically typed. It isn’t untyped.

jerf 2y ago
The sort of "untyped" that your last sentence is referring to is a dead term, though. The only "untyped" language still in common use is assembler, and that's not commonly written by hand anymore (and when it is, it's primarily running on numbers, not complex structs and complex values). There aren't any extant languages anymore that just accept numbers in RAM and just treat them as whatever. So increasingly, this objection is meaningless, because nobody is using "untyped" that way anymore. The way in which people do use the term, Python is only "optionally" typed, and a lot of real-world Python code is "untyped".
- js2 2y ago
  I think the objection is to the conflation of strong/weak with dynamic/static and it being unclear exactly what typed/untyped means, since it can refer to either. Python has always been strongly typed at runtime (dynamic), vs say JavaScript which is relatively weakly typed at runtime. Obviously lihaoyi was referring to static/dynamic when they wrote untyped (as made clear by the reference to type annotations) but kstrauser is objecting to using the term "untyped" since that can be interpreted to mean weak typing as well, which Python is not. $0.02 anyway.
  - lolinder 2y ago
    Strong/weak is a meaningless dichotomy that could be replaced by nice/icky while conveying the same meaning. It just distinguishes whether I, personally, believe a given language has sufficient protections against dumb programmer errors. What counts as strong or weak depends entirely on who's talking. Some will say that everything from C on is strong, others draw the line at Java, still others aren't comfortable until you get to Haskell, and then there are some who want to go even further before it's truly "strong". Typed versus untyped is, on the other hand, a rigorously defined academic distinction, and one that very clearly places pre-type-hints Python in the untyped category. That's not a bad thing—untyped isn't inherently a derogatory term—but because untyped languages have fallen out of vogue there's a huge effort to rebrand them.
    - poincaredisk 2y ago
      ...but Python is obviously typed. It has types. In fact everything has a type, and even the types are of "type" type. It has type errors. Saying it's "untyped" invokes a wrong impression. Your usage is very non-standard in programmer circles. What's wrong with universally understood and well defined concepts of "statically" and "dynamically" typed languages?
      - lolinder 2y ago
        As I said in another comment [0], it depends on what definition of types we're using. But if we're going to pedantically jump down someone's throat correcting their usage (in this case OP's usage of "untyped"), we should at least use the most pedantically correct definition, which is the one used by academics who study type systems and which pointedly excludes dynamic checks. I have no problem with people using the other terminology in casual usage—I do so myself more often than not. I do have a problem with people pedantically correcting usage that is actually more correct than their preferred usage. I dislike pedantry in general, but I especially dislike incorrect pedantry. [0] https://news.ycombinator.com/item?id=42367659 https://news.ycombinator.com/item?id=42367659
lolinder 2y ago
It depends on what definition of "type system" you're using. Colloquially many programmers use it to refer to any system that checks whether objects have specific shapes. Academics, on the other hand, have a very specific definition of a type system that excludes dynamic detect languages. From TAPL (one of the authoritative works on the subject): > A type system is a tractable syntactic method for proving the absence of certain program behaviors by classifying phrases according to the kinds of values they compute. And later on: > A type system can be regarded as calculating a kind of static approximation to the runtime behaviors of the terms in a program. ... Terms like "dynamically typed" are arguably misnomers and should probably be replaced by "dynamically checked," but the usage is standard. In other words, you're both correct in your definitions depending on who you're talking to, but if we're going to get pedantic (which you seem to be) OP is slightly more correct. Personally, it feels like dynamically typed language advocates have been getting more and more vocal about their language of choice being "typed" as static typing has grown in popularity in recent years. This seems like misdirected energy—static typing advocates know what they're advocating for and know that dynamically typed languages don't fill their need. You're not accomplishing much by trying to force them to use inclusive language. Rather than trying to push Python as a typed language it seems like it would be more effective to show why dynamic checks have value.
- kstrauser 2y ago
  Here's a Stack Overflow question about it from 15 years ago: https://stackoverflow.com/questions/2025353/is-python-a-weakly-typed-language-as-variables-can-switch-types https://stackoverflow.com/questions/2025353/is-python-a-weak... It was an old discussion before then, even. It has nothing to do with advocacy and it's certainly not recent. It's about accuracy so that people stop hearing and then repeating the same incorrect ideas. There's no common definition of types by which Python is untyped, as though it doesn't have types at all when in fact every Python object has a type.
  - lolinder 2y ago
    > There's no common definition of types by which Python is untyped You mean besides the one used by every programming languages researcher and hobbyist? Sure, you can define "common" to exclude them, but I would give at least some credence to the definitions put forward by the teams of people who invent type theory. As I've said here and elsewhere, I have no problem with people casually using "dynamically typed" as a term—I do so as well. But there's no cause to correct someone for using the more correct terminology. If hearing it makes you feel defensive of python, that implies that you perceive "untyped" as a pejorative that needs defending against. In that case, your efforts would be better spent correcting the evolving consensus that (statically) typed is better than they would be spent trying to shout people down for using the academic definitions of typed and untyped.
grumpyprole 2y ago
This "strong typing" message from the Python community has always sounded like propaganda to me - designed to confuse management. Strong typing is about machine checked proofs of invariants, not whether you avoid a few daft built-in coercions.
- ithkuil 2y ago
  There is a static vs dynamic distinction and strong vs weak typing There is also a semi humorously named "stringly typed" which means weakly typed in such a way that incompatible types are promoted to strings before being operated on. I'm not aware of any static weakly typed language, but it's logically possible to have one

ajayvk 2y ago

I have been building an internal tools development and deployment platform [1]. It is built in Go, using Starlark for configuration and API business logic. Starlark has been great to build with. You get the readability of having a simple subset of python, without python's dependency management challenges. It is easily extensible with plugin APIs. Concurrent API performance is great without the python async challenges. One challenge wrt using Starlark as an general purpose embedded scripting language is that it does not support usual error handling features. There are no exceptions and no multi value return for error values, all errors result in an abort. This works for a config language, where a fail-fast behavior is good. But for a general purpose script, you need more fine grained error handling. Since I am using Starlark for API logic only, I came up with a user definable error handling behavior. This uses thread locals to keep track of error state [2], which might not work for more general purpose scripting use cases. [1] https://github.com/claceio/clace https://github.com/claceio/clace [2] https://clace.io/docs/plugins/overview/#automatic-error-handling https://clace.io/docs/plugins/overview/#automatic-error-hand...

jcmfernandes 2y ago
This is also one of my major complaints at this point. I'm building a developer tool with starklark resting at its core and I had already came across your work. I wish the starklark team had addressed it at this point.
- mahmoudimus 2y ago
  We solved this by introducing a Result library. load("@.../result", result=result) def throw(arg): return 1/0 if result.Result(throw).map(arg).is_ok: # proceed else: fail("...")
- ajayvk 2y ago
  I wrote up the approach I used at https://clace.io/blog/errors/ https://clace.io/blog/errors/, started a discussion at https://news.ycombinator.com/item?id=42370488 https://news.ycombinator.com/item?id=42370488 since it could apply outside of Starlark also
2y ago

Rochus 2y ago

> a large Starlark codebase is a large Python codebase, and large Python codebases are imperative, untyped, and can get messy even without all the things mentioned above. Even though your Starlark is pure and deterministic, it still easily ends up a rats nest of sphagetti This brings it to the point. I'm still wondering why the achievements of software engineering of the past fifty years, like modularization and static type checking had apparently so little influence on build systems. I implemented https://github.com/rochus-keller/BUSY https://github.com/rochus-keller/BUSY for this reason, but it seems to require a greater cultural shift than most developers are already willing to make.

klooney 2y ago
The tyranny of the one-small-change use case having outsized importance. Usually the build system is no one's job, which means that all hurdles grow.
kccqzy 2y ago
I think it's a cultural thing. People like to think of a language for the build system as a little language that somehow doesn't "deserve" a type system. And even they do think a type system is necessary, they think such a language doesn't "deserve" a complicated type system (say Java-like with subtyping and generics) which makes that type system less useful. I'm curious, what kind of type system does BUSY use?
- Rochus 2y ago
  It's a rather traditional type system; the specification is here: http://software.rochus-keller.ch/busy_spec.html http://software.rochus-keller.ch/busy_spec.html. The main advantage are the combination of modularization, types and formal declarations, so that if you make a change in a large build (such as my https://github.com/rochus-keller/LeanQt https://github.com/rochus-keller/LeanQt or https://github.com/rochus-keller/LeanCreator https://github.com/rochus-keller/LeanCreator systems) incompatibilites are immediately found by the compiler. Without these features you can never be sure whether all effects were checked.

marssaxman 2y ago

> a "hermetic" subset of Python That's funny: I've been using bazel (and previously blaze) for well over a decade, but it has never once occurred to me to think of starlark as having anything at all to do with Python! I can't see anything about it which is distinctively pythonic.

IshKebab 2y ago
Erm... the syntax?
- marssaxman 2y ago
  I'm curious - what about it strikes you that way? To my eyes, starlark bears more resemblance to YAML, or TOML, or any other generic configuration language, than to Python.
  - laurentlb 2y ago
    You've probably looked only at the Bazel BUILD files. They are indeed quite declarative (as the syntax is restricted even more). If you open other Starlark files that have functions (in Bazel, that would be in .bzl files), you should recognize the Python syntax (e.g. `def` statements + space indentation).
  - IshKebab 2y ago
    Erm what? It's very very obviously based on Python. The docs even explicitly say that. This is the example they give: def fizz_buzz(n): """Print Fizz Buzz numbers from 1 to n.""" for i in range(1, n + 1): s = "" if i % 3 == 0: s += "Fizz" if i % 5 == 0: s += "Buzz" print(s if s else i) fizz_buzz(20) Does that look like YAML or TOML?

IshKebab 2y ago

The Rust version of Starlark used in Buck2 apparently supports type annotations. I've never used it though and I have no idea about IDE support.

davidjfelix 2y ago
Allegedly it has an LSP and vscode support but I also have never used either. https://github.com/facebook/buck2/tree/main/starlark-rust/vscode https://github.com/facebook/buck2/tree/main/starlark-rust/vs...