7 ms·

Given it is known how slow Python at starting up, I am puzzled why Mozilla continue to use it in build scripts. Perl is just as portable but starts up like 10 t

Given it is known how slow Python at starting up, I am puzzled why Mozilla continue to use it in build scripts. Perl is just as portable but starts up like 10 times faster.

feedmeseymour 9y ago
Perl is “just as portable” in the same way that a motorcycle can just as easily drive under a steamroller... it’s not gonna be pretty and there’s no easy way out if you do it.
- ryl00 9y ago
  I write Perl scripts for Windows and Linux, and I don't find portability to be especially onerous. Of course there are platform differences to keep in mind, but is that any different from any other cross-platform scripting language?
jdashg 9y ago
Compile time is absolutely dominated by c++ (and now rust) compilation and linking. I doubt build system language choice will ever bubble up to relevance, so why optimize for it?
ch4s3 9y ago
I imagine there are two aspects to this, they probably started in python and have a lot of it already, and it's probably easier to gen new folks involved which I think is one of their goals.
sametmax 9y ago
I've built firefox from source. Python start up time is not really a problem, it's so long to build anything anyway.
- kevincox 8y ago
  I imagine a lot of the pain is for incremental builds where the build system overheads can matter a lot more.
coldtea 9y ago
Build scripts, especially for a C++ app like Firefox, are a place where "slow startup times" are totally irrelevant.
- sciurus 9y ago
  Did you read the link we're discussing? It's in large part about why the slow startup times for Firefox's build scripts are a problem. > Changing gears, my day job is maintaining Firefox's build system. We use Python heavily in the build system. And again, Python startup overhead is problematic. I don't have numbers offhand, but we invoke likely a few hundred Python processes as part of building Firefox. It should be several thousand. But, we've had to "hack" parts of the build system to "batch" certain build actions in single process invocations in order to avoid Python startup overhead. This undermines the ability of some build tools to formulate a reasonable understanding of the DAG and it causes a bit of pain for build system developers and makes it difficult to achieve "no-op" and fast incremental builds because we're always invoking certain Python processes because we've had to move DAG awareness out of the build backend and into Python. At some point, we'll likely replace Python code with Rust so the build system is more "pure" and easier to maintain and reason about.
  - coldtea 9y ago
    >Did you read the link we're discussing? It's in large part about why the slow startup times for Firefox's build scripts are a problem. The link we're discussing is about Python runs for the Mercurial test suite. The Firefox build is mentioned in passing. It's not what the post is discussing (and doesn't get into numbers there).
    - dannyw 9y ago
      You claim that it is “totally irrelevant”. A Firefox maintainer, even in passing, claims otherwise.
      - coldtea 9y ago
        Maybe not totally irrelevant, but it's in most cases not a factor compared to build times. For tests, yes. I can't fathom what Mozilla does for thousands of Python invocations during build. Maybe it's test runs as well?
indygreg2 9y ago
I wrote the linked post and maintain the Firefox build system. The reason is that in 2018 (and for the past 10 years honestly) and it is far easier to find people who know Python than Perl. Python is essentially the lingua franca in Firefox land for systems-level tasks that don't warrant a compiled language. As I said in the post, Rust will likely infringe on Python over time due to performance, lower defect rate, long-term maintenance advantages, etc.
- fnord123 9y ago
  >As I said in the post, Rust will likely infringe on Python over time due to performance, lower defect rate, long-term maintenance advantages, etc. Indeed. As hg is moving to use more and more Rust: https://www.mercurial-scm.org/wiki/OxidationPlan https://www.mercurial-scm.org/wiki/OxidationPlan
  - cpeterso 9y ago
    indygreg knows because he is the author of that wiki page. :) https://www.mercurial-scm.org/wiki/OxidationPlan?action=info https://www.mercurial-scm.org/wiki/OxidationPlan?action=info
    - fnord123 9y ago
      I didn't realize that so thanks for pointing it out. But sometimes I also comment for third parties reading along to pick up some info as well.
- petre 9y ago
  Can't wait for a static, dependency-free hg, but sadly they aren't going to rewrite it entirely in Rust, just the speed sensitive parts. I don't care if it's written in Rust, D, Go or even C. When we picked fossil over hg and git, the Python dependency made us shy away from hg, although the CLI and the overall experience was better than git. Now we also fossil as a deployment tool; it can be a statically compiled drop anywhere binary.
  - 8y ago

Given it is known how slow Python at starting up, I am puzzled why Mozilla continue to use it in build scripts. Perl is just as portable but starts up like 10 times faster.

feedmeseymour 9y ago

Perl is “just as portable” in the same way that a motorcycle can just as easily drive under a steamroller... it’s not gonna be pretty and there’s no easy way out if you do it.

ryl00 9y ago
I write Perl scripts for Windows and Linux, and I don't find portability to be especially onerous. Of course there are platform differences to keep in mind, but is that any different from any other cross-platform scripting language?

jdashg 9y ago

Compile time is absolutely dominated by c++ (and now rust) compilation and linking. I doubt build system language choice will ever bubble up to relevance, so why optimize for it?

ch4s3 9y ago

I imagine there are two aspects to this, they probably started in python and have a lot of it already, and it's probably easier to gen new folks involved which I think is one of their goals.

sametmax 9y ago

I've built firefox from source. Python start up time is not really a problem, it's so long to build anything anyway.

kevincox 8y ago
I imagine a lot of the pain is for incremental builds where the build system overheads can matter a lot more.

coldtea 9y ago

Build scripts, especially for a C++ app like Firefox, are a place where "slow startup times" are totally irrelevant.

sciurus 9y ago
Did you read the link we're discussing? It's in large part about why the slow startup times for Firefox's build scripts are a problem. > Changing gears, my day job is maintaining Firefox's build system. We use Python heavily in the build system. And again, Python startup overhead is problematic. I don't have numbers offhand, but we invoke likely a few hundred Python processes as part of building Firefox. It should be several thousand. But, we've had to "hack" parts of the build system to "batch" certain build actions in single process invocations in order to avoid Python startup overhead. This undermines the ability of some build tools to formulate a reasonable understanding of the DAG and it causes a bit of pain for build system developers and makes it difficult to achieve "no-op" and fast incremental builds because we're always invoking certain Python processes because we've had to move DAG awareness out of the build backend and into Python. At some point, we'll likely replace Python code with Rust so the build system is more "pure" and easier to maintain and reason about.
- coldtea 9y ago
  >Did you read the link we're discussing? It's in large part about why the slow startup times for Firefox's build scripts are a problem. The link we're discussing is about Python runs for the Mercurial test suite. The Firefox build is mentioned in passing. It's not what the post is discussing (and doesn't get into numbers there).
  - dannyw 9y ago
    You claim that it is “totally irrelevant”. A Firefox maintainer, even in passing, claims otherwise.
    - coldtea 9y ago
      Maybe not totally irrelevant, but it's in most cases not a factor compared to build times. For tests, yes. I can't fathom what Mozilla does for thousands of Python invocations during build. Maybe it's test runs as well?

indygreg2 9y ago

I wrote the linked post and maintain the Firefox build system. The reason is that in 2018 (and for the past 10 years honestly) and it is far easier to find people who know Python than Perl. Python is essentially the lingua franca in Firefox land for systems-level tasks that don't warrant a compiled language. As I said in the post, Rust will likely infringe on Python over time due to performance, lower defect rate, long-term maintenance advantages, etc.

fnord123 9y ago
>As I said in the post, Rust will likely infringe on Python over time due to performance, lower defect rate, long-term maintenance advantages, etc. Indeed. As hg is moving to use more and more Rust: https://www.mercurial-scm.org/wiki/OxidationPlan https://www.mercurial-scm.org/wiki/OxidationPlan
- cpeterso 9y ago
  indygreg knows because he is the author of that wiki page. :) https://www.mercurial-scm.org/wiki/OxidationPlan?action=info https://www.mercurial-scm.org/wiki/OxidationPlan?action=info
  - fnord123 9y ago
    I didn't realize that so thanks for pointing it out. But sometimes I also comment for third parties reading along to pick up some info as well.
petre 9y ago
Can't wait for a static, dependency-free hg, but sadly they aren't going to rewrite it entirely in Rust, just the speed sensitive parts. I don't care if it's written in Rust, D, Go or even C. When we picked fossil over hg and git, the Python dependency made us shy away from hg, although the CLI and the overall experience was better than git. Now we also fossil as a deployment tool; it can be a statically compiled drop anywhere binary.
- 8y ago