9 ms·

> The Python Steering Council and Python Software Foundation will work together to contract a developer This would represent a significant increase in the numb

by pansa2 6y ago

> The Python Steering Council and Python Software Foundation will work together to contract a developer This would represent a significant increase in the number of people paid to work on the Python language. Two years ago it was estimated that it only had (the equivalent of) two full-time employees working on it. https://discuss.python.org/t/official-list-of-core-developers/924 https://discuss.python.org/t/official-list-of-core-developer...

josephg 6y ago
Its such a statement about the free rider problem that python can have only 2 people working fulltime on it. Python is one of the most popular languages in the world - there's got to be trillions of dollars worth of companies depending on it. Small improvements to the language would result in millions of dollars of productivity gains in aggregate globally. And yet ... collectively we can only find enough money to pay two full time salaries?! Seriously?
- edwardwatson 6y ago
  Python is as popular as it is because it's free, not because it's good. If companies had to contribute, many would pick something else.
- tasubotadas 6y ago
  So... why don't you pay?
  - bzb6 6y ago
    Someone else should!
  - feanaro 6y ago
    Why do you assume he isn't paying? The payment should obviously be proportional, though. You can't expect a single individual to finance what a gigantic multibillion corporation could.
    - tasubotadas 6y ago
      Just because some random individual came up with some arbitratry agenda of whom should finance what, doesn't mean that it makes sense. There are so many better (and worse) things to finance than Python Foundation.
      - feanaro 6y ago
        It's not very arbitrary. The reasoning (in one of the upstream posts) is that Python is one of the world's most used programming languages. It definitely makes sense for large multi-billion corporations which are the users of this language to help finance it. Your point would make more sense if the claim was that a significant portion of the corporation's profits should go towards financing Python. Then I would agree that "there are so many better things to finance than Python". But if we're discussing a salary or two in the context of corporations that employ hundreds of thousands of people, then your argument makes much less sense in the current context. There are very few things that would have better marginal utility then paying for the (effectively) third full-time Python developer.
  - josephg 6y ago
    I say this as someone who's spent decades of my life donating time to opensource projects. - But the model where big companies have their technology stacks underpinned by charity is ridiculous. Obviously ridiculous, if python only has 2 full-time-equivalent staff. This problem isn't solved by little old me pulling out my check book. My bank and my insurance company don't need charity from me in order to fund the technology they've built their businesses on. They need to learn to pony up and start funding the "free" software they use. Paying myself would only help kick the can down the road - and encourage and enable this sort of corporate mooching to keep happening. In 60 years most of the volunteer maintainers of the software we depend on every day will be dead. Some time between now and then we need solution to this problem of opensource sustainability. Delaying buys us nothing.
    - sokoloff 6y ago
      If python continues to be this popular, there will be more maintainers who join the project. Surely you’ve seen volunteers join existing projects out of their own interest and need now. The decades that you contributed (thank you, BTW): were those all on projects you started, or did you also join some existing efforts? I don’t see any reason to think that will stop in 10, 20, or 60 years.
      - josephg 6y ago
        > If python continues to be this popular, there will be more maintainers who join the project. Will there? I feel like most of the opensource attention flows gradually up stack / toward trendy things on github and away from infrastructure older than github. When heartbleed happened a few years ago it was revealed that OpenSSL was maintained by basically one guy. The same is true of the linux timezone database. Apparently python is mostly 2 people. I'd put money on this being is a theme for lots of tools we take for granted. How many people work on bash? Or busybox, grub, libpthread or libicu? I bet the number is tiny.
        sokoloff 6y ago
        Cpython had 5 pull requests opened yesterday. 6 the day before that (from 10 total distinct contributors). That doesn’t seem like a project starving for contributors. If everyone with commit access up and quit, there’d be turmoil for several weeks and something reasonable would emerge. People want to work on popular, useful things. Python is both of those.
- Barrin92 6y ago
  because software that is provided for free cannot capture the positive externalities it creates and thus is under-supplied, it's just very straight-forward economics. Yet it's become such a widespread sentiment to expect software for free that charging for something like Python would probably be seen as obscene and kill the language through loss of reputation alone.
- _tom_ 6y ago
  And large improvements to the language could result in millions of dollars of productivity losses! :-) I'm not a big fan of "Now this language has gotten popular, let's add all the features to it that made our other language so unpopular."
  - huffmsa 6y ago
    It taking 10 years to transition from 2 to 3 wasn't necessarily a bad thing. Definitely concerns that more hands could make another Angular, which had 2 incompatible branches, with one going from v2 to v11 in 5 years.
    - josephg 6y ago
      > It taking 10 years to transition from 2 to 3 wasn't necessarily a bad thing. The python3 interpreter could have been largely (or completely) source-compatible with python2 code. They could have done it rust2018 style - where source code files can opt in to a newer parser, and in return get access to the new features. Then the whole python3 drama would have been avoided. MacOS still ships with a python 2.7 interpreter because of all that nonsense. But if they only had 2 full-time people working on it, no wonder they didn't have the man power to do the transition any other way. Disappointing.
      - my123 6y ago
        Which puts a: > WARNING: Python 2.7 is not recommended. > This version is included in macOS for compatibility with legacy software. > Future versions of macOS will not include Python 2.7. > Instead, it is recommended that you transition to using 'python3' from within Terminal. banner on launch nowadays...
        josephg 6y ago
        Right; none of this should be necessary. I don’t need an old version of my C compiler in order to compile old C files. Rust made breaking changes to their language and yet a modern rust compiler will happily compile the older 2015 rust syntax. Python could have gone down the same road; but I suppose all of python’s corporate users chose to fight python compatibility changes locally rather than donate time / money fixing python itself.
- andi999 6y ago
  I am seriously wondering: is it actually possible for stock listed companies to donate? I mean aren't they legally obliged not to spend more than necessary (towards their stockholders obliged). Also what does the IRS say on that?
  - foota 6y ago
    No. This is a misinterpretation of the responsibility. Companies have great latitude in what they do, as long as they're forthright.
  - OJFord 6y ago
    They have a duty to try to operate efficiently, but that doesn't make corporate social responsibility, a bad hire, or wasting something nobody liked in the canteen illegal.
    - andi999 6y ago
      Well, bad hire, bad food can always be justified (avoiding hindsight fallacy). But giving away money for nothing received (meaning of donation) is something different (at least to me)
      - keeganpoppen 6y ago
        companies donate money to stuff all the time what are you on about?
        andi999 6y ago
        So I read up a little. To tax deduct the nonprofit must have registered as 501(c)(3). Is this the case here?
        sokoloff 6y ago
        I can pay you to put my company name on your business website or a physical banner at a trade show and account for this as a business expense (deduct it) without you being a non-profit. I have broad latitude to make business expenditures in a manner that I believe is profit-seeking or optimizing for my business. People do this with Google and Facebook all the time. We call them ads. Not every ad campaign has a positive RoI.
- kortilla 6y ago
  > And yet ... collectively we can only find enough money to pay two full time salaries?! Seriously? But what’s the problem here? Is python struggling as a language? If there were clear calls for help companies could find it a lot easier to justify donating money or dev time.
  - coldtea 6y ago
    >Is python struggling as a language? As a language, interpreter, runtime, yes. In terms of adoption, no.
    - kortilla 6y ago
      With what? Why haven’t they said so? There are a bunch of tech companies that would love the exposure of employing core developers.
- passtheglass 6y ago
  To me that is more like evidence of a well-engineered codebase.
- II2II 6y ago
  Is that actually a bad thing? Much of the popularity of Python is based upon third party libraries that have been contributed by both individuals and organizations. These are still contributions, and there are many cases where the contributors are paid. Should Python receive more funding and pay more people to work on the language? Probably, yet it is doubtful that having a large team working on the core language will improve the health of the language.
- coldtea 6y ago
  >Its such a statement about the free rider problem that python can have only 2 people working fulltime on it It's also a big statement about how non realistic were the ideas that FOSS will thrive on volunteers working together. Back in the day where the "Cathredral and the Bazaar" was published, VALinux got its IPO, and FOSS was the latest hype, people really believed that this will work without corporate sponsorship, just by volunteers doing the work. Then again, they also believed in many eyes available through the magic of FOSS making bugs shallow (Torvalds), and you get 20+ years of some obvious bug in something as critical as OpenSSL.
ramraj07 6y ago
That's insane. A thought occured recently - some lines of code in the python code base have been executed an innumerable number of times (trillion-trillion? Can't even start to calculate the order of magnitude). Even the subtlest of optimizations to these lines could collectively save probably hundreds or thousands of tons of CO2 from being released. That alone might deserve a couple of People looking into such massively common codebases fulltime.
- Radim 6y ago
  And yet… Did you know about this (serious) proposal to make CPython 5x faster? https://mail.python.org/archives/list/python-dev@python.org/thread/RDXLCH22T2EZDRCBM6ZYYIUTBWQVVVWH/ https://mail.python.org/archives/list/python-dev@python.org/... Needed funding though.
  - tpetry 6y ago
    Why does this not happen or anything like this? Ruby got MJT in a big effort to make Ruby a lot faster. PHP is constantly increasing it‘s performance and tries to change a lot to get faster (basic building blocks for JIT in PHP8). Why is python not iterating so much on it‘s performance? Or is it doing this but it‘s not as public in promoting it like my former two examples.
    - dmurray 6y ago
      There's already PyPy. It's faster for, I estimate, 90% of workloads that don't use NumPy or Tensorflow, which offload work to faster implementations in different ways. It can certainly be 5 times faster. It doesn't have that much adoption, which suggests that performance isn't critical to Python's user base. If PyPy got significant market share, perhaps the CPython developers would prioritise an approach like this. (Or perhaps they'd have even more justification to say look, there's already a performance-focused implementation, we'll do our thing).
      - tpetry 6y ago
        Pypy isnt used that much because you may run into problems. And nobody wants to research strange runtime errors in production. But the speed improvements would be welcome if they were in cpython because you can be quite sure there will be no problems.
ThePhysicist 6y ago
That's a pretty distorted view IMHO. There are a lot of people that are working in the Python ecosystem and improve the language, and while they are not directly paid by the PSF their employers give them time to devote to working on Python. I think e.g. GvR had plenty of freedom to work on his Python projects while at Dropbox. I know a lot of companies in Germany that give people ample time to go to Python conferences and engage with the community, as it's also a way to meet and recruit good Python developers. So it might be true that there are only a few FTEs working on Python, but the larger community of people contributing to it is much larger.
- pansa2 6y ago
  > GvR had plenty of freedom to work on his Python projects while at Dropbox. That post was made while Guido was at Dropbox, and it claims he had one day a week?
  - ThePhysicist 6y ago
    They talk about maintaining Python, which includes things like preparing and managing releases. Why would you need more than a few people to do that? The core language itself also evolves at a slow pace (which is good), it's not that there are tons of new features every year that get merged into the codebase, so a few FTEs is probably adequate to handle that kind of work. Considering the larger Python ecosystem the community does a lot of the work, so saying there are only two people working full-time on Python is quite misleading IMHO, as Python has probably one of the largest communities in the world for a programming language, the Github repo alone lists almost 1.500 contributors.
dragonwriter 6y ago
> Two years ago it was estimated that it only had (the equivalent of) two full-time employees working on it. The discussion thread on that reveals that it was a bit low even by its explicit premise (core devs that had allocated time for OSS that was used for Python), and that methodology itself undercounted reality because it ignored people who had and made use of employer permission to use paid time but did not have a set allocation.
vnparker 6y ago
And these caused a lot of disagreement in the core team. Many thought that they were introducing too much churn. Coupled with the fact that they get paid for the alleged churn and others have to catch up on unpaid time, this can ruin a project. The Python dev community has certainly gotten far less vibrant (and productive) in the last decade due to stifling corporate influence.
sokoloff 6y ago
Only two people working full-time on python is not the same as “(the equivalent of) two full-time employees”. One is two people. The other is a few thousand person-hours of effort per year.
- pansa2 6y ago
  I’m not sure what you mean. How is “two people working full-time“ not equivalent to “two full-time employees”?
  - sokoloff 6y ago
    Two people working full time on Python have two names. The equivalent of two full time employees could be 10 people with 20% of their time. I suspect there are a lot of patches submitted to python that were developed by someone being paid while they did the work that isn’t accounted for in the 1 actual full-time plus 5 20% time people. IOW, paying for 1 additional named person to work ~2000 hours in 2021 will almost surely not be a net increase of 50% of capacity.

by pansa2 6y ago

josephg 6y ago

Its such a statement about the free rider problem that python can have only 2 people working fulltime on it. Python is one of the most popular languages in the world - there's got to be trillions of dollars worth of companies depending on it. Small improvements to the language would result in millions of dollars of productivity gains in aggregate globally. And yet ... collectively we can only find enough money to pay two full time salaries?! Seriously?

edwardwatson 6y ago
Python is as popular as it is because it's free, not because it's good. If companies had to contribute, many would pick something else.
tasubotadas 6y ago
So... why don't you pay?
- bzb6 6y ago
  Someone else should!
- feanaro 6y ago
  Why do you assume he isn't paying? The payment should obviously be proportional, though. You can't expect a single individual to finance what a gigantic multibillion corporation could.
  - tasubotadas 6y ago
    Just because some random individual came up with some arbitratry agenda of whom should finance what, doesn't mean that it makes sense. There are so many better (and worse) things to finance than Python Foundation.
    - feanaro 6y ago
      It's not very arbitrary. The reasoning (in one of the upstream posts) is that Python is one of the world's most used programming languages. It definitely makes sense for large multi-billion corporations which are the users of this language to help finance it. Your point would make more sense if the claim was that a significant portion of the corporation's profits should go towards financing Python. Then I would agree that "there are so many better things to finance than Python". But if we're discussing a salary or two in the context of corporations that employ hundreds of thousands of people, then your argument makes much less sense in the current context. There are very few things that would have better marginal utility then paying for the (effectively) third full-time Python developer.
- josephg 6y ago
  I say this as someone who's spent decades of my life donating time to opensource projects. - But the model where big companies have their technology stacks underpinned by charity is ridiculous. Obviously ridiculous, if python only has 2 full-time-equivalent staff. This problem isn't solved by little old me pulling out my check book. My bank and my insurance company don't need charity from me in order to fund the technology they've built their businesses on. They need to learn to pony up and start funding the "free" software they use. Paying myself would only help kick the can down the road - and encourage and enable this sort of corporate mooching to keep happening. In 60 years most of the volunteer maintainers of the software we depend on every day will be dead. Some time between now and then we need solution to this problem of opensource sustainability. Delaying buys us nothing.
  - sokoloff 6y ago
    If python continues to be this popular, there will be more maintainers who join the project. Surely you’ve seen volunteers join existing projects out of their own interest and need now. The decades that you contributed (thank you, BTW): were those all on projects you started, or did you also join some existing efforts? I don’t see any reason to think that will stop in 10, 20, or 60 years.
    - josephg 6y ago
      > If python continues to be this popular, there will be more maintainers who join the project. Will there? I feel like most of the opensource attention flows gradually up stack / toward trendy things on github and away from infrastructure older than github. When heartbleed happened a few years ago it was revealed that OpenSSL was maintained by basically one guy. The same is true of the linux timezone database. Apparently python is mostly 2 people. I'd put money on this being is a theme for lots of tools we take for granted. How many people work on bash? Or busybox, grub, libpthread or libicu? I bet the number is tiny.
      - sokoloff 6y ago
        Cpython had 5 pull requests opened yesterday. 6 the day before that (from 10 total distinct contributors). That doesn’t seem like a project starving for contributors. If everyone with commit access up and quit, there’d be turmoil for several weeks and something reasonable would emerge. People want to work on popular, useful things. Python is both of those.
Barrin92 6y ago
because software that is provided for free cannot capture the positive externalities it creates and thus is under-supplied, it's just very straight-forward economics. Yet it's become such a widespread sentiment to expect software for free that charging for something like Python would probably be seen as obscene and kill the language through loss of reputation alone.
_tom_ 6y ago
And large improvements to the language could result in millions of dollars of productivity losses! :-) I'm not a big fan of "Now this language has gotten popular, let's add all the features to it that made our other language so unpopular."
- huffmsa 6y ago
  It taking 10 years to transition from 2 to 3 wasn't necessarily a bad thing. Definitely concerns that more hands could make another Angular, which had 2 incompatible branches, with one going from v2 to v11 in 5 years.
  - josephg 6y ago
    > It taking 10 years to transition from 2 to 3 wasn't necessarily a bad thing. The python3 interpreter could have been largely (or completely) source-compatible with python2 code. They could have done it rust2018 style - where source code files can opt in to a newer parser, and in return get access to the new features. Then the whole python3 drama would have been avoided. MacOS still ships with a python 2.7 interpreter because of all that nonsense. But if they only had 2 full-time people working on it, no wonder they didn't have the man power to do the transition any other way. Disappointing.
    - my123 6y ago
      Which puts a: > WARNING: Python 2.7 is not recommended. > This version is included in macOS for compatibility with legacy software. > Future versions of macOS will not include Python 2.7. > Instead, it is recommended that you transition to using 'python3' from within Terminal. banner on launch nowadays...
      - josephg 6y ago
        Right; none of this should be necessary. I don’t need an old version of my C compiler in order to compile old C files. Rust made breaking changes to their language and yet a modern rust compiler will happily compile the older 2015 rust syntax. Python could have gone down the same road; but I suppose all of python’s corporate users chose to fight python compatibility changes locally rather than donate time / money fixing python itself.
andi999 6y ago
I am seriously wondering: is it actually possible for stock listed companies to donate? I mean aren't they legally obliged not to spend more than necessary (towards their stockholders obliged). Also what does the IRS say on that?
- foota 6y ago
  No. This is a misinterpretation of the responsibility. Companies have great latitude in what they do, as long as they're forthright.
- OJFord 6y ago
  They have a duty to try to operate efficiently, but that doesn't make corporate social responsibility, a bad hire, or wasting something nobody liked in the canteen illegal.
  - andi999 6y ago
    Well, bad hire, bad food can always be justified (avoiding hindsight fallacy). But giving away money for nothing received (meaning of donation) is something different (at least to me)
    - keeganpoppen 6y ago
      companies donate money to stuff all the time what are you on about?
      - andi999 6y ago
        So I read up a little. To tax deduct the nonprofit must have registered as 501(c)(3). Is this the case here?
        sokoloff 6y ago
        I can pay you to put my company name on your business website or a physical banner at a trade show and account for this as a business expense (deduct it) without you being a non-profit. I have broad latitude to make business expenditures in a manner that I believe is profit-seeking or optimizing for my business. People do this with Google and Facebook all the time. We call them ads. Not every ad campaign has a positive RoI.
kortilla 6y ago
> And yet ... collectively we can only find enough money to pay two full time salaries?! Seriously? But what’s the problem here? Is python struggling as a language? If there were clear calls for help companies could find it a lot easier to justify donating money or dev time.
- coldtea 6y ago
  >Is python struggling as a language? As a language, interpreter, runtime, yes. In terms of adoption, no.
  - kortilla 6y ago
    With what? Why haven’t they said so? There are a bunch of tech companies that would love the exposure of employing core developers.
passtheglass 6y ago
To me that is more like evidence of a well-engineered codebase.
II2II 6y ago
Is that actually a bad thing? Much of the popularity of Python is based upon third party libraries that have been contributed by both individuals and organizations. These are still contributions, and there are many cases where the contributors are paid. Should Python receive more funding and pay more people to work on the language? Probably, yet it is doubtful that having a large team working on the core language will improve the health of the language.
coldtea 6y ago
>Its such a statement about the free rider problem that python can have only 2 people working fulltime on it It's also a big statement about how non realistic were the ideas that FOSS will thrive on volunteers working together. Back in the day where the "Cathredral and the Bazaar" was published, VALinux got its IPO, and FOSS was the latest hype, people really believed that this will work without corporate sponsorship, just by volunteers doing the work. Then again, they also believed in many eyes available through the magic of FOSS making bugs shallow (Torvalds), and you get 20+ years of some obvious bug in something as critical as OpenSSL.

ramraj07 6y ago

That's insane. A thought occured recently - some lines of code in the python code base have been executed an innumerable number of times (trillion-trillion? Can't even start to calculate the order of magnitude). Even the subtlest of optimizations to these lines could collectively save probably hundreds or thousands of tons of CO2 from being released. That alone might deserve a couple of People looking into such massively common codebases fulltime.

Radim 6y ago
And yet… Did you know about this (serious) proposal to make CPython 5x faster? https://mail.python.org/archives/list/python-dev@python.org/thread/RDXLCH22T2EZDRCBM6ZYYIUTBWQVVVWH/ https://mail.python.org/archives/list/python-dev@python.org/... Needed funding though.
- tpetry 6y ago
  Why does this not happen or anything like this? Ruby got MJT in a big effort to make Ruby a lot faster. PHP is constantly increasing it‘s performance and tries to change a lot to get faster (basic building blocks for JIT in PHP8). Why is python not iterating so much on it‘s performance? Or is it doing this but it‘s not as public in promoting it like my former two examples.
  - dmurray 6y ago
    There's already PyPy. It's faster for, I estimate, 90% of workloads that don't use NumPy or Tensorflow, which offload work to faster implementations in different ways. It can certainly be 5 times faster. It doesn't have that much adoption, which suggests that performance isn't critical to Python's user base. If PyPy got significant market share, perhaps the CPython developers would prioritise an approach like this. (Or perhaps they'd have even more justification to say look, there's already a performance-focused implementation, we'll do our thing).
    - tpetry 6y ago
      Pypy isnt used that much because you may run into problems. And nobody wants to research strange runtime errors in production. But the speed improvements would be welcome if they were in cpython because you can be quite sure there will be no problems.

ThePhysicist 6y ago

That's a pretty distorted view IMHO. There are a lot of people that are working in the Python ecosystem and improve the language, and while they are not directly paid by the PSF their employers give them time to devote to working on Python. I think e.g. GvR had plenty of freedom to work on his Python projects while at Dropbox. I know a lot of companies in Germany that give people ample time to go to Python conferences and engage with the community, as it's also a way to meet and recruit good Python developers. So it might be true that there are only a few FTEs working on Python, but the larger community of people contributing to it is much larger.

pansa2 6y ago
> GvR had plenty of freedom to work on his Python projects while at Dropbox. That post was made while Guido was at Dropbox, and it claims he had one day a week?
- ThePhysicist 6y ago
  They talk about maintaining Python, which includes things like preparing and managing releases. Why would you need more than a few people to do that? The core language itself also evolves at a slow pace (which is good), it's not that there are tons of new features every year that get merged into the codebase, so a few FTEs is probably adequate to handle that kind of work. Considering the larger Python ecosystem the community does a lot of the work, so saying there are only two people working full-time on Python is quite misleading IMHO, as Python has probably one of the largest communities in the world for a programming language, the Github repo alone lists almost 1.500 contributors.

dragonwriter 6y ago

> Two years ago it was estimated that it only had (the equivalent of) two full-time employees working on it. The discussion thread on that reveals that it was a bit low even by its explicit premise (core devs that had allocated time for OSS that was used for Python), and that methodology itself undercounted reality because it ignored people who had and made use of employer permission to use paid time but did not have a set allocation.

vnparker 6y ago

And these caused a lot of disagreement in the core team. Many thought that they were introducing too much churn. Coupled with the fact that they get paid for the alleged churn and others have to catch up on unpaid time, this can ruin a project. The Python dev community has certainly gotten far less vibrant (and productive) in the last decade due to stifling corporate influence.

sokoloff 6y ago

Only two people working full-time on python is not the same as “(the equivalent of) two full-time employees”. One is two people. The other is a few thousand person-hours of effort per year.

pansa2 6y ago
I’m not sure what you mean. How is “two people working full-time“ not equivalent to “two full-time employees”?
- sokoloff 6y ago
  Two people working full time on Python have two names. The equivalent of two full time employees could be 10 people with 20% of their time. I suspect there are a lot of patches submitted to python that were developed by someone being paid while they did the work that isn’t accounted for in the 1 actual full-time plus 5 20% time people. IOW, paying for 1 additional named person to work ~2000 hours in 2021 will almost surely not be a net increase of 50% of capacity.