13 ms·

I've always been disappointed by how large software projects, both FOSS and commercial, lose their "can do" spirit with age. Long-time contributors become very

I've always been disappointed by how large software projects, both FOSS and commercial, lose their "can do" spirit with age. Long-time contributors become very quick with a "no". They dismiss longstanding problems as illegitimate use cases and reject patches with vague and impervious arguments about "maintainability" or "complexity". Maybe in some specific cases these concerns might be justified, but when everything garners this reaction, the overall effect is that progress stalls, crystallized at the moment the last bit of technical boldness flowed away. You can see this attitude of "no" on this very HN thread. Read the comments! Instead of talking about ways we can make Python startup faster, we're seeing arguments that Python shouldn't be fast, we shouldn't try to make it faster, and that programs (and, by implication, programmers) who want Python startup to be fast are somehow illegitimate. It's a dismal perspective. We should be exercising our creativity as a way to solve problems, not finding creative ways to convince ourselves to accept mediocrity.

draw_down 9y ago
For a large enough project or area of concern, saying "no" to many things is essential to saying "yes" to anything.
gnulinux 9y ago
I think your comment is well-intentioned (I upvoted) but I respectfully disagree. I think wanting Python to be a bit faster is similar to wanting Haskell to have a little bit of mutability. Engineering with restrictions is a good thing, we can do great systems in Haskell because it's a very neat language even though it lacks mutability. We also can do great systems in Python because it's a very neat language even though it's a bit slow. Sure, you can always optimize Python's performance, that's a legitimate problem and it takes a few engineers to solve it. But it's more interesting to work around Python's slowness by engineering tricks such as better algorithms etc.
- jsharpe 9y ago
  That's not a great analogy. Haskell is a neat language in part because it doesn't have mutability. Python is a neat language despite being slow. I can't imagine anyone would object if Python could magically be 10x faster. I can't say the same thing for the Haskell thing.
  - gnulinux 9y ago
    My whole point is that 10x thing cannot just magically happen. The reason Python is slow is not incompetent programming or lack of magic. We know why that's happening. Because every variable in Python interpreter is a hash map and pretty much every operation is a hash map lookup. How do you optimize this? The only way is to remove language features like `setattr` and my whole point is that some people use Python because it's flexible enough to do that so they need their `setattr`.
    - civility 9y ago
      > The reason Python is slow is not incompetent programming or lack of magic. We know why that's happening. Because every variable in Python interpreter is a hash map and pretty much every operation is a hash map lookup. This statement is easily refuted by PyPy. Here's a simple program which runs 70 times faster in PyPy than Python on my machine - including startup time: https://pastebin.com/2n0PL9hY > How do you optimize this? Semantically, everything in JavaScript and Lua is also hash map lookups. Yet very smart people have made those languages very fast. CPython is not slow for the reasons you stated.
      - rrcaptain 9y ago
        Not all python programs are compatible with PyPy though. Nor do they get much of a performance boost by switching. Pandas and NumPy for example didn't work on PyPy until less than a year ago. And a good chunk of Python codebases are going to use one of those at some point.
        civility 9y ago
        > Not all python programs are compatible with PyPy though. I'm not sure how this is relevant to my reply... I didn't say PyPy was a replacement for Python. I said PyPy, Lua, and JavaScript are existence proofs that dynamic "hash table" languages don't have to be slow. Therefore, CPython must be slow for some other reason.
        9y ago
- weberc2 9y ago
  > Sure, you can always optimize Python's performance, that's a legitimate problem and it takes a few engineers to solve it. But it's more interesting to work around Python's slowness by engineering tricks such as better algorithms etc. Surely you're not implying that improving Python's performance would preclude finding interesting algorithms, nor that this is a suitable rationale for keeping Python slow? Anyway, algos can only get you so far when they're built on slow primitives (all data scattered haphazardly across the heap, every property access is a hash table lookup, every function call is a dozen C function calls, etc).
- dingo_bat 9y ago
  > I think wanting Python to be a bit faster is similar to wanting Haskell to have a little bit of mutability I'm sorry but that makes zero sense. Haskell is defined by immutability. People want to use haskell because of that characteristic. I don't want to use python because it is slow.
  - gnulinux 9y ago
    Sorry I disagree. There is definitely a sense in which Haskell is desirable because it is immutable, e.g. I myself love immutable data structures, it certainly makes it desirable. But my point is that it puts a restriction. Now you cannot implement algorithms that need mutability such as hash maps. It is easy to circumvent such problems but one other way is basically introducing mutability to Haskell which totally doesn't make sense. I think same goes for python, if you want to make it significantly faster then you need to face certain trade-offs: maybe data model should be optimized, or maybe `int` shouldn't be arbitrary precision integers, or maybe there should be primitive types like `int`, `double` as in Java to increase performance. Truth is these are not Pythonic solutions, and just like mutability is not Haskell-esque, optimizing Python sacrifising these trade-offs is not Pythonic.
    - tome 9y ago
      > one other way is basically introducing mutability to Haskell which totally doesn't make sense It makes perfect sense, because Haskell is not "an immutable language". It's a language in which some things are immutable, and those that are not are explicitly indicated in the type system.
netheril96 9y ago
This is why large companies like Google often reinvents the wheel. Open source gives everyone the right to use, but not the power of control. Sure, you can fork, but then your version will diverge from the official, and the pain of maintaining compatibility may be greater than writing your own from scratch.
flukus 9y ago
Everyone is focusing on python, but where is this "can do" spirit from mozilla? Their are languages with better startup times, bash, perl, lua, awk to name a few, and could likely do whatever the python scripts are doing.
- catach 9y ago
  > but where is this "can do" spirit from mozilla? The mail included both > At some point, we'll likely replace Python code with Rust so the build system is more "pure" and easier to maintain and reason about. and > Since I am disproportionately impacted by this issue, if there's anything I can do to help, let me know.
chocolatebunny 9y ago
It's a byproduct of how many people you have to answer to. I was kind of having discussion with a coworker about an app that had a lot of features that made it seem kind of cluttered but useful. I think small projects can make bolder choices and enable more options because they have a smaller userbase that would be impacted by their changes and they want to be able to reach more people so adding a feature is generally a net benefit. But a larger project cannot risk hurting the large userbase they have already established so they have to be more cautious about the changes that they make.
sjellis 9y ago
I get the impression that backwards-compatibility does weigh pretty heavily on the Python core developers these days. There are so many Python installations out there doing so much that the default answer to a change has to be "no". The fact that macOS and popular Linux distributions ship with copies of Python is great, but once something is effectively a component of operating systems, boldness is not a viable strategy. Arguably, one of the reasons why the transition to Python 3 has been so drawn out is that every time somebody installs macOS or one of many Linux distributions, a new Python 2 system is born. I've seen .NET Core developers explain that having .NET Framework shipped in Windows put them under massive constraints, and this was one of the motivations for a new runtime.
crististm 9y ago
It may also be that they simply don't have an attack on the startup problem.
angrygoat 9y ago
With major infrastructure like Python there's a tendency to over-emphasise compatibility between releases. Look at this post in the same list thread: https://mail.python.org/pipermail/python-dev/2018-May/153300.html https://mail.python.org/pipermail/python-dev/2018-May/153300... Python 3.6 is trying an enormous number of potential paths that code for imports might be found at. Why is that fixed in stone? Couldn't Python 3.(n+1) change that, if it's slow and historical, cutting out a bunch of slow system calls? As someone who makes use of Python to deploy software, it's entirely possible that could cause me a few issues... which I'd fix quite easily. It should be totally reasonable to expect the community using the software to cope with those sorts of changes after a major release; the alternative is ossification. Django suffered from maintaining too much compatibility, and releasing too slowly, and they fixed it. Three or four years ago everyone was talking about moving away from it; now they release often, deprecate stuff when they need to, and the project is as vibrant as it ever was. Time for cPython to learn the same lesson.
jdtang13 9y ago
I think part of what explains this attitude in people is "lack of imagination". In the sense that sometimes, especially when an existing project or organization or bureaucracy has become huge and daunting, people cannot imagine excellence anymore, so they believe it to be literally impossible.
remoteorbust 9y ago
I think it's because they've seen exactly where saying "yes" leads them and they don't like that place.
- dingo_bat 9y ago
  They hate fast code?
  - catach 9y ago
    Perhaps the known opportunities for dramatic perf increases require compatibility breaks and some expressiveness downgrades. That Python 3 transition was a fun time, yeah?
    - makapuf 9y ago
      If python 3 transition did bring dramatic perf increases I guess people would have been way faster to sell that upgrade to their hierarchies. Not that I blame python team for the lack of it. But fictional dev history would have been different
      - catach 9y ago
        No doubt, huge perf improvements for free would have made the transition more compelling. However, what I'm suggesting is that large perf wins are not free. Breaking compatibility too much more could have doomed Python 3. And now that the devs know how painful a transition is, they're far less likely to break compatibility again for any reason.
maxerickson 9y ago
To be fair, they are frequently saying no to things other people think they should do (rather than saying no to things like contributions of startup improvements).
wgthatcher 9y ago
Python3 has the exact opposite problem: Too many devs willing to say "yes" to features and a small number of devs who try to keep things fast and maintainable. Remember that Python2 was faster.
- poooogles 9y ago
  That changed with the dict improvements in 3.6.
geofft 9y ago
This isn't an attitude of "no" - it's an attitude of "yes" to other things. The arguments are that making Python startup fast makes other things worse, and we care about those other things. Here are some other things we can say "yes" to: - Rewrite as much of Mercurial in Rust as possible, which will provide performance improvements well beyond what Python can possibly offer. https://www.mercurial-scm.org/wiki/OxidationPlan https://www.mercurial-scm.org/wiki/OxidationPlan - Spend resources on developing PyPy, which (being a JIT) has relatively slow startup but much faster performance in general, for people who want fast performance. - Write compilers from well-typed Python to native code. - Keep CPython easy to hack on, so that more people with a "can do" spirit can successfully contribute to CPython instead of it being a mess of special cases in Guido's head. Will you join me in saying "yes" to these things and not convincing ourselves to accept mediocrity?
- x404 9y ago
  I agree with you. As the limitation of developing resources, say "no" is difficult but important.
- MadWombat 9y ago
  I am slightly afraid to ask, but what is a "well typed python"?
  - geofft 9y ago
    Have you seen MyPy, static type annotations / checking for Python? http://mypy-lang.org/ http://mypy-lang.org/ In context, what I'm really getting at "a sufficiently non-dynamic subset of Python that it can be compiled statically, but also a sufficiently large one that real Python programs can have a chance of being in the subset." PyPy has a thing called RPython that fits the former but not really the latter (I don't know of any non-PyPy-related codebases that work in RPython). In general, adding complete type annotations to a codebase is pretty correlated with making it static enough to do meta-level things on like compiling and optimizing it - for instance if you have a variable that changes types as the program runs, at least now you've enumerated its possible types. It's not the only way of doing so, but it seems to work well in practice and there seems to be a correlation between compiled vs. interpreted languages and static vs. dynamic typing.
qaq 9y ago
We is very abstract term I am sure if you proposed a patch that addressed the issue without adverse side-effects it would get accepted.
crdoconnor 9y ago
This is true but python's relative slowness (along with the GIL) is an issue that is regularly blown out of all proportion. Part of the reason for the language's success is because it made intelligent tradeoffs that often went against the grain of the opinions of the commentariat and focused on its strengths rather than pandering to the kinds of people who write language performance comparison blog posts. If speed were of primary importance then PyPy would be a lot more popular.
- quotemstr 9y ago
  You're conflating two kinds of "performance", startup latency and steady state throughput. We're talking about the former, and you're proposing improvements for the latter. In fact, moving to pypy is exactly what you shouldn't do to improve startup. It's surprising but frequently true that startup latency has a greater effect on the perception of performance than actual throughput. Nobody likes to type a command and then be kept waiting, even if the started program could in principle demonstrate amazing feats of computation once warmed up.
- jacquesm 9y ago
  Python derives a good chunk of its speed (if not all of it) from carefully tuned libraries written in other languages (or even for other architectures in the case of many machine learning packages). As soon as you try to do a lot of heavy processing python even the compiled versions quickly bog down. IMO the best way to use python is to use it to cleverly glue together highly optimized code. That way you spend the minimum amount of effort and you get maximum performance.
  - netheril96 9y ago
    Multithreading is the glue I need. How am I supposed to write optimized native module to spawn threads to do computation in numpy and pandas?
  - crdoconnor 9y ago
    Yeah, that was kind of my point :/
- 9y ago
abuckenheimer 9y ago
FWIW no one who replied to this email thread said something even close to "no". Victor Stinner points out that startup time is something that comes up a lot and mentions some recent work in the area [1]. Python is a big ship, it may not be as nimble as a young FOSS project but it is always improving and investments in things like start up time pays dividends to a large ecosystem. [1] https://mail.python.org/pipermail/python-dev/2018-May/153300.html https://mail.python.org/pipermail/python-dev/2018-May/153300...
deleted 9y ago
[deleted]
takluyver 9y ago
I'm not denying this phenomenon, but part of it is surely that widely used projects get more conservative because any change risks breaking something for someone somewhere. And the maintainers tend to feel a sense of responsibility to help people deal with these breakages.
ubernostrum 9y ago
I'll bring a slightly different perspective, as someone who's been using Python professionally for over a decade: there is no such thing as just saying "yes" or "no". Every "yes" to one group is at least an implicit "no" to some other group, and vice-versa. The Python 2/3 transition is a great example of this. Python 2 continued an earlier tradition of saying "yes" to almost everything from one particular group of programmers: people working on Unix who wanted a high-level language they could use to write Unix utilities, administrative tools, daemons, etc. In doing that, Python said "no" to people in a lot of other domains. Python 3 switched to saying "yes" to those other domains much more often. Which came with the inherent cost of saying "no" (or, more often, "not anymore") to the Unix-y crowd Python 2 had catered to. Life got harder for those programmers with Python 3. There's been work since then to mitigate some of the worst of it, but some of the changes that made Python nice to use for other domains are just always going to be messy for people doing the traditional Unix-type stuff. Personally, I think it was the right choice, and not just because my own problem domain got some big improvements from Python 3. In order to keep growing, and really even to maintain what it already had, Python had to become more than just a language that was good for traditional Unix-y things. Not changing in that respect would have been a guaranteed dead end. This doesn't mean it has to feel good to be someone from the traditional Unix programming domain who now feels like the language only ever says "no". But it does mean that it's worth having the perspective that this was how a lot of us felt in that golden age when you think Python said "yes" to everything, because really it was Python saying "yes" to you and "no" to me. And it's worth understanding that what feels like "no" doesn't mean the language is against you; it means the language is trying to balance the competing needs of a very large community.
- eesmith 9y ago
  "people working on Unix .... In doing that, Python said "no" to people in a lot of other domains." Could you elaborate on this? I thought Python was pretty good about supporting non-Unix OSes from early on. It was originally developed on SGI IRIX and MacOS. From the README for version 0.9: > There are built-in modules that interface to the operating system and to various window systems: X11, the Mac window system (you need STDWIN for these two), and Silicon Graphics' GL library. It runs on most modern versions of UNIX, on the Mac, and I wouldn't be surprised if it ran on MS-DOS unchanged. I developed it mostly on an SGI IRIS workstation (using IRIX 3.1 and 3.2) and on the Mac, but have tested it also on SunOS (4.1) and BSD 4.3 (tahoe). though it looks like there wasn't "painless" DOS support until 1994, with the comment "Many portability fixes should make it painless to build Python on several new platforms, e.g. NeXT, SEQUENT, WATCOM, DOS, and Windows." I also thought that PythonWin had very good Windows support quite early on. The 1.5a3 release notes say: > - Mark Hammond will release Python 1.5 versions of PythonWin and his other Windows specific code: the win32api extensions, COM/ActiveX support, and the MFC interface. > - As always, the Macintosh port will be done by Jack Jansen. He will make a separate announcement for the Mac specific source code and the binary distribution(s) when these are ready.
loeg 9y ago
I've always been disappointed at how quickly people make sweeping generalizations from a single anecdote. (I also think Python can do better here, but the generalization isn't justifiable.)
Rapzid 9y ago
Competition. Hiphop VM lit a fire under the PHP team.

draw_down 9y ago

For a large enough project or area of concern, saying "no" to many things is essential to saying "yes" to anything.

gnulinux 9y ago

I think your comment is well-intentioned (I upvoted) but I respectfully disagree. I think wanting Python to be a bit faster is similar to wanting Haskell to have a little bit of mutability. Engineering with restrictions is a good thing, we can do great systems in Haskell because it's a very neat language even though it lacks mutability. We also can do great systems in Python because it's a very neat language even though it's a bit slow. Sure, you can always optimize Python's performance, that's a legitimate problem and it takes a few engineers to solve it. But it's more interesting to work around Python's slowness by engineering tricks such as better algorithms etc.

jsharpe 9y ago
That's not a great analogy. Haskell is a neat language in part because it doesn't have mutability. Python is a neat language despite being slow. I can't imagine anyone would object if Python could magically be 10x faster. I can't say the same thing for the Haskell thing.
- gnulinux 9y ago
  My whole point is that 10x thing cannot just magically happen. The reason Python is slow is not incompetent programming or lack of magic. We know why that's happening. Because every variable in Python interpreter is a hash map and pretty much every operation is a hash map lookup. How do you optimize this? The only way is to remove language features like `setattr` and my whole point is that some people use Python because it's flexible enough to do that so they need their `setattr`.
  - civility 9y ago
    > The reason Python is slow is not incompetent programming or lack of magic. We know why that's happening. Because every variable in Python interpreter is a hash map and pretty much every operation is a hash map lookup. This statement is easily refuted by PyPy. Here's a simple program which runs 70 times faster in PyPy than Python on my machine - including startup time: https://pastebin.com/2n0PL9hY > How do you optimize this? Semantically, everything in JavaScript and Lua is also hash map lookups. Yet very smart people have made those languages very fast. CPython is not slow for the reasons you stated.
    - rrcaptain 9y ago
      Not all python programs are compatible with PyPy though. Nor do they get much of a performance boost by switching. Pandas and NumPy for example didn't work on PyPy until less than a year ago. And a good chunk of Python codebases are going to use one of those at some point.
      - civility 9y ago
        > Not all python programs are compatible with PyPy though. I'm not sure how this is relevant to my reply... I didn't say PyPy was a replacement for Python. I said PyPy, Lua, and JavaScript are existence proofs that dynamic "hash table" languages don't have to be slow. Therefore, CPython must be slow for some other reason.
      - 9y ago
weberc2 9y ago
> Sure, you can always optimize Python's performance, that's a legitimate problem and it takes a few engineers to solve it. But it's more interesting to work around Python's slowness by engineering tricks such as better algorithms etc. Surely you're not implying that improving Python's performance would preclude finding interesting algorithms, nor that this is a suitable rationale for keeping Python slow? Anyway, algos can only get you so far when they're built on slow primitives (all data scattered haphazardly across the heap, every property access is a hash table lookup, every function call is a dozen C function calls, etc).
dingo_bat 9y ago
> I think wanting Python to be a bit faster is similar to wanting Haskell to have a little bit of mutability I'm sorry but that makes zero sense. Haskell is defined by immutability. People want to use haskell because of that characteristic. I don't want to use python because it is slow.
- gnulinux 9y ago
  Sorry I disagree. There is definitely a sense in which Haskell is desirable because it is immutable, e.g. I myself love immutable data structures, it certainly makes it desirable. But my point is that it puts a restriction. Now you cannot implement algorithms that need mutability such as hash maps. It is easy to circumvent such problems but one other way is basically introducing mutability to Haskell which totally doesn't make sense. I think same goes for python, if you want to make it significantly faster then you need to face certain trade-offs: maybe data model should be optimized, or maybe `int` shouldn't be arbitrary precision integers, or maybe there should be primitive types like `int`, `double` as in Java to increase performance. Truth is these are not Pythonic solutions, and just like mutability is not Haskell-esque, optimizing Python sacrifising these trade-offs is not Pythonic.
  - tome 9y ago
    > one other way is basically introducing mutability to Haskell which totally doesn't make sense It makes perfect sense, because Haskell is not "an immutable language". It's a language in which some things are immutable, and those that are not are explicitly indicated in the type system.

netheril96 9y ago

This is why large companies like Google often reinvents the wheel. Open source gives everyone the right to use, but not the power of control. Sure, you can fork, but then your version will diverge from the official, and the pain of maintaining compatibility may be greater than writing your own from scratch.

flukus 9y ago

Everyone is focusing on python, but where is this "can do" spirit from mozilla? Their are languages with better startup times, bash, perl, lua, awk to name a few, and could likely do whatever the python scripts are doing.

catach 9y ago
> but where is this "can do" spirit from mozilla? The mail included both > At some point, we'll likely replace Python code with Rust so the build system is more "pure" and easier to maintain and reason about. and > Since I am disproportionately impacted by this issue, if there's anything I can do to help, let me know.

chocolatebunny 9y ago

It's a byproduct of how many people you have to answer to. I was kind of having discussion with a coworker about an app that had a lot of features that made it seem kind of cluttered but useful. I think small projects can make bolder choices and enable more options because they have a smaller userbase that would be impacted by their changes and they want to be able to reach more people so adding a feature is generally a net benefit. But a larger project cannot risk hurting the large userbase they have already established so they have to be more cautious about the changes that they make.

sjellis 9y ago

I get the impression that backwards-compatibility does weigh pretty heavily on the Python core developers these days. There are so many Python installations out there doing so much that the default answer to a change has to be "no". The fact that macOS and popular Linux distributions ship with copies of Python is great, but once something is effectively a component of operating systems, boldness is not a viable strategy. Arguably, one of the reasons why the transition to Python 3 has been so drawn out is that every time somebody installs macOS or one of many Linux distributions, a new Python 2 system is born. I've seen .NET Core developers explain that having .NET Framework shipped in Windows put them under massive constraints, and this was one of the motivations for a new runtime.

crististm 9y ago

It may also be that they simply don't have an attack on the startup problem.

angrygoat 9y ago

With major infrastructure like Python there's a tendency to over-emphasise compatibility between releases. Look at this post in the same list thread: https://mail.python.org/pipermail/python-dev/2018-May/153300.html https://mail.python.org/pipermail/python-dev/2018-May/153300... Python 3.6 is trying an enormous number of potential paths that code for imports might be found at. Why is that fixed in stone? Couldn't Python 3.(n+1) change that, if it's slow and historical, cutting out a bunch of slow system calls? As someone who makes use of Python to deploy software, it's entirely possible that could cause me a few issues... which I'd fix quite easily. It should be totally reasonable to expect the community using the software to cope with those sorts of changes after a major release; the alternative is ossification. Django suffered from maintaining too much compatibility, and releasing too slowly, and they fixed it. Three or four years ago everyone was talking about moving away from it; now they release often, deprecate stuff when they need to, and the project is as vibrant as it ever was. Time for cPython to learn the same lesson.

jdtang13 9y ago

I think part of what explains this attitude in people is "lack of imagination". In the sense that sometimes, especially when an existing project or organization or bureaucracy has become huge and daunting, people cannot imagine excellence anymore, so they believe it to be literally impossible.

remoteorbust 9y ago

I think it's because they've seen exactly where saying "yes" leads them and they don't like that place.

dingo_bat 9y ago
They hate fast code?
- catach 9y ago
  Perhaps the known opportunities for dramatic perf increases require compatibility breaks and some expressiveness downgrades. That Python 3 transition was a fun time, yeah?
  - makapuf 9y ago
    If python 3 transition did bring dramatic perf increases I guess people would have been way faster to sell that upgrade to their hierarchies. Not that I blame python team for the lack of it. But fictional dev history would have been different
    - catach 9y ago
      No doubt, huge perf improvements for free would have made the transition more compelling. However, what I'm suggesting is that large perf wins are not free. Breaking compatibility too much more could have doomed Python 3. And now that the devs know how painful a transition is, they're far less likely to break compatibility again for any reason.

maxerickson 9y ago

To be fair, they are frequently saying no to things other people think they should do (rather than saying no to things like contributions of startup improvements).

wgthatcher 9y ago

Python3 has the exact opposite problem: Too many devs willing to say "yes" to features and a small number of devs who try to keep things fast and maintainable. Remember that Python2 was faster.

poooogles 9y ago
That changed with the dict improvements in 3.6.

geofft 9y ago

This isn't an attitude of "no" - it's an attitude of "yes" to other things. The arguments are that making Python startup fast makes other things worse, and we care about those other things. Here are some other things we can say "yes" to: - Rewrite as much of Mercurial in Rust as possible, which will provide performance improvements well beyond what Python can possibly offer. https://www.mercurial-scm.org/wiki/OxidationPlan https://www.mercurial-scm.org/wiki/OxidationPlan - Spend resources on developing PyPy, which (being a JIT) has relatively slow startup but much faster performance in general, for people who want fast performance. - Write compilers from well-typed Python to native code. - Keep CPython easy to hack on, so that more people with a "can do" spirit can successfully contribute to CPython instead of it being a mess of special cases in Guido's head. Will you join me in saying "yes" to these things and not convincing ourselves to accept mediocrity?

x404 9y ago
I agree with you. As the limitation of developing resources, say "no" is difficult but important.
MadWombat 9y ago
I am slightly afraid to ask, but what is a "well typed python"?
- geofft 9y ago
  Have you seen MyPy, static type annotations / checking for Python? http://mypy-lang.org/ http://mypy-lang.org/ In context, what I'm really getting at "a sufficiently non-dynamic subset of Python that it can be compiled statically, but also a sufficiently large one that real Python programs can have a chance of being in the subset." PyPy has a thing called RPython that fits the former but not really the latter (I don't know of any non-PyPy-related codebases that work in RPython). In general, adding complete type annotations to a codebase is pretty correlated with making it static enough to do meta-level things on like compiling and optimizing it - for instance if you have a variable that changes types as the program runs, at least now you've enumerated its possible types. It's not the only way of doing so, but it seems to work well in practice and there seems to be a correlation between compiled vs. interpreted languages and static vs. dynamic typing.

qaq 9y ago

We is very abstract term I am sure if you proposed a patch that addressed the issue without adverse side-effects it would get accepted.

crdoconnor 9y ago

This is true but python's relative slowness (along with the GIL) is an issue that is regularly blown out of all proportion. Part of the reason for the language's success is because it made intelligent tradeoffs that often went against the grain of the opinions of the commentariat and focused on its strengths rather than pandering to the kinds of people who write language performance comparison blog posts. If speed were of primary importance then PyPy would be a lot more popular.

quotemstr 9y ago
You're conflating two kinds of "performance", startup latency and steady state throughput. We're talking about the former, and you're proposing improvements for the latter. In fact, moving to pypy is exactly what you shouldn't do to improve startup. It's surprising but frequently true that startup latency has a greater effect on the perception of performance than actual throughput. Nobody likes to type a command and then be kept waiting, even if the started program could in principle demonstrate amazing feats of computation once warmed up.
jacquesm 9y ago
Python derives a good chunk of its speed (if not all of it) from carefully tuned libraries written in other languages (or even for other architectures in the case of many machine learning packages). As soon as you try to do a lot of heavy processing python even the compiled versions quickly bog down. IMO the best way to use python is to use it to cleverly glue together highly optimized code. That way you spend the minimum amount of effort and you get maximum performance.
- netheril96 9y ago
  Multithreading is the glue I need. How am I supposed to write optimized native module to spawn threads to do computation in numpy and pandas?
- crdoconnor 9y ago
  Yeah, that was kind of my point :/
9y ago

abuckenheimer 9y ago

FWIW no one who replied to this email thread said something even close to "no". Victor Stinner points out that startup time is something that comes up a lot and mentions some recent work in the area [1]. Python is a big ship, it may not be as nimble as a young FOSS project but it is always improving and investments in things like start up time pays dividends to a large ecosystem. [1] https://mail.python.org/pipermail/python-dev/2018-May/153300.html https://mail.python.org/pipermail/python-dev/2018-May/153300...

deleted 9y ago

[deleted]

takluyver 9y ago

I'm not denying this phenomenon, but part of it is surely that widely used projects get more conservative because any change risks breaking something for someone somewhere. And the maintainers tend to feel a sense of responsibility to help people deal with these breakages.

ubernostrum 9y ago

I'll bring a slightly different perspective, as someone who's been using Python professionally for over a decade: there is no such thing as just saying "yes" or "no". Every "yes" to one group is at least an implicit "no" to some other group, and vice-versa. The Python 2/3 transition is a great example of this. Python 2 continued an earlier tradition of saying "yes" to almost everything from one particular group of programmers: people working on Unix who wanted a high-level language they could use to write Unix utilities, administrative tools, daemons, etc. In doing that, Python said "no" to people in a lot of other domains. Python 3 switched to saying "yes" to those other domains much more often. Which came with the inherent cost of saying "no" (or, more often, "not anymore") to the Unix-y crowd Python 2 had catered to. Life got harder for those programmers with Python 3. There's been work since then to mitigate some of the worst of it, but some of the changes that made Python nice to use for other domains are just always going to be messy for people doing the traditional Unix-type stuff. Personally, I think it was the right choice, and not just because my own problem domain got some big improvements from Python 3. In order to keep growing, and really even to maintain what it already had, Python had to become more than just a language that was good for traditional Unix-y things. Not changing in that respect would have been a guaranteed dead end. This doesn't mean it has to feel good to be someone from the traditional Unix programming domain who now feels like the language only ever says "no". But it does mean that it's worth having the perspective that this was how a lot of us felt in that golden age when you think Python said "yes" to everything, because really it was Python saying "yes" to you and "no" to me. And it's worth understanding that what feels like "no" doesn't mean the language is against you; it means the language is trying to balance the competing needs of a very large community.

eesmith 9y ago
"people working on Unix .... In doing that, Python said "no" to people in a lot of other domains." Could you elaborate on this? I thought Python was pretty good about supporting non-Unix OSes from early on. It was originally developed on SGI IRIX and MacOS. From the README for version 0.9: > There are built-in modules that interface to the operating system and to various window systems: X11, the Mac window system (you need STDWIN for these two), and Silicon Graphics' GL library. It runs on most modern versions of UNIX, on the Mac, and I wouldn't be surprised if it ran on MS-DOS unchanged. I developed it mostly on an SGI IRIS workstation (using IRIX 3.1 and 3.2) and on the Mac, but have tested it also on SunOS (4.1) and BSD 4.3 (tahoe). though it looks like there wasn't "painless" DOS support until 1994, with the comment "Many portability fixes should make it painless to build Python on several new platforms, e.g. NeXT, SEQUENT, WATCOM, DOS, and Windows." I also thought that PythonWin had very good Windows support quite early on. The 1.5a3 release notes say: > - Mark Hammond will release Python 1.5 versions of PythonWin and his other Windows specific code: the win32api extensions, COM/ActiveX support, and the MFC interface. > - As always, the Macintosh port will be done by Jack Jansen. He will make a separate announcement for the Mac specific source code and the binary distribution(s) when these are ready.

loeg 9y ago

I've always been disappointed at how quickly people make sweeping generalizations from a single anecdote. (I also think Python can do better here, but the generalization isn't justifiable.)

Rapzid 9y ago

Competition. Hiphop VM lit a fire under the PHP team.