4 ms·

I don't work on large python projects, mostly just small scripts that need to work well (integrating with a 3rd party rest api is a good example). I don't do CI

by _8j50 4y ago

I don't work on large python projects, mostly just small scripts that need to work well (integrating with a 3rd party rest api is a good example). I don't do CI or unittests but I use git. This is because it takes time and honestly no one outside of myself would care for small stuff like that. But I do run autopep8 and pylint it (I ignore stuff like line being too long,broad exception handling or lack of docs). My concern is a) It needs to be reliable (don't wanna spend a ton of time chasing bugs later on) b) How can I write the actual code better? I see what pro devs write and they use smarter language features or better organization of the code itself that makes it faster and reliable, I wish I could learn that explicitly somewhere. I mean, just the 2.7->3.0 jump was big for me because since I don't code regularly that meant googling errors a lot basically. Even now, I dread new python versions because some dependency would start using those features and that means I have to use venv to get that small script to work and then figure out how to troubleshoot bugs in that other lib's code with the new feature so I can do a PR for them. I love python but this is exactly why I prioritize languages that don't churn out new drastic features quickly. Those are just not suitable for people whose day job is not coding and migrating to new versions, supporting code bases, messing with build systems, unit tests, qa,ci,etc... coding is a tool for me, not the centerpiece of all I do. But python is still great despite all that.

_bohm 4y ago
For your dependency/versioning issue, use a virtualenv per-project and pin your dependency versions in requirements.txt
- LtWorf 4y ago
  Don't pin unless it's needed. I have a library… most downloaded version is 3 years old. The newer versions are massively faster but nobody uses them.
  - anderskaseorg 4y ago
    You should not pin the public requirements that get uploaded with a library (listed in setup.py, setup.cfg, or pyproject.toml), since that will restrict your downstream users, leading to version conflicts and persistent security vulnerabilities. But it’s totally reasonable to pin the private requirements that you develop it against (listed in requirements.txt, poetry.lock, or similar), updating them every so often during the course of development, so that contributors can use a consistent set of tools.
    - LtWorf 4y ago
      So leaf packages can pin vulnerable or slow stuff why?
      - anderskaseorg 4y ago
        The context is > For your dependency/versioning issue, use a virtualenv per-project and pin your dependency versions in requirements.txt requirements.txt is not uploaded to PyPI and has no effect on your package’s dependencies when a user installs it (leaf package or no). It’s only used for developing the package itself, typically in a unique virtual environment.
  - _bohm 4y ago
    Agree in principle, but I'm giving advice to someone who programs on occasion and is primarily concerned with their programs breaking due to dependency version upgrades when they come back to them after a little while.
    - LtWorf 4y ago
      My advice would be to use stuff that is in distributions… it hopefully (not necessarily) is maintained by less noob people who don't break API all the time.
selcuka 4y ago
> Even now, I dread new python versions because some dependency would start using those features If a dependency breaks compatibility with earlier Python versions because the author wants to use a fancy new feature is not really the fault of Python, is it? Library authors should target the earliest supported Python version they can. Being backwards compatible (at which Python has been doing a good job since the 2->3 fiasco) is one thing, but trying to be forwards compatible is something else. Are you suggesting that Python developers should only ship bug fixes so that Python 3.0 can still run code written for Python 3.11?
- forgotusername6 4y ago
  This isn't a general problem, but I have been seriously burnt by a change from a minor version upgrade. In 3.8 someone decided that they didn't like the way people were excepting the Exception for cancelled asycnio tasks. So they changed the cancelled task exception to inherit from base exception instead of exception. This meant a bunch of well used libraries immediately had a load of subtle bugs that in normal operation just didn't happen. I can't remember the exact details but I think when the bug did happen the task queue would just continue to grow until we ran out of memory. This change wasn't a bug fix, more an optimization or an attempt to get people to code a certain way. I'm all in favour of bug fixes, but Devs shouldn't have to worry about minor upgrades breaking everything. See https://bugs.python.org/issue32528 https://bugs.python.org/issue32528
- SPBS 4y ago
  It doesn’t mean it’s Python’s fault, but it fosters a culture where Python developers who regularly follow the language and are some of the ecosystem’s biggest authors are enticed to trying out the fancy new features (even if the old way still works) because “this is cleaner, this is how I want to do things from now on”.
powersnail 4y ago
To be fair, 2.7->3.0 was big for everyone. Python quite literally became a different language. Since then, nothing has been as dramatic as that.
black3r 4y ago
> I love python but this is exactly why I prioritize languages that don't churn out new drastic features quickly. What do you mean by "drastic" features "quickly"? Python releases new version once a year these days, and upgrading our Django-based source code with 150 dependencies from 3.4 to 3.11 literally meant switching out the python version in our CI configuration and README.rst every once in a while, no code changes were necessary for any of those jumps... Our developer README also contains a guide how to set-up and use pyenv and it's virtualenv plugin which makes installing new python versions and managing virtualenvs easy, just pyenv install, pyenv virtualenv, pyenv local, and your shell automatically uses the correct virtualenv whenever you're anywhere inside your project folder... jumping to python3 was big, but you had plenty of time to prepare for that and plenty of good utilities to make the jump easier (2to3, six, ...). python2.7 itself was released 18 months after python3.0, and by the time python2.7's support ended, python3.8 was already out...
- _8j50 4y ago
  Firsr, to this date, stuff I absolutley need that is in 2.7 i have to either try to fix or venv or somehow get it to work is one of my biggest headaches (Not my code). Second, yes, all you have to do is switch out the python version to upgrade but let's say you start using f-strings that means all of your users (doesn't apply to django since it is server software) have to upgrade to the right python version including all the deps. But what if your project is a library? That means all other libraries need to use the same or greater python version but what if your distro doesn't yet support the very latesr python version? It's such a nightmare. New versions should come out no more often than every 3-4 years imho and even then every effort should be made to have those features backward compatible like have a tool that will degrade scripts to be usable on a previous language version.
  - black3r 4y ago
    > Firsr, to this date, stuff I absolutley need that is in 2.7 i have to either try to fix or venv or somehow get it to work is one of my biggest headaches (Not my code). 2.7 was supported for 10 years and it's support ended 2 years ago. There's been ample time to upgrade the code or look for an alternative. If I "absolutely needed" to use a piece of code that I didn't write, is for an unsupported platform and is itself unsupported, I'd absolutely find the time for it. As a developer if I use a library that hasn't been touched for 3 years it's a red flag and I start to look for alternative libraries or forking the code. > That means all other libraries need to use the same or greater python version but what if your distro doesn't yet support the very latesr python version? It's such a nightmare. if your distro doesn't support the latest python version you're probably on a very old distro. For example python3.11 installs fine on all supported versions of Ubuntu (18.04+) and Debian (10+) and both Windows (8.1+) and macOS (10.15+). And python3.9 installs fine even on centos7 (released in 2014) and still supports the vast majority of python libraries. If you're on an OS nearing or past its end of support, you can't reasonably expect all the latest software to work on it. And it's usually fine to just use an older version of python / libraries until you're ready to update. > New versions should come out no more often than every 3-4 years imho If new versions came out every 3-4 years, that would mean they would have more drastic changes, because the smaller changes would accumulate over that duration. The longer the "new features" are out, the longer users have to upgrade their system and the longer developers can take getting used to them. But in the end, it doesn't really matter how often a new version comes out but rather how long the old versions should be supported, right? And I think it's up to the library authors to decide how long to support older versions, not the authors of the programming language.

by _8j50 4y ago

_bohm 4y ago

For your dependency/versioning issue, use a virtualenv per-project and pin your dependency versions in requirements.txt

LtWorf 4y ago
Don't pin unless it's needed. I have a library… most downloaded version is 3 years old. The newer versions are massively faster but nobody uses them.
- anderskaseorg 4y ago
  You should not pin the public requirements that get uploaded with a library (listed in setup.py, setup.cfg, or pyproject.toml), since that will restrict your downstream users, leading to version conflicts and persistent security vulnerabilities. But it’s totally reasonable to pin the private requirements that you develop it against (listed in requirements.txt, poetry.lock, or similar), updating them every so often during the course of development, so that contributors can use a consistent set of tools.
  - LtWorf 4y ago
    So leaf packages can pin vulnerable or slow stuff why?
    - anderskaseorg 4y ago
      The context is > For your dependency/versioning issue, use a virtualenv per-project and pin your dependency versions in requirements.txt requirements.txt is not uploaded to PyPI and has no effect on your package’s dependencies when a user installs it (leaf package or no). It’s only used for developing the package itself, typically in a unique virtual environment.
- _bohm 4y ago
  Agree in principle, but I'm giving advice to someone who programs on occasion and is primarily concerned with their programs breaking due to dependency version upgrades when they come back to them after a little while.
  - LtWorf 4y ago
    My advice would be to use stuff that is in distributions… it hopefully (not necessarily) is maintained by less noob people who don't break API all the time.

selcuka 4y ago

> Even now, I dread new python versions because some dependency would start using those features If a dependency breaks compatibility with earlier Python versions because the author wants to use a fancy new feature is not really the fault of Python, is it? Library authors should target the earliest supported Python version they can. Being backwards compatible (at which Python has been doing a good job since the 2->3 fiasco) is one thing, but trying to be forwards compatible is something else. Are you suggesting that Python developers should only ship bug fixes so that Python 3.0 can still run code written for Python 3.11?

forgotusername6 4y ago
This isn't a general problem, but I have been seriously burnt by a change from a minor version upgrade. In 3.8 someone decided that they didn't like the way people were excepting the Exception for cancelled asycnio tasks. So they changed the cancelled task exception to inherit from base exception instead of exception. This meant a bunch of well used libraries immediately had a load of subtle bugs that in normal operation just didn't happen. I can't remember the exact details but I think when the bug did happen the task queue would just continue to grow until we ran out of memory. This change wasn't a bug fix, more an optimization or an attempt to get people to code a certain way. I'm all in favour of bug fixes, but Devs shouldn't have to worry about minor upgrades breaking everything. See https://bugs.python.org/issue32528 https://bugs.python.org/issue32528
SPBS 4y ago
It doesn’t mean it’s Python’s fault, but it fosters a culture where Python developers who regularly follow the language and are some of the ecosystem’s biggest authors are enticed to trying out the fancy new features (even if the old way still works) because “this is cleaner, this is how I want to do things from now on”.

powersnail 4y ago

To be fair, 2.7->3.0 was big for everyone. Python quite literally became a different language. Since then, nothing has been as dramatic as that.

black3r 4y ago

> I love python but this is exactly why I prioritize languages that don't churn out new drastic features quickly. What do you mean by "drastic" features "quickly"? Python releases new version once a year these days, and upgrading our Django-based source code with 150 dependencies from 3.4 to 3.11 literally meant switching out the python version in our CI configuration and README.rst every once in a while, no code changes were necessary for any of those jumps... Our developer README also contains a guide how to set-up and use pyenv and it's virtualenv plugin which makes installing new python versions and managing virtualenvs easy, just pyenv install, pyenv virtualenv, pyenv local, and your shell automatically uses the correct virtualenv whenever you're anywhere inside your project folder... jumping to python3 was big, but you had plenty of time to prepare for that and plenty of good utilities to make the jump easier (2to3, six, ...). python2.7 itself was released 18 months after python3.0, and by the time python2.7's support ended, python3.8 was already out...

_8j50 4y ago
Firsr, to this date, stuff I absolutley need that is in 2.7 i have to either try to fix or venv or somehow get it to work is one of my biggest headaches (Not my code). Second, yes, all you have to do is switch out the python version to upgrade but let's say you start using f-strings that means all of your users (doesn't apply to django since it is server software) have to upgrade to the right python version including all the deps. But what if your project is a library? That means all other libraries need to use the same or greater python version but what if your distro doesn't yet support the very latesr python version? It's such a nightmare. New versions should come out no more often than every 3-4 years imho and even then every effort should be made to have those features backward compatible like have a tool that will degrade scripts to be usable on a previous language version.
- black3r 4y ago
  > Firsr, to this date, stuff I absolutley need that is in 2.7 i have to either try to fix or venv or somehow get it to work is one of my biggest headaches (Not my code). 2.7 was supported for 10 years and it's support ended 2 years ago. There's been ample time to upgrade the code or look for an alternative. If I "absolutely needed" to use a piece of code that I didn't write, is for an unsupported platform and is itself unsupported, I'd absolutely find the time for it. As a developer if I use a library that hasn't been touched for 3 years it's a red flag and I start to look for alternative libraries or forking the code. > That means all other libraries need to use the same or greater python version but what if your distro doesn't yet support the very latesr python version? It's such a nightmare. if your distro doesn't support the latest python version you're probably on a very old distro. For example python3.11 installs fine on all supported versions of Ubuntu (18.04+) and Debian (10+) and both Windows (8.1+) and macOS (10.15+). And python3.9 installs fine even on centos7 (released in 2014) and still supports the vast majority of python libraries. If you're on an OS nearing or past its end of support, you can't reasonably expect all the latest software to work on it. And it's usually fine to just use an older version of python / libraries until you're ready to update. > New versions should come out no more often than every 3-4 years imho If new versions came out every 3-4 years, that would mean they would have more drastic changes, because the smaller changes would accumulate over that duration. The longer the "new features" are out, the longer users have to upgrade their system and the longer developers can take getting used to them. But in the end, it doesn't really matter how often a new version comes out but rather how long the old versions should be supported, right? And I think it's up to the library authors to decide how long to support older versions, not the authors of the programming language.