7 ms·

How long does python upgrading take you? I think Python 3.7 to 3.8 was change my docker version and push to CI... 5 minutes later it told me all tests passed an

How long does python upgrading take you? I think Python 3.7 to 3.8 was change my docker version and push to CI... 5 minutes later it told me all tests passed and I deployed to production??

Scarblac 7y ago
But what version does production have, and all the other servers and projects?
- kevinmgranger 7y ago
  That wouldn't matter if you're using docker, unless I'm missing something here.
  - brianwawok 7y ago
    Exactly. Each of my apps has their own python runtime. Running on top of kubernetes. Man in 2020 if you are SSHing between boxes and upgrading python by hand, you really should invest in some devops. Even 5 hours a week.....
    - aprdm 7y ago
      I hope you realize not everyone running python is running web apps. There are huge companies that use python for automation and they all run from some computer grid storage (usually NFS) and are imported in the same environment as the client instead of having a server side environment separated from the client side environment.
      - brianwawok 7y ago
        98% of my containers are not web apps. Still a lot easier to manage as docker volumes in k8s.
    - Scarblac 7y ago
      We have VMs and deploy using Ansible, devops doesn't have to mean running everything in containers using Kubernetes. Our applications are mostly monoliths and the number of servers they use is quite constant. But it does mean our Python version really only changes when we switch to the next Ubuntu LTR, and isn't always the same between different projects.
  - Scarblac 7y ago
    Yes, I was only thinking of my own situation (we use Docker in development, but not in production except for one project).
thoraway1010 7y ago
You'd be amazed out how many stale docker vm's are out there running old python versions - docker doesn't solve this issue and in fact I've seen docker images get updated less rather than more often ironically!
- solarkraft 7y ago
  > docker doesn't solve this issue Well. The container doesn't magically update itself. But FROM python:3.6 -> FROM python:3.8 doesn't seem that hard. Then again a flexible package manager (with recent repos) can do this as well.
- brianwawok 7y ago
  I am using my own docker image though. I control what is installed on it.
- flukus 7y ago
  For a while my company was considering docker precisely to keep an old python app (described as python 1.5) running. The dependencies of that app haven't been maintained in a decade, will never be upgraded and will likely disappear soon as the hosts remove their mercurial repos. Having the base machine running something modern with the old python stuff in a docker image seemed like a promising solution but the stuck with RHEL7 instead. Thankfully it's no longer my problem.
RMPR 7y ago
This is underestimating the number of packages not ported yet. Last time I checked, pygame and opencv were among those.
- riquito 7y ago
  > Last time I checked pygame supports python3 since 2016, python-opencv since 2018 at least
  - RMPR 7y ago
    The comment I'm replying to is about the transition from 3.7 to 3.8 > I think Python 3.7 to 3.8 was change my docker version and push to CI
mywittyname 7y ago
I upgraded from 3.7 to 3.8 and found out they removed a function in the time package required by crypto. I do strongly feel that removing functions from the core library is a huge no-no for point releases.
- elcritch 7y ago
  From afar, Python 3.x releases seem more like major updates than point releases. There’s entirely blocks of functionality/syntax between 3.x releases.
  - brianwawok 7y ago
    Except it is mostly additive. Adding new features will seldom break old stuff. It is the removing part that is hard. (With the exception being when like a variable broke because it became a keyword, but if you made a variable something like async I am not sure you are entirely innocent).
- xxpor 7y ago
  You're supposed to pay attention to DeprecationWarnings.
  - MattGaiser 7y ago
    The problem is that the entire ecosystem needs to pay attention to such warnings and it doesn't happen. As a result, these changes end up breaking code in places the program authors never touched.
    - xxpor 7y ago
      That's true in one respect, but you as an end user can use them to know not to upgrade before your deps have changed what they need to.
      - scoot_718 7y ago
linsomniac 7y ago
It's not how long it takes me, it's how long it takes Ubuntu. Usually, a few years... If I don't want to use OS packages, I have to decide to package, build, deploy, and maintain the Python packages, often in a way that separates it from the system Python package, while also managing any Python libraries we need (some of which may be C extensions and require compiling). Whenever possible, I like to leverage upstream packaging so that I don't have to track security updates on my own. I say this as someone who used to maintain the official Python.org RPM packages.
mehrdadn 7y ago
Python 3.7 to 3.8 broke some stuff. I was pretty annoyed to discover (the hard way) that e.g. this wouldn't run anymore: python3 -c "import ast; print(ast.arguments([]).args)" It seemed completely unnecessary too. They could've just kept the structure the same across minor versions...
- brianwawok 7y ago
  That is what is great about having a good test setup. You would know pretty quick if stuff broke. Which has been basically nothing for me (3.6 -> 3.7 -> 3.8)
  - mehrdadn 7y ago
    To be clear, the pain here was hunting down the root cause and fixing it when it shouldn't have broken in the first place, not the speed of discovering the problem after the code had already broken.
  - scoot_718 7y ago
    I'm glad your hello world is still functional.
- takeda 7y ago
  I think that probably lesson learned from python 2 -> 3 migration. It's probably better to introduce many small breaks than a big one? The python despite having 3 digit versions apparently is not following semantic versioning. The major version number generally had not much significance (the python 3 was exception, 1->2 and what they are assuring us 3->4 won't be a big change)
  - mehrdadn 7y ago
    "Better" depends on your agenda. If it's to shove the next major version down users' throats, maybe it's easier to swallow small bites (bytes?) than one giant cow. If it's to provide a predictable and pleasant user experience, I'm less confident this is the right approach.

How long does python upgrading take you? I think Python 3.7 to 3.8 was change my docker version and push to CI... 5 minutes later it told me all tests passed and I deployed to production??

Scarblac 7y ago

But what version does production have, and all the other servers and projects?

kevinmgranger 7y ago
That wouldn't matter if you're using docker, unless I'm missing something here.
- brianwawok 7y ago
  Exactly. Each of my apps has their own python runtime. Running on top of kubernetes. Man in 2020 if you are SSHing between boxes and upgrading python by hand, you really should invest in some devops. Even 5 hours a week.....
  - aprdm 7y ago
    I hope you realize not everyone running python is running web apps. There are huge companies that use python for automation and they all run from some computer grid storage (usually NFS) and are imported in the same environment as the client instead of having a server side environment separated from the client side environment.
    - brianwawok 7y ago
      98% of my containers are not web apps. Still a lot easier to manage as docker volumes in k8s.
  - Scarblac 7y ago
    We have VMs and deploy using Ansible, devops doesn't have to mean running everything in containers using Kubernetes. Our applications are mostly monoliths and the number of servers they use is quite constant. But it does mean our Python version really only changes when we switch to the next Ubuntu LTR, and isn't always the same between different projects.
- Scarblac 7y ago
  Yes, I was only thinking of my own situation (we use Docker in development, but not in production except for one project).

thoraway1010 7y ago

You'd be amazed out how many stale docker vm's are out there running old python versions - docker doesn't solve this issue and in fact I've seen docker images get updated less rather than more often ironically!

solarkraft 7y ago
> docker doesn't solve this issue Well. The container doesn't magically update itself. But FROM python:3.6 -> FROM python:3.8 doesn't seem that hard. Then again a flexible package manager (with recent repos) can do this as well.
brianwawok 7y ago
I am using my own docker image though. I control what is installed on it.
flukus 7y ago
For a while my company was considering docker precisely to keep an old python app (described as python 1.5) running. The dependencies of that app haven't been maintained in a decade, will never be upgraded and will likely disappear soon as the hosts remove their mercurial repos. Having the base machine running something modern with the old python stuff in a docker image seemed like a promising solution but the stuck with RHEL7 instead. Thankfully it's no longer my problem.

RMPR 7y ago

This is underestimating the number of packages not ported yet. Last time I checked, pygame and opencv were among those.

riquito 7y ago
> Last time I checked pygame supports python3 since 2016, python-opencv since 2018 at least
- RMPR 7y ago
  The comment I'm replying to is about the transition from 3.7 to 3.8 > I think Python 3.7 to 3.8 was change my docker version and push to CI

mywittyname 7y ago

I upgraded from 3.7 to 3.8 and found out they removed a function in the time package required by crypto. I do strongly feel that removing functions from the core library is a huge no-no for point releases.

elcritch 7y ago
From afar, Python 3.x releases seem more like major updates than point releases. There’s entirely blocks of functionality/syntax between 3.x releases.
- brianwawok 7y ago
  Except it is mostly additive. Adding new features will seldom break old stuff. It is the removing part that is hard. (With the exception being when like a variable broke because it became a keyword, but if you made a variable something like async I am not sure you are entirely innocent).
xxpor 7y ago
You're supposed to pay attention to DeprecationWarnings.
- MattGaiser 7y ago
  The problem is that the entire ecosystem needs to pay attention to such warnings and it doesn't happen. As a result, these changes end up breaking code in places the program authors never touched.
  - xxpor 7y ago
    That's true in one respect, but you as an end user can use them to know not to upgrade before your deps have changed what they need to.
    - scoot_718 7y ago

linsomniac 7y ago

It's not how long it takes me, it's how long it takes Ubuntu. Usually, a few years... If I don't want to use OS packages, I have to decide to package, build, deploy, and maintain the Python packages, often in a way that separates it from the system Python package, while also managing any Python libraries we need (some of which may be C extensions and require compiling). Whenever possible, I like to leverage upstream packaging so that I don't have to track security updates on my own. I say this as someone who used to maintain the official Python.org RPM packages.

mehrdadn 7y ago

Python 3.7 to 3.8 broke some stuff. I was pretty annoyed to discover (the hard way) that e.g. this wouldn't run anymore: python3 -c "import ast; print(ast.arguments([]).args)" It seemed completely unnecessary too. They could've just kept the structure the same across minor versions...

brianwawok 7y ago
That is what is great about having a good test setup. You would know pretty quick if stuff broke. Which has been basically nothing for me (3.6 -> 3.7 -> 3.8)
- mehrdadn 7y ago
  To be clear, the pain here was hunting down the root cause and fixing it when it shouldn't have broken in the first place, not the speed of discovering the problem after the code had already broken.
- scoot_718 7y ago
  I'm glad your hello world is still functional.
takeda 7y ago
I think that probably lesson learned from python 2 -> 3 migration. It's probably better to introduce many small breaks than a big one? The python despite having 3 digit versions apparently is not following semantic versioning. The major version number generally had not much significance (the python 3 was exception, 1->2 and what they are assuring us 3->4 won't be a big change)
- mehrdadn 7y ago
  "Better" depends on your agenda. If it's to shove the next major version down users' throats, maybe it's easier to swallow small bites (bytes?) than one giant cow. If it's to provide a predictable and pleasant user experience, I'm less confident this is the right approach.