3 ms·

I started programming with C and C++ in the 90s and I absolutely loved it. Then big projects with multithreading started to blow up because it became to complex

I started programming with C and C++ in the 90s and I absolutely loved it. Then big projects with multithreading started to blow up because it became to complex to manage. 2010 I started with Python and absolutely fell in love. Since then I've been using it and never regretted it, until I had to upgrade servers and with it migrate from 2.7 to 3.8, now again to 3.10. There are issues with these migrations which aren't negletible if you're running over 50 different servers, doing different things, where suddenly for no reason at all a function parameter gets removed from some asyncio function, or all the virtualenvs are no longer usable because they were set up with 3.8, which got replaced by 3.10. This makes upgrading an OS very painful as it requires many weeks of preparation, which is something you first need to become aware if. I now have tooling which monitors every running Python process and prints out which modules are used an other things, because there was so much to look after. Basically each Python script has to register it's execution at a central server, but because it can be down, they have to use a local caching proxy server for this. Then again, two weeks ago I started learning Rust and I'm absolutely blown away. I'm completely absorbed by it. I'm now thinking of migrating some of the Python projects to Rust so that I can have a single executable which I can then copy around without having to care about virtual environments. I mean, I loved C and C++ for its precision, and love Python for it's liveliness (I use `reload` a lot so all the projects are living things which support hot reloading), but Rust seems to hit the sweet spot between C++ and Python. Between performance and little dependencies due to the lack of an interpreter, and a somewhat easy to use threading and async / await with Tokio, with all the safety guarantees Rust offers. I'm currently in heaven for this and I hope it lasts.

eru 4y ago
It's probably best not to run Python bare-metal on your operating system. Or at least not to rely on your operating system's Python. Use a Python package manager, and/or run your Python processes in containers. That way you can upgrade your OS without having to upgrade all Python programs at the same time.
- qwertox 4y ago
  Thank you for bringing this approach onto my radar. It sounds very promising for solving the OS upgrade issue but also let's me finally use the latest version for my projects. I will read up on this. Are there some recommended keywords or things to avoid which I could use to orient myself? Like "use deadsnakes" or "avoid deadsnakes"?
  - eru 4y ago
    I suggest you look into Python's Poetry package manager. There's some ways to keep multiple isolated Python environments around on the same OS (and Poetry can help with that, too). But I would suggest that you just stick every Python program in its own dedicated docker container, if you can get away with the overhead. (There are some ways to make rather slim containers.) Of course, it doesn't have to be docker. Any other containerisation or VMs also work. In general, I found it a good idea to keep the host operation system as small as possible, and stick everything into containers that you can upgrade separately.

eru 4y ago

It's probably best not to run Python bare-metal on your operating system. Or at least not to rely on your operating system's Python. Use a Python package manager, and/or run your Python processes in containers. That way you can upgrade your OS without having to upgrade all Python programs at the same time.

qwertox 4y ago
Thank you for bringing this approach onto my radar. It sounds very promising for solving the OS upgrade issue but also let's me finally use the latest version for my projects. I will read up on this. Are there some recommended keywords or things to avoid which I could use to orient myself? Like "use deadsnakes" or "avoid deadsnakes"?
- eru 4y ago
  I suggest you look into Python's Poetry package manager. There's some ways to keep multiple isolated Python environments around on the same OS (and Poetry can help with that, too). But I would suggest that you just stick every Python program in its own dedicated docker container, if you can get away with the overhead. (There are some ways to make rather slim containers.) Of course, it doesn't have to be docker. Any other containerisation or VMs also work. In general, I found it a good idea to keep the host operation system as small as possible, and stick everything into containers that you can upgrade separately.