8 ms·

For the lazy who just want to know if this makes Python faster yet, this is foundational work to enable later improvements: > The initial benchmarks show somet

For the lazy who just want to know if this makes Python faster yet, this is foundational work to enable later improvements: > The initial benchmarks show something of a 2-9% performance improvement. > I think that whilst the first version of this JIT isn’t going to seriously dent any benchmarks (yet), it opens the door to some huge optimizations and not just ones that benefit the toy benchmark programs in the standard benchmark suite.

Topfi 3y ago
Honestly, 2-9% already seems like a very signficant improvement, especially since as they mention "remember that CPython is already written in C". Whilst it's great to look at the potential for even greater gains by building upon this work, I feel we shouldn't undersell what's been accomplished.
- adastra22 3y ago
  What is being accomplished then?
  - gray_-_wolf 3y ago
    2-9%
- bomewish 3y ago
  Also recall that a 50% speed improvement in SQLite was caused by 50-100 different optimisations that each eeked out 0.5-1% speedups. On phone now don’t have the ref but it all adds up.
  - boxed 3y ago
    Many small improvements is the way to go in most situations. It's not great clickbait, but we should remember that we got from a single cell at some time to humans through many small changes. The world would be a lot better if people just embraced the grind of many small improvements...
  - toyg 3y ago
    Marginal gains. https://www.bbc.co.uk/news/magazine-34247629 https://www.bbc.co.uk/news/magazine-34247629
  - Akronymus 3y ago
    I tried searching for that article because I vaguely recall it, but can't find it either. But yeah, a lot of small improvements add up. Reminds me of this talk: https://www.youtube.com/watch?v=NZ5Lwzrdoe8 https://www.youtube.com/watch?v=NZ5Lwzrdoe8
    - Topfi 3y ago
      Here is a source for the SQLite case: https://topic.alibabacloud.com/a/sqlite-387-a-large-number-of-minor-optimizations-improving-performance-by-more-than-50_1_46_32753417.html https://topic.alibabacloud.com/a/sqlite-387-a-large-number-o...
      - Akronymus 3y ago
        That looks like blogspam to me, rather than an actual source.
      - formerly_proven 3y ago
        https://sqlite-users.sqlite.narkive.com/CVRvSKBs/50-faster-than-3-7-17# https://sqlite-users.sqlite.narkive.com/CVRvSKBs/50-faster-t...
  - IshKebab 3y ago
    That's true, and Rust compiler speed has seen similar speedups from lots of 1% improvements. But even if you can get a 2x improvement from lots of 1% improvements (if you work really really hard), you're never going to get a 10x improvement. Rust is never going to compile remotely as quickly as Go. Python is never going to be remotely as fast as Rust, C++, Go, Java, C#, Dart, etc.
    - inglor_cz 3y ago
      Does it matter? Trains are never going to beat jets in pure speed. But in certain scenarios, trains make a lot more sense to use than jets, and in those scenarios, it is usually preferable having a 150 mph train to a 75 mph train. Looking at the world of railways, high-speed rail has attracted a lot more paying customers than legacy railways, even though it doesn't even try to achieve flight-like speeds. Same with programming languages, I guess.
      - fl0ki 3y ago
        What is the programming analogy here? Two decades ago, you could (as e.g. Paul Graham did at the time) argue that dynamically typed languages can get your ideas to market faster so you become viable and figure out optimization later. It's been a long time since that argument held. Almost every dynamic programming language still under active development is adding some form of gradual typing because the maintainability benefits alone are clearly recognized, though such languages still struggle to optimize well. Now there are several statically typed languages to choose from that get those maintainability benefits up-front and optimize very well. Different languages can still be a better fit for different projects, e.g. Rust, Go, and Swift are all statically typed compiled languages better fit for different purposes, but in your analogy they're all jets designed for different tactical roles, none of them are "trains" of any speed. Analogies about how different programming languages are like different vehicles or power tools or etc go way back and have their place, but they have to recognize that sometimes one design approach largely supersedes another for practical purposes. Maybe the analogy would be clearer comparing jets and trains which each have their place, to horse-drawn carriages which still exist but are virtually never chosen for their functional benefits.
- spacechild1 3y ago
  > "remember that CPython is already written in C" What is this supposed to say? Most scripting language interpreters are written in low level languages (or assembly), but that alone doesn't say anything about the performance of the language itself.
  - nertirs 3y ago
    This means, that a lot of python libraries like polars or tensorflow are written not in python. So python programs, that already spend most of its cpu time running these libraries code, won't see much of an impact.
    - gh02t 3y ago
      Isn't the point that if pure Python was faster they wouldn't need to be written in other [compiled] languages? Having dealt with Cython it's not bad, but if I could write more of my code in native Python my development experience would be a lot simpler. Granted we're still very far from that and probably won't ever reach it, but there definitely seems to be a lot of progress.
      - __MatrixMan__ 3y ago
        Since Nim compiles to C, a middle step worth being aware of is Nim + nimporter which isn't anywhere near "just python" but is (maybe?) closer than "compile a C binary and call it from python". Or maybe it's just syntactic sugar around that. But sugar can be nice.
  - eequah9L 3y ago
    I think they mean that a lot of runtime of any benchmark is going to be spent in the C bits of the standard library, and therefore not subject to the JIT. Only the glue code and the bookkeeping or whatnot that the benchmark introduces would be improved by the JIT. This reduces the impact that the JIT can make.
blagie 3y ago
From the write-up, I honestly don't understand how this paves the way. I don't see an architectural path from a cut-and-paste JIT to something optimizing. That's the whole point of a cut-and-paste JIT.
- fulafel 3y ago
  Support for generating machine code at all seems like a necessary building block to me and probably is quite a bit of effort to work on top of a portable interpreter code base.
- lumpa 3y ago
  There's a lot of effort going on to improve CPython performance, with optimization tiers, etc. It seems the JIT is how at least part of that effort will materialize: https://github.com/python/cpython/issues/113710 https://github.com/python/cpython/issues/113710 > We're getting a JIT. Now it's time to optimize the traces to pass them to the JIT.
- guenthert 3y ago
  Isn't it the case that Python allows for type specifier (type hints) since 3.5, albeit the CPython interpreter ignores them? The JIT might take advantage of them, which ought to improve performance significantly for some code. That what makes Python flexible is what makes it slow. Restricting the flexibility were possible offers opportunities to improve performance (and allows for tools and humans to spot errors more easily).
  - a-french-anon 3y ago
    Isn't CL a good counter-example to that "dynamism inherently stunts performances" mantra?
    - guenthert 3y ago
      To the contrary. In CL some flexibility was given up (compared to other LISP dialects) in favor of enabling optimizing compilers, e.g. the standard symbols cannot be reassigned (also preserving the sanity of human readers). CL also offers what some now call 'gradual typing', i.e. optional type declarations. And remaining flexibility, e.g. around the OO support, limits how well the compiler can optimize the code.
deleted 3y ago
[deleted]
lifthrasiir 3y ago
An important context here is that the same code was reused for interpreter and JIT implementations (that's a main selling point for copy-and-patch JIT). In the other words, this 2--9% improvement mostly represents the core interpreter overhead that JIT should significant reduce. It was even possible that JIT itself might have no performance impact by itself, so this result is actually very encouraging; any future opcode specialization and refinement should directly translate to a measurable improvement.
- formerly_proven 3y ago
  Copy&patch seems not much worse than compiling pure Python with Cython, which roughly corresponds to "just call whatever CPython API functions the bytecode interpreter would call for this bunch of Python", so that's roughly a baseline for how much overhead you get from the interpeter bit.
  - lifthrasiir 3y ago
    There is no reason to use copy-and-patch JIT if that were the case, because the good old threaded interpreter would have been fine. There are other optimization works in parallel with this JIT effort, including finer-grained micro operations (uops) that can replace usual opcodes at higher tiers. Uops themselves can be used without JIT, but the interpreter overhead is proportional to the number of (u)ops executed and would be too large for uops. The hope is that copy-and-patch JIT combined with uops have to be much faster than threaded code.
    - ncruces 3y ago
      A threaded interpreter still has one branch per bytecode instruction; a copy-and-patch JIT removes this overhead.
vanderZwan 3y ago
You're right, and in this case "foundational work" even undersells how minimal this work really is compared to the results it already gets. I recommend that people watch Brandt Bucher's "A JIT Compiler for CPython" from last year's CPython Core Developer Sprint[0]. It gives a good impression of the current implementation and its limitations, and some hints at what may or may not work out. It also indirectly gives a glimpse into the process of getting this into Python through the exchanges during the Q&A discussion. One thing to especially highlight is that this copy-and-patch has a much, much lower implementation complexity for the maintainers, as a lot of the heavy lifting is offloaded to LLVM. Case in point: as of the talk this was all just Brandt Bucher's work. The implementation at the time was ~700 lines of "complex" Python, ~100 lines of "complex" C, plus of course the LLVM dependency. This produces ~3000 lines of "simple" generated C, requires an additional ~300 lines of "simple" hand-written C to come together, and no further dependencies (so no LLVM necessary to run the JIT. Also "complex" and "simple" qualifiers are Bucher's terms, not mine). Another thing to note is that these initial performance improvements are just from getting this first version of the copy-and-patch JIT to work at all, without really doing any further fine-tuning or optimization. This may have changed a bit in the months since, but the situation is probably still comparable. So if one person can get this up and running in a few klocs, most of which are generated, I think it's reasonable to have good hopes for its future. [0] https://www.youtube.com/watch?v=HxSHIpEQRjs https://www.youtube.com/watch?v=HxSHIpEQRjs
sylware 3y ago
I removed the SDKs of some big (big for the wrong reasons) open source projects which generates a lot of code using python3 scripts. In those custom SDKs, I do generate all the code at the start of the build, which takes a significant amount of time for mostly non-pertinent anymore/inappropiately done code generation.. I will really feel python3 speed improvement for those builds.
attractivechaos 3y ago
I wouldn't be so enthusiastic. Look at other languages that have JIT now: Ruby and PHP. After years of efforts, they are still an order of magnitude slower than V8 and even PyPy [1]. It seems to me that you need to design a JIT implementation from ground up to get good performance – V8, Dart and LuaJIT are like this; if you start with a pure interpreter, it may be difficult to speed it up later. [1] https://github.com/attractivechaos/plb2 https://github.com/attractivechaos/plb2
- vlovich123 3y ago
  PyPy is designed from the ground up and is still slower than V8 AFAIK. Don’t forget that v8 has enormous amounts of investment from professionally paid developers whereas PyPy is funded by government grants. Not sure about Ruby & PHP and it’s entirely possible that the other JIT implementations are choosing simplicity of maintenance over eking out every single bit of performance. Python also has structural challenges like native extensions (don’t exist in JavaScript) where the API forces slow code or massive hacks like avoiding the C API at all costs (if I recall correctly I read that’s being worked on) and the GIL. One advantage Python had is the ability to use multiple cores way before JS but the JS ecosystem remained single threaded longer & decided to use message passing instead to build WebWorkers which let the JIT remain fast.
  - attractivechaos 3y ago
    PyPy is only twice as slow as v8 and is about an order of magnitude faster than CPython. It is quite an achievement. I would be very happy if CPython could get this performance but I doubt.
chaxor 3y ago
Anyone know if there will be any better tools for cross-compiling python projects? The package management and build tools for python have been so atrociously bad (environments add far too much complexity to the ecosystem) that it turns many developers away from the language altogether. A system like Rust's package management, build tools, and cross compilation capability is an enormous draw, even without the memory safety. The fact that it actually works (because of the package management and build tools) is the main reason to use the language really. Python used to do that ~10 years ago. Now absolutely nothing works. It takes weeks to get simple packages working, only can do anything under extremely brittle conditions that nullify the project you're trying to use this other package for, etc. If python could ever get it's act together and make better package management, and allow for cross-compiling, it could make a big difference. (I am aware of the very basic fact that it's interpreted rather than compiled yada yada - there are still ways to make executables, they are just awful). Since python is data science centric, it would be good to have decent data management capabilities too, but perhaps that could be after fundamental problem are dealt with. I tried looking at mojo, but it's not open source, so I'm quite certain that kills any hope of it ever being useful at all to anyone. The fact that I couldn't even install it without making an account made me run away as fast as possible.
- simonw 3y ago
  "It takes weeks to get simple packages working" Can you expand on what you mean by that? I have trouble imagining a Python packaging problem that takes weeks to resolve - I'd expect them to either be resolvable in relatively short order or for them to prove effectively impossible such that people give up.
  - chaxor 3y ago
    - Trying to figure out what versions the scripts used and specifying them in a new poetry project - Realizing some OS-dependent software is needed so making a docker file/docker-compose.yml - Getting some of it working in the container with a poetry environment - Realizing that other parts of the code work with other versions, so making a different poetry environment for those parts - Trying to tie this package/container as a dependency of another project - Oh actually, this is a dependency of a dependency - How do you call a function from a package running in a container with multiple poetry environments in a package? - What was I doing again? - 2 weeks have passed trying to get this to work, perhaps I'll just do something else Rinse and repeat. ¯\_(ツ)_/¯ That's python!
deleted 3y ago
[deleted]
deleted 3y ago
[deleted]

Topfi 3y ago

Honestly, 2-9% already seems like a very signficant improvement, especially since as they mention "remember that CPython is already written in C". Whilst it's great to look at the potential for even greater gains by building upon this work, I feel we shouldn't undersell what's been accomplished.

adastra22 3y ago
What is being accomplished then?
- gray_-_wolf 3y ago
  2-9%
bomewish 3y ago
Also recall that a 50% speed improvement in SQLite was caused by 50-100 different optimisations that each eeked out 0.5-1% speedups. On phone now don’t have the ref but it all adds up.
- boxed 3y ago
  Many small improvements is the way to go in most situations. It's not great clickbait, but we should remember that we got from a single cell at some time to humans through many small changes. The world would be a lot better if people just embraced the grind of many small improvements...
- toyg 3y ago
  Marginal gains. https://www.bbc.co.uk/news/magazine-34247629 https://www.bbc.co.uk/news/magazine-34247629
- Akronymus 3y ago
  I tried searching for that article because I vaguely recall it, but can't find it either. But yeah, a lot of small improvements add up. Reminds me of this talk: https://www.youtube.com/watch?v=NZ5Lwzrdoe8 https://www.youtube.com/watch?v=NZ5Lwzrdoe8
  - Topfi 3y ago
    Here is a source for the SQLite case: https://topic.alibabacloud.com/a/sqlite-387-a-large-number-of-minor-optimizations-improving-performance-by-more-than-50_1_46_32753417.html https://topic.alibabacloud.com/a/sqlite-387-a-large-number-o...
    - Akronymus 3y ago
      That looks like blogspam to me, rather than an actual source.
    - formerly_proven 3y ago
      https://sqlite-users.sqlite.narkive.com/CVRvSKBs/50-faster-than-3-7-17# https://sqlite-users.sqlite.narkive.com/CVRvSKBs/50-faster-t...
- IshKebab 3y ago
  That's true, and Rust compiler speed has seen similar speedups from lots of 1% improvements. But even if you can get a 2x improvement from lots of 1% improvements (if you work really really hard), you're never going to get a 10x improvement. Rust is never going to compile remotely as quickly as Go. Python is never going to be remotely as fast as Rust, C++, Go, Java, C#, Dart, etc.
  - inglor_cz 3y ago
    Does it matter? Trains are never going to beat jets in pure speed. But in certain scenarios, trains make a lot more sense to use than jets, and in those scenarios, it is usually preferable having a 150 mph train to a 75 mph train. Looking at the world of railways, high-speed rail has attracted a lot more paying customers than legacy railways, even though it doesn't even try to achieve flight-like speeds. Same with programming languages, I guess.
    - fl0ki 3y ago
      What is the programming analogy here? Two decades ago, you could (as e.g. Paul Graham did at the time) argue that dynamically typed languages can get your ideas to market faster so you become viable and figure out optimization later. It's been a long time since that argument held. Almost every dynamic programming language still under active development is adding some form of gradual typing because the maintainability benefits alone are clearly recognized, though such languages still struggle to optimize well. Now there are several statically typed languages to choose from that get those maintainability benefits up-front and optimize very well. Different languages can still be a better fit for different projects, e.g. Rust, Go, and Swift are all statically typed compiled languages better fit for different purposes, but in your analogy they're all jets designed for different tactical roles, none of them are "trains" of any speed. Analogies about how different programming languages are like different vehicles or power tools or etc go way back and have their place, but they have to recognize that sometimes one design approach largely supersedes another for practical purposes. Maybe the analogy would be clearer comparing jets and trains which each have their place, to horse-drawn carriages which still exist but are virtually never chosen for their functional benefits.
spacechild1 3y ago
> "remember that CPython is already written in C" What is this supposed to say? Most scripting language interpreters are written in low level languages (or assembly), but that alone doesn't say anything about the performance of the language itself.
- nertirs 3y ago
  This means, that a lot of python libraries like polars or tensorflow are written not in python. So python programs, that already spend most of its cpu time running these libraries code, won't see much of an impact.
  - gh02t 3y ago
    Isn't the point that if pure Python was faster they wouldn't need to be written in other [compiled] languages? Having dealt with Cython it's not bad, but if I could write more of my code in native Python my development experience would be a lot simpler. Granted we're still very far from that and probably won't ever reach it, but there definitely seems to be a lot of progress.
    - __MatrixMan__ 3y ago
      Since Nim compiles to C, a middle step worth being aware of is Nim + nimporter which isn't anywhere near "just python" but is (maybe?) closer than "compile a C binary and call it from python". Or maybe it's just syntactic sugar around that. But sugar can be nice.
- eequah9L 3y ago
  I think they mean that a lot of runtime of any benchmark is going to be spent in the C bits of the standard library, and therefore not subject to the JIT. Only the glue code and the bookkeeping or whatnot that the benchmark introduces would be improved by the JIT. This reduces the impact that the JIT can make.

blagie 3y ago

From the write-up, I honestly don't understand how this paves the way. I don't see an architectural path from a cut-and-paste JIT to something optimizing. That's the whole point of a cut-and-paste JIT.

fulafel 3y ago
Support for generating machine code at all seems like a necessary building block to me and probably is quite a bit of effort to work on top of a portable interpreter code base.
lumpa 3y ago
There's a lot of effort going on to improve CPython performance, with optimization tiers, etc. It seems the JIT is how at least part of that effort will materialize: https://github.com/python/cpython/issues/113710 https://github.com/python/cpython/issues/113710 > We're getting a JIT. Now it's time to optimize the traces to pass them to the JIT.
guenthert 3y ago
Isn't it the case that Python allows for type specifier (type hints) since 3.5, albeit the CPython interpreter ignores them? The JIT might take advantage of them, which ought to improve performance significantly for some code. That what makes Python flexible is what makes it slow. Restricting the flexibility were possible offers opportunities to improve performance (and allows for tools and humans to spot errors more easily).
- a-french-anon 3y ago
  Isn't CL a good counter-example to that "dynamism inherently stunts performances" mantra?
  - guenthert 3y ago
    To the contrary. In CL some flexibility was given up (compared to other LISP dialects) in favor of enabling optimizing compilers, e.g. the standard symbols cannot be reassigned (also preserving the sanity of human readers). CL also offers what some now call 'gradual typing', i.e. optional type declarations. And remaining flexibility, e.g. around the OO support, limits how well the compiler can optimize the code.

deleted 3y ago

[deleted]

lifthrasiir 3y ago

An important context here is that the same code was reused for interpreter and JIT implementations (that's a main selling point for copy-and-patch JIT). In the other words, this 2--9% improvement mostly represents the core interpreter overhead that JIT should significant reduce. It was even possible that JIT itself might have no performance impact by itself, so this result is actually very encouraging; any future opcode specialization and refinement should directly translate to a measurable improvement.

formerly_proven 3y ago
Copy&patch seems not much worse than compiling pure Python with Cython, which roughly corresponds to "just call whatever CPython API functions the bytecode interpreter would call for this bunch of Python", so that's roughly a baseline for how much overhead you get from the interpeter bit.
- lifthrasiir 3y ago
  There is no reason to use copy-and-patch JIT if that were the case, because the good old threaded interpreter would have been fine. There are other optimization works in parallel with this JIT effort, including finer-grained micro operations (uops) that can replace usual opcodes at higher tiers. Uops themselves can be used without JIT, but the interpreter overhead is proportional to the number of (u)ops executed and would be too large for uops. The hope is that copy-and-patch JIT combined with uops have to be much faster than threaded code.
  - ncruces 3y ago
    A threaded interpreter still has one branch per bytecode instruction; a copy-and-patch JIT removes this overhead.

vanderZwan 3y ago

You're right, and in this case "foundational work" even undersells how minimal this work really is compared to the results it already gets. I recommend that people watch Brandt Bucher's "A JIT Compiler for CPython" from last year's CPython Core Developer Sprint[0]. It gives a good impression of the current implementation and its limitations, and some hints at what may or may not work out. It also indirectly gives a glimpse into the process of getting this into Python through the exchanges during the Q&A discussion. One thing to especially highlight is that this copy-and-patch has a much, much lower implementation complexity for the maintainers, as a lot of the heavy lifting is offloaded to LLVM. Case in point: as of the talk this was all just Brandt Bucher's work. The implementation at the time was ~700 lines of "complex" Python, ~100 lines of "complex" C, plus of course the LLVM dependency. This produces ~3000 lines of "simple" generated C, requires an additional ~300 lines of "simple" hand-written C to come together, and no further dependencies (so no LLVM necessary to run the JIT. Also "complex" and "simple" qualifiers are Bucher's terms, not mine). Another thing to note is that these initial performance improvements are just from getting this first version of the copy-and-patch JIT to work at all, without really doing any further fine-tuning or optimization. This may have changed a bit in the months since, but the situation is probably still comparable. So if one person can get this up and running in a few klocs, most of which are generated, I think it's reasonable to have good hopes for its future. [0] https://www.youtube.com/watch?v=HxSHIpEQRjs https://www.youtube.com/watch?v=HxSHIpEQRjs

sylware 3y ago

I removed the SDKs of some big (big for the wrong reasons) open source projects which generates a lot of code using python3 scripts. In those custom SDKs, I do generate all the code at the start of the build, which takes a significant amount of time for mostly non-pertinent anymore/inappropiately done code generation.. I will really feel python3 speed improvement for those builds.

attractivechaos 3y ago

I wouldn't be so enthusiastic. Look at other languages that have JIT now: Ruby and PHP. After years of efforts, they are still an order of magnitude slower than V8 and even PyPy [1]. It seems to me that you need to design a JIT implementation from ground up to get good performance – V8, Dart and LuaJIT are like this; if you start with a pure interpreter, it may be difficult to speed it up later. [1] https://github.com/attractivechaos/plb2 https://github.com/attractivechaos/plb2

vlovich123 3y ago
PyPy is designed from the ground up and is still slower than V8 AFAIK. Don’t forget that v8 has enormous amounts of investment from professionally paid developers whereas PyPy is funded by government grants. Not sure about Ruby & PHP and it’s entirely possible that the other JIT implementations are choosing simplicity of maintenance over eking out every single bit of performance. Python also has structural challenges like native extensions (don’t exist in JavaScript) where the API forces slow code or massive hacks like avoiding the C API at all costs (if I recall correctly I read that’s being worked on) and the GIL. One advantage Python had is the ability to use multiple cores way before JS but the JS ecosystem remained single threaded longer & decided to use message passing instead to build WebWorkers which let the JIT remain fast.
- attractivechaos 3y ago
  PyPy is only twice as slow as v8 and is about an order of magnitude faster than CPython. It is quite an achievement. I would be very happy if CPython could get this performance but I doubt.

chaxor 3y ago

Anyone know if there will be any better tools for cross-compiling python projects? The package management and build tools for python have been so atrociously bad (environments add far too much complexity to the ecosystem) that it turns many developers away from the language altogether. A system like Rust's package management, build tools, and cross compilation capability is an enormous draw, even without the memory safety. The fact that it actually works (because of the package management and build tools) is the main reason to use the language really. Python used to do that ~10 years ago. Now absolutely nothing works. It takes weeks to get simple packages working, only can do anything under extremely brittle conditions that nullify the project you're trying to use this other package for, etc. If python could ever get it's act together and make better package management, and allow for cross-compiling, it could make a big difference. (I am aware of the very basic fact that it's interpreted rather than compiled yada yada - there are still ways to make executables, they are just awful). Since python is data science centric, it would be good to have decent data management capabilities too, but perhaps that could be after fundamental problem are dealt with. I tried looking at mojo, but it's not open source, so I'm quite certain that kills any hope of it ever being useful at all to anyone. The fact that I couldn't even install it without making an account made me run away as fast as possible.

simonw 3y ago
"It takes weeks to get simple packages working" Can you expand on what you mean by that? I have trouble imagining a Python packaging problem that takes weeks to resolve - I'd expect them to either be resolvable in relatively short order or for them to prove effectively impossible such that people give up.
- chaxor 3y ago
  - Trying to figure out what versions the scripts used and specifying them in a new poetry project - Realizing some OS-dependent software is needed so making a docker file/docker-compose.yml - Getting some of it working in the container with a poetry environment - Realizing that other parts of the code work with other versions, so making a different poetry environment for those parts - Trying to tie this package/container as a dependency of another project - Oh actually, this is a dependency of a dependency - How do you call a function from a package running in a container with multiple poetry environments in a package? - What was I doing again? - 2 weeks have passed trying to get this to work, perhaps I'll just do something else Rinse and repeat. ¯\_(ツ)_/¯ That's python!

deleted 3y ago

[deleted]

deleted 3y ago

[deleted]