5 ms·

I miss those days of deep granular CS projects that strove to create the most efficient minimalist systems possible - seems the opposite of today's prolific jun

I miss those days of deep granular CS projects that strove to create the most efficient minimalist systems possible - seems the opposite of today's prolific jungles of libraries and linkers. Then again I consider Jonathan Blow a prophet in the desert of the real...

llm_trw 2y ago
The emperor has no clothes. Numpy 2.0 came out two days ago and it's chaos in the whole AI ecosystem. I'm not sure how much money we're wasting on that but I wouldn't be surprised if it's on the order of a billion dollars - suppose there are 50,000 people getting paid on the order of $1,000 per day each spending the two week dealing with fires over the next year: $500,000,000
- rbanffy 2y ago
  Did something major break on 2.0?
  - FuriouslyAdrift 2y ago
    The made breaking changes in the ABI... not like there weren't warnings. "Breaking changes to the NumPy ABI. As a result, binaries of packages that use the NumPy C API and were built against a NumPy 1.xx release will not work with NumPy 2.0. On import, such packages will see an ImportError with a message about binary incompatibility. It is possible to build binaries against NumPy 2.0 that will work at runtime with both NumPy 2.0 and 1.x. See NumPy 2.0-specific advice for more details." https://numpy.org/devdocs/dev/depending_on_numpy.html#numpy-2-abi-handling https://numpy.org/devdocs/dev/depending_on_numpy.html#numpy-...
    - topspin 2y ago
      > The made breaking changes in the ABI... They bumped the major number. That's fair play. There has always been a lot of slouching wrt versions in python. That's not numpy's fault. Too bad they're getting the black eye for it. They could have avoided it by making a new dependency name ("numpy2"), but that sets a shameful precedent, so I give them credit for not copping out.
      - rikthevik 2y ago
        It's a tough situation. Once you have an installed base, your hands get tied. I miss the early days when I could rework the UI and UX of the product regularly. Now there is a lot of legwork to make sure we're not changing existing workflows (documented or undocumented). That said, I like that we're making money, so I can't complain too much. I viscerally understand how companies can get stuck and unable to change their products significantly because their installed base will revolt. It's a tough situation.
        topspin 2y ago
        It's a self inflicted situation by dependees. Tools in widespread use, JupyterLab for instance, has users tossing dependencies onto the pile, offering next to no guidance on dependency version control, as if expecting them to have to ponder such issues is unreasonable. Well, if that's how you're going to behave, foregoing entirely obvious and solved engineering problems, you get to glue all the pieces back together when reality asserts itself. I don't imagine for one minute numpy folks didn't know what this would look like, and they did it anyway. Good on them. Breaking changes are necessary. That's unqualified. Necessary. If you can't afford a big blowup in your work then manage your work.
        rbanffy 2y ago
        One of the reasons I made pip-chill was to have a tool to make it easy to me to have a minimal set of non-versioned requirements for canary builds on the latest-and-greatest versions. When the canary breaks, you realise you have work to do.
        rbanffy 2y ago
        This is a symptom of cruft. Clean breaks are needed to keep software fresh. The Tree of Elegance needs to be refreshed with the blood of both users and programers. Or something like that.
        llm_trw 2y ago
  - IshKebab 2y ago
    It broke our builds. Because Numpy doesn't use semver there's no way to specify compatible versions constraints. One of our dependencies had a `numpy>=1.something` constraint but wasn't actually compatible with Numpy 2. We need to use a lock file really, but `pip` doesn't support that - you need to use a better package installed like `uv` to get this standard feature.

llm_trw 2y ago

The emperor has no clothes. Numpy 2.0 came out two days ago and it's chaos in the whole AI ecosystem. I'm not sure how much money we're wasting on that but I wouldn't be surprised if it's on the order of a billion dollars - suppose there are 50,000 people getting paid on the order of $1,000 per day each spending the two week dealing with fires over the next year: $500,000,000

rbanffy 2y ago
Did something major break on 2.0?
- FuriouslyAdrift 2y ago
  The made breaking changes in the ABI... not like there weren't warnings. "Breaking changes to the NumPy ABI. As a result, binaries of packages that use the NumPy C API and were built against a NumPy 1.xx release will not work with NumPy 2.0. On import, such packages will see an ImportError with a message about binary incompatibility. It is possible to build binaries against NumPy 2.0 that will work at runtime with both NumPy 2.0 and 1.x. See NumPy 2.0-specific advice for more details." https://numpy.org/devdocs/dev/depending_on_numpy.html#numpy-2-abi-handling https://numpy.org/devdocs/dev/depending_on_numpy.html#numpy-...
  - topspin 2y ago
    > The made breaking changes in the ABI... They bumped the major number. That's fair play. There has always been a lot of slouching wrt versions in python. That's not numpy's fault. Too bad they're getting the black eye for it. They could have avoided it by making a new dependency name ("numpy2"), but that sets a shameful precedent, so I give them credit for not copping out.
    - rikthevik 2y ago
      It's a tough situation. Once you have an installed base, your hands get tied. I miss the early days when I could rework the UI and UX of the product regularly. Now there is a lot of legwork to make sure we're not changing existing workflows (documented or undocumented). That said, I like that we're making money, so I can't complain too much. I viscerally understand how companies can get stuck and unable to change their products significantly because their installed base will revolt. It's a tough situation.
      - topspin 2y ago
        It's a self inflicted situation by dependees. Tools in widespread use, JupyterLab for instance, has users tossing dependencies onto the pile, offering next to no guidance on dependency version control, as if expecting them to have to ponder such issues is unreasonable. Well, if that's how you're going to behave, foregoing entirely obvious and solved engineering problems, you get to glue all the pieces back together when reality asserts itself. I don't imagine for one minute numpy folks didn't know what this would look like, and they did it anyway. Good on them. Breaking changes are necessary. That's unqualified. Necessary. If you can't afford a big blowup in your work then manage your work.
        rbanffy 2y ago
        One of the reasons I made pip-chill was to have a tool to make it easy to me to have a minimal set of non-versioned requirements for canary builds on the latest-and-greatest versions. When the canary breaks, you realise you have work to do.
      - rbanffy 2y ago
        This is a symptom of cruft. Clean breaks are needed to keep software fresh. The Tree of Elegance needs to be refreshed with the blood of both users and programers. Or something like that.
        llm_trw 2y ago
- IshKebab 2y ago
  It broke our builds. Because Numpy doesn't use semver there's no way to specify compatible versions constraints. One of our dependencies had a `numpy>=1.something` constraint but wasn't actually compatible with Numpy 2. We need to use a lock file really, but `pip` doesn't support that - you need to use a better package installed like `uv` to get this standard feature.