6 ms·

Yeah, I've been using python for the first time in a while to try out some of the llm stuff and I can't believe how bad the dependency hell is. It's probably pa

Yeah, I've been using python for the first time in a while to try out some of the llm stuff and I can't believe how bad the dependency hell is. It's probably particularly bad due to the pace of change in this field. But I spend an hour getting dependencies fixed every time I touch anything. 80% of the Google Collabs I find are just outright broken. I wish there were other viable non python options to try out these things.

bboygravity 4y ago
No idea what a Google Collab is, but does the code come with an environment or at least a specifications of which packages and versions to use (requirements.txt)? It sounds unnecessarily weird to me that people would share Python code that simply doesn't work out at all out of the box.
- version_five 4y ago
  Its rarely as easy as sharing a requirements.txt. There are lots of things that can still break - for examples you get weird situations where different modules require different versions of a third module. Or all the Cuda toolkit version issues thsy seem to come up with gpu stuff. When we share python, we tend to share a docker image, and even this isn't foolproof. A big problem I think is that it doesn't incentivize building something portable. And it's very hard to test across different machines. Add to that all the different practices re virtual environments, venv, conda, etc, everyone tries to install the dependencies differently or is starting from some nonstandard state. It's a mess.
  - pablo1107 4y ago
    Maybe using Nix it's a better experience for creating such an environment where you depending also on system utilities.
    - superkuh 4y ago
      Everyone is using llama.cpp because we reject the idea of giving up on system libraries like nix does. That kind of tomfoolery (at least in the desktop context) is only required when you use software projects that use libraries/languages which break forwards compatibility every 3 years. If you just write straight c++ (without c++xx, or anything like it) you can compile the code on machines from decades ago if you want.
      - deleted 4y ago
        [deleted]
      - remexre 4y ago
        What's c++xx?
        opless 4y ago
        C++11, and greater.
        remexre 4y ago
        Huh, I was proficient in Rust before "properly" learning C++, so maybe that accounts for it, but I didn't realize C++11 was controversial. Is it just move semantics, or are there some library things that are hard to implement?
        int_19h 4y ago
        I think what OP is saying is that decades-old systems wouldn't have C++11-compatible compilers on them.
        bboygravity 4y ago
        And maybe that "C++" is now basically a bunch of different incompatible languages instead of just 1 language, depending on what "xx" is (11, 14, 17, 20, 23, etc). It's like Python 2 vs Python 3 except even worse.
- vkou 4y ago
  > No idea what a Google Collab is It's ~equivalent to a Jupyter notebook.
hunta2097 4y ago
You're using virtual environments, right? ML libraries are particularly bad, most other stuff works well. Friends don't let friends install pip into /usr/lib.
- AnthonyMouse 4y ago
  This just goes to show what a mess this is. Suppose you have a big piece of compute hardware (e.g. at a university) which is shared by multiple users. They all want to come in and play with these models. Each one is tens to hundreds of gigabytes. Is each user supposed to have their own copy in their home directory?
  - Accujack 4y ago
    This is not exactly a new problem.
    - AnthonyMouse 4y ago
      That's kind of the point. We solved this problem decades ago. You have a system package manager that installs a system-wide copy of the package that everybody can use. But now we encounter this broken nonsense because solved problems get unsolved by bad software.
nyarlathotep_ 4y ago
IME the ML world with Python is a whole mess on top of the existing dependency issues. I've been very _careful_ too (using pyenv/virtualenvs etc) for dependency management, but with Nvidia driver dependencies and "missing 'sqlite3/bz2' issues related to the underlying interpreter (not to mention issues with different Python3.x versions) I'm lucky to be able to even run a 'hello world' ML sample after an afternoon of fighting with it. My Ubuntu install w/ Nvidia card only seems to recognize the GPU in some circumstances even when using the same `conda` env. Often this is remedied by rebooting the machine(?). No idea how companies manage this stuff in production. Absolute minefield that seems to catastrophically break if you sneeze at it. I'll admit I am not an expert in managing ML envs, but I've dealt with a lot of python environments for typical CRUD stuff, and while rough at times, it was never this bad.

bboygravity 4y ago

No idea what a Google Collab is, but does the code come with an environment or at least a specifications of which packages and versions to use (requirements.txt)? It sounds unnecessarily weird to me that people would share Python code that simply doesn't work out at all out of the box.

version_five 4y ago
Its rarely as easy as sharing a requirements.txt. There are lots of things that can still break - for examples you get weird situations where different modules require different versions of a third module. Or all the Cuda toolkit version issues thsy seem to come up with gpu stuff. When we share python, we tend to share a docker image, and even this isn't foolproof. A big problem I think is that it doesn't incentivize building something portable. And it's very hard to test across different machines. Add to that all the different practices re virtual environments, venv, conda, etc, everyone tries to install the dependencies differently or is starting from some nonstandard state. It's a mess.
- pablo1107 4y ago
  Maybe using Nix it's a better experience for creating such an environment where you depending also on system utilities.
  - superkuh 4y ago
    Everyone is using llama.cpp because we reject the idea of giving up on system libraries like nix does. That kind of tomfoolery (at least in the desktop context) is only required when you use software projects that use libraries/languages which break forwards compatibility every 3 years. If you just write straight c++ (without c++xx, or anything like it) you can compile the code on machines from decades ago if you want.
    - deleted 4y ago
      [deleted]
    - remexre 4y ago
      What's c++xx?
      - opless 4y ago
        C++11, and greater.
        remexre 4y ago
        Huh, I was proficient in Rust before "properly" learning C++, so maybe that accounts for it, but I didn't realize C++11 was controversial. Is it just move semantics, or are there some library things that are hard to implement?
        int_19h 4y ago
        I think what OP is saying is that decades-old systems wouldn't have C++11-compatible compilers on them.
        bboygravity 4y ago
        And maybe that "C++" is now basically a bunch of different incompatible languages instead of just 1 language, depending on what "xx" is (11, 14, 17, 20, 23, etc). It's like Python 2 vs Python 3 except even worse.
vkou 4y ago
> No idea what a Google Collab is It's ~equivalent to a Jupyter notebook.

hunta2097 4y ago

You're using virtual environments, right? ML libraries are particularly bad, most other stuff works well. Friends don't let friends install pip into /usr/lib.

AnthonyMouse 4y ago
This just goes to show what a mess this is. Suppose you have a big piece of compute hardware (e.g. at a university) which is shared by multiple users. They all want to come in and play with these models. Each one is tens to hundreds of gigabytes. Is each user supposed to have their own copy in their home directory?
- Accujack 4y ago
  This is not exactly a new problem.
  - AnthonyMouse 4y ago
    That's kind of the point. We solved this problem decades ago. You have a system package manager that installs a system-wide copy of the package that everybody can use. But now we encounter this broken nonsense because solved problems get unsolved by bad software.

nyarlathotep_ 4y ago

IME the ML world with Python is a whole mess on top of the existing dependency issues. I've been very _careful_ too (using pyenv/virtualenvs etc) for dependency management, but with Nvidia driver dependencies and "missing 'sqlite3/bz2' issues related to the underlying interpreter (not to mention issues with different Python3.x versions) I'm lucky to be able to even run a 'hello world' ML sample after an afternoon of fighting with it. My Ubuntu install w/ Nvidia card only seems to recognize the GPU in some circumstances even when using the same `conda` env. Often this is remedied by rebooting the machine(?). No idea how companies manage this stuff in production. Absolute minefield that seems to catastrophically break if you sneeze at it. I'll admit I am not an expert in managing ML envs, but I've dealt with a lot of python environments for typical CRUD stuff, and while rough at times, it was never this bad.