3 ms·

How is managing python packages or ruby gems on NixOS? I remember the experience was not polished a few years ago.

by kovek 5y ago

kevincox 5y ago
In general it is very nice. A common method is you create a `default.nix` file in the project you are working on and use tools that manage the deps for you. For example: Rust+Cargo: https://gitlab.com/kevincox/watchlog/-/blob/22c877065f763b3d191dbea2e6ff06104e8720c3/default.nix#L8-10 https://gitlab.com/kevincox/watchlog/-/blob/22c877065f763b3d... Node+NPM: https://gitlab.com/kevincox/kevincox-web-compiler/-/blob/9faccec9f145a1815f2ab57570976fd32c5e8ccc/default.nix#L8-19 https://gitlab.com/kevincox/kevincox-web-compiler/-/blob/9fa... My only Ruby project is private but I just rolled my own with: export "GEM_HOME=$out" bundle install --jobs $NIX_BUILD_CORES Then I can set the resulting derivation as $GEM_HOME in the build step. And sorry, but I don't do much Python. For development you then use nix-shell to build the environment with all of the dependencies available. The downside is that you either need to commit the default.nix to the project, or configure it to be ignored. But the advantage is that it handles everything for you. For example you can configure native libraries such as PostgreSQL client or openssl without worrying if they are installed on your system. This means that the entire dev environment is tested in CI (assuming that you run nix-build in CI). This is also nicer than docker/VM based dev environments as it just prefixes your usual environment with the deps, so you still have your regular commands and development tools available.
- reasonabl_human 5y ago
  Are you up for explaining these gists? We use nix extensively at work and yet I’m having trouble wrapping my head around these. Node dependencies have been a thorn in our nixification of some packages, C / C++ has come easily.
  - kevincox 5y ago
    Sure. NPM is the easy case because the package-lock.json file can easily be read by Nix and contains hashes for all of the packages. This means that simply be importing the file into Nix you can have a reproducible build. No Nix-specific maintenance required. In the linked case I use this library to manage that https://github.com/nmattia/napalm https://github.com/nmattia/napalm (in that example I use master but for production I would pin a version). It simply parses the package-lock.json, downloads the packages and uses npm to build the node_modules folder. It also provides some convenient functions for building packages with "bin" files or just linking node_modules inside a build. Note that this is more for project development. It doesn't use the "system" packages (intentionally) for Node, it fetches whatever versions you have specified from NPM. Nix will only provide the "native" stuff like Node and NPM themselves and any native libraries.
adisbladis 5y ago
IMO (shameless plug, I'm the author) the best way to manage Python environments with Nix is poetry2nix: https://github.com/nix-community/poetry2nix https://github.com/nix-community/poetry2nix.
aitchnyu 5y ago
I see isolated dev environments in terminal. Can Pycharm (IDE and GUI) pick up these environments? Services and dev tools want to hog system ports. Can I run multiple dev projects each managing own services in Pycharm?

How is managing python packages or ruby gems on NixOS? I remember the experience was not polished a few years ago.

by kovek 5y ago

How is managing python packages or ruby gems on NixOS? I remember the experience was not polished a few years ago.

In general it is very nice. A common method is you create a `default.nix` file in the project you are working on and use tools that manage the deps for you. For example: Rust+Cargo: https://gitlab.com/kevincox/watchlog/-/blob/22c877065f763b3d191dbea2e6ff06104e8720c3/default.nix#L8-10 https://gitlab.com/kevincox/watchlog/-/blob/22c877065f763b3d... Node+NPM: https://gitlab.com/kevincox/kevincox-web-compiler/-/blob/9faccec9f145a1815f2ab57570976fd32c5e8ccc/default.nix#L8-19 https://gitlab.com/kevincox/kevincox-web-compiler/-/blob/9fa... My only Ruby project is private but I just rolled my own with: export "GEM_HOME=$out" bundle install --jobs $NIX_BUILD_CORES Then I can set the resulting derivation as $GEM_HOME in the build step. And sorry, but I don't do much Python. For development you then use nix-shell to build the environment with all of the dependencies available. The downside is that you either need to commit the default.nix to the project, or configure it to be ignored. But the advantage is that it handles everything for you. For example you can configure native libraries such as PostgreSQL client or openssl without worrying if they are installed on your system. This means that the entire dev environment is tested in CI (assuming that you run nix-build in CI). This is also nicer than docker/VM based dev environments as it just prefixes your usual environment with the deps, so you still have your regular commands and development tools available.

reasonabl_human 5y ago
Are you up for explaining these gists? We use nix extensively at work and yet I’m having trouble wrapping my head around these. Node dependencies have been a thorn in our nixification of some packages, C / C++ has come easily.
- kevincox 5y ago
  Sure. NPM is the easy case because the package-lock.json file can easily be read by Nix and contains hashes for all of the packages. This means that simply be importing the file into Nix you can have a reproducible build. No Nix-specific maintenance required. In the linked case I use this library to manage that https://github.com/nmattia/napalm https://github.com/nmattia/napalm (in that example I use master but for production I would pin a version). It simply parses the package-lock.json, downloads the packages and uses npm to build the node_modules folder. It also provides some convenient functions for building packages with "bin" files or just linking node_modules inside a build. Note that this is more for project development. It doesn't use the "system" packages (intentionally) for Node, it fetches whatever versions you have specified from NPM. Nix will only provide the "native" stuff like Node and NPM themselves and any native libraries.

adisbladis 5y ago

IMO (shameless plug, I'm the author) the best way to manage Python environments with Nix is poetry2nix: https://github.com/nix-community/poetry2nix https://github.com/nix-community/poetry2nix.

aitchnyu 5y ago

I see isolated dev environments in terminal. Can Pycharm (IDE and GUI) pick up these environments? Services and dev tools want to hog system ports. Can I run multiple dev projects each managing own services in Pycharm?