4 ms·

I've been wanting to use Nix(OS) on my Pi for a year now. But from what I understand, there is no binary cache for ARM yet. Building every required package on P

I've been wanting to use Nix(OS) on my Pi for a year now. But from what I understand, there is no binary cache for ARM yet. Building every required package on Pi is out of the question and I don't have another persistent system for cross-compiling (Pi is the only always-on system I own). That being said, I've been using NixOS on my main system and loving it! I've been through format-and-resetup dance far too many times to understand NixOS is worst system management tool, except all the others.

jeppesen-io 5y ago
> Building every required package on Pi is out of the question Are you sure? TBH I don't know if there's a binary cache, but I installed without issue. It was longer than say a Debian install, but can't say it took that long What timeframe do you need to meet your needs?
- jeppesen-io 5y ago
  Just tested on my pi4, seems to use cache > ssh pi4 (nelson@pi4) Password: Last login: Sun Aug 15 10:44:23 2021 from 192.168.86.235 nelson@pi4:~/ > sudo nix-channel --update [sudo] password for nelson: unpacking channels... created 1 symlinks in user environment nelson@pi4:~/ > sudo nixos-rebuild switch ... > copying path '/nix/store/qpd30425yap9y2mhsa4lg8cfid6703cz-glibc-locales-2.33-49' from 'https://cache.nixos.org https://cache.nixos.org'... copying path '/nix/store/59mpcrm3ndbg32834038v1c6lzjh9yi7-linux-5.10.52-1.20210805' from 'https://cache.nixos.org https://cache.nixos.org'... copying path ect
  - CameronNemo 5y ago
    They have an aarch64 cache iirc. Not an 32 bit arm cache. There are several variants.
  - e3bc54b2 5y ago
    Thanks, I was not aware of this. I will try it out. On another note, do you have any idea why Pi foundation still recommends 32bit images for Raspbian?
    - earleybird 5y ago
      annecdata fwiw, re 64bit Raspios vs 32bit. I have a 4 node 64 bit k3s cluster running folding-at-home as a testing workload where nodes will stop responding after some indeterminate time (1-3 days) - at present, just power cycling to recover
- e3bc54b2 5y ago
  I'm not worried about initial setup time, but consecutive updates. But looks like I was wrong and there is a cache as long as I use aarch64. I'm not sure why Pi foundation recommends 32bit image, but NixOS should be fine imo.
jeppesen-io 5y ago
> I've been through format-and-resetup dance far too many times to understand I'm curious to hear more about this - can you explain?
- stardenburden 5y ago
  Your system state in nixos is described in a config file. When you reinstall you won't need to remember what packages you installed, what files you touched in /etc and worse of all: how you configured your text editor. All of that state can be described with nix. So if I wanted to reinstall right now, I'd have the exact same system as before. Naturally you'll have to put in some work to make your system 100% nix, but it's worth it. That's also why I think it's perfect for projects like this one, people will merely have to copy your config file, change nothing if deploying on the same hardware (except maybe some IP address here and there) and it should all just work:tm:
  - djhworld 5y ago
    > Your system state in nixos is described in a config file. When you reinstall you won't need to remember what packages you installed, what files you touched in /etc and worse of all: how you configured your text editor. I don't see how this is different to ansible, you install an OS then run an ansible playbook remotely via SSH to get your Pi/whatever to the state you want. I have an ansible setup that manages 3 Pis this way, all from one command. Granted, writing YAML is not fun, but i'd like to see a stronger argument why Nix is worth the effort.
    - e3bc54b2 5y ago
      I am not well versed with Ansible, but I understand it to be imperative while Nix is declarative. Put another way, if you add a package to your playbook and then remove it, the package is still there on your system until you remove it old fashioned way. With Nix, if something removed from config, it also automatically gets removed from system. Next and imo biggest benefit of NixOS is generations. Every time you change or update your system, the whole system gets rebuilt and creates a new generation. If kernel update broke your display, you can reboot and select previous entry from boot menu and go about as if update never happened. I'm not aware if Ansible can do that yet. Last, Nix can let you create isolated islands of package config, where you can have your project dependencies setup without affecting or being affected by system dependencies. Obvious examples are Python and Ruby, but these islands or 'nix shell's as they are called can be used to prepare any combination of languages unlike single language tools like pip or cargo. I recommend giving Nix a go for any system for few months. Its not the most pleasant learning experience (documentation is crap, to put nicely), but if you manage to climb that hill, the view will probably make you stay.
stardenburden 5y ago
There is a binary cache, at least for the rpi 3b (aarch64) and I know from friends that it also exists for the 4.
- e3bc54b2 5y ago
  I was not aware! Thanks, I should be able to make full use of it now :)

jeppesen-io 5y ago

> Building every required package on Pi is out of the question Are you sure? TBH I don't know if there's a binary cache, but I installed without issue. It was longer than say a Debian install, but can't say it took that long What timeframe do you need to meet your needs?

jeppesen-io 5y ago
Just tested on my pi4, seems to use cache > ssh pi4 (nelson@pi4) Password: Last login: Sun Aug 15 10:44:23 2021 from 192.168.86.235 nelson@pi4:~/ > sudo nix-channel --update [sudo] password for nelson: unpacking channels... created 1 symlinks in user environment nelson@pi4:~/ > sudo nixos-rebuild switch ... > copying path '/nix/store/qpd30425yap9y2mhsa4lg8cfid6703cz-glibc-locales-2.33-49' from 'https://cache.nixos.org https://cache.nixos.org'... copying path '/nix/store/59mpcrm3ndbg32834038v1c6lzjh9yi7-linux-5.10.52-1.20210805' from 'https://cache.nixos.org https://cache.nixos.org'... copying path ect
- CameronNemo 5y ago
  They have an aarch64 cache iirc. Not an 32 bit arm cache. There are several variants.
- e3bc54b2 5y ago
  Thanks, I was not aware of this. I will try it out. On another note, do you have any idea why Pi foundation still recommends 32bit images for Raspbian?
  - earleybird 5y ago
    annecdata fwiw, re 64bit Raspios vs 32bit. I have a 4 node 64 bit k3s cluster running folding-at-home as a testing workload where nodes will stop responding after some indeterminate time (1-3 days) - at present, just power cycling to recover
e3bc54b2 5y ago
I'm not worried about initial setup time, but consecutive updates. But looks like I was wrong and there is a cache as long as I use aarch64. I'm not sure why Pi foundation recommends 32bit image, but NixOS should be fine imo.

jeppesen-io 5y ago

> I've been through format-and-resetup dance far too many times to understand I'm curious to hear more about this - can you explain?

stardenburden 5y ago
Your system state in nixos is described in a config file. When you reinstall you won't need to remember what packages you installed, what files you touched in /etc and worse of all: how you configured your text editor. All of that state can be described with nix. So if I wanted to reinstall right now, I'd have the exact same system as before. Naturally you'll have to put in some work to make your system 100% nix, but it's worth it. That's also why I think it's perfect for projects like this one, people will merely have to copy your config file, change nothing if deploying on the same hardware (except maybe some IP address here and there) and it should all just work:tm:
- djhworld 5y ago
  > Your system state in nixos is described in a config file. When you reinstall you won't need to remember what packages you installed, what files you touched in /etc and worse of all: how you configured your text editor. I don't see how this is different to ansible, you install an OS then run an ansible playbook remotely via SSH to get your Pi/whatever to the state you want. I have an ansible setup that manages 3 Pis this way, all from one command. Granted, writing YAML is not fun, but i'd like to see a stronger argument why Nix is worth the effort.
  - e3bc54b2 5y ago
    I am not well versed with Ansible, but I understand it to be imperative while Nix is declarative. Put another way, if you add a package to your playbook and then remove it, the package is still there on your system until you remove it old fashioned way. With Nix, if something removed from config, it also automatically gets removed from system. Next and imo biggest benefit of NixOS is generations. Every time you change or update your system, the whole system gets rebuilt and creates a new generation. If kernel update broke your display, you can reboot and select previous entry from boot menu and go about as if update never happened. I'm not aware if Ansible can do that yet. Last, Nix can let you create isolated islands of package config, where you can have your project dependencies setup without affecting or being affected by system dependencies. Obvious examples are Python and Ruby, but these islands or 'nix shell's as they are called can be used to prepare any combination of languages unlike single language tools like pip or cargo. I recommend giving Nix a go for any system for few months. Its not the most pleasant learning experience (documentation is crap, to put nicely), but if you manage to climb that hill, the view will probably make you stay.

stardenburden 5y ago

There is a binary cache, at least for the rpi 3b (aarch64) and I know from friends that it also exists for the 4.

e3bc54b2 5y ago
I was not aware! Thanks, I should be able to make full use of it now :)