4 ms·

Pacman aint slow. Seems to basically run at the speed required to download and copy files. Then again it has no concept of required versions, since Arch just re

Pacman aint slow. Seems to basically run at the speed required to download and copy files. Then again it has no concept of required versions, since Arch just requires you to stay up to date. So there's no extensive solving of dependencies over version requirements, just the packages themselves. When I was on Ubuntu though, which does have version requirements taken into account in packages, I didn't find apt to be slow either. You know what's slow? The conda package manager. Or installing software - practically any software - on Windows. I regularly budget half an hour to an hour just to install something on the Windows computers I'm forced to use.

Falcorian 7y ago
That's my experience as well: apt is only "slow" in that sometimes it has to download and write gigs of stuff to disk. Now brew and conda, they often take minutes to solve for the environment, which is what I think of when you say "slow". That part is probably mostly algorithmic, and it should be fast, unlike network and disk.
phiresky 7y ago
The main performance problem in pacman as far as I can tell is really slow xz decompression. This is mostly important for large packages: $ time xzcat /var/cache/pacman/pkg/cuda-10.1.243-1-x86_64.pkg.tar.xz >/dev/null 1:17,33 total Just decompressing that 1.8GB package takes 77 seconds. There was a discussion in April to switch compression to zstd - which reduces the decompression time for that package to only 4 seconds (!) while keeping the same file size. That way the disk is the bottleneck again, as it should be. That switch would be awesome, but it looks like it's not happening currently.
- sigotirandolas 7y ago
  If you have a fast connection, xz decompression may be the bottleneck, but in some places, downloading that 1.8GB can easily take an hour or more. So I’m not sure whether this is the bottleneck in the usual case. I would also be interested to see if it’s using parallel decompresssion when possible. I have an intuition it doesn’t, since I have always found Arch defaulting to single thread decompression, e.g in makepkg or for compressing with mkinitcpio.
- aiNohY6g 7y ago
  link: https://lists.archlinux.org/pipermail/arch-dev-public/2019-March/029520.html https://lists.archlinux.org/pipermail/arch-dev-public/2019-M...
bryanlarsen 7y ago
Put `apt-get update && apt-get install -y jq` at the top of your CI script which runs hundreds of times a day and then tell me that apt isn't slow. Should I have to maintain a container just for `FROM node:10 ; RUN apt install jq`? If I do so everybody will assume that it's some sort of specialized testing container rather than just a very generic `node:10` and think they can't run test scripts on their local machine. If node:10 was based on Alpine rather than Ubuntu the equivalent `apk install jq` is blazing fast.
- boring_twenties 7y ago
  1. You can and probably should base your image on node:10-alpine instead of node:10 2. Even if you don't, why would you have to maintain a separate container? If `RUN apt install jq` is indeed the second entry in your Dockerfile after the `FROM ...`, then it should be cached and reused indefinitely -- that's half the point of Docker in the first place. It won't work if you have a command like `COPY` before the `RUN apt`, though.
bisby 7y ago
It's also not fair to show download speeds. Since the whole point of all of these is that there are a ton of mirrors. Yesterday I had a nearby arch mirror fail and timeout. And then the secondary backup mirror downloaded things at 20M/s. Also. just tried qemu on arch and there definitely wasn't 124mb of data transferred. (Total download size: 7.85mb) so this can vary heavily depending on which deps you have pre-installed for something elsewhere. Doesn't seem to be the best methodology.

Falcorian 7y ago

That's my experience as well: apt is only "slow" in that sometimes it has to download and write gigs of stuff to disk. Now brew and conda, they often take minutes to solve for the environment, which is what I think of when you say "slow". That part is probably mostly algorithmic, and it should be fast, unlike network and disk.

phiresky 7y ago

The main performance problem in pacman as far as I can tell is really slow xz decompression. This is mostly important for large packages: $ time xzcat /var/cache/pacman/pkg/cuda-10.1.243-1-x86_64.pkg.tar.xz >/dev/null 1:17,33 total Just decompressing that 1.8GB package takes 77 seconds. There was a discussion in April to switch compression to zstd - which reduces the decompression time for that package to only 4 seconds (!) while keeping the same file size. That way the disk is the bottleneck again, as it should be. That switch would be awesome, but it looks like it's not happening currently.

sigotirandolas 7y ago
If you have a fast connection, xz decompression may be the bottleneck, but in some places, downloading that 1.8GB can easily take an hour or more. So I’m not sure whether this is the bottleneck in the usual case. I would also be interested to see if it’s using parallel decompresssion when possible. I have an intuition it doesn’t, since I have always found Arch defaulting to single thread decompression, e.g in makepkg or for compressing with mkinitcpio.
aiNohY6g 7y ago
link: https://lists.archlinux.org/pipermail/arch-dev-public/2019-March/029520.html https://lists.archlinux.org/pipermail/arch-dev-public/2019-M...

bryanlarsen 7y ago

Put `apt-get update && apt-get install -y jq` at the top of your CI script which runs hundreds of times a day and then tell me that apt isn't slow. Should I have to maintain a container just for `FROM node:10 ; RUN apt install jq`? If I do so everybody will assume that it's some sort of specialized testing container rather than just a very generic `node:10` and think they can't run test scripts on their local machine. If node:10 was based on Alpine rather than Ubuntu the equivalent `apk install jq` is blazing fast.

boring_twenties 7y ago
1. You can and probably should base your image on node:10-alpine instead of node:10 2. Even if you don't, why would you have to maintain a separate container? If `RUN apt install jq` is indeed the second entry in your Dockerfile after the `FROM ...`, then it should be cached and reused indefinitely -- that's half the point of Docker in the first place. It won't work if you have a command like `COPY` before the `RUN apt`, though.

bisby 7y ago

It's also not fair to show download speeds. Since the whole point of all of these is that there are a ton of mirrors. Yesterday I had a nearby arch mirror fail and timeout. And then the secondary backup mirror downloaded things at 20M/s. Also. just tried qemu on arch and there definitely wasn't 124mb of data transferred. (Total download size: 7.85mb) so this can vary heavily depending on which deps you have pre-installed for something elsewhere. Doesn't seem to be the best methodology.