25 ms·

Compiling LLVM for what purpose? Hacking on it? Producing an optimized build? Installing a build on a user's machine? You should be more specific. The default

by da-x 5y ago

Compiling LLVM for what purpose? Hacking on it? Producing an optimized build? Installing a build on a user's machine? You should be more specific. The default clean build works just fine. On my machine I can clone the Git repo, do `mkdir build && cmake ../llvm && make -j64` and end up with working binaries after 5 minutes. The build take 37GB by default here, and the binaries under bin/ are immediately usable work. If you want a smaller build with shared library just add a parameter. Want to throw in sub-projects such as LLD or Clang? Just another parameter. It's all very well-documented.

FooBarWidget 5y ago
> `make -j64` and end up with working binaries after 5 minutes. I think the problematic part for most people lies in the first two words in that quoted sentence.
- meinersbur 5y ago
  `mkdir build && cmake ../llvm -GNinja && ninja` better?
  - lazypenguin 5y ago
    Most people don’t have 64 cores available to them to compile LLVM in 5 minutes. Switching to ninja doesn’t really solve that problem.
  - bregma 5y ago
    You missed the part where you upgrade your RAM to at least 128 GB and up the swap to 512 GB and use gold as your linker because the OOM killer will still raise it's head for ld under those extreme restrictions.
    - shipp02 5y ago
      Sorry, what are you talking about. I have built ponylang, build llvm with a couple of patches from source on my 2018 Mac book air. Just don't put -j64. The first build will take a couple of hours but that is par for the course for a project of llvm's size and complexity. The subsequent buttons take a couple minutes.
    - jcelerier 5y ago
      I don't understand why anyone would use ld or gold in 2022. Every distro has shipped lld for years and it links in less than seconds. It's a default to not break compatibility, not because it's good ; makes as much sense to use as ed(1). For reference on a 8C/16T CPU from 2016 an llvm initial build is ~10-12 minutes here.. not instant but not the end of the world either.
      - xxpor 5y ago
        When LLVM doesn't support your platform :( Granted, that means no Rust support today so probably not relevent to most people reading this thread, but yeah I'm still salty :|
        jcelerier 5y ago
        ... which platform is that ?
        bregma 5y ago
        I for one maintain the official toolchains for QNX. I've been trying to port LLVM, and it's not possible to bootstrap LLVM using LLVM because it hasn't been ported yet.
        jcelerier 5y ago
        > it's not possible to bootstrap LLVM using LLVM I don't understand, LLVM would be on the host machine under some Linux or Windows on x86 where the development happens, not on the QNX hardware anyways, no ? Unless for some reason one would want to port Mesa to have LLVMPIPE or something like this, but I have a hard time imagining the use case.
        bregma 5y ago
        What QNX hardware are you thinking of? The self-hosted x86_64 PC being used to develop on?
    - Szpadel 5y ago
      I compile llvm on machine with 4gb ram
OskarS 5y ago
64 cores, and it takes 5 minutes? That’s… not awesome. Most people do not have 64 (presumably high performance) cores lying around for the purpose of building dependencies.
- xxpor 5y ago
  10 minutes of a c6i.32xlarge costs less than a dollar at on-demand rates in the us-east-2 AWS region. That'll get you 128 vCPUs and 256 gb of memory. I think people put up with a lot of stuff that there's no reason to any more with on-demand computing.
  - westurner 5y ago
    Q: "Does my data fit in RAM?" https://news.ycombinator.com/item?id=22309883 https://news.ycombinator.com/item?id=22309883 A: https://yourdatafitsinram.net https://yourdatafitsinram.net Is there a one-liner to build llvm e.g. with k8s and containers, or just a `git push` to a PR? #GitOps And another to publish a locally-signed tagged build? `conda install rust` does install `cargo`. `conda install -c conda-forge -y mamba; mamba install -y rust libllvm14 lit llvm llvm-tools llvmdev` conda-forge/llvmdev-feedstock; ninja, cmake, python >= 3 https://github.com/conda-forge/llvmdev-feedstock/blob/main/recipe/meta.yaml https://github.com/conda-forge/llvmdev-feedstock/blob/main/r... .scripts/run_docker_build.sh is called by build-locally.py: https://github.com/conda-forge/llvmdev-feedstock/blob/main/.scripts/run_docker_build.sh https://github.com/conda-forge/llvmdev-feedstock/blob/main/....
  - cesarb 5y ago
    > 10 minutes of a c6i.32xlarge costs less than a dollar at on-demand rates in the us-east-2 AWS region. That assumes you have an AWS account. There are legitimate reasons to not have an AWS account; for instance, you might not want to risk accidentally racking up huge bills, or you might not have access to a payment method AWS accepts, or you might want to avoid using services under USA jurisdiction (due to GDPR concerns or similar).
    - fs111 5y ago
dboreham 5y ago
Pretty good April 1 post!

by da-x 5y ago

FooBarWidget 5y ago

> `make -j64` and end up with working binaries after 5 minutes. I think the problematic part for most people lies in the first two words in that quoted sentence.

meinersbur 5y ago
`mkdir build && cmake ../llvm -GNinja && ninja` better?
- lazypenguin 5y ago
  Most people don’t have 64 cores available to them to compile LLVM in 5 minutes. Switching to ninja doesn’t really solve that problem.
- bregma 5y ago
  You missed the part where you upgrade your RAM to at least 128 GB and up the swap to 512 GB and use gold as your linker because the OOM killer will still raise it's head for ld under those extreme restrictions.
  - shipp02 5y ago
    Sorry, what are you talking about. I have built ponylang, build llvm with a couple of patches from source on my 2018 Mac book air. Just don't put -j64. The first build will take a couple of hours but that is par for the course for a project of llvm's size and complexity. The subsequent buttons take a couple minutes.
  - jcelerier 5y ago
    I don't understand why anyone would use ld or gold in 2022. Every distro has shipped lld for years and it links in less than seconds. It's a default to not break compatibility, not because it's good ; makes as much sense to use as ed(1). For reference on a 8C/16T CPU from 2016 an llvm initial build is ~10-12 minutes here.. not instant but not the end of the world either.
    - xxpor 5y ago
      When LLVM doesn't support your platform :( Granted, that means no Rust support today so probably not relevent to most people reading this thread, but yeah I'm still salty :|
      - jcelerier 5y ago
        ... which platform is that ?
        bregma 5y ago
        I for one maintain the official toolchains for QNX. I've been trying to port LLVM, and it's not possible to bootstrap LLVM using LLVM because it hasn't been ported yet.
        jcelerier 5y ago
        > it's not possible to bootstrap LLVM using LLVM I don't understand, LLVM would be on the host machine under some Linux or Windows on x86 where the development happens, not on the QNX hardware anyways, no ? Unless for some reason one would want to port Mesa to have LLVMPIPE or something like this, but I have a hard time imagining the use case.
        bregma 5y ago
        What QNX hardware are you thinking of? The self-hosted x86_64 PC being used to develop on?
  - Szpadel 5y ago
    I compile llvm on machine with 4gb ram

OskarS 5y ago

64 cores, and it takes 5 minutes? That’s… not awesome. Most people do not have 64 (presumably high performance) cores lying around for the purpose of building dependencies.

xxpor 5y ago
10 minutes of a c6i.32xlarge costs less than a dollar at on-demand rates in the us-east-2 AWS region. That'll get you 128 vCPUs and 256 gb of memory. I think people put up with a lot of stuff that there's no reason to any more with on-demand computing.
- westurner 5y ago
  Q: "Does my data fit in RAM?" https://news.ycombinator.com/item?id=22309883 https://news.ycombinator.com/item?id=22309883 A: https://yourdatafitsinram.net https://yourdatafitsinram.net Is there a one-liner to build llvm e.g. with k8s and containers, or just a `git push` to a PR? #GitOps And another to publish a locally-signed tagged build? `conda install rust` does install `cargo`. `conda install -c conda-forge -y mamba; mamba install -y rust libllvm14 lit llvm llvm-tools llvmdev` conda-forge/llvmdev-feedstock; ninja, cmake, python >= 3 https://github.com/conda-forge/llvmdev-feedstock/blob/main/recipe/meta.yaml https://github.com/conda-forge/llvmdev-feedstock/blob/main/r... .scripts/run_docker_build.sh is called by build-locally.py: https://github.com/conda-forge/llvmdev-feedstock/blob/main/.scripts/run_docker_build.sh https://github.com/conda-forge/llvmdev-feedstock/blob/main/....
- cesarb 5y ago
  > 10 minutes of a c6i.32xlarge costs less than a dollar at on-demand rates in the us-east-2 AWS region. That assumes you have an AWS account. There are legitimate reasons to not have an AWS account; for instance, you might not want to risk accidentally racking up huge bills, or you might not have access to a payment method AWS accepts, or you might want to avoid using services under USA jurisdiction (due to GDPR concerns or similar).
  - fs111 5y ago

dboreham 5y ago

Pretty good April 1 post!