4 ms·

Anyone cross-compiling OCaml with OpenEmbedded and Yocto? The security properties are attractive for IoT and embedded, but it's an ongoing challenge to build t

Anyone cross-compiling OCaml with OpenEmbedded and Yocto? The security properties are attractive for IoT and embedded, but it's an ongoing challenge to build the OCaml toolchain from scratch and build apps without leakage of pre-compiled, native, bootstrap toolchain binaries. With the rise of Arm and RISC-V, it would be nice to see the OCaml community contributing to an upstream OE/YP layer.

ernst_klim 6y ago
Cross architecture compilation is still not a breeze and not a first class feature (there are toolchains for android and ios, but you still can't build a generic toolchain easily as you could with Rust). Though we are going this way slowly but steadily. Afaik some people are working on build system, so I hope soon we'll have a GCC/Rust-like experience.
- EduardoRFS 6y ago
  I have some tools that you can use to cross compile pretty easily, not a first class yet but it works, you can build generic toolchains quite easily, just a new `sysroot` with the compiler + some flags https://github.com/EduardoRFS/reason-mobile https://github.com/EduardoRFS/reason-mobile
avsm 6y ago
This is indeed on our development list for this winter to integrate more. Over in MirageOS (mirage.io), we've been working hard on getting RISC-V support in shape, so we can build unikernels for bare metal embedded devices with a simple "dune build @riscv-solo5". As part of that work, a few things are going on: - use "dune workspaces" (essentially a cross compilation context) to build all C bindings in OCaml packages with custom CFLAGS/LDFLAGS. Ongoing work here: https://github.com/mirage/mirage/pull/1153 https://github.com/mirage/mirage/pull/1153 - upstream in OCaml, the core developers have specced out the design for integrating cross-compilation more naturally into the compiler. - OCaml 4.11.0 has just been released with a native code RISC-V backend (and of course has had ARM for years). Mirage doesn't really use OpenEmbedded or Yocto, but I expect it should be straightforward to embed the workflow in there once it's stable upstream in Mirage itself. Any pointers to something I should read about how to get involved in Yocto?
- deleted 6y ago
  [deleted]
- transpute 6y ago
  There's a non-upstream OpenEmbedded layer for building OCaml components, but it jumps through hoops to coordinate OCaml toolchain with upstream OE releases, https://github.com/OpenXT/meta-openxt-ocaml-platform https://github.com/OpenXT/meta-openxt-ocaml-platform This could be generalized to create an upstream OE layer that uses the upcoming cross-compiling improvements from upstream Ocaml, which would then enable MirageOS unikernels in OE meta-virtualization, https://layers.openembedded.org/layerindex/branch/master/layer/meta-virtualization/ https://layers.openembedded.org/layerindex/branch/master/lay.... RISC-V support is in early stages for meta-virt, Western Digital added some support. Intro to meta-virt: https://youtube.com/watch?feature=emb_title&v=fIbWvrlWJYg https://youtube.com/watch?feature=emb_title&v=fIbWvrlWJYg Intro to Yocto: https://elinux.org/images/a/a8/Getting_Started_with_Embedded_Linux-_Using_the_Yocto_Project_to_Build_your_Own_Custom_Embedded_Linux_Distribution.pdf https://elinux.org/images/a/a8/Getting_Started_with_Embedded... Meta-virt mailing list would be a good place for discussion, https://lists.yoctoproject.org/g/meta-virtualization https://lists.yoctoproject.org/g/meta-virtualization
  - avsm 6y ago
    Very handy, thanks. I notice that the openxt-ocaml-platform repo uses ocamlbuild/ocamlfind. Life should get a lot easier when we move that to use ocaml/dune directly -- cross-compilation really needs a lot of help from the build system, and dune is moving fast to support these workflows.
    - transpute 6y ago
      Hopefully there can be a way to use ocaml/dune directly without importing "everything" into each per-component build tree. OE best practices employ per-recipe sysroots for building each component, where a minimal set of dependencies is present for the build environment. This helps with reproducible builds and generation of manifests for supply chain integrity.
greenarc 6y ago
We ended up building an ARM image with the OCaml compiler installed, running it with qemu to build our OCaml code as a second step. Basically from yocto have a "rootfs" which is your standard image + gcc, use it to build a docker container or chroot in it, add qemu-arm-static, build the OCaml compiler from opam and then use it to build binaries. Not the most straightforward, but it works... fine. And in the end it's just a docker image like any other that is used to build our binaries. With that said this has been running for a few years, I didn't check if upstream OCaml cross-compiling made progress since then.
- avsm 6y ago
  That's a pretty reasonable approach. We publish multiarch base images for OCaml/opam regularly (e.g. see `latest` at https://hub.docker.com/r/ocaml/opam2/tags https://hub.docker.com/r/ocaml/opam2/tags or https://hub.docker.com/r/ocurrent/opam/tags https://hub.docker.com/r/ocurrent/opam/tags for slimmer CI images).

ernst_klim 6y ago

Cross architecture compilation is still not a breeze and not a first class feature (there are toolchains for android and ios, but you still can't build a generic toolchain easily as you could with Rust). Though we are going this way slowly but steadily. Afaik some people are working on build system, so I hope soon we'll have a GCC/Rust-like experience.

EduardoRFS 6y ago
I have some tools that you can use to cross compile pretty easily, not a first class yet but it works, you can build generic toolchains quite easily, just a new `sysroot` with the compiler + some flags https://github.com/EduardoRFS/reason-mobile https://github.com/EduardoRFS/reason-mobile

avsm 6y ago

This is indeed on our development list for this winter to integrate more. Over in MirageOS (mirage.io), we've been working hard on getting RISC-V support in shape, so we can build unikernels for bare metal embedded devices with a simple "dune build @riscv-solo5". As part of that work, a few things are going on: - use "dune workspaces" (essentially a cross compilation context) to build all C bindings in OCaml packages with custom CFLAGS/LDFLAGS. Ongoing work here: https://github.com/mirage/mirage/pull/1153 https://github.com/mirage/mirage/pull/1153 - upstream in OCaml, the core developers have specced out the design for integrating cross-compilation more naturally into the compiler. - OCaml 4.11.0 has just been released with a native code RISC-V backend (and of course has had ARM for years). Mirage doesn't really use OpenEmbedded or Yocto, but I expect it should be straightforward to embed the workflow in there once it's stable upstream in Mirage itself. Any pointers to something I should read about how to get involved in Yocto?

deleted 6y ago
[deleted]
transpute 6y ago
There's a non-upstream OpenEmbedded layer for building OCaml components, but it jumps through hoops to coordinate OCaml toolchain with upstream OE releases, https://github.com/OpenXT/meta-openxt-ocaml-platform https://github.com/OpenXT/meta-openxt-ocaml-platform This could be generalized to create an upstream OE layer that uses the upcoming cross-compiling improvements from upstream Ocaml, which would then enable MirageOS unikernels in OE meta-virtualization, https://layers.openembedded.org/layerindex/branch/master/layer/meta-virtualization/ https://layers.openembedded.org/layerindex/branch/master/lay.... RISC-V support is in early stages for meta-virt, Western Digital added some support. Intro to meta-virt: https://youtube.com/watch?feature=emb_title&v=fIbWvrlWJYg https://youtube.com/watch?feature=emb_title&v=fIbWvrlWJYg Intro to Yocto: https://elinux.org/images/a/a8/Getting_Started_with_Embedded_Linux-_Using_the_Yocto_Project_to_Build_your_Own_Custom_Embedded_Linux_Distribution.pdf https://elinux.org/images/a/a8/Getting_Started_with_Embedded... Meta-virt mailing list would be a good place for discussion, https://lists.yoctoproject.org/g/meta-virtualization https://lists.yoctoproject.org/g/meta-virtualization
- avsm 6y ago
  Very handy, thanks. I notice that the openxt-ocaml-platform repo uses ocamlbuild/ocamlfind. Life should get a lot easier when we move that to use ocaml/dune directly -- cross-compilation really needs a lot of help from the build system, and dune is moving fast to support these workflows.
  - transpute 6y ago
    Hopefully there can be a way to use ocaml/dune directly without importing "everything" into each per-component build tree. OE best practices employ per-recipe sysroots for building each component, where a minimal set of dependencies is present for the build environment. This helps with reproducible builds and generation of manifests for supply chain integrity.

greenarc 6y ago

We ended up building an ARM image with the OCaml compiler installed, running it with qemu to build our OCaml code as a second step. Basically from yocto have a "rootfs" which is your standard image + gcc, use it to build a docker container or chroot in it, add qemu-arm-static, build the OCaml compiler from opam and then use it to build binaries. Not the most straightforward, but it works... fine. And in the end it's just a docker image like any other that is used to build our binaries. With that said this has been running for a few years, I didn't check if upstream OCaml cross-compiling made progress since then.

avsm 6y ago
That's a pretty reasonable approach. We publish multiarch base images for OCaml/opam regularly (e.g. see `latest` at https://hub.docker.com/r/ocaml/opam2/tags https://hub.docker.com/r/ocaml/opam2/tags or https://hub.docker.com/r/ocurrent/opam/tags https://hub.docker.com/r/ocurrent/opam/tags for slimmer CI images).