3 ms·

I'm still missing some details to understand that story, as I don't believe that there is any platform that Go supports on which C sqlite would not run, more th

by acqq 3y ago

I'm still missing some details to understand that story, as I don't believe that there is any platform that Go supports on which C sqlite would not run, more the opposite, the memory requirements of wazero.io way must be clearly higher than of the C sqlite library? It seems to me it's not about would C sqlite run or not on that platform but that there is still some other convenience, and that's what I'd still like to learn, as I'm sure ncruces does solve some specific problem.

ncruces 3y ago
To compile C for a given platform, you need a C compiler for that platform. Which definitely exists but you may not have it at hand. To cross compile from your platform A to that platform B, you need a cross compiler, which may be even harder to come by. If you're working on a Go project, you already have a Go compiler that can do this (cross compilation) very easily, out of the box. But if you add a C dependency, you also need a C cross compiler. SQLite is easy to configure, and very portable, but this still adds some friction. Then if you want to setup CI, you need all this infra in CI. If you want to automate releases for a bunch of platforms, you need all of it in your release process. I'm not saying it's a huge pain but it's enough that some projects avoid cgo as a matter of principle.
- acqq 3y ago
  Many thanks for this good answer, that level of detail is exactly what I have been missing up to now. I'm surely aware of the fact that C cross-compilation is often very clumsy implemented. But I also have to mention that there is a project that also tries to solve that too, while also creating a whole new language: https://zig.news/kristoff/building-sqlite-with-cgo-for-every-os-4cic https://zig.news/kristoff/building-sqlite-with-cgo-for-every...
  - glenjamin 3y ago
    it's also worth noting that even on the same platform, compiling with CGO enabled creates a dynamic link against `glibc`, which creates a dependency on having the matching version in your deployment target. We ran into this recently as our CI system is running a newer OS than we currently deploy to - and disabling CGO was an easy way to sidestep this requirement
    - acqq 3y ago
      Is this something inherent in CGO implementation? If your dependency happened because the compiled library depended on it, do you know that that is probably avoidable (I haven't tried your use case, so I could be wrong) by using zig as a compiler as it can "cross compile" for lower glibc version and also to musl, avoiding glibc altogether?
      - ncruces 3y ago
        With zig I think you can avoid libc deps on Linux, at least, by statically linking musl instead. Not sure SQLite is tested in this configuration, for the fans of “this didn't pass TH3 so we can't trust it at all.”
        acqq 3y ago
        AFAIK: you also link to a different (lower!) glibc version from the one on which you do the compilation, probably solving the problem you had without having an excuse that you linked an "untested" library.

by acqq 3y ago

ncruces 3y ago

To compile C for a given platform, you need a C compiler for that platform. Which definitely exists but you may not have it at hand. To cross compile from your platform A to that platform B, you need a cross compiler, which may be even harder to come by. If you're working on a Go project, you already have a Go compiler that can do this (cross compilation) very easily, out of the box. But if you add a C dependency, you also need a C cross compiler. SQLite is easy to configure, and very portable, but this still adds some friction. Then if you want to setup CI, you need all this infra in CI. If you want to automate releases for a bunch of platforms, you need all of it in your release process. I'm not saying it's a huge pain but it's enough that some projects avoid cgo as a matter of principle.

acqq 3y ago
Many thanks for this good answer, that level of detail is exactly what I have been missing up to now. I'm surely aware of the fact that C cross-compilation is often very clumsy implemented. But I also have to mention that there is a project that also tries to solve that too, while also creating a whole new language: https://zig.news/kristoff/building-sqlite-with-cgo-for-every-os-4cic https://zig.news/kristoff/building-sqlite-with-cgo-for-every...
- glenjamin 3y ago
  it's also worth noting that even on the same platform, compiling with CGO enabled creates a dynamic link against `glibc`, which creates a dependency on having the matching version in your deployment target. We ran into this recently as our CI system is running a newer OS than we currently deploy to - and disabling CGO was an easy way to sidestep this requirement
  - acqq 3y ago
    Is this something inherent in CGO implementation? If your dependency happened because the compiled library depended on it, do you know that that is probably avoidable (I haven't tried your use case, so I could be wrong) by using zig as a compiler as it can "cross compile" for lower glibc version and also to musl, avoiding glibc altogether?
    - ncruces 3y ago
      With zig I think you can avoid libc deps on Linux, at least, by statically linking musl instead. Not sure SQLite is tested in this configuration, for the fans of “this didn't pass TH3 so we can't trust it at all.”
      - acqq 3y ago
        AFAIK: you also link to a different (lower!) glibc version from the one on which you do the compilation, probably solving the problem you had without having an excuse that you linked an "untested" library.