5 ms·

Fixing an old .NET Core native library loading issue on Alpine

68 points by ingve 1y ago

ZeroConcerns 1y ago
SQLite is pretty much the only remaining native dependency in my C# codebases, and as much as I love the engine, I wish that could go away. Replacing System.Data.SQLite with Microsoft.Data.Sqlite already helped with Apple ARM builds (despite all the small differences that only showed up in actual use), but pretty much the only native debugging I do these days is related to the "batteries" -- the linked article outlines the general strategy pretty well. On the one hand, I feel bad about turning into a "pure-Java only" kind of developer (I mean, limiting yourself to H2, the horror...), but on the other hand, I'm increasingly starting to see their point. Oh, well, if AI actually works, I'm sure Microsoft.Native.Data.Sqlite is just around the corner (and, later edit, to prevent confusion: the abuse of 'Native' here is mostly making fun of Microsoft naming conventions -- they'd call it 'Interop' if it were like truly native).
- mcraiha 1y ago
  Would the WebAssembly version of SQLite be OK for you?
  - ZeroConcerns 1y ago
    I haven't really kept up with the state of WebAssembly in .NET (mostly because I'm entirely uninterested in Blazor), but I don't think we're at a point yet where we can simply reference a .wasm file and invoke code in it regardless of the underlying platform, right? Until that is the case: no, not really.
- pjmlp 1y ago
  Exactly because of the benefits of this, there is the meme CGO is not Go, as any package with native dependencies kills the possibility to cross-compile with the Go toolchain. Same applies to the pain of using native dependencies in Perl, Python, Ruby,... Many .NET developers are only now slowly the pain of having so many dependencies to C++ DLLs, COM and C++/CLI. One of the reasons why we still do so many .NET Framework projects at my employer agency.
  - andix 1y ago
    > One of the reasons why we still do so many .NET Framework projects at my employer agency. What's the issue with porting them to .net (core)? Most of that stuff is still supported, if you only have native windows DLLs you would still be constrained to windows only, but still better than staying on ancient .net framework.
    - stackskipton 1y ago
      If you are stuck on Windows and upgrade to Core requires more then 2 hours of dev time, it's likely not worth it. Core biggest feature is running on Linux. If you can't, who cares? Framework is not going anywhere. It will be supported till 2035 for now.
      - andix 1y ago
        No, linux support is not the only new feature. Read the change logs, it's thousands of huge improvements everywhere.
        stackskipton 1y ago
        As someone who deals with this, Framework -> Core on Windows is small % performance improvement. Framework Windows -> Core on Linux is huge. Most of it coming from not Windows. Yes, there is other nice language features but obviously 15 years of Framework code base has probably put up guard rails around those sharp edges. My point still stands, I can't imagine most companies green lighting .Net Framework -> Core conversion if they can't switch to Linux. If you are stuck on Windows, you have probably developed all the tooling to deal with Windows so it's all sunk costs.
        bob1029 1y ago
        Linux was rarely part of the conversation when I was doing these conversions. Getting access to things like Kestrel (breaking out of IIS jail) is way more critical. Also, self contained deployments mean you can stop shipping magical blessed machine images around. It's not even about the performance. It's about having technology that doesn't actively hate you.
- andix 1y ago
  I guess you would need to switch to a dotnet native database, like litedb. Even if you would use postgres, there would be native code left, decoupled from the dotnet application though. It would be interesting though, if it's possible to run webassembly inside the CLR (dotnet runtime). But I don't really get the issue with native code inside a dotnet application. In the end everything you do in dotnet ends up being executed as native code. Even a simple console.writeline() is implemented in native code.
  - ZeroConcerns 1y ago
    I've really tried to like LiteDB (mostly because it can use an IO.Stream as the database backing store, which enables lots of fun scenarios), but even light usage mostly resulted in data corruption and inconsistent result sets, something I've literally never seen with SQLite. Plus, I think the project is pretty much dead? And yes, of course everything ultimately runs as native code, but deployment is a major issue. As long as you only deploy IL (or, possibly at some point, WASM), you only need to worry about the relatively lightweight CLR (the dotnet executable and its direct dependencies) -- it does get a lot more complex once you go beyond that, unfortunately.
josteink 1y ago
I fail to understand why they feel the need to test their setup with the latest Alpine while at the same time using out of date and unsupported versions of .NET. On the flip side, good debugging!
- andix 1y ago
  I really don't get why people still bother with unsupported dotnet versions. There might be a few edge cases that prevent upgrading, but in 99% an upgrade from dotnet 3.1 to dotnet 10 is completely smooth. Running in an unsupported dotnet version also means that there won't be any security patches. Not great.
  - SideburnsOfDoom 1y ago
    > in 99% an upgrade from dotnet 3.1 to dotnet 10 is completely smooth. > Running in an unsupported dotnet version is not great Uh, dotnet 10 is currently versioned "10.0.0-preview.7". It won't be released until November 2025. It's therefor 1) Not guaranteed smooth and 2) unsupported. Source: https://dotnet.microsoft.com/en-us/download/dotnet/10.0 https://dotnet.microsoft.com/en-us/download/dotnet/10.0 Perhaps you mean .NET 9. Yes, it's a smooth update in many scenarios.
    - orphea 1y ago
      > Running in an unsupported dotnet version is not great They meant netcoreapp3.1 and net5.0.
      - SideburnsOfDoom 1y ago
        Oh, sorry, I read "unsupported dotnet version" as "an unsupported dotnet version as of today", which incudes 10.0 preview 7.
    - andix 1y ago
      .NET 10 is the next LTS version. It’s going to be released approximately in November and already quite stable. Now is the time to upgrade to 10, do some testing and then deploy once 10 is released. 9 is a non-lts version and 8 (LTS) is already 2 years old (you’ve lost 2 years of support already). Edit: Microsoft officially supports .net release candidates for production use, the first RC is probably going to be released within the next few weeks.
junto 1y ago
Nice walkthrough. You come across a lot of these old .NET based architectures in consulting. Not just older .NET Core projects but still a lot of .NET Framework too. It’s a hard sell to a client that they should use their budget for an upgrade to the core of their software without any visible feature additions. Hence they tend to live on in this limbo state. Beneficial for consultants however. This is the heart of the problem with outsourcing that the powers that be rarely recognize. Yes they might pay less by offshoring the initial development costs (CAPEX), but the long terms support and maintenance costs (OPEX) is then way more costly. It also gets harder and harder to find developers that want to work in this aging tech.
- andix 1y ago
  I've seen the struggle with not upgrading frameworks and staying on an old version. Usually the cost/risk for upgrading is perceived much higher than it actually is. And the cost/risk of technical debt is mostly ignored. I've seen this issue so often: "we can't use this external component, because we would need to upgrade the framework. Let's just work around it, it will only take 2 days (and 50 more days to maintain it over a decade)"

68 points by ingve 1y ago

SQLite is pretty much the only remaining native dependency in my C# codebases, and as much as I love the engine, I wish that could go away. Replacing System.Data.SQLite with Microsoft.Data.Sqlite already helped with Apple ARM builds (despite all the small differences that only showed up in actual use), but pretty much the only native debugging I do these days is related to the "batteries" -- the linked article outlines the general strategy pretty well. On the one hand, I feel bad about turning into a "pure-Java only" kind of developer (I mean, limiting yourself to H2, the horror...), but on the other hand, I'm increasingly starting to see their point. Oh, well, if AI actually works, I'm sure Microsoft.Native.Data.Sqlite is just around the corner (and, later edit, to prevent confusion: the abuse of 'Native' here is mostly making fun of Microsoft naming conventions -- they'd call it 'Interop' if it were like truly native).

mcraiha 1y ago
Would the WebAssembly version of SQLite be OK for you?
- ZeroConcerns 1y ago
  I haven't really kept up with the state of WebAssembly in .NET (mostly because I'm entirely uninterested in Blazor), but I don't think we're at a point yet where we can simply reference a .wasm file and invoke code in it regardless of the underlying platform, right? Until that is the case: no, not really.
pjmlp 1y ago
Exactly because of the benefits of this, there is the meme CGO is not Go, as any package with native dependencies kills the possibility to cross-compile with the Go toolchain. Same applies to the pain of using native dependencies in Perl, Python, Ruby,... Many .NET developers are only now slowly the pain of having so many dependencies to C++ DLLs, COM and C++/CLI. One of the reasons why we still do so many .NET Framework projects at my employer agency.
- andix 1y ago
  > One of the reasons why we still do so many .NET Framework projects at my employer agency. What's the issue with porting them to .net (core)? Most of that stuff is still supported, if you only have native windows DLLs you would still be constrained to windows only, but still better than staying on ancient .net framework.
  - stackskipton 1y ago
    If you are stuck on Windows and upgrade to Core requires more then 2 hours of dev time, it's likely not worth it. Core biggest feature is running on Linux. If you can't, who cares? Framework is not going anywhere. It will be supported till 2035 for now.
    - andix 1y ago
      No, linux support is not the only new feature. Read the change logs, it's thousands of huge improvements everywhere.
      - stackskipton 1y ago
        As someone who deals with this, Framework -> Core on Windows is small % performance improvement. Framework Windows -> Core on Linux is huge. Most of it coming from not Windows. Yes, there is other nice language features but obviously 15 years of Framework code base has probably put up guard rails around those sharp edges. My point still stands, I can't imagine most companies green lighting .Net Framework -> Core conversion if they can't switch to Linux. If you are stuck on Windows, you have probably developed all the tooling to deal with Windows so it's all sunk costs.
        bob1029 1y ago
        Linux was rarely part of the conversation when I was doing these conversions. Getting access to things like Kestrel (breaking out of IIS jail) is way more critical. Also, self contained deployments mean you can stop shipping magical blessed machine images around. It's not even about the performance. It's about having technology that doesn't actively hate you.
andix 1y ago
I guess you would need to switch to a dotnet native database, like litedb. Even if you would use postgres, there would be native code left, decoupled from the dotnet application though. It would be interesting though, if it's possible to run webassembly inside the CLR (dotnet runtime). But I don't really get the issue with native code inside a dotnet application. In the end everything you do in dotnet ends up being executed as native code. Even a simple console.writeline() is implemented in native code.
- ZeroConcerns 1y ago
  I've really tried to like LiteDB (mostly because it can use an IO.Stream as the database backing store, which enables lots of fun scenarios), but even light usage mostly resulted in data corruption and inconsistent result sets, something I've literally never seen with SQLite. Plus, I think the project is pretty much dead? And yes, of course everything ultimately runs as native code, but deployment is a major issue. As long as you only deploy IL (or, possibly at some point, WASM), you only need to worry about the relatively lightweight CLR (the dotnet executable and its direct dependencies) -- it does get a lot more complex once you go beyond that, unfortunately.

josteink 1y ago

I fail to understand why they feel the need to test their setup with the latest Alpine while at the same time using out of date and unsupported versions of .NET. On the flip side, good debugging!

andix 1y ago
I really don't get why people still bother with unsupported dotnet versions. There might be a few edge cases that prevent upgrading, but in 99% an upgrade from dotnet 3.1 to dotnet 10 is completely smooth. Running in an unsupported dotnet version also means that there won't be any security patches. Not great.
- SideburnsOfDoom 1y ago
  > in 99% an upgrade from dotnet 3.1 to dotnet 10 is completely smooth. > Running in an unsupported dotnet version is not great Uh, dotnet 10 is currently versioned "10.0.0-preview.7". It won't be released until November 2025. It's therefor 1) Not guaranteed smooth and 2) unsupported. Source: https://dotnet.microsoft.com/en-us/download/dotnet/10.0 https://dotnet.microsoft.com/en-us/download/dotnet/10.0 Perhaps you mean .NET 9. Yes, it's a smooth update in many scenarios.
  - orphea 1y ago
    > Running in an unsupported dotnet version is not great They meant netcoreapp3.1 and net5.0.
    - SideburnsOfDoom 1y ago
      Oh, sorry, I read "unsupported dotnet version" as "an unsupported dotnet version as of today", which incudes 10.0 preview 7.
  - andix 1y ago
    .NET 10 is the next LTS version. It’s going to be released approximately in November and already quite stable. Now is the time to upgrade to 10, do some testing and then deploy once 10 is released. 9 is a non-lts version and 8 (LTS) is already 2 years old (you’ve lost 2 years of support already). Edit: Microsoft officially supports .net release candidates for production use, the first RC is probably going to be released within the next few weeks.

junto 1y ago

Nice walkthrough. You come across a lot of these old .NET based architectures in consulting. Not just older .NET Core projects but still a lot of .NET Framework too. It’s a hard sell to a client that they should use their budget for an upgrade to the core of their software without any visible feature additions. Hence they tend to live on in this limbo state. Beneficial for consultants however. This is the heart of the problem with outsourcing that the powers that be rarely recognize. Yes they might pay less by offshoring the initial development costs (CAPEX), but the long terms support and maintenance costs (OPEX) is then way more costly. It also gets harder and harder to find developers that want to work in this aging tech.

andix 1y ago
I've seen the struggle with not upgrading frameworks and staying on an old version. Usually the cost/risk for upgrading is perceived much higher than it actually is. And the cost/risk of technical debt is mostly ignored. I've seen this issue so often: "we can't use this external component, because we would need to upgrade the framework. Let's just work around it, it will only take 2 days (and 50 more days to maintain it over a decade)"