5 ms·

Interesting nugget about golang from their FAQ: https://github.com/canonical/dqlite/blob/master/doc/faq.md https://github.com/canonical/dqlite/blob/master/doc/

by bb88 7y ago

Interesting nugget about golang from their FAQ: https://github.com/canonical/dqlite/blob/master/doc/faq.md https://github.com/canonical/dqlite/blob/master/doc/faq.md Why C? The first prototype implementation of dqlite was in Go, leveraging the hashicorp/raft implementation of the Raft algorithm. The project was later rewritten entirely in C because of performance problems due to the way Go interoperates with C: Go considers a function call into C that lasts more than ~20 microseconds as a blocking system call, in that case it will put the goroutine running that C call in waiting queue and resuming it will effectively cause a context switch, degrading performance (since there were a lot of them happening). See also this issue in the Go bug tracker. The added benefit of the rewrite in C is that it's now easy to embed dqlite into project written in effectively any language, since all major languages have provisions to create C bindings.

MuffinFlavored 7y ago
> since all major languages have provisions to create C bindings. Does WebAssembly or any of its runtimes provide a way to do this?
- szczepano 7y ago
  you can probably compile c to webassembly using clang/llvm
  - derefr 7y ago
    That's not what the parent was asking. They were asking if WASM has an FFI interface for runtimes that want to not execute the WASM code entirely sandboxed from the host, but rather want to allow you to dlload(2) code into the OS process hosting the WASM interpreter and call it through WASM ops. Presumably, https://github.com/CraneStation/wasmtime https://github.com/CraneStation/wasmtime would benefit from such an FFI API being specified.
    - MuffinFlavored 7y ago
      Precisely. Do you think this is a feature the WebAssembly team should standardize or is it a bad fit for the language?
    - deleted 7y ago
      [deleted]
    - syrusakbary 7y ago
      That should be doable! (Disclaimer: I work at Wasmer https://github.com/wasmerio/wasmer https://github.com/wasmerio/wasmer )
    - zbentley 7y ago
      That sounds like ActiveX 2.0 at best, and an utter security nightmare at worst.
- tracker1 7y ago
  Yes, emscripten/llvm builds C code into webassembly. I'd say a significant portion of webassembly is C/C++ sourced.
  - MuffinFlavored 7y ago
    You can't use emscripten/llvm with, #include <windows.h> to provide a GUI per se, can you?
    - tracker1 7y ago
      Nope. There are iirc specific bindings for that in a browser context. I think there's been a lot of work for other runtimes.
- thosakwe 7y ago
  Genuine question here: if you’re already going to be shipping native code, why would you need WebAssembly in the mix, especially if you’re not targeting a browser?
  - funny_falcon 7y ago
    For example, if you want to execute "native" code in a restricted environment. WebAssembly is quite fast when (jit)compiled. Therefore, it's very tempting to use it to embed extension logic.
SomeOldThrow 7y ago
I don't want to flame, but I did find it curious they went with C rather than Rust. In my experience the transition is straightforward and the string handling (particularly with unicode encodings) is way better (in addition to the normal ownership benefits), and the result (an easily linkable library exposing a C ABI) is roughly the same.
- stefan_ 7y ago
  Yes, and one library will be 100 MiB, the other 1.
  - SomeOldThrow 7y ago
    A) disk space seems like a reasonable tradeoff for many situations, especially when binaries typically aren't the source of data consumption B) I don't think I've seen a rust binary more than 10 megs. Cargo, rust, and ripgrep are both about 6 mb on my disk; fd is 2.5mb. These seem like a reasonable standins for a binary of significant size and complexity. sqlite3 itself is about 1.3mb. C) Dqlite doesn't seem particularly concerned with disk-constrained systems, though I may be interpreting their site incorrectly, and the low footprint should be equally achievable with the rust runtime—surely the database itself would be a much larger concern. This just seems like an unusually good fit for the benefits of the language—reliable client glue you can import into many runtimes where being able to prove data flow would be an strong defensive coding pattern. That said, I think that C is a good, conservative approach here, I'm certainly not knocking anyone's judgement. Overall the parent poster is absolutely correct: there's a strong correlation between use of rust's type system and size of outputted code. EDIT: phrasing.
    - josegonzalez 7y ago
      > Dqlite is a fast, embedded, persistent SQL database with Raft consensus that is perfect for fault-tolerant IoT and Edge devices. Seems like, at least for embedded devices, you'd want something as small as possible so as to avoid consuming all available disk (not to say any other language will balloon it significantly or not).
nickpsecurity 7y ago
They could also use D or Rust for this. If borrow-checker is too much, Rust can still do automatic, memory management with other benefits remaining. Both also support letting specific modules be unsafe where performance is critical.

by bb88 7y ago

MuffinFlavored 7y ago

> since all major languages have provisions to create C bindings. Does WebAssembly or any of its runtimes provide a way to do this?

szczepano 7y ago
you can probably compile c to webassembly using clang/llvm
- derefr 7y ago
  That's not what the parent was asking. They were asking if WASM has an FFI interface for runtimes that want to not execute the WASM code entirely sandboxed from the host, but rather want to allow you to dlload(2) code into the OS process hosting the WASM interpreter and call it through WASM ops. Presumably, https://github.com/CraneStation/wasmtime https://github.com/CraneStation/wasmtime would benefit from such an FFI API being specified.
  - MuffinFlavored 7y ago
    Precisely. Do you think this is a feature the WebAssembly team should standardize or is it a bad fit for the language?
  - deleted 7y ago
    [deleted]
  - syrusakbary 7y ago
    That should be doable! (Disclaimer: I work at Wasmer https://github.com/wasmerio/wasmer https://github.com/wasmerio/wasmer )
  - zbentley 7y ago
    That sounds like ActiveX 2.0 at best, and an utter security nightmare at worst.
tracker1 7y ago
Yes, emscripten/llvm builds C code into webassembly. I'd say a significant portion of webassembly is C/C++ sourced.
- MuffinFlavored 7y ago
  You can't use emscripten/llvm with, #include <windows.h> to provide a GUI per se, can you?
  - tracker1 7y ago
    Nope. There are iirc specific bindings for that in a browser context. I think there's been a lot of work for other runtimes.
thosakwe 7y ago
Genuine question here: if you’re already going to be shipping native code, why would you need WebAssembly in the mix, especially if you’re not targeting a browser?
- funny_falcon 7y ago
  For example, if you want to execute "native" code in a restricted environment. WebAssembly is quite fast when (jit)compiled. Therefore, it's very tempting to use it to embed extension logic.

SomeOldThrow 7y ago

I don't want to flame, but I did find it curious they went with C rather than Rust. In my experience the transition is straightforward and the string handling (particularly with unicode encodings) is way better (in addition to the normal ownership benefits), and the result (an easily linkable library exposing a C ABI) is roughly the same.

stefan_ 7y ago
Yes, and one library will be 100 MiB, the other 1.
- SomeOldThrow 7y ago
  A) disk space seems like a reasonable tradeoff for many situations, especially when binaries typically aren't the source of data consumption B) I don't think I've seen a rust binary more than 10 megs. Cargo, rust, and ripgrep are both about 6 mb on my disk; fd is 2.5mb. These seem like a reasonable standins for a binary of significant size and complexity. sqlite3 itself is about 1.3mb. C) Dqlite doesn't seem particularly concerned with disk-constrained systems, though I may be interpreting their site incorrectly, and the low footprint should be equally achievable with the rust runtime—surely the database itself would be a much larger concern. This just seems like an unusually good fit for the benefits of the language—reliable client glue you can import into many runtimes where being able to prove data flow would be an strong defensive coding pattern. That said, I think that C is a good, conservative approach here, I'm certainly not knocking anyone's judgement. Overall the parent poster is absolutely correct: there's a strong correlation between use of rust's type system and size of outputted code. EDIT: phrasing.
  - josegonzalez 7y ago
    > Dqlite is a fast, embedded, persistent SQL database with Raft consensus that is perfect for fault-tolerant IoT and Edge devices. Seems like, at least for embedded devices, you'd want something as small as possible so as to avoid consuming all available disk (not to say any other language will balloon it significantly or not).

nickpsecurity 7y ago

They could also use D or Rust for this. If borrow-checker is too much, Rust can still do automatic, memory management with other benefits remaining. Both also support letting specific modules be unsafe where performance is critical.