ncruces

69 ms·

961.
by ncruces 3y ago
With zig I think you can avoid libc deps on Linux, at least, by statically linking musl instead. Not sure SQLite is tested in this configuration, for the fans of “this didn't pass TH3 so we can't trust it at all.”
962.
by ncruces 3y ago
One problem with WASM is that, at this point (2023), the only easy way of building that, is to statically link everything C into one WASM blob (so you can't use my blob). But you can try to use my build scripts, and tweak my code, to
963.
by ncruces 3y ago
I'm sorry. The debugging experience of trying to whip up PInvoke wrappers for the COM library for Kinect 2 that worked with Unity Engine's awful version of Mono was charring. Also had “fun” building some Xamarin Android PInvoke-to
964.
by ncruces 3y ago
Also, while you're copying you can inverse grep filter to avoid duplicates. And since you're copying, you could maybe ensure sorting, but I don't think doing that with "bash" (plus off the shelf utils) makes a lot o
965.
by ncruces 3y ago
I don't mean debugging the C bits (although having a stack trace and an exact line number when I cause SQLite to crash is rather nice). I mean developing a complex wrapper/binding that involves Go calls C, which then calls back to
966.
by ncruces 3y ago
It's not just about cross compilation. There's also debuggability and other aspects of development. It's an anecdote, but I've had a better experience debugging issues while developing SQLite extensions and VFSes for t
967.
by ncruces 3y ago
This is a great answer to the parent question, I'll just add that I (try) to provide both a database/sql driver, and a lower level direct mapping of the SQLite C API. Pretty much the only thing the lower level wrapper does for y
968.
by ncruces 3y ago
To compile C for a given platform, you need a C compiler for that platform. Which definitely exists but you may not have it at hand. To cross compile from your platform A to that platform B, you need a cross compiler, which may be even hard
969.
by ncruces 3y ago
There's pretty bad posterization in the background. If you can't see it, kick up your contrast. You don't need HDR levels of contrast to notice it.
970.
by ncruces 3y ago
One thing I tried to make sure, to avoid the pitfall modernc is having, is to make sure building "the WASM BLOB" is easily reproducible with widely available tools: https://github.com/ncruces/go-sqlite3/b
971.
by ncruces 3y ago
The major one is (cross) compilation and portability. You can (e.g.) easily whip up a small CLI that embeds an SQLite database in the binary [1], cross compile this to all the platforms Go supports [2], copy one file, and be reasonably sure
972.
by ncruces 3y ago
Thank you for the feedback! I tried to improve the copy a bit, but this is clearly not my thing…
973.
by ncruces 3y ago
The nitty gritty is here, mostly: https://news.ycombinator.com/item?id=38652566 About my driver, the major compromise to be aware of is that WAL mode support is limited. No tiptoeing here: I call that caveat out in the fron
974.
by ncruces 3y ago
The nitty gritty is in another answer. The commits implementing the new function cache are these two: https://github.com/ncruces/go-sqlite3/compare/0b093b7c...986... Creating good, representative, benchmarks
975.
by ncruces 3y ago
No, I meant in wazero, when using their AOT WASM-to-native compiler engine. So, my bindings wrap a WASM build of SQLite. The WASM module exports functions (like sqlite3_exec) for Go to call, and imports Go functions (like go_full_pathname)
976.
Show HN: My Go SQLite driver did poorly on a benchmark, so I fixed it(github.com)
234 points by ncruces 3y ago|58 comments
977.
by ncruces 3y ago
To follow up, I made a new release with performance fixes, and here are the new results: https://github.com/ncruces/go-sqlite-bench/blob/98553a08199b...
978.
by ncruces 3y ago
OK, so if I'm looking at this right, a smarter, wider cache goes a great length to fixing the issue. In [1] I implemented a simple PLRU bit cache, and I'm seeing an 8x performance improvement in some of the tests I was doing worse
979.
by ncruces 3y ago
It doesn't support everything. No driver likely does. An example of one thing missing is the pointer passing interface which limits the ability to create complex extensions. If support was added, the way function creation and virtual t
980.
by ncruces 3y ago
The performance difference was larger than I had expected. But this is good. To fix a recent crash [1] that was happening due to a particular case of reentrancy, which only showed up when I implemented virtual tables and queried other table
981.
by ncruces 3y ago
I would suspect some unnecessary copying. Nothing else explains a “lower level” binding (zombie) performing worse than modernc. My (WASM) binding also has a lower level API. But I worked hard to make database/sql work first class, so I
982.
by ncruces 3y ago
For the WASM one (disclaimer: I built it) it's at least reasonably simple to reproduce the build [1], and the resulting blob is fully cross platform. Also, although I do (lightly) patch SQLite which might invalidate results in your vie
983.
by ncruces 3y ago
I'm the author or the WASM (+wazero) based github.com/ncruces/go-sqlite3. Happy to field questions. W.r.t. benchmark results. wazero's current compiler is somewhat naive, which may explain a large performance delta in CP
984.
by ncruces 3y ago
30% is in the ballpark of what I'm expecting, actually.
985.
by ncruces 3y ago
I don't really buy the security argument if: “some traditional WebAssembly compilers can decorate memory accesses with bound-checking code. w2c2 currently can’t, but it totally could.” Also, in my experience, naive bounds checking will
986.
by ncruces 3y ago
How does it compare to μTP (Micro Transport Protocol)? https://en.wikipedia.org/wiki/Micro_Transport_Protocol
987.
by ncruces 3y ago
You have options that don't use cgo. Disclaimer, I made the second one. https://modernc.org/sqlite https://github.com/ncruces/go-sqlite3
988.
by ncruces 3y ago
I don't have them at hand, will post them to the wiki when I get a chance. That said there's 2 aspects to SQLite performance: compute and IO. For compute, wazero's current compiler is pretty naive, so CPU bound benchmarks wil
989.
by ncruces 3y ago
Some discussion on reddit: https://www.reddit.com/r/golang/comments/182516r/go_bindings...
990.
Show HN: Go bindings to SQLite using wazero(github.com)
9 points by ncruces 3y ago|3 comments

More ›