5 ms·
LumoSQL
- Lio 1y agoI'd say the most interesting thing on that site is the Not-Forking idea[1]. 1. https://lumosql.org/src/not-forking/doc/trunk/README.md https://lumosql.org/src/not-forking/doc/trunk/README.md
- actionfromafar 1y agoI have never seen PikChr diagrams before, very interesting.
- sksrbWgbfK 1y agohttps://pikchr.org/home/doc/trunk/homepage.md https://pikchr.org/home/doc/trunk/homepage.md Made by the same people who brought us SQLite.
- stonemetal12 1y agoThe US Navy?
- aredox 1y agoI was going to say that. We really need a way to customise software at the source code level without forking.
- Arwill 1y agoYou mean like #define + #ifdef?
- LoganDark 1y agoVery niche but: https://github.com/SpongePowered/Mixin https://github.com/SpongePowered/Mixin It's not really possible to implement in the same way for many other languages, but something like this but for source code transformations (rather than bytecode, or machine code for compiled languages) is probably the kind of thing they're thinking of. Mixin allows you to insert code into methods, modify calls to functions, read/write to local variables, modify constants, and a lot more in that type of vein. It is the way mods are made in the Fabric mod loader for Minecraft. I believe Forge also reluctantly added support for Mixin back in 1.16 or so.
- SOLAR_FIELDS 1y agoBack in the day I used AspectJ to do something like this. The interface was decently friendly and IMO decorator style patterns are going to be the most user friendly approach to something like this. You won’t ever find a universal solution to this though without doing transpilation to some common format that gets morphed (ASM in your example, but I could see also a theoretical world where WASM would offer this)
- eddd-ddde 1y agoI got to the end of that without really understanding what this is solving, what it does, or how. How do you handle changing upstream files locally without forking? Do you just, keep changes in a separate configuration format that is applied lazily at built time? I've never had issues with maintaining a fork anyways.
- theamk 1y agoit's a replacement for checkout + patch process. The main advantage over plain "patch" is that it is more powerful in the face of upstream changes. For example if you rename the upstream file, you have no good way to represent this in .patch, but that project allows is. There also a way to specify patch using function name, which should make it more robust in face of upstream changes. As for my opinion, this seems like an incremental improvement over existing tools. I'd prefer a simple shell script that does "git checkout ... && mv ... && cp .. && patch ..." over something fancy like this.
- 90s_dev 1y agoI'm confused, is this just patch and apply patch?
- alexjurkiewicz 1y agoYes. It's a small DSL to fork a repository and apply a series of textfile transformations (replace file, replace partial file). But if you give this a cool name, it's a New Idea.
- worldsayshi 1y agoSo it's a fork but with somewhat semantic diff specifications?
- SOLAR_FIELDS 1y agoYes, specifications which could almost certainly be applied with a custom merge drivers or strategy rather than trying to come up with a clever name and hand baking a custom toolchain to do what you can already do with Git
- SoftTalker 1y agoNot all projects use git.
- 90s_dev 1y agoWeird.
- SOLAR_FIELDS 1y agoHow many of those projects are attempts at maintaining patches across multiple upstreams, where none of those upstream projects use Git? The author freely admits in their post that all upstream projects in their case are using git, like most of the FOSS world. They just chose fossil for their downstream project because Reasons.
- 1y ago
- aidenn0 1y agoArticle doesn't tell me why it's better than git's subtree merge.
- theamk 1y agolet's say you need to copy config.h.in to config.h and patch a few lines. In case of "subtree merge", you will have to do it manually, every time you upgrade upstream.
- e63f67dd-065b 1y agoDon't we just call these out-of-tree patches? Lots of those for Linux lying about, it's not a new idea. I guess the difference is that they have multiple upstreams, so really they're more of a new project that consists of: - A set of sqlite patches, - Other upstreams and patches? - A custom toolchain to build all the above together into one artefact
- SOLAR_FIELDS 1y agoIf a trunk is a series of patches then isn’t maintaining a series of patches against multiple upstreams just maintaining multiple forks? Feels like mostly semantics to try to differentiate at this point. I mean what is the thing that you would get from doing it this other way? Isn’t it fundamentally still going to require pulling upstream changes regularly and resolving conflicts when they occur? Reading the treatise they say “this allows us to address conflicts that normally you would have to resolve manually”. So Git has tools to pick and choose and auto resolve conflicts, you just have to customize the behavior of your tooling a bit. Seems like they’re just ditching the inbuilt tools git/github offers to achieve this and doing the exact same thing with custom bespoke tooling. Instead of doing that, I’d be more interested in leveraging what Git has baked in with some sort of wrapper tool to perform my automation with some bespoke algorithm. There are merge drivers and strategies end users can implement themselves for this kind of power user behavior that don’t require some weird reinvention of concepts already built into Git
- riffraff 1y agoAFAIU, it's a build strategy. Instead of having a fork, you have a configuration that says "checkout x then apply changes a,b,c".
- herodoturtle 1y agoThis looks really good. The per-row checksums is particularly neat. Good luck to these folks.
- noduerme 1y agoPossibly the wrong place to ask this, but: I've played with SQLite when it was still available in-browser, and I felt that was on the brink of being a game-changer. If it was still supported in-browser and we had replication from the browser, peer-to-peer, I think we'd be living in a much more useful world. It's a lovely tech, but I never built anything serious around it. At this point, as a front-end web technology that seems to be gone. I know I could conceivably use it to back a NodeJS server, keep all the data in memory and local files, but I don't see a great use case for that. I do lots of small projects that could use SQLite, but I usually scaffold them around a single shot Mysql DB for testing, which is easy to spin up and easy to detach from any given back-end instance. So I'm not sure what I'd gain by trying to make a tiny databse on the back-end live in Sqlite files. I'm totally enchanted by stuff like Litestream, and I'm actually dying to find a reason to try it. But every good use case for Sqlite that I could think of sort of died when it stopped being a viable client-side store. TL;DR, what are people using SQLite for? What's the advantage over spinning up a tiny MySQL instance in a cloud somewhere, where you don't have to deal with managing replication and failover by yourself?
- nicoburns 1y agoMost uses of SQLite are client-side apps. Basically everything that's not web: from mobile apps and desktop apps to embedded software in things like cars, tvs, kiosks, etc. There are probably more apps using SQLite than not using SQLite for these kind of apps.
- noduerme 1y agoAre these things just running sql.js? Doesn't that use a kind of unstable webstorage instead of writing client-side files? I don't have a good handle on the state of SQLite these days as a way to store semi-permanent data on the client. In a locked-in environment or a backend I feel like it might make sense, but... isn't there like a 50Mb limit on localStorage, and how does that play nicely with a potentially larger DB...?
- nicoburns 1y ago
- alberth 1y ago[flagged]
- pmbanugo 1y agoI'd try it just for the LMDB mixing
- tehlike 1y agoI'd try it to see if it can be made to work against rocksdb or foundationdb - mvsqlite kind of does that but foundationdb is missing things like compression that rocksdb provides.
- maxloh 1y agoSQLite itself is open source, but the last time I checked, its test suite remains proprietary. How does a "fork" like this be tested if everything stays working and compatible to upstream after the change?
- rogerbinns 1y agoThe general test suite is not proprietary, and is a standard part of the code. You can run make test. It uses TCL to run the testing, and covers virtually everything. There is a separate TH3 test suite which is proprietary. It generates C code of the tests so you can run the testing in embedded and similar environments, as well as coverage of more obscure test cases. https://sqlite.org/th3.html https://sqlite.org/th3.html
- OJFord 1y agoWhy is that? Surely that leads to conversations with open source contributors like 'this fails the test suite, but I can't show you, please fix it'?
- jitl 1y agoSQLite doesn’t accept contributions
- LiamPowell 1y agoThis isn't an issue as SQLite doesn't accept contributions because they don't want to risk someone submitting proprietary code and lying about its origin. I've never understood why other large open-source projects are just willing to accept contributions from anyone. What's the plan when someone copy-pastes code from some proprietary codebase and the rights holders finds it?
- Vvector 1y agoThe "plan" is to take out the contaminated code and rewrite it.
- mmooss 1y ago> LumoSQL exists to demonstrate changes to SQLite that might be useful, but which SQLite probably cannot consider for many years because of SQLite's unique position of being used by a majority of the world's population. > SQLite is used by thousands of software projects, just three being Google's Android, Mozilla's Firefox and Apple's iOS which between them have billions of users. That is a main reason why SQLite is so careful and conservative with all changes. That's a great perspective. How well does the SQLite team work with them? How well does it work in production, especially if you need SQLite compatibility? And
- Lord_Zero 1y agoThe home page doesn't say what it actually is or does.
- philjohn 1y agoMain thing seems to be pluggable backends.
- advisedwang 1y agoright, but what do the alternative backends do that the OG doesn't
- kragen 1y agoThe home page says, "LumoSQL can swap back end key-value store engines in and out of SQLite. LMDB is the most famous (but not the only) example of an alternative key-value store, and LumoSQL can combine dozes of versions of LMDB and SQLite source code(...)" and also, "Encryption and corruption detection, optionally per-row".
- systems 1y ago"LumoSQL can swap back end key-value store engines in and out of SQLite." => "LumoSQL can swap SQLite backend, with Key-value store engines" === "LMDB is the most famous (but not the only) example of an alternative key-value store" => "We currently only support LMDB as an alternative KV store" === "and LumoSQL can combine dozes of versions of LMDB and SQLite source code like this:" => "LumoSQL will allow you to use different versions of SQLite and LMDB in parallel as different backends"
- diggan 1y agoI feel like this best explains what added benefits LumoSQL tries to add: > LumoSQL is a derivative of SQLite, the most-used software in the world. Our focus is on privacy, at-rest encryption, reproducibility and the things needed to support these goals. [...] https://lumosql.org/src/lumosql/file?name=doc/project-announce-phase2.md&ci=trunk https://lumosql.org/src/lumosql/file?name=doc/project-announ... I'm unsure what Phase 1 was about, or if there is a planned Phase 3, but seems to outline what they're currently aiming for at least.
- blacklion 1y agoAFAIR, LMDB is very buggy. There was one person who showed that and maintained fork of LMDB with many bugs fixed, but he is very opinionated and think that world outside Russia is "evil" and forbid to use his fork for evil... Oh, license was changed to Apache 2.0! But still github account has note which equals Hitler to Soros... Why should SQLite backend be replaced with LMDB? UPD: Ooops, LMDB was forked a long time ago, so, maybe, LMDB can be fixed already!
- jasonwatkinspdx 1y agoAFAIK the Symas version is the official now. It's part of the OpenLDAP repo but they have a read only mirror on github for people who want to use it stand alone. OpenLDAP is very heavily used by a lot of companies, in particular mobile operators that put it under very heavy load, so I'd be reasonably confident of its reliability.
- canadiantim 1y agoCan LumoSQL be used with Litestream for replication?
- canadiantim 1y agoHow does LumoSQL compare to LibSQL/Turso?
- cluckindan 1y agoKV backend suggestion: BadgerDB https://github.com/hypermodeinc/badger https://github.com/hypermodeinc/badger
- alexpadula 1y agoHow does LumoSQL deal with locking? If I proposed Wildcat as a storage engine which is lockless. Can LumoSQL be optimized for atomicy and multi writer? C shared library too if you're curious https://wildcatdb.com https://wildcatdb.com