11 ms·
SQLite B-Tree Module
- plq 5y agoThis has been shared without context but I guess the SQLite team is starting to modularize the btree code in order to facilitate work like SQLightning: https://github.com/LMDB/sqlightning https://github.com/LMDB/sqlightning At the time SQLightning greatly improved SQLite performance but due to LMDB's requirement to have keys fit in 2/3 of a page it wasn't really useful as a general purpose replacement of SQLite's internal b-tree implementation. EDIT: It looks like SQLightning got adopted and has been worked on by the SQLite team under the name LumoSQL. Here's the project's readme: https://lumosql.org/src/lumosql/doc/trunk/README.md https://lumosql.org/src/lumosql/doc/trunk/README.md which contains at the end "A Brief History of LumoSQL"
- wootest 5y agoWhere does it say that the authors/contributors of LumoSQL is the SQLite team or from the SQLite team? There are meeting notes at https://lumosql.org/src/lumosql/file?name=doc/meetings/lumo-meeting-notes-2022-01-14.md&ci=tip https://lumosql.org/src/lumosql/file?name=doc/meetings/lumo-... which does not overlap with the SQLite developers page: https://sqlite.org/crew.html https://sqlite.org/crew.html or with commits going a year back on SQLite's Fossil timeline. The strongest the README gets is saying "The LumoSQL and SQLite projects are cooperating", which is closer than any other effort I've seen, and welcome if the SQLite project ever wants to swap out the underlying storage engine, but doesn't really mean that the SQLite team "works on LumoSQL" or vice versa. Certainly it looks like LumoSQL has put significant work into the cooperation by using Fossil and by "not forking" which may have made the cooperation palatable. Also, the SQLite project has been consistent on wanting to write all the code for SQLite themselves and not merge in patches (https://sqlite.org/copyright.html https://sqlite.org/copyright.html). Them working on another exploratory project would be a way for them to absorb those changes back into SQLite in a way that wouldn't be incompatible, but it would have to be the same team doing the changes for that to be consistent.
- fivea 5y ago> EDIT: It looks like SQLightning got adopted and has been worked on by the SQLite team under the name LumoSQL. I see none of your claims being supported by your link. It seems LumoSQL is just an umbrella term that refers to a bunch of patches applied over SQLite. I'm not sure if that's enough to not call it a fork. Also,I saw zero references to the people actually involved in LumoSQL, other than the project being supported by the NLNet foundation. I'd expect that a small project managed and hosted by the SQLite team to be hosted in www.sqlite.org, but this looks like a completely independent and unrelated effort.
- plq 5y ago> It looks like SQLightning got adopted and has been worked on by the SQLite team under the name LumoSQL. Sorry about this misinformation, it looks like Sunday morning sleepiness got me. It rather looks like LumoSQL is maintained by an independent group of engineers who seem to be quite familiar with the sqlite codebase and tooling.
- arcticbull 5y agoThe comment at the top... This is obsolete documentation, retained only for historical reference. Do not consider anything on this page to be authoritative. Implies this is old or abandoned, but I could not find any date information to validate that against.
- SQLite 5y agoEdit history: https://sqlite.org/docsrc/finfo?name=pages/btreemodule.in https://sqlite.org/docsrc/finfo?name=pages/btreemodule.in That last substantive edit to the document was in 2009. There were some spelling corrections in 2010. We finally got around to removing it from the documentation set in 2016.
- jstsch 5y agoA bit off-topic, but SQLite continues to impress me. This year, we've added a simple form builder to our product which allows creators to quickly gather feedback/input from their audiences. Basically each form has it's own small sqlite database as a datastore and a json file for the form definition (field styling/constraints, etc). It's 'low tech', but performs great. Not having an additional service running also means having one less thing to maintain.
- ruslan_talpa 5y agoBeen feeling lately the fatigue of managing 3-5 different services for a every single trivial project. This pushed me to explore adding SQLite support in my product [1], i think I'll wrap a Litestream [2] process in the same container and call it "Backed in a Box" :) [1] https://github.com/subzerocloud/blue-steel https://github.com/subzerocloud/blue-steel [2] https://github.com/benbjohnson/litestream https://github.com/benbjohnson/litestream edit: typo
- ergocoder 5y agoI used Sqlite and wasn't impressed. Their SQL dialect is lacking. Parsing a date or regex extraction are extremely difficult. You have to resort to WITH RECURSIVE.
- quietbritishjim 5y agoYou can just supply your own function to parse the date. In most database systems that would be a disaster, because the database would have a send a request over the network back to your program for each argument it needs to process. For SQLite, it's no big deal, because it's executing in the same address space as your program so it's a simple function call. I'll admit that it does make it harder to port your program to another language, so it's more of a workaround than an ideal solution.
- stavros 5y agoOK I love SQLite but there has been a long-standing misconception, or issue, that I'd like to get clarification on: Is using SQLite from multiple processes safe? For a long while, either SQLite itself or the Python bindings weren't safe for concurrent access, is this still the case? Can I use SQLite for my Django app? With the backup system on the Tailscale post yesterday, the operational burden is much much lower than Postgres for many use cases.
- drunkpotato 5y agoThis is just what I’ve heard so take with a grain of salt, but as I understand it sqlite is “mostly” safe to use concurrently. There are situations where it doesn’t behave correctly under concurrent load, the one I’ve heard about being when the sqlite is on a network-mounted drive. I’d love to hear from someone who knows more though.
- stavros 5y agoThat's kind of expected, I think. Due to its design, SQLite relies on filesystem semantics to provide atomicity. If the filesystem doesn't provide the semantics, it makes sense that atomicity will fail. I'm more asking whether it's still unsafe to use in a filesystem that DOES provide those semantics.
- tyilo 5y agoMy experience is that it is only safe to have one connection to a SQLite database. Having multiple connections mostly works, however if the database is stored on a slow medium, such as an SD-card, concurrent queries will fail.
- sydthrowaway 5y agoHas anyone actually ever fixed a TODO they wrote in their life?
- rrdharan 5y agoYes, I have, maybe 20-30% of them but not zero as your comment would seem to imply.