6 ms·
GitDB, a distributed embeddable database on top of Git
- dgllghr 4y agoLooks very cool! Reminds me of Irmin (https://irmin.org/ https://irmin.org/) but with a much simpler and more intuitive interface
- anentropic 4y agoI had same thought! I dislike Go, but I bet gitdb is easier to use
- encryptluks2 4y agoI've been imagining a similar concept, but using linting rules to create an optimized file tree with a SQL API to manipulate and query the data programmatically. You could even use a traditional text editor to manage the data.
- tzahifadida 4y agoReally don't get it and would like to understand. Let's say I don't need a full DB. Why use it instead of, let's say, sqlite? I know devops uses git itself for storing and pulling out stuff for their automations. Is it in that direction?
- beagle3 4y agoDisconnected remote work. Git pull to receive updates, Git push to send your updates. No idea how/if it handles merge conflicts. Also - you could have db “branches”.
- qbasic_forever 4y agoPutting a sqlite db in a git repo doesn't work well, it's just an opaque blob to git so you can't merge changes beyond "wipe out every other change and accept this as the new complete state of the world".
- ltbarcly3 4y agoHow is this distributed in anything but the most trivial sense? In the documentation it literally says this is not distributed, and there is a centralized git server. If the creator doesn't know how common database terms are used they probably don't know much about writing a database.
- lifty 4y agoCurious how this compares to Dolt or Noms.
- mwexler 4y agoUsing git for so many things (it's a database! it's a blockchain! it's a dessert topping! It's a floor wax! [1]) does start to smell of "I have a hammer, and luckily everything looks like a nail if I squint enough". But it is fun to see how folks have stretched git into so many directions, even if some of them are more useful than others. [1] SNL, 1976: https://www.nbc.com/saturday-night-live/video/shimmer-floor-wax/n8625 https://www.nbc.com/saturday-night-live/video/shimmer-floor-... (may not work in all regions, sorry) (edited link)
- danielvaughn 4y agoI don't see it that way - version control in general turns out to be highly applicable in many areas far beyond source code. I think it's still under-utilized.
- qbasic_forever 4y agoGit is an abstraction over file trees (or even more generally just chunks of data), and it turns out a ton of problems in software are easily modeled and dealt with as files in a tree.
- bastawhiz 4y agoLots of problems in software are easily modeled as ordered lists, but that doesn't mean linked lists are the right tool for accomplishing that.
- rmbyrro 4y agoIf the rest of the world were like the software industry, we would see news like this on Facebook: "KayaCar, a distributed kayak with wheels to drive on the streets of San Francisco"
- sophacles 4y agoI would consider using facebook if there were articles like that. Not sure why you consider this a problem... "Because I can" is a good enough reason to do anything that doesn't cause harm to others.
- chrsig 4y ago> "Because I can" is a good enough reason to do anything that doesn't cause harm to others. It's hard to tell in advance how any action may impact others. The extreme example: jumping off a bridge doesn't physically harm others. But having to look at, examine, or otherwise interact with your corpse may inflict trauma on others.
- deleted 4y ago[deleted]
- elpakal 4y agoIsn't there also an environmental cost for all things blockchain because of power consumption? That doesn't directly impact others but might eventually.
- TickleSteve 4y ago"proof of work" is the computationally expensive thing... a block-chain is simply a data structure with many uses outside of cryptocurrencies.
- swyx 4y agodid you just compare releasing a new OSS project to suicide?
- hiccuphippo 4y agoI guess this would allow git to have its own issue tracker and forum like fossil does?
- qbasic_forever 4y agoYou can already store and sync issues in git repo metadata, check out git-bug: https://github.com/MichaelMure/git-bug https://github.com/MichaelMure/git-bug
- layer8 4y agoI believe one can say that it’s actual Git data, not just metadata.
- uniqueuid 4y agoVery cool. Now we need exactly this but for sqlite. Decentral offline databases that can sync.
- phlyingpenguin 4y agoIt doesn't appear to be doing much more than replication though. https://litestream.io https://litestream.io does that for sqlite already. (Edit for clarity) GitDB seem to have more "distributed" ideas in their roadmap for v3 maybe.
- uniqueuid 4y agoLitestream is awesome, but I don't see that it enables multi-way merges yet. Am I wrong?
- benbjohnson 4y agoLitestream author here. You're correct. Also, there's no plans for multi-master replication right now. I do think it'd be an interesting project to make a eventually consistent distributed database using the SQLite session extension[1] but I haven't put much thought into that. [1]: https://www.sqlite.org/sessionintro.html https://www.sqlite.org/sessionintro.html
- uniqueuid 4y agoI keep hoping that you'll eventually write this :) Let me know when you do so I can chip in with some money.
- qbasic_forever 4y agoAre you sure you want to use sqlite if your workload is mostly multiple, independent writers?
- uniqueuid 4y agoYes because I want to do offline sync (i.e. latency between syncs anything between 1 second and 1 week), yet index and query the current local state (which includes previously synced items). If this sounds like an append-only log that's precisely correct; I just want it to work efficiently with fixed memory and ideally as an embedded library. Thus sqlite.
- amelius 4y agoHow does merging work? Would it even be acceptable to allow automatic merging in all types of application?
- uniqueuid 4y agoTo be honest, I would be completely fine with a use case that offloads conflict prevention to the application. I think there's a lot of cases where you can guarantee that your data conforms to some sort of CRDT [1] and therefore massively reduce complexity in the DB. [1] even if it's last-write-wins