23 ms·
OrbitDB: Peer-to-peer databases for the decentralized web
- dang 6y agoRelated from 2018: https://news.ycombinator.com/item?id=18748542 https://news.ycombinator.com/item?id=18748542
- deleted 6y ago[deleted]
- xchaotic 6y agoeventually consistent database Is an oxymoron
- ArnoVW 6y agoIn case you didn't know, it's in used as a technical term in distributed computing circles. https://en.m.wikipedia.org/wiki/Eventual_consistency https://en.m.wikipedia.org/wiki/Eventual_consistency
- seangrogg 6y agoAlternatives to strong consistency in databases has been around for quite a while...
- trollbaitmate 6y agoi feel bad for you
- sjkelleyjr 6y agoLol...hackernews commenters...
- dchyrdvh 6y agoIirc, Spanner is eventually consistent.
- deleted 6y ago[deleted]
- AtlasBarfed 6y agoWhen you get down to it, between cache coherency across CPUs and memory, disk flush delays and disk caches, every database is eventually consistent. And if you want to operate over a distributed network, which means you WILL have network partitions, then you are subject to CAP and will need eventual consistency mechanisms.
- jacobparker 6y ago> When you get down to it, between cache coherency across CPUs and memory, disk flush delays and disk caches, every database is eventually consistent. This is a false. None of the things you listed preclude consistency. > And if you want to operate over a distributed network, which means you WILL have network partitions, then you are subject to CAP and will need eventual consistency mechanisms. That's not how CAP works. Plenty of distributed CP databases exist.
- vhiremath4 6y agoAsk me how I know you've never scaled a high throughput Elasticsearch cluster.
- dang 6y agoPlease don't post unsubstantive comments here.
- qorrect 6y agoWhat is the use case for this ?
- mirekrusin 6y agoThey want to change the world, pied-piper style.
- aeden 6y agoDecentralized structured data store it seems. If you want a P2P application you have to store data somewhere. No idea if this fits the bill.
- s3n4 6y agoAlso because OrbitDB runs in browser JavaScript, it's great for building local-first distributed applications.
- haadcode 6y agoThe use case is shared, mutable data structures that don't rely on central coordination or control.
- deleted 6y ago[deleted]
- collyw 6y agoDoes that even make sense? Why wouldn't you want centralized coordination of your data structures? We could go back to people emailing emailing Excels to each other is that's what you want.
- jariel 6y agoThis question really needs to be answered in pragmatic and illustrative terms.
- continuations 6y agop2p DB in JS using CRDT. That sounds a lot like GUN. How does OrbitDB compare to GUN?
- lildata 6y agoOne of the main difference is it is based on IPFS
- ekseda 6y agoAnother one is that the source code of OrbitDB is not completely obfuscated.
- evv 6y agoCan you elaborate? Source code to GUN is right here: https://github.com/amark/gun/tree/master/src https://github.com/amark/gun/tree/master/src
- ekseda 6y agoYes sorry for not doing this. In the source code folder, please open any random file and try to understand what the code does. I once tried to fix a bug that I found in GUN but I gave up after just trying to figure out what the code is supposed to do.
- vorpalhex 6y agoThat's not an issue with the source code linked above which is just basic javascript.
- iovrthoughtthis 6y agoWow. You are right. What the heck.
- xsdklfsdfklj 6y agoThat is very readable javascript. old school javascript like this have an advantage: you can ship as is, without the need to obfuscate under the excuse of performance. if you load two hundred build dependencies and then write twenty word long variables, it might look nice for you at development, but a user trying to debug or validate code on the fly would be in a very bad position as you would have to 'minify' (i.e. obfuscate) to not have a 70mb production file. Code like this is very readable if you know some conventions, things like `cb` being a callback pointer. Its not even close to the cryptic knowledge needed to read assembly on weird platforms, for example. And a far cry from obfuscated.
- noworriesnate 6y agoIt looks like they have an identity provider, so this must support authentication. Link: https://github.com/orbitdb/orbit-db-identity-provider https://github.com/orbitdb/orbit-db-identity-provider What I'd like to know is do they support authorization?
- rudolph9 6y agoI believe it generates a pub/priv key pair and stores it in localstorage.
- vasa_develop 6y agoYup. You can actually extend the Access Control to create your own custom authentication/authorization methods. So, basically you can have any sort of Web 2.0 or Web 3,0 based authentication/authorization methods. Oauth, JWT, DIDs...anything you can imagine. Link to Access Controller: https://github.com/orbitdb/orbit-db-access-controllers#creating-a-custom-access-controller https://github.com/orbitdb/orbit-db-access-controllers#creat...
- markhenderson 6y agoHey thanks for taking a look at this. We support identity providers (the one you linked), and access controllers.[1] Identity providers work by cross-signing an external keypair with the generated OrbitDB key-pairs, and access controllers generally work by exposing a `canAppend` function that facilitates any kind of auth you want to perform. There's support for Metamask for example. OrbitDB is also used _in_ Metamask under the hood by our good friends at 3box.[2] TallyLab, my application project, uses these in its IAM system and you can see the repos here.[3] 1. https://github.com/orbitdb/orbit-db-access-controllers https://github.com/orbitdb/orbit-db-access-controllers 2. https://3box.io/ https://3box.io/ 3. https://github.com/tallylab https://github.com/tallylab
- vuldin 6y agoOrbitDB is one of the key dependencies in 3box, an awesome tool for building decentralized apps where the user controls their own data. https://3box.io/ https://3box.io/
- xsdklfsdfklj 6y agothat link doesn't load if I block google ad requests.
- s3n4 6y agoYou can always check out the 3Box docs site: https://docs.3box.io/ https://docs.3box.io/
- pw6hv 6y agoHow do you block these requests? Just asking as I have pihole blocking google ads but I have no issue accessing the website.
- doublerabbit 6y agoI use uMatrix, however the whole site fails to load in Palemoon.
- s3n4 6y agoI'm one of the maintainers of 3Box! Let me know if you have any questions. You can learn more at: Web: http://3box.io http://3box.io Github: https://github.com/3box https://github.com/3box Discord chat: https://discord.gg/669BQHv https://discord.gg/669BQHv
- jsf01 6y agoI am excited by this from what all the docs say but every one of the demos is either broken on my iPhone or so extremely slow to load that I am not able to demo it.
- x5engine 6y ago3box makes OrbitDB really easy to have a persistent easy to use! If you try to go with just OrbitDb you will have to handle manage the ipfs nodes and pin data so it stays persistent and loads faster but also handle the permissions too. But with 3Box makes it easier to do all of the permissions and loading all data faster + always persistent and/or public or private data, and you know what's amazing? the way you can store data? for example if you want to build a chat app you can use Messaging (Threads) to make quickly a chat app! https://docs.3box.io/api/messaging https://docs.3box.io/api/messaging Enjoy!
- LoSboccacc 6y agolast time I tested it needed to fetch the whole db to do even the most basic reads, so even simple but long running applications had a very long, network intensive initialization time. is that still the case?
- markhenderson 6y agoOrbitDB works with a CRDT stored in IPFS. In order to calculate the state of the database, it does need to reduce the CRDT oplog which requires fetching all the entries. This was indeed very time consuming, particularly for remote requests, since we used a "nexts" list of addresses to load. HOWEVER! Our latest release, 0.23.0, mitigated this by using a power-of-2 skiplist to load things in parallel, which gave us a nice 4-5x boost there.
- aboodman 6y agoDoes this mean that in order to initialize the database, the entire version history must be synced and reduced, to get the current values? (Is there a design doc for OrbitDB anywhere?)
- eeZah7Ux 6y agojavascript? No thanks.
- dang 6y agoCould you please stop posting unsubstantive comments to Hacker News? You've been doing it repeatedly, and we ban that sort of account because we're trying for something a bit better than that here. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- tln 6y agoDid any of the demos work for anyone? HN hug of death?
- markhenderson 6y agoYeah, sorry about that :( There's this though: https://github.com/haadcode/orbit-db-control-center https://github.com/haadcode/orbit-db-control-center (which also seems to be putting https://ipfs.io https://ipfs.io under strain. Sorry, ipfs.io!
- collyw 6y agoDecentralized with a control center?
- brabel 6y agoShouldn't a distributed, non-centralized DB handle HN hugs without a hiccup?
- collyw 6y agoExactly what I was thinking. If it can't do that it just sounds like complexity for the sake of complexity.
- markhenderson 6y agoHey folks! I'm https://twitter.com/aphelionz https://twitter.com/aphelionz. One of the maintainers of OrbitDB. Happy to answer any questions you might have. I'll also be in the thread answering folks as well.
- Nican 6y agoHello! Thanks for maintaining an open source project. I am excited to see IPFS take off. Some basic questions, as I am still struggling to see the use case of this project. * How did this project get started? What problem is it trying to solve? * Are there any real world examples of where this project is used?
- markhenderson 6y agotl;dr: You use this any time you want to have mutable data shared across a peer-to-peer network. I wasn't there at the beginning but I believe the project came out of trying to achieve said mutable state within IPFS (which, for other readers is content-addressed and therefore append-only) http://orbitdb.org http://orbitdb.org lists all of our current users, the biggest one is Metamask by way of https://3box.io https://3box.io. https://tallylab.com https://tallylab.com is building with it for remote encrypted backup and shared tallies. https://github.com/dappkit/aviondb https://github.com/dappkit/aviondb is a MongoDB-like interface for it.
- haadcode 6y ago> * How did this project get started? What problem is it trying to solve? OrbitDB got started because we wanted to build serverless applications, especially for the web (ie. applications that run in the browser.) Serverless meaning "no server" and no central authority, ie. something that can't be shut down. OrbitDB gives tools to build systems and applications where the user owns their data, that is, the data that is not controlled by a service. As a way of simple example, imagine Twitter that doesn't have one massive database for all tweets, but rather you'd have one database for each user.
- haadcode 6y agoOne great piece of writing to think about the use cases and what kind of systems and applications can be built following the concepts applied in OrbitDB is this "Local-first software" https://www.inkandswitch.com/local-first.html https://www.inkandswitch.com/local-first.html (there's prolly a thread somewhere here too on that).
- budabudimir 6y agoA database implemented in JS; very encouraging. I guess performance and robustness were not a priority.
- markhenderson 6y agoWell, I imagine you'll be happy to know that some of us have begun IPFS implementation in Rust: https://github.com/ipfs-rust/rust-ipfs https://github.com/ipfs-rust/rust-ipfs
- budabudimir 6y agoDefinitely a much better choice
- haadcode 6y agoCan you elaborate why you think Rust is a better choice?
- budabudimir 6y agoI would have said that for any statically typed language. It simply leads to more robust software because compiler rejects large set of incorrect programs that would otherwise end up in production. Your test coverage has to be much larger just to verify that your program is sane on the most basic level. Rusts compiler is even stricter than most. Not to mention resilience to change, JS is simply a blunder
- haadcode 6y ago> I guess performance and robustness were not a priority. You'd be surprised how well JS does on both fronts, in addition to being able to run across platforms :)
- brabel 6y agoPerformance-wise, JS is not too bad when you have lots of IO going on, as in this case... but robustness? You gotta be kidding! JS is the poster child of a language does NOT have robustness as one of its attributes. Just about every single one of its design decisions makes robustness difficult.
- deleted 6y ago[deleted]
- jayd16 6y agoPretty neat if you think about the technology as a smarter cache. One could easily see certain applications, such as blog networks or Twitter alternatives. Security is easy in some cases. Sign your log appends. Security around transactions without a central authority is pretty complex. Whats the solution there? Block-chain type things? It would have to be application specific I suppose. I never gave FirebaseDB a shot because it felt like too much vendor lock in but this might be interesting to try one day.
- haadcode 6y agoGreat comments! Indeed, "sign your log appends" gives everything needed for authority. Re. transactions, from OrbitDB's perspective this would be application specific, so you could hook into traditional, centralized consensus or use a blockchain or other types of decentralized consensus mechanisms. Or, given the core data structure is a log, build a "custom database" on OrbitDB that models and provides an interface for a consensus algorithm, eg. "append 1: head is X" <- "append 2: ack head is X" <- "append 3: ack from me too that head is X" etc.