8 ms·
Fluree DB – A scalable blockchain database
- manojlds 8y agoIsn't blockchain just a slow database in the first place. What do you mean by blockchain meet database.
- arxpoetica 8y agoNot having read the white paper, it appears the defining difference here is data versioning.
- atomical 8y agoI would assume it means that it's a database where it's impossible to modify the version history and maintain database integrity. It would offer some protection against tampering.
- prepend 8y agoHaving a multi-node database that syncs (even slowly) and handles merge conflicts is useful in many applications. Specifically, this is useful in global health situations where you have lots of data collectors with tablets entering data that all has to eventually come together. But has lots of offline/online changes and has an untrusted data entry issue where you don’t want anything deleted.
- he0001 8y agoHow can it handle merge conflicts automatically?
- dustingetz 8y agoi assume he means https://en.m.wikipedia.org/wiki/Conflict-free_replicated_data_type https://en.m.wikipedia.org/wiki/Conflict-free_replicated_dat...
- prepend 8y agoI don’t need it to merge automatically, but storing them in a way that can be reconciled later is necessary. In some situations merges could be automatic if the records don’t have any conflicts.
- MichaelGG 8y agoYou've described merging/replication which has nothing to do with blockchain.
- prepend 8y agoThat’s right, the blockchain part just adds in immutability of data and being able to trust different nodes. What database models have replication/merging models provide assurance of data integrity and perpetual versioning?
- MichaelGG 8y agoBlockchain adds neither things you mention. Both can be accomplished with digital signatures and timestamping, which are not new. Blockchain alone doesn't prevent you from modifying data -- just rewrite the entire chain. As an example, Bitcoin only prevents this with proof of work, but only following accumulated difficulty prevents someone from thinking my privately-mined chain is true.
- manigandham 8y agoWe have lots of distributed databases with merging/syncing abilities. You can also append-only and disable delets. No blockchain necessary. I don't get your use-case regarding data collection people, what exactly will a blockchain prevent them from doing? Are they all entering the same piece of data to then pick the version they all input the most?
- prepend 8y agoIt’s very likely that I’m just not aware. What databases enforce no deletes or edits in a verifiable way? I would appreciate you sharing some examples. In my use case you have say 5 people covering different territory collecting info that usually doesn’t conflict. Imagine doing a census of a few villages. You don’t want data changed or altered especially time stamps and want it to be shared across all nodes when they are in proximity so they can sync to a remote instance when back in coverage. In some situations it’s literally the group meets at a tree at noon with a coordinator that syncs up all the collectors and moves on to the next group. There are some conflicts that need to be merged, but mostly just syncing lots of nodes asynchronously and you don’t want anyone dropping records, maliciously editing them, or accidentally overwriting.
- ilikebits 8y agoNo deletes or edits is a really bad idea, especially for health data. In theory, it's pretty neat; in practice, it's a legal, compliance, and privacy nightmare.
- evanweaver 8y agoSeems more like a blockchain as a service like Chain's Sequence than an actual database.
- jrq 8y agoSince one of these buzzword bingo projects are posted so often, can someone explain what this does, and why I shouldn't just use datomic anyways?
- Ihfhcub 8y agoIt does absolutely nothing interesting. It's a broken database with a buzzword attached
- jrq 8y agoWhat do these projects hope to gain from pushing bad tech or broken ideas? Why do so many founders WANT to bust a Theranos-esque move? Its nuts
- manigandham 8y agoMoney. Bad tech and broken ideas have never stopped businesses before and some have become rather successful. Using blockchain hype along with access to unregulated ICO markets just makes it easier.
- jcousins 8y agoEpicenter podcast with the founders for those interested: https://epicenter.tv/episode/216/ https://epicenter.tv/episode/216/
- wslh 8y agoFundamental red flag: performance/speed metrics not available, it reads like they have found a way to obtain better performance in a database + blockchain software. To probe that they should base their claims on solid research. Currently one of the best performant BFT software is BFT-SMaRt [1][2] which is not scalable to many nodes without degrading the performance significantly. If someone wants to claim a breakthrough it would be simpler to identify current blockchain restrictions and show how they are improved. Once this is clear we can only hope that the relatively few researchers in the field take a look to see if there are obvious inconsistencies. [1] https://github.com/bft-smart/library https://github.com/bft-smart/library [2] http://repositorio.ul.pt/bitstream/10455/6897/1/TR-2013-07.pdf http://repositorio.ul.pt/bitstream/10455/6897/1/TR-2013-07.p...
- shiado 8y agohttp://doyouneedablockchain.com/ http://doyouneedablockchain.com/
- prepend 8y agoNormally I like this flowchart, but the permanent record is useful even within trusted parties. There aren’t many (any?) distributed databases with permanence built-in. So getting that for free is nice to not have to build yourself. Especially self-hosted. If this allowed self hosted, it would be cool. Paying them seems not very useful.
- endymi0n 8y agoOMG, looks like „permanence“ is now to blockchainers what „web scale“ was to MongoDB fanboys. Now if you were so kind to explain to me what about modern, distributed databases isn‘t „permanent“? Permanence is the very definition of a database‘s job. If you really want to make extra sure it‘s permanent, give write-only permissions to the DB user, make backups and if you want to allow mutations, enable change data capture. Now if there only was a way to short blockchain technology as a financial asset...
- prepend 8y agoWhat I mean by permanence is that you can’t delete. Databases typically allow delete and mod and this is a big problem for me. What is a more accurate term? I always used persist for writing to disk, like databases do. With the “workaround” you describe, although a big pain in the ass for lots of nodes, still doesn’t work. How do you know that someone’s backup is correct? Or how do you verify that a remote node didn’t write/delete/sync? There’s many situations that make your idea fail that a blockchain would work. Although, I agree that lots of “blockchain” solutions just need a decent distributed cache since you don’t care much about integrity.
- MichaelGG 8y agoOf course you can delete, just create a new DB and insert the records except the ones you don't want. If you want to have integrity control, you can use digital signatures without blockchain. If all you're trying to fix is making it hard to issue a DELETE statement, then make it easier to set permissions. Which isn't really hard in any mainstream database... It seems like this is a log, which might work better, although existing RDBMS already have high performance versioning/change tracking.
- marknadal 8y agoThis sounds a lot like (full disclosure: mine) https://github.com/amark/gun https://github.com/amark/gun ? How do you compare? - Except ours is Open Source (MIT/ZLIB/Apache2) and ranked #2 on GitHub in Blockchain and Cryptography https://github.com/topics/blockchain https://github.com/topics/blockchain - Performance benchmarks are posted publicly https://github.com/amark/gun/wiki/100000-ops-sec-in-IE6-on-2GB-Atom-CPU https://github.com/amark/gun/wiki/100000-ops-sec-in-IE6-on-2... - It has been load tested doing 100M+ records/day for $10/total costs (CPU, disk, backup) https://www.youtube.com/watch?v=x_WqBuEA7s8 https://www.youtube.com/watch?v=x_WqBuEA7s8 - Explains how consensus, CRDTs, and DAGs work http://gun.js.org/distributed/matters.html http://gun.js.org/distributed/matters.html
- wslh 8y agoCan you link your project foundations to a peer reviewed paper with adversarial attack analysis?
- marknadal 8y agohttp://stanford.edu/~nadal/A-Decentralized-Data-Synchronization-Protocol.pdf http://stanford.edu/~nadal/A-Decentralized-Data-Synchronizat...
- wslh 8y agoWhere are the independent security reviews? Also, I don't see past references where you based your new idea. Please take into account that I am respecting you for your work but this field involves more than a claim. If you look, for example, to simmetric encryption algorithms history, there were a lot of proposals and people involved in trying to crack them. Many of them sounded great but were break by professional cryptographers. The blockchain scene is full of proposals but there are very few security reviews.
- marknadal 8y agoGo have anybody you want look at the code, it is open source. My friends like Dominic Tarr (of Secure Scuttlebutt), security teams at MIT (Christian and Dmitri), and others have consulted with me about it. If you are willing to contribute donations to pay a firm, please do!
- prepend 8y agoIs my math right that a 1 gb database sitting there doing nothing is 1.4M tokens per month (2/MB/hour). And tokens are $5/200k, so $35/month before you do anything. This seems pretty crazy. But I really need a blockchain db for the permanence.
- jasonkolb 8y agoMe too. Have you find anything good so far?
- prepend 8y agoI just learned of the gun,https://github.com/amark/gun https://github.com/amark/gun project from a downthread post. I was kind of surprised that it didn’t exist and had some really janky sql+git stuff.
- manigandham 8y ago> But I really need a blockchain db for the permanence. Do you really? What exactly is so permanent and who holds all the copies?
- prepend 8y agoI explain it on the thread, but I have health records for disease outbreaks where the version is very important never to delete. The copies end up being stored amongst a few countries and Health partners. In some situations, countries can’t be trusted to archive or not alter records as it could be politically sensitive (eg, “maybe we just edit the db to say 10 cases of disease X is really 10 cases of disease Y.”). Or prevent situations where one site accidentally deletes some records, but not all, and it isn’t caught for weeks later where there have been other adds and edits over the time. Figuring out the restore was really hard. Being able to immediately detect changes across all nodes is valuable.
- f00_ 8y agomaybe you'd be interested in opentimestamps or factom, they use the bitcoin blockchain to timestamp documents https://petertodd.org/2016/opentimestamps-announcement https://petertodd.org/2016/opentimestamps-announcement
- shubidubi 8y agoi find it strange that none of the team members is an engineer
- nwatson 8y agoBrian Platz, Fluree CEO, is an entrepreneur and engineer. I know Fluree's written mostly in Clojure, and he wrote the core. (He worked out of the same co-working space I'm in but recently the company graduated to their own space.) Brian Platz (https://angel.co/brian-platz https://angel.co/brian-platz) also previously founded SilkRoad Technology (not the other Silk Road), an HR management solution (among perhaps other things) ... https://www.silkroad.com/ https://www.silkroad.com/. Many institutions use Silk Road, e.g., the institution where my wife teaches, Wake Forest U ... here's their Silk Road portal: https://wakejobs.silkroad.com/WFU/Employment_Listings.html https://wakejobs.silkroad.com/WFU/Employment_Listings.html.
- he0001 8y agoImmutable eh? What about GDPR?
- alexchamberlain 8y agoI've not read too much into Fluree itself, but GDPR doesn't actually say you have to actually delete data. You can cryptographically shred it, among a few other options.
- he0001 8y agoHow do you cryptographically shred it in a immutable blockhain? And what are the other options? It explicitly says you have to “erase the personal data”, GDPR Chapt 3 Art. 17
- nickpeterson 8y agoI think you would put the surrogate key in the block chain interactions, and then have a side place where you store the encrypted gdpr data. Then you can just overwrite the fields with null and keep the key.
- warkdarrior 8y agoIf you can overwrite the data fields, then it is no longer immutable, is it?
- nickpeterson 8y agoThere us no such thing as immutable data. The only constant is change. I'm suggesting that in the modeling of the data structures in a system, you rely only on the surrogate value and not the actual values. You're holding an immutable key to mutable data, that can be altered, but isn't necessary to be constant for the system to work. There is really no alternative, because laws are making it increasingly likely that immutable data stores will not be legally sound. Data stores like datomic have the notion of excision to handle this. Blockchains may require immutability, but that simply means you shouldn't put data in that must be removed by a court order.
- lightedman 8y agoThe second I hear scalable blockchain, I walk away, because the math has been done and blockchains cannot scale if you're expecting everyone to act as a node/voter/verifier. Just the physical infrastructure and equipment to utilize the blockchain alone would take up more resources than we have currently mined and stockpiled, and then the insane energy usage to boot would be outrageous. If someone can come up with a solution to the log x exp growth rate that is incurred per additional user/voter/system added, then maybe one can say they've fixed the scaling issue. The math says good luck catching up!
- aboodman 8y agoIIIUC the fundemental datamodel here is a log of changes. A "query engine" is layered on top to get reasonable query performance, but that amounts to maintaining a materialized view for any data that might want to be queried. Otherwise the blockchain itself is not practically queriable by itself.
- KingMob 8y agoSounds Datomic, but way less efficient.
- filleokus 8y agoSomewhat related Ask HN-commenters: Are there any widely used append-only DBMS's out there? Perhaps even with built in support for hash chains? I would like to have basically a single node, non-POW, "blockchain" database. Or, described differently, I guess, ≈≈ git for a database.
- aboodman 8y agohttps://github.com/attic-labs/noms https://github.com/attic-labs/noms (full disclosure: my project)
- aboodman 8y ago(not widely used though)
- purerandomness 8y agoYou basically want a linked list.
- filleokus 8y agoWell, not really. It would be nice to be able to query the latest state in an efficient way, model relations and data constraints and have transaction handling etc as we are used to with traditional databases.
- purerandomness 8y agoAh, so you want CQRS + Event Sourcing?
- dustingetz 8y agogit as a database is Datomic http://www.dustingetz.com/:datomic-in-four-snippets http://www.dustingetz.com/:datomic-in-four-snippets