4 ms·

Hrm.. interesting idea! The tl:dr; appears to be "we built a blockchain based query logger" where essentially the consensus appears to just be that it is valid

Hrm.. interesting idea! The tl:dr; appears to be "we built a blockchain based query logger" where essentially the consensus appears to just be that it is valid SQL, and defines an ordering of the statements. Then once everyone agrees to apply the change, then it is applied... With (AFAICT) no interaction with the DB during the consensus process? This seems to be a bit naive of features like locks and transactions that are critical to most DBs, so while maybe you can define a global ordering of statements, the execution of those statements may be nonsensical and result in a lot of failed transactions. For example, it seems like you could grind the usability of the system to a halt by flooding the validators with 'valid' SQL designed to create as many conflicts as possible that would cause a ton of writes to fail to commit. I agree with the desire for more services to become decentralized, and this is a novel idea, but (IMHO) more and more the technical hurdles of blockchains often bring more challenges than they solve.

q3k 7y ago
This is naive of any real-world, large scale use of RDBMS systems. Not to mention any actually distributed storage systems.
- addisonj 7y ago
  Agreed. You aren't going to get this to work with Wikipedia. But I could see some utility on the small scale (if it weren't easy to break with conflicts). A Discourse install for local discussion in an oppressive regime run across a few hundred commodity computers could be pretty hard to shut down and reasonable to have work for 50k people, the weakness here is that one bad actor could disrupt it just by flooding it with valid but conflicting writes (not to mention being able to just issue deletes against all the state if all actors are treated equally)
- pagutierrezn 7y ago
  And don't expect it to support normal transactions or participate in distributed ones
ryscheng 7y ago
great point! This was about as simple as it got in order to ship something quickly that was minimally intrusive. So the prototype as it is now buys you sequential consistency, but I want to detangle some great points you made. 1. Performance: The performance is probably going to be limited. In the sense that most databases have better performance than most blockchain implementations. So you're arbitrarily bottlenecking your writes. There are a ton of really great papers that show you can do WAY better if you are more careful with your distributed design. They usually involve getting into the weeds of redesigning the internals database. Just for fun, I really liked these papers: http://nms.csail.mit.edu/~stavros/pubs/hstore.pdf http://nms.csail.mit.edu/~stavros/pubs/hstore.pdf https://irenezhang.net/papers/tapir-tocs18.pdf https://irenezhang.net/papers/tapir-tocs18.pdf https://www.usenix.org/system/files/conference/nsdi17/nsdi17-wei-michael.pdf https://www.usenix.org/system/files/conference/nsdi17/nsdi17... So there is a ton of space to improve the performance here, I just wanted to build something quickly. 2. Authentication: Definitely an interesting technical challenge here. Most web apps are written assuming they are one-of-many servers accessing the database, but honest and has root authority. So how do we make sure that a validator has the authority to write to a particular table/row? Right now, this is still an open challenge and would love to engage the community on ideas here. There might be some way to tag/sign all tables/rows with creator, and do some write access control based on that? I'm not sure how many web apps would break though. 3. Write-conflicts: Even without good access control as specified above, most web apps are written assuming they are not the only web server to be accessing the database concurrently. So I would anticipate honest servers would have either good failover logic or be wrapping operations in SQL transactions if they really needed them to be atomic. Either way, I don't see any issues being serialized and appended to the log.

q3k 7y ago

This is naive of any real-world, large scale use of RDBMS systems. Not to mention any actually distributed storage systems.

addisonj 7y ago
Agreed. You aren't going to get this to work with Wikipedia. But I could see some utility on the small scale (if it weren't easy to break with conflicts). A Discourse install for local discussion in an oppressive regime run across a few hundred commodity computers could be pretty hard to shut down and reasonable to have work for 50k people, the weakness here is that one bad actor could disrupt it just by flooding it with valid but conflicting writes (not to mention being able to just issue deletes against all the state if all actors are treated equally)
pagutierrezn 7y ago
And don't expect it to support normal transactions or participate in distributed ones

ryscheng 7y ago

great point! This was about as simple as it got in order to ship something quickly that was minimally intrusive. So the prototype as it is now buys you sequential consistency, but I want to detangle some great points you made. 1. Performance: The performance is probably going to be limited. In the sense that most databases have better performance than most blockchain implementations. So you're arbitrarily bottlenecking your writes. There are a ton of really great papers that show you can do WAY better if you are more careful with your distributed design. They usually involve getting into the weeds of redesigning the internals database. Just for fun, I really liked these papers: http://nms.csail.mit.edu/~stavros/pubs/hstore.pdf http://nms.csail.mit.edu/~stavros/pubs/hstore.pdf https://irenezhang.net/papers/tapir-tocs18.pdf https://irenezhang.net/papers/tapir-tocs18.pdf https://www.usenix.org/system/files/conference/nsdi17/nsdi17-wei-michael.pdf https://www.usenix.org/system/files/conference/nsdi17/nsdi17... So there is a ton of space to improve the performance here, I just wanted to build something quickly. 2. Authentication: Definitely an interesting technical challenge here. Most web apps are written assuming they are one-of-many servers accessing the database, but honest and has root authority. So how do we make sure that a validator has the authority to write to a particular table/row? Right now, this is still an open challenge and would love to engage the community on ideas here. There might be some way to tag/sign all tables/rows with creator, and do some write access control based on that? I'm not sure how many web apps would break though. 3. Write-conflicts: Even without good access control as specified above, most web apps are written assuming they are not the only web server to be accessing the database concurrently. So I would anticipate honest servers would have either good failover logic or be wrapping operations in SQL transactions if they really needed them to be atomic. Either way, I don't see any issues being serialized and appended to the log.