4 ms·

After just giving scathing remarks to another database system, I'm pleasantly surprised by LF! LF looks like a well-reasoned project with a solid direction tha

After just giving scathing remarks to another database system, I'm pleasantly surprised by LF! LF looks like a well-reasoned project with a solid direction that is unique and applicable. I do disagree with some of its design choices (DAGs aren't very scalable, full replication versus partial/sharded, etc., and GPL instead of MIT - which my decentralized DB, http://gun.eco http://gun.eco is MIT/Apache2), but for the problem it is trying to solve, it seem like you've made the right choices for LF and its use cases. Congrats and keep it up! Excited to hear about this over time. :)

heyoni 7y ago
Oh hi Mark!
twokei 7y ago
Interesting - would love your thoughts, why is it that DAGs aren't very scalable?
- marknadal 7y ago
  DAGs aren't very scalable, because like with git, you have to store the whole history. In some scenarios, you can rebase/snapshot to clean up history, but these usually require a type of centralization or consensus, which defeats the point of using something like git. As a result, DAGs can only be used to replace a subset of apps/tools out there. The most important/used apps, though, are indexed lists. Things like: - Google rankings - Reddit homepages - AirBnB listings - Ubers nearby If you're updating a geo-index 100s of times a second, like in the case of cars' GPS locations, then you're just wasting resources with a DAG that you'll never use and wind up bottlenecking the system, preventing scaling beyond a certain threshold. I've dealt with this in practice, and it was no fun. We switched off DAGs, and our biggest production deployment now has been with 15 million monthly users. This is far far far beyond the scale of any of the previous systems.
tasubotadas 7y ago
I think that as it currently stands, the GPL3 licence makes the project unusable. There aren't any externally packaged drivers with a different licence, and you can't include their native code in your commercial projects without having to share the code due to the inclusion of the GPL3 licenced code.
- api 7y ago
  We are considering a license more like that adopted by CockroachDB for all our projects in the future. Haven't decided on the particulars yet.
olah_1 7y ago
Really strange that this reply is greyed out. I specifically came to the comments hoping that someone compared this project to Gun. Who better than Mark?

heyoni 7y ago

Oh hi Mark!

twokei 7y ago

Interesting - would love your thoughts, why is it that DAGs aren't very scalable?

marknadal 7y ago
DAGs aren't very scalable, because like with git, you have to store the whole history. In some scenarios, you can rebase/snapshot to clean up history, but these usually require a type of centralization or consensus, which defeats the point of using something like git. As a result, DAGs can only be used to replace a subset of apps/tools out there. The most important/used apps, though, are indexed lists. Things like: - Google rankings - Reddit homepages - AirBnB listings - Ubers nearby If you're updating a geo-index 100s of times a second, like in the case of cars' GPS locations, then you're just wasting resources with a DAG that you'll never use and wind up bottlenecking the system, preventing scaling beyond a certain threshold. I've dealt with this in practice, and it was no fun. We switched off DAGs, and our biggest production deployment now has been with 15 million monthly users. This is far far far beyond the scale of any of the previous systems.

tasubotadas 7y ago

I think that as it currently stands, the GPL3 licence makes the project unusable. There aren't any externally packaged drivers with a different licence, and you can't include their native code in your commercial projects without having to share the code due to the inclusion of the GPL3 licenced code.

api 7y ago
We are considering a license more like that adopted by CockroachDB for all our projects in the future. Haven't decided on the particulars yet.

olah_1 7y ago

Really strange that this reply is greyed out. I specifically came to the comments hoping that someone compared this project to Gun. Who better than Mark?