carlsverre

5 ms·

1. 31.▲
   by carlsverre 2y ago

   I think you could! Graft just provides transactions over sets of pages. As long as you can model your data in a compatible way it should work with Graft. A better question is how easy the system allows extensions to intercept filesystem ope
2. 32.▲
   by carlsverre 2y ago

   Exciting to see all the continued energy around Graft today! If you want to continue the conversation feel free to join the Discord [1] or post a discussion on GitHub [2]. I'll be at Antithesis BugBash [3] for the next few days talking
3. 33.▲
   by carlsverre 2y ago

   Thank you! Glad we see the world the same way! I've also always wanted a general purpose sync layer that provided a simple consistency model to build on top of. And I made Graft open source to enable exactly that! Deploy it anywhere! J
4. 34.▲
   by carlsverre 2y ago

   Thanks! And great idea. I haven't deep dived into DuckDB's storage format yet, however I know that they split up the data into row groups which internally are columnar. I'm not sure how that maps to the pages in the file. If
5. 35.▲
   by carlsverre 2y ago

   There is simply so much to talk about here! Thanks for such an excellent question. First, a caveat: Graft currently has no permissions. Anyone with access to the Graft PageStore and MetaStore can read/write to any volume. This is obvio
6. 36.▲
   by carlsverre 2y ago

   Finally got a chance to watch this. LMAO so good!
7. 37.▲
   by carlsverre 2y ago

   Thank you!
8. 38.▲
   by carlsverre 2y ago

   Thank you! You're 1000% correct, Graft's approach is not a good fit for high write contention on a single Volume. Graft instead is designed for architectures that can either partition writes[^1] or can represent writes as bundled
9. 39.▲
   by carlsverre 2y ago

   Woah that's awesome. Using Effect to represent mutations is brilliant! Any chance your project is public? I'd love to dig into the details. Alternatively would you be willing to nerd out on this over a call sometime? You can reach
10. 40.▲
    by carlsverre 2y ago

    Hey Phil! Also you're 100% right. I should use a different word than scale. I was meaning scale in the sense that they "scale" durability and availability. But obviously it sounds like I say they are scaling performance. I&#x
11. 41.▲
    by carlsverre 2y ago

    Per the Rust FAQ [1]: > The Apache license includes important protection against patent aggression, but it is not compatible with the GPL, version 2. To avoid problems using Rust with GPL2, it is alternately MIT licensed. The Rust API gu
12. 42.▲
    by carlsverre 2y ago

    Thank you!! From an extremely brief scan, it appears that Erlang wrappers around SQLite should be able to use the Graft SQLite extension just fine. Alternatively, it would be reasonably straight forward to wrap Graft Client (Rust library) d
13. 43.▲
    by carlsverre 2y ago

    This is a great point. Solutions like Graft which involve syncing data to the edge work poorly when the dataset size is too large and not partitioned enough to be consistency partial. This is why Graft isn't just focused on client sync
14. 44.▲
    by carlsverre 2y ago

    Thank you! And I agree haha. I love writing SQLite VFS's. In SQLSync, the major issue was precisely what you describe: pulling in the full snapshot to replay. This is the main driver behind the "partial" aspect of the Graft d
15. 45.▲
    by carlsverre 2y ago

    Glad you enjoyed Graft! The system your describing sounds very cool! It's actually quite similar to SQLSync. The best description of how SQLSync represents and replays intentions (SQLSync calls them mutations) is this talk I did at Was
16. 46.▲
    by carlsverre 2y ago

    Thank you! I made them myself using http://excalidraw.com/ .
17. 47.▲
    by carlsverre 2y ago

    Just to make sure I understand correctly, would you agree that if clients always synchronously commit (i.e. wait until the MetaStore commit returns ok) before acknowledging any commit locally, the client will experience Strict Serializabili
18. 48.▲
    by carlsverre 2y ago

    Hey friends! Author of Graft here. Just want to say, huge thanks for all the great comments, stars, and support. Feels really nice to finally be building in public again. I'm going to force myself to sign off for the evening. Will be a
19. 49.▲
    by carlsverre 2y ago

    You're right - it's a bit confusing! I took a crack at explaining it in the blog post under the Consistency section: https://sqlsync.dev/posts/stop-syncing-everything/#consisten... The key idea is that i
20. 50.▲
    by carlsverre 2y ago

    For now, Graft suffers from the same minimum payload size of 4KB. However, there are ways to eliminate that. I've filed an issue to resolve this in Graft ( https://github.com/orbitinghail/graft/issues/35 )
21. 51.▲
    by carlsverre 2y ago

    Thanks for bringing that up! Cloud-Backed SQLite (CBS) is an awesome project and perhaps even more importantly a lot more mature than Graft. But here is my overview of what's different: CBS uses manifests and blocks as you point out. T
22. 52.▲
    by carlsverre 2y ago

    It's been a fun project to work on, and I'm excited to see how deep this rabbit hole goes. :) I'd love to hear how that goes! I haven't tried getting the SQLite extension running on mobile yet, so any help there would be
23. 53.▲
    by carlsverre 2y ago

    Thank you! I'm extremely excited and interested to explore applying Graft to solutions outside of SQLlite/SQLSync. That was a driving factor behind why I decided to make it more general. But you're absolutely right, I'm
24. 54.▲
    by carlsverre 2y ago

    Thank you! Generally Turso has focused on operating more like a traditional network attached backend. Although that has changed recently with libsql and embedded replicas. I think at this point the main delta is they use traditional wal bas
25. 55.▲
    by carlsverre 2y ago

    ASAP :)
26. 56.▲
    Weird Rust Expressions(github.com)

    3 points by carlsverre 2y ago|0 comments
27. 57.▲
    by carlsverre 2y ago

    Inspired by this blog post I just added an `#[implicit]` field feature to the `thiserror` crate. It makes it easy to automatically annotate errors with things like code location (per this blog post), a timestamp, or a backtrace without requ
28. 58.▲
    by carlsverre 2y ago

    This is really cool. I'd love to see this done for the movie "Lola rennt" (Run Lola Run) which is studied for it's color symbolism throughout the film.
29. 59.▲
    by carlsverre 2y ago

    I had so much fun finishing this. Much more interesting than other 2048 variants I've tried. I got a score of 9341.
30. 60.▲
    by carlsverre 2y ago

    Also... can you expand this to comments? Doesn't seem too different from logs. And then add in the fairly obvious auto-PR logic with the ability to respond to PR comments and now you're eliminating maintenance burden. Caveat: I du