lxpz

7 ms·

31.
by lxpz 5y ago
It's not a hard requirement, you might just have a harder time handling voluminous files as Garage nodes will in all case have to transfer data internally in the cluster (remember that files have to be sent to three nodes when they are
32.
by lxpz 5y ago
Exists, already linked in another comment : https://github.com/deuxfleurs-org/garage
33.
by lxpz 5y ago
I'd much rather have a system that doesn't try to solve human conflict by algorithms. Blockchains do this: you write code, the code becomes the law, and then there is no recourse when things go wrong. I deeply believe that we shou
34.
by lxpz 5y ago
No, we are simply using synapse-s3-storage-provider: https://github.com/matrix-org/synapse-s3-storage-provider
35.
by lxpz 5y ago
Garage is designed for self-hosting by collectives of system administrators, what we could call "inter-hosting". Basically you ask your friends to put a server box at their home and achieve redundancy that way.
36.
by lxpz 5y ago
True; we don't solve this. If there is a fault and data is stored only on node A and not nodes B and C, the view might be inconsistent until the next repair procedure is run (which happens on each read operation if an inconsistency is
37.
by lxpz 5y ago
You are absolutely correct concerning the trust model of Garage: any administrator of one node is an administrator of the entire cluster, and can manipulate data arbitrarily in the system. From an ideological perspective, we are strongly at
38.
by lxpz 5y ago
License-wise, Garage is AGPLv3 like many similar software projects such as MinIO and MongoDB. We are not lawyers, but in the spirit, this should not preclude from using it in commercial software as long as it is not modified. In the case wh
39.
by lxpz 5y ago
Very interesting, thanks for sharing.
40.
by lxpz 5y ago
Exposing Prometheus metrics is an ongoing work which we haven't had much time to advance yet. For now we can check on replication progress or overall cluster health by inspecting Garage's state from the command line.
41.
by lxpz 5y ago
I also had some things to answer to that comment which I'll put back here for the record. > I have been building distributed systems for 20 years, and they are not more reliable. They are probabilistically more likely to fail. It de
42.
by lxpz 5y ago
Yes, in which case you just make your cluster bigger or delete some data. Seems like a reasonable compromise to me. Garage has advanced functionnality to control how much data is stored on each node, and is also very flexible with respect t
43.
by lxpz 5y ago
Thanks! I'll give a bit more details about CRDTs, then. The thing is, if you don't have CRDT, the only way to replicate things over nodes in such a way that they end up in consistent states, is to have a way of ordering operations
44.
by lxpz 5y ago
Raft consensus apparently needs more round-trips than that (maybe two round-trips to another node per write?), as evidenced by this benchmark we made against Minio: https://garagehq.deuxfleurs.fr/documentation/design&#x
45.
by lxpz 5y ago
Thanks for the feedback! We will definitely be adjusting our pitch in further iterations.
46.
by lxpz 5y ago
Sure! CRDTs are often presented as a way of building asynchronous replication system with eventual consistency. This means that when modifying a CRDT object, you just update your local copy, and then synchronize in background with other nod
47.
by lxpz 5y ago
We ensure the CRDT is synced with at least two nodes in different geographical areas before returning an OK status to a write operation. We are using CRDTs not so much for their asynchronous replication properties (what is usually touted as
48.
by lxpz 5y ago
> Raft is for synchronized writes. It's slow in the case of a single-write being slow because you have to wait for an "ok" from replicas, which is a good thing (compared to async-replication in, say, cassandra/dynamod
49.
by lxpz 5y ago
We don't currently have a comparison with SeaweedFS. For the record, we have been developping Garage for almost two years now, and we hadn't heard of SeaweedFS at the time we started. To me, the two key differentiators of Garage o
50.
by lxpz 5y ago
For people interested in digging in, we have an interesting benchmark here showing how it significantly outperforms Minio in the geo-distributed setting: https://garagehq.deuxfleurs.fr/documentation/design/benchmar
51.
Garage, our self-hosted distributed object storage solution(garagehq.deuxfleurs.fr)
409 points by lxpz 5y ago|130 comments
52.
by lxpz 5y ago
It's "closure" in the mathematical sense, not in the functionnal programming sense. It would have been clearer with the full term: "transitive closure" instead of just "closure".
53.
by lxpz 5y ago
This should be integrated with Cargo to reduce the size of the target directories which are becoming ridiculously large.
54.
by lxpz 5y ago
Thank you for stating this. I absolutely agree with you, and this is what we are trying to do with the Deuxfleurs collective which is also getting funded by the EU to work on our project Garage [0, 1]. We are working on making self-hosting
55.
by lxpz 5y ago
Hey, thanks for sharing your perspective. I'm sorry you interpreted my comment in that way, I did not mean to say that TeXmacs had shortcomings in the way it imports LaTeX, in fact I do believe it does its best given the constraints it
56.
by lxpz 5y ago
I've successfully used TeXmacs to take more than 100 pages of notes live during a math class. The keyboard shortcuts used to build math formulas are extremely effective and quite easy to learn. I still use it from time to time when I n
57.
by lxpz 6y ago
Algorithms that tolerate Byzantine failures generally require t<n/3, i.e. at least 4 nodes to tolerate 1 byzantine fault. This is in fact required for primitives as simple as reliable broadcast. Systems that use leader election such
58.
by lxpz 6y ago
I'm not the best qualified to answer this question as I do spend a lot of time reading about programming languages in general, and even though I was able to grasp Rust's async/await very fast, I probably owe it to previous kn
59.
by lxpz 6y ago
The advantage of async tasks for structured concurrency lies in task cancellation, which is intrinsically linked to the notion of "task ownership". If you are using an OS thread to offload some task, and then realize that you don&
60.
by lxpz 6y ago
That .NET syntax using Task.WhenAll seems quite bad, which might be part of the reason why not many people bother (disclaimer: I don't do C# or ASP.NET). In Rust it would be: let p, c, m = join!( GetUserPermission(usernam

More ›