7 ms·

What does it look like to actually write a program ontop of Matrix? Eg i have a key value store that i'd like to use Matrix (or IPFS/etc) for the P2P layer. Is

What does it look like to actually write a program ontop of Matrix? Eg i have a key value store that i'd like to use Matrix (or IPFS/etc) for the P2P layer. Is it a lot like IRC channels that you connect to, pushing data into them and getting data out from other clients in the same channel? Ie is it a series of messages? Or is it something more advanced; with built in conflict resolution or other P2P-y features?

SubGenius 6y ago
Hey, I can't comment about Matrix's P2P implementation as I haven't played with it yet. It's still a heavy work-in-progress I believe. Regular Matrix is pretty straight-forward - you join rooms and send events. Events get propagated to all participating/federated servers that are in the room. You watch for incoming events. The Matrix devs should be able to say more about their P2P implementation. :)
- ncmncm 6y ago
  Hey, there's no prob, with Bob! I can't tell what language Hummingbard is coded in. Hint? GPL with targeted relaxations is usually a good way to start. You can always relax the license more later if you exercise the foresight to get contributors to allow relaxations per your judgment; and, don't rely on any strict GPL dependencies. Being GPL but not able to use GPL libraries is a funny position to get into, but that's our world.
  - grey_earthling 6y ago
    A reminder: for software designed to be used via the network, the GPL behaves like a permissive licence — you're not distributing binaries to users, so the requirement to publish source code isn't triggered. If you want forks accessed via the network to publish source code, use the AGPL, otherwise you may as well just use the MIT licence.
southerntofu 6y ago
> is it a series of messages? Or is it something more advanced; with built in conflict resolution or other P2P-y features? From the client's perspective, matrix is just a basic stream of events (like XMPP). The magic happens on the server side where a room is replicated across servers and conflict-resolution voodoo is applied. That's also why synapse server is really really resource-intensive (gigabytes of RAM) compared to equivalent-in-features (apart from decentralized rooms) XMPP/ActivityPub servers ; dendrite is fairing better but i haven't benchmarked it yet.
- Arathorn 6y ago
  this isn’t really correct (but is a common misunderstanding). from the client’s perspective, a matrix room is a persistent timeline of events which you can long-poll (/sync) and navigate forwards and backwards (/messages). Events can also update key/value state associated with a room (eg that room’s name, topic, avatar, membership, or any other metadata), which you query via /state. This is not a stream of events like XMPP but a replicated DAG of events. Synapse’s RAM usage is a) way better these days, b) not due primarily to the merge resolution algorithm (state res), but due to slapping in-memory caches everywhere rather than being smarter on the db schema. Dendrite does 5x better by being smarter in the schema, and Conduit looks to be even better via cunning use of using key/value dbs for persistence (sled).
  - adkadskhj 6y ago
    For the key-value storage, would it make sense to use it as a "large" KV database? Eg, for sake of simplicity, imagine you're storing Git in Matrix (similar to what i'm doing, if i use Matrix). I've got two primary concepts: 1. A mutable set of pointers. 2. A series of key values, where the keys are content addresses, and the values are a byte array. My initial assumption would be to use the messages to post all newly created blocks and changes of pointers. Eg, Send a message with a new content address and the bytes. Send a message with a new address for a pointer. For a Git-like to work in that model, it just needs to slurp up any messages in the datastream. Making sure to also slurp in old messages properly that maybe are out of order (ie, reconnecting after being offline for a while). But being a Git-like, it's a fairly simple model. With that said, perhaps the metadata storage in Matrix would better suite some of this? I just imagine Matrix isn't intended to host TiBs of `/state` metadata. Thoughts?

SubGenius 6y ago

Hey, I can't comment about Matrix's P2P implementation as I haven't played with it yet. It's still a heavy work-in-progress I believe. Regular Matrix is pretty straight-forward - you join rooms and send events. Events get propagated to all participating/federated servers that are in the room. You watch for incoming events. The Matrix devs should be able to say more about their P2P implementation. :)

ncmncm 6y ago
Hey, there's no prob, with Bob! I can't tell what language Hummingbard is coded in. Hint? GPL with targeted relaxations is usually a good way to start. You can always relax the license more later if you exercise the foresight to get contributors to allow relaxations per your judgment; and, don't rely on any strict GPL dependencies. Being GPL but not able to use GPL libraries is a funny position to get into, but that's our world.
- grey_earthling 6y ago
  A reminder: for software designed to be used via the network, the GPL behaves like a permissive licence — you're not distributing binaries to users, so the requirement to publish source code isn't triggered. If you want forks accessed via the network to publish source code, use the AGPL, otherwise you may as well just use the MIT licence.

southerntofu 6y ago

> is it a series of messages? Or is it something more advanced; with built in conflict resolution or other P2P-y features? From the client's perspective, matrix is just a basic stream of events (like XMPP). The magic happens on the server side where a room is replicated across servers and conflict-resolution voodoo is applied. That's also why synapse server is really really resource-intensive (gigabytes of RAM) compared to equivalent-in-features (apart from decentralized rooms) XMPP/ActivityPub servers ; dendrite is fairing better but i haven't benchmarked it yet.

Arathorn 6y ago
this isn’t really correct (but is a common misunderstanding). from the client’s perspective, a matrix room is a persistent timeline of events which you can long-poll (/sync) and navigate forwards and backwards (/messages). Events can also update key/value state associated with a room (eg that room’s name, topic, avatar, membership, or any other metadata), which you query via /state. This is not a stream of events like XMPP but a replicated DAG of events. Synapse’s RAM usage is a) way better these days, b) not due primarily to the merge resolution algorithm (state res), but due to slapping in-memory caches everywhere rather than being smarter on the db schema. Dendrite does 5x better by being smarter in the schema, and Conduit looks to be even better via cunning use of using key/value dbs for persistence (sled).
- adkadskhj 6y ago
  For the key-value storage, would it make sense to use it as a "large" KV database? Eg, for sake of simplicity, imagine you're storing Git in Matrix (similar to what i'm doing, if i use Matrix). I've got two primary concepts: 1. A mutable set of pointers. 2. A series of key values, where the keys are content addresses, and the values are a byte array. My initial assumption would be to use the messages to post all newly created blocks and changes of pointers. Eg, Send a message with a new content address and the bytes. Send a message with a new address for a pointer. For a Git-like to work in that model, it just needs to slurp up any messages in the datastream. Making sure to also slurp in old messages properly that maybe are out of order (ie, reconnecting after being offline for a while). But being a Git-like, it's a fairly simple model. With that said, perhaps the metadata storage in Matrix would better suite some of this? I just imagine Matrix isn't intended to host TiBs of `/state` metadata. Thoughts?