3 ms·

I've been toying around an architecture that sets things up such that the data for each user is actually stored with each user and only materialized on demand,

I've been toying around an architecture that sets things up such that the data for each user is actually stored with each user and only materialized on demand, such that many data leaks would yield little since the server doesn't actually store most of the user data. I mention this since this sorts of leaks are inevitable as long as people are fallible. I feel the correct solution is to not store user data to begin with. some problems I've identified: 1. suppose you have x users and y groups, of which require some subset of x. joining the data on demand can become expensive, O(x*y). 2. the main usefulness of such an architecture is if the data itself is stored with the user, but as group sizes y increase, a single user's data being offline makes aggregate usecases more difficult. this would lend itself to replicating the data server side, but that would defeat the purpose 3. assuming the previous two are solved, which is very difficult to say the least, how do you secure the data for the user such that someone who knows about this architecture can't just go to the clients and trivially scrape all of the data (per user)? 4. how do you allow for these features without allowing people to modify their data in ways you don't want to allow? encryption? a concrete example of this would be if HN had it so that each user had a sqlite database that stored all of the posts made per user. then, HN server would actually go and fetch the data for each of the posters to then show the regular page. presumably here if a data of a given user is inaccessible then their data would be omitted.

yellow_postit 6mo ago
I’ve always liked this idea but I think it eventually ends back up with essentially our current system. Users have multiple devices so you quickly get to needing a sync service. Once that gets complex enough, then people will outsource to a third party and then we are back to a FB/Google/Apple sign in and data mgmt world.
requilence 6mo ago
Disclosure: I work at Anytype. This is the architectural bet we took, so I'll answer your four problems directly. Premise: treat it as certain that the server will eventually be compromised, subpoenaed, or misconfigured. So the server must hold nothing that can be decrypted or linked to a specific user's content. Users hold their own encryption keys, the server stores ciphertext, and there is no UUID→identity mapping at the sync layer. Sync runs over any-sync, which is peer-to-peer-capable; intermediate nodes see ciphertext. On your four problems: 1. O(x*y) joins - pushed to the client, because the server can't decrypt enough to do them. 2. Offline members - eventual-consistency sync and CRDT. 3. Client-side theft - if an attacker has the user's keys, they have the data. Intentional: no server-side gate to break means no server-side gate to exfiltrate at scale. We're considering optional 2FA at the infrastructure layer as an additional barrier to data retrieval. 4. Unwanted modifications - content is signed with user keys and validated on read. Real cost is on the product side: no server-side AI over your notes, no server-side full-text search, slower cold-start, and harder to build product analytics (no access to user data). Granular ACLs are also harder — permissions are enforced by key possession, so revoking access often requires key rotation rather than a permission-flag change. But the exact bug this post is about (a server endpoint that maps a public UUID to an email) is structurally impossible in this model, because there's no such mapping on our servers to misuse. any-sync and our data format (any-block) are MIT, if you want to poke at how it works: https://github.com/anyproto https://github.com/anyproto

yellow_postit 6mo ago

I’ve always liked this idea but I think it eventually ends back up with essentially our current system. Users have multiple devices so you quickly get to needing a sync service. Once that gets complex enough, then people will outsource to a third party and then we are back to a FB/Google/Apple sign in and data mgmt world.

requilence 6mo ago

Disclosure: I work at Anytype. This is the architectural bet we took, so I'll answer your four problems directly. Premise: treat it as certain that the server will eventually be compromised, subpoenaed, or misconfigured. So the server must hold nothing that can be decrypted or linked to a specific user's content. Users hold their own encryption keys, the server stores ciphertext, and there is no UUID→identity mapping at the sync layer. Sync runs over any-sync, which is peer-to-peer-capable; intermediate nodes see ciphertext. On your four problems: 1. O(x*y) joins - pushed to the client, because the server can't decrypt enough to do them. 2. Offline members - eventual-consistency sync and CRDT. 3. Client-side theft - if an attacker has the user's keys, they have the data. Intentional: no server-side gate to break means no server-side gate to exfiltrate at scale. We're considering optional 2FA at the infrastructure layer as an additional barrier to data retrieval. 4. Unwanted modifications - content is signed with user keys and validated on read. Real cost is on the product side: no server-side AI over your notes, no server-side full-text search, slower cold-start, and harder to build product analytics (no access to user data). Granular ACLs are also harder — permissions are enforced by key possession, so revoking access often requires key rotation rather than a permission-flag change. But the exact bug this post is about (a server endpoint that maps a public UUID to an email) is structurally impossible in this model, because there's no such mapping on our servers to misuse. any-sync and our data format (any-block) are MIT, if you want to poke at how it works: https://github.com/anyproto https://github.com/anyproto