6 ms·
One other major downside of end-to-end encryption: everything has to be done client-side, and the server becomes very close to a dumb pipe and block storage. (T
by chrismorgan 4y ago
One other major downside of end-to-end encryption: everything has to be done client-side, and the server becomes very close to a dumb pipe and block storage. (The only parts the server can do anything with are those that aren’t encrypted, so the explicit goal of E2EE is to reduce the server to a dumb pipe and block storage.) Got ten gigabytes of stuff you want to search through? Without E2EE, the server can implement search and store an index, so your search can be done on a fast server, and you get a response in 100ms taking maybe 10kB of network traffic. With E2EE, all the server can do is hand you encrypted blobs. Maybe you have to download the full ten gigabytes, or maybe you also store the index and can download just that, which is probably smaller, but now you’ve got to do complicated index synchronisation when making changes (complicated because you want to minimise the amount of extra sync required). So your search now runs on your probably-slow phone processor and storage (we don’t all use iPhones), maybe takes a few seconds, and requires that you have downloaded (and store) maybe 10GB, maybe only 100MB, but a whole lot more.
It kinda amuses me that mobile spent a long time moving both storage, computation and network traffic from the client to the server as much as possible (because the mobile devices had too little of each), and E2EE significantly forces it back the other way. One would hope this would mean that you’d get more just-software rather than software-as-a-service, but no, mostly you just end up with the worst of both worlds.
I’m glad that they mentioned “In the end, trust is Still Required™”, and the last sentence of that section is good stuff. But I like to state it more strongly in two ways: if the software is served over the web or to any platform with automatic updates, E2EE is fundamentally broken; and, first-party end-to-end encryption is snake oil.
- thrwawy74 4y agoWe've been hearing about homomorphic encryption for years, but I haven't seen it applied in a very practical way.
- polygamous_bat 4y agoWhen I last looked into fully homomorphic encryption in 2017, the algorithms were still not at an applicable place. Sure, we may have polynomial time algorithms for FHE, but it doesn't help anything in practice if those polynomials are not reasonable themselves.
- Anunayj 4y agoThere was a pretty neat recent post on a pratical Homomorphic Encryption project [1]. Maybe not practical, but pretty cool nevertheless. 1. https://news.ycombinator.com/item?id=31668814 https://news.ycombinator.com/item?id=31668814
- coldtea 4y ago>One other major downside of end-to-end encryption: everything has to be done client-side, and the server becomes very close to a dumb pipe and block storage. That sounds like a feature...
- chrismorgan 4y agoIt tries to be, but in practice as I say you mostly just end up with the worst of both worlds, because of perverse incentives (the software provider wants to charge a monthly figure and doesn’t want their server to be fungible). Anyway, it’s still much more commonly a limitation than an advantage. Perhaps the most obvious example: I want to be able to search through 10GB of emails without needing to download them all (or a not-much-lighter index), because downloading it takes a long time, costs a fair bit (traffic isn’t generally), and requires that I have that much spare space on whatever device I’m using at the time. The fact of the matter is that E2EE in all its forms is consistently inconvenient—and double racheting makes it even worse than earlier non-key-cycling techniques. In some situations it may be worth the inconvenience, but I wish there was more acknowledgement of the fact that it’s seriously not the ideal goal, and I wish the innovation would head in the direction of proper and deliberately thorough decentralisation instead, because if you run your own server, you don’t need E2EE: its purpose disappears completely, leaving only the inconvenience. (Reduce it instead to just encryption at rest and transport encryption, which are still useful and entirely sufficient.)
- jdvh 4y agoI know I'm biased because I'm working on an E2EE todo/planning app[1], but over the years I've become convinced that E2EE apps are the future. All the syncing code you have to write anyway if you want your app to work offline. Despite all the buzz about big data, most individuals and most companies generate only a modest amount of data. Even when you have a long tail of archived data (e.g. emails that go back 10 years) the key benefit is in having all recent data with you, available offline, and end-to-end encrypted. Self-hosting is a lot more difficult in practice than in theory. Somebody has to make backups, maintain a server architecture with redundancy, apply security patches, etc. For hobby purposes you can get away with just not doing any of that, but for businesses and people who don't want to tinker with their own servers outsourcing all this makes a ton of sense. I don't agree with your claims that traffic is expensive. In any case, downloading data once and syncing subsequent deltas is still a lot more data efficient than roundtripping JSON for every user action. [1] https://thymer.com https://thymer.com
- ilammy 4y ago> With E2EE, all the server can do is hand you encrypted blobs. Encrypted search is a thing. You need to know in advance what kind of queries you want to do against your DB, but there are off-the-shelf solutions for encrypted search which do not involve homomorphic encryption. Here's an overview: https://www.cossacklabs.com/blog/secure-search-over-encrypted-data-acra-se/ https://www.cossacklabs.com/blog/secure-search-over-encrypte...
- yunohn 4y agoIt is possible, but it isn’t very feasible right now. Maybe in the future?
- woojoo666 4y agoThe top-level comment should at least acknowledge that it's possible. We shouldn't just put up our hands and say "it"s impossible/impractical to have E2EE data without overloading our clients", we should be pushing companies to prioritize research into techniques like encrypted search, so that the "future" can come sooner
- yunohn 4y agoI mean it’s like Crypto enthusiasts claiming their coins are green because they can theoretically run on renewables, when the reality is that the majority of miners use fossil fuel sources, and either way the energy is wasted. Similarly, unless we desire E2EE for everything, it’s both algorithmically and computationally more feasible to stick to centralised unencrypted formats.
- _8j50 4y agoHomomorphic search is a thing, new "web3" stuff shouldn't even store on a server, servers become app hosters and routers basically.
- crabmusket 4y agoI pity the one who has 10GB of notes. It's a fair point though. E2EE is a tradeoff that may make sense in some cases but not others, and yes, it doesn't do away with the need for trust.
- basilgohar 4y ago> It kinda amuses me that mobile spent a long time moving both storage, computation and network traffic from the client to the server as much as possible (because the mobile devices had too little of each), and E2EE significantly forces it back the other way. One would hope this would mean that you’d get more just-software rather than software-as-a-service, but no, mostly you just end up with the worst of both worlds. This kind of pendulum like, tick-tocking back and forth between things like fat clients/thin-clients, dumb termials/PCs, centralized vs. decentralized (well, cloud-based [i.e., someone else's computer]) applications, and so on, are part-and-parcel of computing that goes as far back as possible. This is just another "tock" in that cycle. Technology happens in leaps and bounds and not at all uniformly, so this is a natural result. I personally am for the best of both worlds in whatever makes the most sense, but definitely skew towards privacy, security, and user-authority.
- lasereyes136 4y ago> This kind of pendulum like, tick-tocking back and forth between things like fat clients/thin-clients, dumb termials/PCs, centralized vs. decentralized (well, cloud-based [i.e., someone else's computer]) applications, and so on, are part-and-parcel of computing that goes as far back as possible. This is just another "tock" in that cycle. True, this is because all software design and architecture is a set of trade-offs and each trade-off has a downside. Software design and architecture is also very fad-ish in that "the it thing" changed and everyone rushes to use it in order to keep their skills competitive. The fads of IT is a large part of the churn that makes low quality software.
- eduction 4y agoI see this kind of pushback from engineers quite often when advocating for privacy first technologies. I think this comment is a good example of the sort of reflexive dismissal many developers have to changing how they work, from being asked to move a bit away from the dominant paradigm of the last 20-30 years (cleartext private data on servers). 1. The idea that it’s impractical to download the data. We are talking about searching text notes. 10gb? That’s nearly 7 million pages of notes. Doesn’t seem a reasonable figure. Even hundreds of megs of text would require a lot of time to acquire and even that would rarely need to be downloaded in one go (new device). 2. The idea that tech that tends to live on the server is somehow magical. Search would take “a few seconds” on a “slow” phone processor you say. Phone professors are incredibly fast these days, but in any case, since when is text search slow? I feel like many developers treat text search as a scary magical black box but it’s rather straightforward. Read about inverted indexes and consider that the servers of the mid 90s were serving at least tens of thousands of text searchers apiece with processors probably slower than what’s in your phone. There are libraries you can use locally to make text search pretty trivial. Apple supports text search (spotlight) across the data and apps on your phone. It’s not rocket science. 3. The idea that anything other than conceptually perfect encryption is useless. “first-party end-to-end encryption is snake oil” - assuming you mean crypto where the user does not handle the keys directly and a first party causes them to be generated, first party e2ee describes some of the biggest privacy wins of the last 15 years - iMessage, FaceTime, Signal, WhatsApp.
- sleepybrett 4y agoI generally agree with your points but would point out that when I exported all my evernote data a few years ago it came to about 15gb, mostly because I use a lot of photographs and diagrams in my notes. I don't know anything about this note platform but if it allows multimedia embedding then the data can balloon quite rapidly.
- remram 4y agoThe photographs can be retrieved as needed while still being encrypted and not impeding text search.
- sipjca 4y agoI think this depends on if homomorphic encryption is feasible and fast. This would alleviate most of the concerns here.
- jeroenhd 4y ago> your probably-slow phone processor and storage A $100 Android (terrible buy by the way, get a second hand top model instead, unless you're fussy about new features) still ships with a ridiculously fast CPU. The Motorola E6 Play comes with a MediaTek MT6739, a four year old bottle of the barrel phone CPU, and can still decrypt data at about 135MB/s. This thing is slow even in term of modern $100 Android devices, but it's not _that_ slow! Stuffing everything behind E2EE with on-the-fly decryption without giving it a second thought is definitely a bad idea. Building an index isn't that hard, though, especially if you're probably not syncing your personal account between 20 devices if you're on a device like that. A good index may use tens of megabytes at most, which fits into the minimum RAM allocated towards an app on modern Android already. Traversing such an index shouldn't take long either. Sure, decrypting the found results may take a few hundred milliseconds, but the search itself shouldn't be a problem, should it? I feel like we've all been spoiled by high performance electronics to the point where we don't realize the kind of power these devices have. SD cards come with their own ARM cores running at 100MHz or even multiples of that depending on the designed data rate; that's more than enough to do some indexing! The difficult part is getting access to all that power. It means getting rid of the nice and handy frameworks (such as the Android SDK, high level programming languages such as Kotlin, and so on) and getting back to bare metal. Even writing this stuff in Rust may come with severe performance penalties because Rust is full of unexpected and unintentional issues; something like C++20 or Zig would be much better (or C if you really hate writing software). I have an old tablet, a dual core 1GHz CPU. I installed LineageOS on it to get it to Android 7 so basic web functionality still works without too much effort. The flash storage of this thing is known to break down and it shows. Booting up is slow, animations stutter, the settings app hangs. You know what's not slow? Opening a web page in a web view. Latest version of Chrome, inside a minimal browser UI written in native code, runs great. Sure, video playback is crap because the GPU was bad even when it was new, but this thing can fetch megabytes of JSON over HTTP3 like a damn champ if you write software right. Processing is great, it just sucks at rendering (because the drivers have been monkey patched four Android versions above the last supported one). The experience is terrible because the software is heavy and written to make developers' lives easier, not to be efficient. Don't be fooled by the electron applications 300MB note taking apps, "slow" chips are still very capable. Imagine how long even a mid-range phone's battery would last if apps were written to be as quick on my shitty tablet as the Lightning Browser is!
- operator-name 4y ago> if the software is served over the Web or to any platform with automatic updates, E2EE is fundimentallu broken; and, first-party end-to-end encryption is snake oil. This statement, and this way or extremist thinking is the classic no true soctsman fallacy. Taken to the extreme it goes back to trusting trust, and not everyone is a cryptographer who wants to verify the encryption is strong and implimented correctly. Privacy is a sliding scale and people have different threat models. The statement is also so broad that applying it as a specific issue with E2EE seems foolish - after all the same could be said for any website. You're trusting that Amazon doesn't maliciously update their website to start sending passwords or credit card numbers in plaintext. Hopefully this example demonstrates why encryption in transit is useful. Just as encryption in transit has tradeoffs (mainly caching), so does E2EE. The benifit of E2EE is data at rest - the server owner cannot suddenly decide to decrypt your data for their means. As you've outlined, for strong privacy this requires tradeoffs (local calculation, more complex systems). As you've also outlined E2EE is also not a magic bullet (trust the client), but that doesn't suddenly discount all of the benifits.
- kmeisthax 4y agoFirst party E2EE has some value even under a "trusting trust" scenario: it lets service operators keep rogue employees from accessing your data. To compromise your data requires pushing a software update, which on pretty much every platform[0] creates a paper trail of cryptographic signatures leading back to the company's signing keys. If someone finds out about user data being stolen through a modified version of the app, that can be traced back to any developer who had access to key material. Yes, if the organization itself decided to compromise its own scheme, E2EE cannot stop that... but again. That creates evidence and paper trails. The kinds of people with the power to do this want plausible deniability; the last thing they want is mathematical proof that they screwed their own customers on purpose. Same with that rogue employee: they don't want to be known as the guy who signed spyware. [0] Yes, including sideloading-friendly ones. If your rogue update isn't signed it will trip a bunch of scary warnings at install time.
- operator-name 4y ago
- teleforce 4y agoPersonally I firmly believe that in the near future local-first software is going to be the main paradigm of Internet based applications, and end-to-end encryption will be its second nature [1]. From the seminal paper, "Local-first apps, on the other hand, have better privacy and security built in at the core. Your local devices store only your own data, avoiding the centralized cloud database holding everybody’s data. Local-first apps can use end-to-end encryption so that any servers that store a copy of your files only hold encrypted data that they cannot read". [1] Local-first software: You own your data, in spite of the cloud: https://www.inkandswitch.com/local-first/ https://www.inkandswitch.com/local-first/
- brnt 4y agoRemove the (implicit) third party server from your argument and it looks like your objection disappears. I see no need to involve others in storing my notes.
- candiodari 4y agoWell this is what the TPM is supposed to solve. You want to trust the developer (after all if your client app is compromised, end-to-end encryption won't save you), but you don't want to trust the server, and physical access to the server doesn't get you any access to the data.