9 ms·

Show HN: Etebase – An open source and end-to-end encrypted Firebase alternative

308 points by tasn 6y ago

tasn 6y ago
Excited to finally post it on HN. It's something I've been working on for the last year, though in a sense, for much longer! Etebase is an open-source SDK and backend-as-a-service for building end-to-end encrypted applications. Think Firebase but end-to-end encrypted. More users care about the privacy and security of their data every day, and encrypted applications are becoming mainstream. However, talking with developers, many get encryption wrong, and they don't even realise it. As good cryptography and bad cryptography look identical to the untrained eye. The idea behind Etebase is to make an easy-to-use API to enable developers to build encrypted apps, and it's based on our learnings from building EteSync[2] over the years. There's still a lot more to get done, but Etebase is already used in GNOME, KDE, Tasks.org, EteSync, and more. With libraries available for Rust, JavaScript/TypeScript, Java, Python and C/C++. I hope Etebase will enable a new age of privacy-first and end-to-end encrypted applications, and that our data will no longer be used to track, analyze and manipulate us. Your data, yours only. Let's end-to-end encrypt everything! I'd love to hear about your experience building encrypted applications. Got any questions? Suggestions? Please let me know! Docs: https://docs.etebase.com/ https://docs.etebase.com/ [1] https://libsodium.org https://libsodium.org [2] https://www.etesync.com https://www.etesync.com
- slooonz 6y ago
  Is there any technical document describing how it works ? For example, order-preserving indices (for range queries) are often an issue in end-to-end databases ; did your design allow to you to provide them, or did you just consider solving this problem out-of-scope ?
  - tasn 6y ago
    There are docs here: https://docs.etebase.com/ https://docs.etebase.com/ Though they mostly focus on how to use rather than how it works (though we are making progress on that section). As for your specific question: we currently don't solve that, but we plan on doing it in the near future. Our approach will be to maintain a client-side index that will be synced across devices. Etebase already utilities to maintain consistency, so this can be done safely. This question, and ones like it, are exactly why we created Etebase. Building encrypted applications is fundamentally different and comes with a lot of challenges. We plan on continue building all of the tools needed to develop using this different paradigm.
- joel_dice 6y ago
  Nice work! I spent about six months pursuing the exact same idea about five years ago with the hope of building a company around it. I got reasonably far with a prototype, but gave up on the business side of things after doing more extensive market research and interviews. One of the people I talked to had been involved with a startup called Adrenaline Mobility which did something very similar, but ultimately ran out of runway and was acquired by Twitter. He told an interesting story about how they (over-)built a highly scalable service but had a really hard time selling it. Those who lack the training to know the difference between good security and hand waving didn't see the value, and those who did have the training and skills didn't trust a third party to do it right anyway. Nonetheless, a lot has changed in the past five years, and end-to-end encryption is more widely appreciated now. I hope you have better luck with the commercial side of things; this looks like a very promising start.
  - tasn 6y ago
    Thanks a lot! Things have indeed changed, and I feel like the public is really starting to understand the dangers with having their data exposed. I've been running EteSync for years now, and it's still growing. I'd love to have a chat with you to learn from your experience if you are willing! My email is tom at etebase, please drop me a line (or let me know how to best reach you). Thanks again!
    - g_p 6y ago
      This might be very good timing for your product - the European Data Protection Board has just out their first guidance after the Schrems II case. It seems to firmly point towards cloud services where the provider can access the data in plaintext as being "unable to have the risks mitigated" from the perspective of international transfers. Perhaps a straightforward E2EE toolbox could help companies implement their products rapidly so they remain complaint. Clearly this won't help fix issues like Microsoft 365 or GSuite (which need access to the plaintext for email and similar), but it might help some types of SaaS to thicken up their client application and prevent the backend having access to unencrypted data, thus making the transfer permitted. I just want to echo the above comment around how most customers don't have the knowledge to understand the benefit of this, but hopefully this is changing as we see more strict enforcement of penalties for data breaches. Data is fast becoming a liability you don't want to have the ability to see, and systems like yours offer usable solutions for those who don't understand all the technology, but need a solution.
- ChrisMarshallNY 6y ago
  I sincerely wish you luck with this. I wrote something similar, a couple of years ago[0], but it doesn't include encryption; it merely gives a place to add encryption. I didn't want to deal with the legalities of included encryption, and I think others can do far better than I (but it is quite secure, nonetheless[1]). It was really done, just to "retool" my architectural and engineering skills, as I was pivoting from being a manager, back to being a developer. I am now using it as a backend for an app under development (currently closed-source). It has been working a treat. But after the app has been released, I may encourage the folks to look at alternatives for the backend. I am not really into maintaining a server system. I like to stay on the app side of things. [0] https://riftvalleysoftware.com/work/open-source-projects/#baobab https://riftvalleysoftware.com/work/open-source-projects/#ba... [1] https://riftvalleysoftware.com/BAOBAB/PDFs/Security.pdf https://riftvalleysoftware.com/BAOBAB/PDFs/Security.pdf (Downloads a PDF)
  - tasn 6y ago
    Thank you for your kind words! I think the encryption is the key differentiator here, at least for me. I don't want my data saved exposed on someone else's server, AKA the cloud.
    - ChrisMarshallNY 6y ago
      The deal with mine, is the encryption is left to the client. The server just stores a binary "thing." That "thing" can be an encrypted envelope, managed by the client[s]. Since it's a closed-loop system, then a key could be shared programmatically, outside the purview of the server.
      - tasn 6y ago
        Yeah, in Etebase the server also just sees "things", though it tries to make it easy on the client to only get what's changed and etc.
        fabrice_d 6y ago
- Tepix 6y ago
  Congrats! Kudos for making the server component open source, something your competitors aren't doing.
- fakedang 6y ago
  Flutter please lol Anyways, great work!
Aachen 6y ago
Cool to see this launched here, we've been using EteSync for a while and it's nice to see it expanding to more end to end encrypted things!
- tasn 6y ago
  Thanks. :) As said in the other comment, our mission is to encrypt everything, and this is the next step towards that! It's actually something that we've been planning since forever, but it's so nice to finally have it out!
  - blazeWayne 6y ago
    That's an awesome mission. More power to you! The product seems incredible.
    - tasn 6y ago
      Thanks! :)
  - mnahkies 6y ago
    Looks like an awesome product. Is the item metadata encrypted as well, or just the content? Do you have any documentation explaining the encryption process in detail?
    - tasn 6y ago
      Everything is encrypted. We are working on a full spec, it's not yet complete, but some of it is already there: https://docs.etebase.com/protocol-specs/introduction https://docs.etebase.com/protocol-specs/introduction We hope to get all of it done in the few days. Fingers crossed. :)
rathboma 6y ago
I make Beekeeper Studio (https://beekeeperstudio.io https://beekeeperstudio.io). This looks really interesting, especially as it is open source. Do you have many FOSS users self-deploying it?
- tasn 6y ago
  Hard to know exactly as people get it from multiple sources, but there are a few hundred thousands downloads on Dockerhub, people run it from source, and it's packaged for OpenWrt, Arch (AUR), Fedora (COPR) and a few other Linux distributions. So yeah, quite a few users.
- nbclark 6y ago
  Off topic, but just checked out Beekeeper - Seems like a solid SQL IDE! Had been using Postico before, but going to give yours a try.
  - rathboma 6y ago
    Awesome thanks for the kind words!
  - indymike 6y ago
    Beekeeper is really nice.
  - jmnicolas 6y ago
    Same here, I'm fed up from the clunkyness of DBeaver so I'm even ready to consider using an electron app! And since you don't have analytics, please know that there's at least one person that will use the light theme, please keep it ;)
- sali0 6y ago
  I have been looking for something exactly like this. Your origin story is very apt: there really isn't an approachable cross-platform solution for noobs like me. Will definitely be checking this out. Thank you for sharing.
- Already__Taken 6y ago
snissn 6y ago
can you explain key management? is it via the user's password? what happens if passwords are lost or changed?
- tasn 6y ago
  We are working on providing a formal spec, but the gist of it is already documented here: https://docs.etebase.com/protocol-specs/authentication https://docs.etebase.com/protocol-specs/authentication A secure encryption key is derived from the user's password using a random salt an Argon2id, though the data is encrypted with a randomly generated key. Keys are generated for each "layer" of the account (so one for the account, one for collections, and etc), so each part can be re-encrypted (or not) separately if needed. Password change: you can either re-encrypt the data if you want, or more likely, if the password hasn't be compromised, just re-encrypt the main encryption key that's used to encrypt the data. Password lost: tough luck. We can't help you recover that, because we don't have access to your data. Though we have some ideas on how to maybe enable recovery (using key custodians, shamir secret sharing, or another method, haven't decided).
  - fabrice_d 6y ago
    What about switching devices? Is the password enough (I guess not since it's salted)?
    - tasn 6y ago
      The salt is kept on the server, so you can just fetch it again when switching devices. However, on the long term, we plan on considering passwords more of a backup feature, and instead move to a model where you just authorise your devices from running devices (though can always fallback to passwords).
codethief 6y ago
This looks exactly like the application I've been wanting to build for quite some time! Thank god I didn't have to. :) Question: The Collection/Item data structure seems simple but also quite powerful at the same time. However, for lots of applications I imagine it would be nice to have some common abstractions available in the API instead of having to roll my own. I'm thinking of, e.g., data structures like queues, stacks, dictionaries, trees, and also conflict resolution strategies. Are there any plans to add those to the API?
- tasn 6y ago
  This is a great question, and the answer is a resounding yes! The API is already powerful, and powers real applications, but it's quite raw. We plan on gradually building all of the above and more. We just first needed a solid base to work from.
LoSboccacc 6y ago
interesting how can user share data with each other?
- alexh1 6y ago
  I see in another comment that users have a key pair generated for them. What you can do is fetch other users public keys and encrypt data to them, or better yet, generate a symmetric key for a project and encrypt that to trusted users. It's how we did it for Portabella (https://portabella.io https://portabella.io) anyway but that's not based on Etebase/Etesync, just another e2ee tool
  - tasn 6y ago
    Always great to see more end-to-end encrypted applications! Looks very cool, I'd love to chat if you are willing, my email is tom @ etebase, could you please drop me a line?
  - gwking 6y ago
    A quick bit of feedback regarding your website: I was interested in reading the front page but ended up leaving because the “perfect for...” type and delete effect was causing the remainder of the page to jump up and down by one line every few seconds. It just happens to wrap in the wrong place on my phone.
    - alex_portabella 6y ago
      Fixed, sorry about that one.
- tasn 6y ago
  As said in a sibling comment, we currently use public-key encryption and verifying keys. However, we plan on adding support for SPAKE2+ (like e.g. Magic wormhole) among other methods, to make it even easier! See the relevant section of the docs: https://docs.etebase.com/guides/collection_sharing https://docs.etebase.com/guides/collection_sharing
ignoramous 6y ago
This looks great and has a very comprehensive set of libraries. Nice bit of competition for https://userbase.com/ https://userbase.com/ (though their focus is mostly "Serverless" and the Web).
- tasn 6y ago
  Thanks! As for userbase: I think our and their scopes are very different. They are more of an easy way to store data that happens to use some encryption, not a comprehensive end-to-end encryption toolkit. They don't do sharing, integrity management, data de-duplication, and lot of other things that are needed for more serious applications. In addition, they don't have libraries for anything other than JS.
z5h 6y ago
In these schemes, how does one typically offer technical support to users? Give them access to a suite of data tool?
- tasn 6y ago
  This is a very good question, and one thing we had issues with over the years while developing EteSync. At EteSync, we ended up building client side debug utils that we can get users to launch in order to figure out what's going on and selectively report issues. You can, depending on your users' risk profile, also have it more automatic (we kept it semi-automatic). Developer tooling is definitely lacking at the moment for building encrypted applications and it's something we plan on addressing.
  - z5h 6y ago
    Good to know you've experienced the need for this and are planning to address it. Thanks for sharing.
TruthWillHurt 6y ago
How is a self-hosted database suppose to be an alternative to a cloud hosted managed service?
- tasn 6y ago
  You don't have to self-host Etebase, you can also use the hosted version on etebase.com if you wish. However, what we are offering here is an alternative to the Firebase APIs, rather than the fact that it's cloud hosted.
- indymike 6y ago
  One of the shortcomings of Firebase is that you can't self host. Self-hosting is still relevant for developers and opens up the device market where cloud connectivity isn't appropriate. Great timing on Etebase.
aitchnyu 6y ago
Tangential, but is Etesync and Notes a cheap, unbloated and foolproof app combo compared to paid competitors? And would the existing protocol support a (team) password manager? I would buy that trio if its a healthy business at $2 per month and avoided lock-in. And in future enabled "SaaS" products which is a static site + a safe backend.
- tasn 6y ago
  EteSync and EteSync Notes: I'd say yes. To be honest, it's way too cheap for what it is. :) Yes, the protocol can support that, though it's not yet implemented. Not sure what you meant with the rest of your comment.
  - aitchnyu 6y ago
    I'm wary of SaaS that earns a ton of cash and then implodes under their expenses. I'll be happy to get an Etesync account and use apps built for it. Like how draw.io is a mostly static site using Dropbox, GDrive etc as portable backends.
    - tasn 6y ago
      We are not going to implode under our expenses. We keep our expenses in check (and have been for the last four years), and we put a big emphasis on the sustainability of the service.
bastawhiz 6y ago
This is neat, but it seems like it's missing a few things. For one, how do you do migrations? Do I need to include every migration in my application bundle in perpetuity, running them in the client on startup? There doesn't seem to be a mechanism to help with downloading, migrating, and reuploading data. That doesn't feel so nice. More pertinent, though, how do I ensure my users aren't abusing my service? If I allow folks to upload avatars for folks in their address book, how can I stop someone from running up my bill by uploading their MP3 collection? As far as I can tell, if everything is encrypted, I just have some very active users. Obviously there's some amount of tradeoff here, but it would seem like a necessary counterbalance is strong quota controls, which I don't see in the docs.
- tasn 6y ago
  Yeah, as I said in my main comment, there are still things that need to be done. One of them is strong quotas. We plan on having it in the near future (we are also running a service, EteSync, using Etebase) though I think it's not an issue that should prevent you from getting started. :) As for migrations: that's a tough one, and we indeed don't have an easy-to-use mechanism yet. I mentioned it in another comment: the API at the moment is powerful and used in production, but it's still missing some goodies to make the developer experience even better. As for including every migration in your application bundle: yeah, well until you consider an old version EOL. It's the same challenge Microsoft has with Office, and every app has with their local storage.
krimpenrik 6y ago
Awesome, I'll give it a test drive. Probably will be good for adoption to have a dart / flutter package.
- tasn 6y ago
  It's planned, as you are not the first to ask for it! Would love to get help though if you are able!
balupton 6y ago
Why use this over say https://gun.eco https://gun.eco
Ali_Jiwani 6y ago
This looks really good and well timed. What will you do if Firebase builds something similar?
- tasn 6y ago
  Thank you! :) That's a question that's looms over every new company and open-source project. I don't think Firebase will go down this route, it's a very different offering to what they already have, and any encrypted stuff will be incompatible with the rest of it. There's also Google's (rightfully earned) bad public image when it comes to privacy, which will limit adoption. In addition, being open-source is very important for a solution revolving around data ownership and privacy, something that Google will just not do. Our plan is therefore to continue building the best open-source project we can and let the developers decide.
WolfOliver 6y ago
How does Authorization work? Does every user has access to every collection/document?
- tasn 6y ago
  Collections are private to each user, so users only have access to their own collections. They can however share them among eachother.
nathias 6y ago
This is amazing!
piotrkaminski 6y ago
Since the pitch keeps referring to Firebase, I thought I might as well mention this here: a while ago I built an encryption layer for the Firebase RTDB at https://github.com/Reviewable/firecrypt https://github.com/Reviewable/firecrypt. It's not nearly as nice as Etsebase appears to be, but if you're already committed (cough locked in cough) to Firebase then it will get you a good chunk of the way there. (And yes, I still actively use and maintain it.)
- boaticus 6y ago
  VERY interesting! Reading the README first paragraph, I see you mention this doesn't make sense for apps where anon users can see data. But what about when anon users can sign up for an account, THEN have access to sensitive data?
  - piotrkaminski 6y ago
    It might work, if you truly trust whoever signs up. The fundamental issue is that there's just one encryption key for the whole database, and you need to provide it to everyone who you want to have access to encrypted data. If you stop trusting someone for any reason your only recourse is re-encrypting the entire database, which is a painful offline operation. (You might also be able to support multiple encryption keys -- one per user, say -- but the library isn't really set up to make that convenient right now.)
jmnicolas 6y ago
Hi, fwiw I noticed a small typo on https://docs.etebase.com/ https://docs.etebase.com/ you wrote challAnges instead of challEnges. May I ask how do you make money? Just by hosting the service? Is it production ready?
- tasn 6y ago
  Oops, fixed. Thanks! Make money: hosting the service, yeah. Though also from our user facing apps. Production ready: yeah. We use it for EteSync (our user facing apps), and through that it's been integrated to GNOME, KDE, Tasks.org and more.
vekker 6y ago
Very interesting, I've been on the lookout for something like this for a long time and it still surprises me how few out-of-the-box, open source, E2E encryption solutions there are available (especially for us Europeans...). One thing that's not clear to me from first glance at the docs: does this support offline-first apps? So what about sync with local storage, search, CRUD of collections/items offline, conflict resolution, ...? Am I missing something, or is this something that the developer has to implement themselves?
- tasn 6y ago
  Thanks for your questions! What we have now is the first step of our vision. It supports offline-first apps in the sense that you can work offline and sync later. If you are talking about not using a server at all: that's coming too. All the rest of the things you mention: they are all in the works. What we have now is already enough to build great apps (as done with EteSync for example), though we want to build the entire ecosystem around it.
habosa 6y ago
Firebaser here! Just want to say this is (in my personal opinion) pretty cool and even though it's the "competition" you're doing something unique and we'll take the comparison as a compliment.
- tasn 6y ago
  Haha, thanks! <3 I don't think we are really direct competitors, we are targeting a different niche. It's just that you are the best example of a backend-as-a-service and thus the best way to explain what we do. :)
jmnicolas 6y ago
After reading the docs I have a couple more questions: - there's the notion of a local cache but is it synced / de-duped automatically or is it something I still have to do? Let's say I have a ToDo mobile app. While online I enter the task "buy milk". Sometimes later I'm in airplane mode and I modify the task as "buy condensed milk". Do your library updates the changes automagically once online? - how realistic is it to run your own server? Is it a couple hours of config and forget about it or it's a multi-days endeavor without certainty of success? - in your examples you use the name Cyberdyne. If I use Etebase do I unknowingly contribute to Skynet? ;)
- tasn 6y ago
  Local cache: no, you need to do it yourself once the app goes online. We plan on having things like that soon as part of support for automatic conflict resolution and etc. Running your own server: trivial, not even a couple of hours. Check out the repo for instructions, or just use the docker images (created by the community). As for Skynet. We are already too late.
  - jmnicolas 6y ago
    Thanks!
speby 6y ago
This looks really good and I can tell a lot of work went into this. Bravo. Although this does not yet have the same sort of real-time push syncing (or subscriptions) the way Firestore (or Firebase) both have, it looks like it has the right foundation to support that. Also impressed that this has such wide client SDK support already. Again, I can tell a lot of work went into it. I'll be trying this out and following the project! Excited to see where it goes next.
- tasn 6y ago
  Thanks a lot! Yeah, A LOT of work went into this. :) As for the wide client SDK support: I think it's essential for adoption so made it a priority. Great to see it being appreciated!
thelazydogsback 6y ago
How does this compare to https://supabase.io/ https://supabase.io/ ?
- tasn 6y ago
  I think it's very different to supabase.io. They are focusing on being a Firebase alternative, we are focusing on making it easy to build encrypted applications. Not the same target audience and not the same offering. With that being said, Supabase is very cool, and I'm a big fan! If you don't care about end-to-end encryption, you should definitely use Supabase!

308 points by tasn 6y ago

tasn 6y ago

Excited to finally post it on HN. It's something I've been working on for the last year, though in a sense, for much longer! Etebase is an open-source SDK and backend-as-a-service for building end-to-end encrypted applications. Think Firebase but end-to-end encrypted. More users care about the privacy and security of their data every day, and encrypted applications are becoming mainstream. However, talking with developers, many get encryption wrong, and they don't even realise it. As good cryptography and bad cryptography look identical to the untrained eye. The idea behind Etebase is to make an easy-to-use API to enable developers to build encrypted apps, and it's based on our learnings from building EteSync[2] over the years. There's still a lot more to get done, but Etebase is already used in GNOME, KDE, Tasks.org, EteSync, and more. With libraries available for Rust, JavaScript/TypeScript, Java, Python and C/C++. I hope Etebase will enable a new age of privacy-first and end-to-end encrypted applications, and that our data will no longer be used to track, analyze and manipulate us. Your data, yours only. Let's end-to-end encrypt everything! I'd love to hear about your experience building encrypted applications. Got any questions? Suggestions? Please let me know! Docs: https://docs.etebase.com/ https://docs.etebase.com/ [1] https://libsodium.org https://libsodium.org [2] https://www.etesync.com https://www.etesync.com

slooonz 6y ago
Is there any technical document describing how it works ? For example, order-preserving indices (for range queries) are often an issue in end-to-end databases ; did your design allow to you to provide them, or did you just consider solving this problem out-of-scope ?
- tasn 6y ago
  There are docs here: https://docs.etebase.com/ https://docs.etebase.com/ Though they mostly focus on how to use rather than how it works (though we are making progress on that section). As for your specific question: we currently don't solve that, but we plan on doing it in the near future. Our approach will be to maintain a client-side index that will be synced across devices. Etebase already utilities to maintain consistency, so this can be done safely. This question, and ones like it, are exactly why we created Etebase. Building encrypted applications is fundamentally different and comes with a lot of challenges. We plan on continue building all of the tools needed to develop using this different paradigm.
joel_dice 6y ago
Nice work! I spent about six months pursuing the exact same idea about five years ago with the hope of building a company around it. I got reasonably far with a prototype, but gave up on the business side of things after doing more extensive market research and interviews. One of the people I talked to had been involved with a startup called Adrenaline Mobility which did something very similar, but ultimately ran out of runway and was acquired by Twitter. He told an interesting story about how they (over-)built a highly scalable service but had a really hard time selling it. Those who lack the training to know the difference between good security and hand waving didn't see the value, and those who did have the training and skills didn't trust a third party to do it right anyway. Nonetheless, a lot has changed in the past five years, and end-to-end encryption is more widely appreciated now. I hope you have better luck with the commercial side of things; this looks like a very promising start.
- tasn 6y ago
  Thanks a lot! Things have indeed changed, and I feel like the public is really starting to understand the dangers with having their data exposed. I've been running EteSync for years now, and it's still growing. I'd love to have a chat with you to learn from your experience if you are willing! My email is tom at etebase, please drop me a line (or let me know how to best reach you). Thanks again!
  - g_p 6y ago
    This might be very good timing for your product - the European Data Protection Board has just out their first guidance after the Schrems II case. It seems to firmly point towards cloud services where the provider can access the data in plaintext as being "unable to have the risks mitigated" from the perspective of international transfers. Perhaps a straightforward E2EE toolbox could help companies implement their products rapidly so they remain complaint. Clearly this won't help fix issues like Microsoft 365 or GSuite (which need access to the plaintext for email and similar), but it might help some types of SaaS to thicken up their client application and prevent the backend having access to unencrypted data, thus making the transfer permitted. I just want to echo the above comment around how most customers don't have the knowledge to understand the benefit of this, but hopefully this is changing as we see more strict enforcement of penalties for data breaches. Data is fast becoming a liability you don't want to have the ability to see, and systems like yours offer usable solutions for those who don't understand all the technology, but need a solution.
ChrisMarshallNY 6y ago
I sincerely wish you luck with this. I wrote something similar, a couple of years ago[0], but it doesn't include encryption; it merely gives a place to add encryption. I didn't want to deal with the legalities of included encryption, and I think others can do far better than I (but it is quite secure, nonetheless[1]). It was really done, just to "retool" my architectural and engineering skills, as I was pivoting from being a manager, back to being a developer. I am now using it as a backend for an app under development (currently closed-source). It has been working a treat. But after the app has been released, I may encourage the folks to look at alternatives for the backend. I am not really into maintaining a server system. I like to stay on the app side of things. [0] https://riftvalleysoftware.com/work/open-source-projects/#baobab https://riftvalleysoftware.com/work/open-source-projects/#ba... [1] https://riftvalleysoftware.com/BAOBAB/PDFs/Security.pdf https://riftvalleysoftware.com/BAOBAB/PDFs/Security.pdf (Downloads a PDF)
- tasn 6y ago
  Thank you for your kind words! I think the encryption is the key differentiator here, at least for me. I don't want my data saved exposed on someone else's server, AKA the cloud.
  - ChrisMarshallNY 6y ago
    The deal with mine, is the encryption is left to the client. The server just stores a binary "thing." That "thing" can be an encrypted envelope, managed by the client[s]. Since it's a closed-loop system, then a key could be shared programmatically, outside the purview of the server.
    - tasn 6y ago
      Yeah, in Etebase the server also just sees "things", though it tries to make it easy on the client to only get what's changed and etc.
      - fabrice_d 6y ago
Tepix 6y ago
Congrats! Kudos for making the server component open source, something your competitors aren't doing.
fakedang 6y ago
Flutter please lol Anyways, great work!

Aachen 6y ago

Cool to see this launched here, we've been using EteSync for a while and it's nice to see it expanding to more end to end encrypted things!

tasn 6y ago
Thanks. :) As said in the other comment, our mission is to encrypt everything, and this is the next step towards that! It's actually something that we've been planning since forever, but it's so nice to finally have it out!
- blazeWayne 6y ago
  That's an awesome mission. More power to you! The product seems incredible.
  - tasn 6y ago
    Thanks! :)
- mnahkies 6y ago
  Looks like an awesome product. Is the item metadata encrypted as well, or just the content? Do you have any documentation explaining the encryption process in detail?
  - tasn 6y ago
    Everything is encrypted. We are working on a full spec, it's not yet complete, but some of it is already there: https://docs.etebase.com/protocol-specs/introduction https://docs.etebase.com/protocol-specs/introduction We hope to get all of it done in the few days. Fingers crossed. :)

rathboma 6y ago

I make Beekeeper Studio (https://beekeeperstudio.io https://beekeeperstudio.io). This looks really interesting, especially as it is open source. Do you have many FOSS users self-deploying it?

tasn 6y ago
Hard to know exactly as people get it from multiple sources, but there are a few hundred thousands downloads on Dockerhub, people run it from source, and it's packaged for OpenWrt, Arch (AUR), Fedora (COPR) and a few other Linux distributions. So yeah, quite a few users.
nbclark 6y ago
Off topic, but just checked out Beekeeper - Seems like a solid SQL IDE! Had been using Postico before, but going to give yours a try.
- rathboma 6y ago
  Awesome thanks for the kind words!
- indymike 6y ago
  Beekeeper is really nice.
- jmnicolas 6y ago
  Same here, I'm fed up from the clunkyness of DBeaver so I'm even ready to consider using an electron app! And since you don't have analytics, please know that there's at least one person that will use the light theme, please keep it ;)
sali0 6y ago
I have been looking for something exactly like this. Your origin story is very apt: there really isn't an approachable cross-platform solution for noobs like me. Will definitely be checking this out. Thank you for sharing.
Already__Taken 6y ago

snissn 6y ago

can you explain key management? is it via the user's password? what happens if passwords are lost or changed?

tasn 6y ago
We are working on providing a formal spec, but the gist of it is already documented here: https://docs.etebase.com/protocol-specs/authentication https://docs.etebase.com/protocol-specs/authentication A secure encryption key is derived from the user's password using a random salt an Argon2id, though the data is encrypted with a randomly generated key. Keys are generated for each "layer" of the account (so one for the account, one for collections, and etc), so each part can be re-encrypted (or not) separately if needed. Password change: you can either re-encrypt the data if you want, or more likely, if the password hasn't be compromised, just re-encrypt the main encryption key that's used to encrypt the data. Password lost: tough luck. We can't help you recover that, because we don't have access to your data. Though we have some ideas on how to maybe enable recovery (using key custodians, shamir secret sharing, or another method, haven't decided).
- fabrice_d 6y ago
  What about switching devices? Is the password enough (I guess not since it's salted)?
  - tasn 6y ago
    The salt is kept on the server, so you can just fetch it again when switching devices. However, on the long term, we plan on considering passwords more of a backup feature, and instead move to a model where you just authorise your devices from running devices (though can always fallback to passwords).

codethief 6y ago

This looks exactly like the application I've been wanting to build for quite some time! Thank god I didn't have to. :) Question: The Collection/Item data structure seems simple but also quite powerful at the same time. However, for lots of applications I imagine it would be nice to have some common abstractions available in the API instead of having to roll my own. I'm thinking of, e.g., data structures like queues, stacks, dictionaries, trees, and also conflict resolution strategies. Are there any plans to add those to the API?

tasn 6y ago
This is a great question, and the answer is a resounding yes! The API is already powerful, and powers real applications, but it's quite raw. We plan on gradually building all of the above and more. We just first needed a solid base to work from.

LoSboccacc 6y ago

interesting how can user share data with each other?

alexh1 6y ago
I see in another comment that users have a key pair generated for them. What you can do is fetch other users public keys and encrypt data to them, or better yet, generate a symmetric key for a project and encrypt that to trusted users. It's how we did it for Portabella (https://portabella.io https://portabella.io) anyway but that's not based on Etebase/Etesync, just another e2ee tool
- tasn 6y ago
  Always great to see more end-to-end encrypted applications! Looks very cool, I'd love to chat if you are willing, my email is tom @ etebase, could you please drop me a line?
- gwking 6y ago
  A quick bit of feedback regarding your website: I was interested in reading the front page but ended up leaving because the “perfect for...” type and delete effect was causing the remainder of the page to jump up and down by one line every few seconds. It just happens to wrap in the wrong place on my phone.
  - alex_portabella 6y ago
    Fixed, sorry about that one.
tasn 6y ago
As said in a sibling comment, we currently use public-key encryption and verifying keys. However, we plan on adding support for SPAKE2+ (like e.g. Magic wormhole) among other methods, to make it even easier! See the relevant section of the docs: https://docs.etebase.com/guides/collection_sharing https://docs.etebase.com/guides/collection_sharing

ignoramous 6y ago

This looks great and has a very comprehensive set of libraries. Nice bit of competition for https://userbase.com/ https://userbase.com/ (though their focus is mostly "Serverless" and the Web).

tasn 6y ago
Thanks! As for userbase: I think our and their scopes are very different. They are more of an easy way to store data that happens to use some encryption, not a comprehensive end-to-end encryption toolkit. They don't do sharing, integrity management, data de-duplication, and lot of other things that are needed for more serious applications. In addition, they don't have libraries for anything other than JS.

z5h 6y ago

In these schemes, how does one typically offer technical support to users? Give them access to a suite of data tool?

tasn 6y ago
This is a very good question, and one thing we had issues with over the years while developing EteSync. At EteSync, we ended up building client side debug utils that we can get users to launch in order to figure out what's going on and selectively report issues. You can, depending on your users' risk profile, also have it more automatic (we kept it semi-automatic). Developer tooling is definitely lacking at the moment for building encrypted applications and it's something we plan on addressing.
- z5h 6y ago
  Good to know you've experienced the need for this and are planning to address it. Thanks for sharing.

TruthWillHurt 6y ago

How is a self-hosted database suppose to be an alternative to a cloud hosted managed service?

tasn 6y ago
You don't have to self-host Etebase, you can also use the hosted version on etebase.com if you wish. However, what we are offering here is an alternative to the Firebase APIs, rather than the fact that it's cloud hosted.
indymike 6y ago
One of the shortcomings of Firebase is that you can't self host. Self-hosting is still relevant for developers and opens up the device market where cloud connectivity isn't appropriate. Great timing on Etebase.

aitchnyu 6y ago

Tangential, but is Etesync and Notes a cheap, unbloated and foolproof app combo compared to paid competitors? And would the existing protocol support a (team) password manager? I would buy that trio if its a healthy business at $2 per month and avoided lock-in. And in future enabled "SaaS" products which is a static site + a safe backend.

tasn 6y ago
EteSync and EteSync Notes: I'd say yes. To be honest, it's way too cheap for what it is. :) Yes, the protocol can support that, though it's not yet implemented. Not sure what you meant with the rest of your comment.
- aitchnyu 6y ago
  I'm wary of SaaS that earns a ton of cash and then implodes under their expenses. I'll be happy to get an Etesync account and use apps built for it. Like how draw.io is a mostly static site using Dropbox, GDrive etc as portable backends.
  - tasn 6y ago
    We are not going to implode under our expenses. We keep our expenses in check (and have been for the last four years), and we put a big emphasis on the sustainability of the service.

bastawhiz 6y ago

This is neat, but it seems like it's missing a few things. For one, how do you do migrations? Do I need to include every migration in my application bundle in perpetuity, running them in the client on startup? There doesn't seem to be a mechanism to help with downloading, migrating, and reuploading data. That doesn't feel so nice. More pertinent, though, how do I ensure my users aren't abusing my service? If I allow folks to upload avatars for folks in their address book, how can I stop someone from running up my bill by uploading their MP3 collection? As far as I can tell, if everything is encrypted, I just have some very active users. Obviously there's some amount of tradeoff here, but it would seem like a necessary counterbalance is strong quota controls, which I don't see in the docs.

tasn 6y ago
Yeah, as I said in my main comment, there are still things that need to be done. One of them is strong quotas. We plan on having it in the near future (we are also running a service, EteSync, using Etebase) though I think it's not an issue that should prevent you from getting started. :) As for migrations: that's a tough one, and we indeed don't have an easy-to-use mechanism yet. I mentioned it in another comment: the API at the moment is powerful and used in production, but it's still missing some goodies to make the developer experience even better. As for including every migration in your application bundle: yeah, well until you consider an old version EOL. It's the same challenge Microsoft has with Office, and every app has with their local storage.

krimpenrik 6y ago

Awesome, I'll give it a test drive. Probably will be good for adoption to have a dart / flutter package.

tasn 6y ago
It's planned, as you are not the first to ask for it! Would love to get help though if you are able!

balupton 6y ago

Why use this over say https://gun.eco https://gun.eco

Ali_Jiwani 6y ago

This looks really good and well timed. What will you do if Firebase builds something similar?

tasn 6y ago
Thank you! :) That's a question that's looms over every new company and open-source project. I don't think Firebase will go down this route, it's a very different offering to what they already have, and any encrypted stuff will be incompatible with the rest of it. There's also Google's (rightfully earned) bad public image when it comes to privacy, which will limit adoption. In addition, being open-source is very important for a solution revolving around data ownership and privacy, something that Google will just not do. Our plan is therefore to continue building the best open-source project we can and let the developers decide.

WolfOliver 6y ago

How does Authorization work? Does every user has access to every collection/document?

tasn 6y ago
Collections are private to each user, so users only have access to their own collections. They can however share them among eachother.

nathias 6y ago

This is amazing!

piotrkaminski 6y ago

Since the pitch keeps referring to Firebase, I thought I might as well mention this here: a while ago I built an encryption layer for the Firebase RTDB at https://github.com/Reviewable/firecrypt https://github.com/Reviewable/firecrypt. It's not nearly as nice as Etsebase appears to be, but if you're already committed (cough locked in cough) to Firebase then it will get you a good chunk of the way there. (And yes, I still actively use and maintain it.)

boaticus 6y ago
VERY interesting! Reading the README first paragraph, I see you mention this doesn't make sense for apps where anon users can see data. But what about when anon users can sign up for an account, THEN have access to sensitive data?
- piotrkaminski 6y ago
  It might work, if you truly trust whoever signs up. The fundamental issue is that there's just one encryption key for the whole database, and you need to provide it to everyone who you want to have access to encrypted data. If you stop trusting someone for any reason your only recourse is re-encrypting the entire database, which is a painful offline operation. (You might also be able to support multiple encryption keys -- one per user, say -- but the library isn't really set up to make that convenient right now.)

jmnicolas 6y ago

Hi, fwiw I noticed a small typo on https://docs.etebase.com/ https://docs.etebase.com/ you wrote challAnges instead of challEnges. May I ask how do you make money? Just by hosting the service? Is it production ready?

tasn 6y ago
Oops, fixed. Thanks! Make money: hosting the service, yeah. Though also from our user facing apps. Production ready: yeah. We use it for EteSync (our user facing apps), and through that it's been integrated to GNOME, KDE, Tasks.org and more.

vekker 6y ago

Very interesting, I've been on the lookout for something like this for a long time and it still surprises me how few out-of-the-box, open source, E2E encryption solutions there are available (especially for us Europeans...). One thing that's not clear to me from first glance at the docs: does this support offline-first apps? So what about sync with local storage, search, CRUD of collections/items offline, conflict resolution, ...? Am I missing something, or is this something that the developer has to implement themselves?

tasn 6y ago
Thanks for your questions! What we have now is the first step of our vision. It supports offline-first apps in the sense that you can work offline and sync later. If you are talking about not using a server at all: that's coming too. All the rest of the things you mention: they are all in the works. What we have now is already enough to build great apps (as done with EteSync for example), though we want to build the entire ecosystem around it.

habosa 6y ago

Firebaser here! Just want to say this is (in my personal opinion) pretty cool and even though it's the "competition" you're doing something unique and we'll take the comparison as a compliment.

tasn 6y ago
Haha, thanks! <3 I don't think we are really direct competitors, we are targeting a different niche. It's just that you are the best example of a backend-as-a-service and thus the best way to explain what we do. :)

jmnicolas 6y ago

After reading the docs I have a couple more questions: - there's the notion of a local cache but is it synced / de-duped automatically or is it something I still have to do? Let's say I have a ToDo mobile app. While online I enter the task "buy milk". Sometimes later I'm in airplane mode and I modify the task as "buy condensed milk". Do your library updates the changes automagically once online? - how realistic is it to run your own server? Is it a couple hours of config and forget about it or it's a multi-days endeavor without certainty of success? - in your examples you use the name Cyberdyne. If I use Etebase do I unknowingly contribute to Skynet? ;)

tasn 6y ago
Local cache: no, you need to do it yourself once the app goes online. We plan on having things like that soon as part of support for automatic conflict resolution and etc. Running your own server: trivial, not even a couple of hours. Check out the repo for instructions, or just use the docker images (created by the community). As for Skynet. We are already too late.
- jmnicolas 6y ago
  Thanks!

speby 6y ago

This looks really good and I can tell a lot of work went into this. Bravo. Although this does not yet have the same sort of real-time push syncing (or subscriptions) the way Firestore (or Firebase) both have, it looks like it has the right foundation to support that. Also impressed that this has such wide client SDK support already. Again, I can tell a lot of work went into it. I'll be trying this out and following the project! Excited to see where it goes next.

tasn 6y ago
Thanks a lot! Yeah, A LOT of work went into this. :) As for the wide client SDK support: I think it's essential for adoption so made it a priority. Great to see it being appreciated!

thelazydogsback 6y ago

How does this compare to https://supabase.io/ https://supabase.io/ ?

tasn 6y ago
I think it's very different to supabase.io. They are focusing on being a Firebase alternative, we are focusing on making it easy to build encrypted applications. Not the same target audience and not the same offering. With that being said, Supabase is very cool, and I'm a big fan! If you don't care about end-to-end encryption, you should definitely use Supabase!