8 ms·

Magma – An encrypted email server daemon

doomrobo 9y ago
I guess I'll be the one to ask: Why write an email server with security as priority #1 in C? I'd understand if it was built off of an existing daemon, but it seems that this was written from scratch. There are many other options out there that are reasonably fast and don't have the pitfalls of manual memory management and undefined behavior that C has.
- jhasse 9y ago
  One reason for this could be that for an open-source application with focus on security it's very important to have many collaboraters, who look at your code and contribute. C is still very popular and many people know it.
  - doomrobo 9y ago
    There are plenty of popular safe languages. Java, Go, and Python are all viable options here.
  - tptacek 9y ago
    C limits the pool of contributors. It doesn't expand it.
    - devillius 9y ago
      I don't think the Python/Ruby/Node/VB developers are the intended contributors. Source: I'm a Python/Ruby/Node/VB/C/C++ dev
      - tptacek 9y ago
        Why not? (I started my career shipping C code, FWIW.)
    - Alupis 9y ago
      > C limits the pool of contributors. It doesn't expand it. Oh boy, I'm starting to grow weary of folks tirelessly thumping this mantra. It's patently absurd.
      - tptacek 9y ago
        Do go on.
        Alupis 9y ago
        I do.
        tptacek 9y ago
        No, really. It sounds to me like you think the number of C developers is comparable to the number of high-level language developers. Tell me more.
        Alupis 9y ago
        The overwhelming majority of "high-level language" developers are, at an absolute minimum, familiar enough with C, or C-like languages, to be able to read/contribute to a C project. Heck, most of us, yourself included, started off with C... it's just how it was/is taught. Perhaps your C skills get rusty without use, or perhaps you're not a bit-twiddling magician, but you can still read it, and therefore think about it, and therefore write it. It doesn't limit the pool of contributors... quite the opposite. Now, if you want to have a debate about a better choice of language for other reasons, that might be fair game... but to imply there's a shortage of C developers is just absurd.
        tptacek 9y ago
        I could not disagree more strongly with this. Most of the Python and Ruby developers I know are not comfortable writing C code. More importantly, there's nothing in the world more dangerous than someone who knows just enough to get something working in C. C is a harder language to learn than Ruby, Python, or Javascript. Unlike any of those languages, to write in C you have to constantly keep object life cycles, memory hierarchy, and memory layout in mind. Most of the standard libraries of Python or Ruby --- even simple string manipulation --- are idiomatically written and rewritten by hand in C. Even experienced C programmers forget, routinely and to calamitous affect, how machine integers work. Writing something in C reduces the number of programmers who can contribute to a project. And that reduction is going to get worse and worse over time. When I started my career, if you were going to ship code to clients, C was the industry standard way to accomplish that. A giant chunk of all working programmers used C as their daily language. Today, only a small minority of programmers will ever once write a production C module.
- cyphar 9y ago
  The most "interesting" thing is that they decided to effectively combine all of the components of a mail server into a single binary. This is actually advertised as a feature ("packaging is hard"), which is quite worrying because having everything in one address space means a minor bug in one piece now affects the integrity of all of the server components.
- elif 9y ago
  Well, email is not that complicated, three letter agencies in the US have a history of hiding backdoors in technology, and one of the authors had his company destroyed by one of those agencies. If you're writing software for the tinfoil hat crowd, and you're using high level languages and existing libraries to do very basic things, you raise suspicion even if it isn't immediately justifiable... the suspicion that most major routers were routinely pwned wasn't justifiable in April, either. Speed is almost never the primary concern with email, which is a world full of rate limits, throttles, spam filters, transient failures, tarpitting, blacklisting, inbox placement delays, etc.
  - doomrobo 9y ago
    >three letter agencies in the US have a history of hiding backdoors in technology So the solution is to use the simplest tech stack possible and put the vulnerabilities in there yourself? If you're using this software, you have to trust somebody. I think most people would sooner trust that Java isn't backdoored than trust that this new C project doesn't have any buffer overflows or memory leaks.
    - ladarlevison 9y ago
      I disagree. The more layers of abstraction you have, the more places there are for a backdoor to hide. Also, unless you're using OpenJDK, Java means downloading a binary blob which could be hiding any number of surprises, intentional, or accidental. With magma I've taken the approach of trying to limit my dependencies to the kernel, and libc. Anything else I use is bundled and thus gets tested extensively for leaks, overflows, etc. That doesn't mean bugs don't exist, but it does mean if they exist, then the source is there for you to inspect and fix.
    - elif 9y ago
      Different strokes for different folks. I think intentional bugs are far worse than unintended, and scanning a c project for buffer overflows is relatively trivial compared to scanning tons of libraries for flaws so obfuscated they made it through review. In my mind technical analysis is incomplete at finding 100% of intended or unintended flaws in either system, but the DIY approach allows for trust in the authors whereas a deep stack makes attribution muddy.
- cbuq 9y ago
  Your concern of memory management and undefined behavior is not unfounded. But in my opinion all languages have these problems, in the C the blame is shifted to the software engineer instead of the language authors.
djsumdog 9y ago
I see references to ClamAV, and references to soon coming official docker images. So is this a complete MTA and delivery (IMAP/POP/etc) servers? I was digging around on github and the about page, but there doesn't seem to be info on how it does spam detection, or DKIM. Does it support signing messages? I wish they had a more detailed about page and feature list, but I am still excited about this.
- mark242 9y ago
  The ClamAV dependency concerns me; messages can only be passed to clamd via a plaintext stream. This seems to be at odds with the end-to-end encryption.
  - ladarlevison 9y ago
    The ClamAV library is built directly into magma so plain text messages can be scanned without leaving the magma process space. I'd like to eventually have magma fork and scan messages in an isolated process space that doesn't have access to anything else, but that's a long term goal. To clarify. We do not use clamd, or transfer the messages to another daemon.
  - matt_wulfeck 9y ago
    Does the "encryption in flight" include interprocess communication? If so there's not much safety in the world!
    - mark242 9y ago
      There's a difference-- point-to-point encryption is what you're talking about. Anytime a message is transferred, it's done via TLS, but you can't be guaranteed that at any one point the message is kept encrypted. You can have this today -- just about any MTA has a configuration to only allow TLS connections, and then you can set up an IMAP server to only allow encrypted connections. End-to-end encryption is the system whereby the moment the message leaves the sender's hands (phone, desktop, whatever) it's encrypted until it gets into the receiver's hands. This is your OpenPGP/etc. You could think of that as "Signal for email". This is apparently not what Magma is doing. Here's the relevant line from Github: https://github.com/lavabit/magma/blob/develop/src/providers/checkers/clamav.c#L387 https://github.com/lavabit/magma/blob/develop/src/providers/... You can see that Magma is writing the email message to a file and instructing ClamAV to scan the file. This is actually worse than writing to a clamd socket, since (imho) it would be easier for someone to monitor a directory in a filesystem on a compromised Magma server. Compare this to how SpamAssassin communicates with ClamAV-- it streams to either the port or socket: https://wiki.apache.org/spamassassin/ClamAVPlugin https://wiki.apache.org/spamassassin/ClamAVPlugin
SwellJoe 9y ago
One of the interesting things about this is the work on DMTP/DMAP, which are "Dark Mail" protocols that have security built in from the ground up (developed by Zimmerman of PGP fame, some Silent Circle folks, and Ladar Levison of Lavabit). It's a pretty neat thing that I hope sees widespread adoption. Fixing email has proven very difficult, due to the incredible inertia of existing systems...but, at some point we really do need to move forward on secure email communication. So, this is really cool stuff.
- deleted 9y ago
  [deleted]
- ape4 9y ago
  This seems to be a paper about it https://www.net.in.tum.de/fileadmin/TUM/NET/NET-2015-09-1/NET-2015-09-1_03.pdf https://www.net.in.tum.de/fileadmin/TUM/NET/NET-2015-09-1/NE...
robert_foss 9y ago
I was hoping to see JMAP support. Is there a reason to prefer DMAP over JMAP?
- ladarlevison 9y ago
  DMAP is designed to work optimally with encrypted message objects. I looked into making it a JSON protocol but JMAP didn't adapt well and was even more complicated than IMAP (if you can believe it).
- ycmbntrthrwaway 9y ago
  JMAP has no E2E encryption: http://jmap.io/#end-to-end-encryption http://jmap.io/#end-to-end-encryption
- SwellJoe 9y ago
  As far as I understand it (which is quite limited, I haven't been able to find a white paper or whatever about DMAP thus far), there's a fundamental difference in what layer of the stack security resides in. JMAP is a vaguely IMAP compatible protocol, in that the mail storage is probably just Maildirs or whatever (with some indexes to make it fast). It is not a complete re-design of how email works. Encryption is just in the network layer between client and server; the server, if compromised, would compromise the mail (unless other encryption features are layered on top, as in the current email standards). That's a perfectly reasonable design decision, given how email has always worked, and the problems inherent in pushing encryption to the client...particularly if the client is a web browser. DMAP is not merely "IMAP with JSON over HTTP" (JMAP isn't just that, but that's a reasonable enough short description). It's a fundamental redesign of a mail client protocol. Again, I haven't read the formal spec or anything, and there is no source for DMAP yet that I can find in the repo (DMTP is there, but no DMAP), but given what it's promising, it can't possibly just be layered on top of IMAP without significant changes. DMAP seems like a very long-term project; maybe years away. I don't know if there are any implementations, yet. And, JMAP is so new as a public thing that I would guess it wasn't on Levison's radar until recently. It recently began the formal standards process, which is cool. But, it's also not really got any complete implementation (just a PoC proxy and an incomplete implementation in Cyrus with plans for Dovecot to get support at some point, too). So, I think it boils down to a few things: JMAP doesn't solve the problem DMAP sets out to solve (which is end-to-end encryption with at-rest encryption on both ends), JMAP is also very new and has no production-ready implementations on client or server, and these are both open sourcing of projects developed as commercial tools (JMAP comes out of Fastmail development, DMAP comes out of Lavabit and Silent Circle). Also, this does appear to have an HTTP server...so, probably something quite similar to JMAP, though certainly not talking the same protocol (since it pre-dates the publication of JMAP details). It could probably be made to support JMAP without a huge amount of work, since it already supports IMAP and HTTP.
HurrdurrHodor 9y ago
Can somebody tell me what this actually does? All I see is an MTA and some hand waving.
- myhf 9y ago
  Some history here: https://en.wikipedia.org/wiki/Lavabit https://en.wikipedia.org/wiki/Lavabit
  - HurrdurrHodor 9y ago
    I know of lavabit but I want to know what this software does.
    - ladarlevison 9y ago
      It's a monolithic mail daemon that speaks SMTP, POP, IMAP and HTTP, and uses DIME to encrypt messages when they arrive unprotected. It has preliminary support for DMTP and DMAP. Basically it's a replacement for your MTA and access server, which you'd run if you wanted a mail server that protected messages using an individuals password.
    - liopleurodon 9y ago
      Here's the talk Levison gave on it at Defcon a few years ago https://www.youtube.com/watch?v=TWzvXaxR6us https://www.youtube.com/watch?v=TWzvXaxR6us
ianopolous 9y ago
At first glance it seems like this will still expose who is emailing who to the network (and potentially other metadata), as well as relying on dns to function. Is that correct?
kijin 9y ago
If the MTA supports regular SMTP, POP and IMAP, then it cannot be said to perform end-to-end encryption. When a client accesses mail through any of these legacy protocols, the server must decrypt all messages before pushing them down the last hop. This used to be Lavabit's way of offering "encrypted" email. If the MTA also supports some other protocol that affords end-to-end encryption, that's great, but now you need to use special client software that is either proprietary or seriously lacking in features. This is ProtonMail's way of offering encrypted email. ProtonMail is currently working on a proxy called ProtonMail Bridge that runs on the client and talks to legacy MUAs. All communication between Bridge and the outside world is still encrypted. That sounds like a good idea to me, since I have no intention to stop using Thunderbird anytime soon. It's also probably the only way to bring end-to-end encryption to a large number of people, since it embraces and accommodates the inertia around email instead of arrogantly telling people to switch and wondering why they don't. When it comes to email, backward compatibility is everything. I'm not sure if Lavavit's Magma can fill the same need; the documentation is rather sparse. But I would gladly run a local MTA if it means I can have my cake (end-to-end encryption) and eat it too (compatibility with legacy MUAs). DIME also looks like a much more open protocol than the vendor-specific stuff that ProtonMail is using.
- ladarlevison 9y ago
  You should look at the DIME spec: darkmail.info/spec specifically where it describes account modes. What we have working now is trustful mode, where the server acts a proxy to allow legacy clients to continue working. We're building DMAP clients which will allow cautious and paranoid mode, but it's taking awhile to get things built properly.
- saurik 9y ago
  While you lose a lot of the benefits of IMAP if the server does not have access to the content of messages, this is absolutely not necessary for POP and the only information which must be transferred in the clear to use SMTP is the TO address (and even then, only in the envelope, not in the actual message). Claiming that these protocols inherently are incompatible with transferring encrypted content is like claiming that any protocol that sits on top of TCP or HTTP is incapable of end-to-end encryption. What I think you are actually trying to say is "compatibility with existing clients without any additional software or plugins", but that is unrelated to the usage of SMTP, POP, and IMAP.
eliaskg 9y ago
Could someone ELI5 what that actually does and how one would use it?
- thephyber 9y ago
  Magma is an email server, supporting many of the common email protocols, designed with security in mind. It was designed and used by the guys at Lavabit (of Edward Snowden fame). It combines outgoing (SMTP) and incoming (POP, *MAP) functionality, so it would be like using Dovecot and Postfix (rival mail server products) together. It requires that you have a relational database (the documents describe MySql) and a key-value store (the documents describe Memcache) to support it and the documentation suggests it's designed for Linux (specifically with instructions for CentOS, which is a Red Hat distro). A mail server is generally installed on an "always-on" computer connected to the internet-connected network. Note that even though it's "designed with security in mind", I don't see any mention of a security audit on either the linked page or the GitHub repo.
konschubert 9y ago
How does it handle Spam?
nwmcsween 9y ago
mm_set in memory.c won't actually do what you want, the asm statement needs fixing and it can be implemented in terms of memset.
- ladarlevison 9y ago
  mm_set is being used. As I recall that pattern was inspired by looking at a libc implementation for memset. The combination of using volatile and the asm("") statements keep the compiler from optimizing away what it thinks are unnecessary memset (like a wipe operation). It's possible that solution doesn't work on every system though. So, how would you improve it?
  - nwmcsween 9y ago
    http://www.5starstandard.ca/projects/myosin/myosin/string/memsset.c http://www.5starstandard.ca/projects/myosin/myosin/string/me... Basically just the right inline asm. But the project as a whole is a bit of a mess, multiple 'memssets' will not do anything to make things more secure than just one.
- sillysaurus3 9y ago
  Is it used anywhere? Whenever I find an obvious bug in a large codebase, it's usually in a section that turns out not to be used. Or used in a section that isn't itself used anywhere, etc.
mike-cardwell 9y ago
I blogged up some security issues with the Lavabit.com service which uses this Magma server. Some of you may be interested - https://www.grepular.com/Lavabit_Security_Misconfiguration https://www.grepular.com/Lavabit_Security_Misconfiguration
j88439h84 9y ago
Is it reasonable to run a secure personal email server with this?
- __jal 9y ago
  I don't mean to be a pain, but before anyone can try to answer that for you, you need to define 'reasonable' and 'secure' in terms of your environment and needs. Are you Just Some Person(tm) with some technical chops, who otherwise isn't going to attract a lot of unwanted attention? Are you someone who does things that might annoy others enough to make them want to mess with you? Who are they and how annoyed might they become? Are you an aspiring Ed Snowden? Do you have confidence in your skill set to 'know' that you've configured this and everything else on the machine securely? If so, do you watch the disclosure lists for vulnerabilities, enforce configuration, audit and monitor, and generally stay on top of these things? Etc. Again, I'm really not trying to be overly pedantic, but there's just no such thing as 'secure' without qualifying it with a description of 'against what'. Similarly, what's 'reasonable' for me - someone who's worked with this stuff my entire career - is absolutely bonkers to suggest to a non-technical person. Anyway, all that said, this doesn't look ready for prime time to me. I'm certainly interested in it, and might play some when I get a chance, but even after it looks ready for use, I'd likely wait for a while to see what other people find before deciding to use it in production. My mail system runs Postfix and Courier with various support systems. Mortals can indeed run this sort of setup, if they're willing to learn a bit. It isn't rocket surgery, but it takes a bit of work.
  - drdaeman 9y ago
    I don't know about the parent comment's requirements, but I'm: 1. Someone who has some servers and a little tiny bit of technical knowledge. Or, to be exactly, I happen to already run Postfix+Dovecot+rspamd for my email. 2. Someone who is interested in personal email security/privacy, in particular in terms of both in-transit and on-storage encryption. I can't realistically watch for every CVE and patch all the holes on the day 1, so I would always love something that would let me trust my servers and network less. I don't have any specific threat model or a designated adversary. I just see "hey, there's a some new stuff that looks promising" (this submission) and wonder if it could improve my life. However, the documentation is sparse, or, I'd better say, nearly non-existent (or I'm looking at a wrong place). So, asking if someone had already spent time to evaluate or had actually used that and can share their experience is a logical choice. Thus, I just wonder - is Magma for me, can I run a secure personal email server with it? Like, use it to replace my Postfix+Dovecot system? Can I expect it to be no less reliable than my current setup (which is just your average run-off-the-mill configuration, documented in every other "how to set up your own mail server" article)? Is it correct to expect it to add some value in terms of security? What the exact those improvements would be?

I guess I'll be the one to ask: Why write an email server with security as priority #1 in C? I'd understand if it was built off of an existing daemon, but it seems that this was written from scratch. There are many other options out there that are reasonably fast and don't have the pitfalls of manual memory management and undefined behavior that C has.

jhasse 9y ago
One reason for this could be that for an open-source application with focus on security it's very important to have many collaboraters, who look at your code and contribute. C is still very popular and many people know it.
- doomrobo 9y ago
  There are plenty of popular safe languages. Java, Go, and Python are all viable options here.
- tptacek 9y ago
  C limits the pool of contributors. It doesn't expand it.
  - devillius 9y ago
    I don't think the Python/Ruby/Node/VB developers are the intended contributors. Source: I'm a Python/Ruby/Node/VB/C/C++ dev
    - tptacek 9y ago
      Why not? (I started my career shipping C code, FWIW.)
  - Alupis 9y ago
    > C limits the pool of contributors. It doesn't expand it. Oh boy, I'm starting to grow weary of folks tirelessly thumping this mantra. It's patently absurd.
    - tptacek 9y ago
      Do go on.
      - Alupis 9y ago
        I do.
        tptacek 9y ago
        No, really. It sounds to me like you think the number of C developers is comparable to the number of high-level language developers. Tell me more.
        Alupis 9y ago
        The overwhelming majority of "high-level language" developers are, at an absolute minimum, familiar enough with C, or C-like languages, to be able to read/contribute to a C project. Heck, most of us, yourself included, started off with C... it's just how it was/is taught. Perhaps your C skills get rusty without use, or perhaps you're not a bit-twiddling magician, but you can still read it, and therefore think about it, and therefore write it. It doesn't limit the pool of contributors... quite the opposite. Now, if you want to have a debate about a better choice of language for other reasons, that might be fair game... but to imply there's a shortage of C developers is just absurd.
        tptacek 9y ago
        I could not disagree more strongly with this. Most of the Python and Ruby developers I know are not comfortable writing C code. More importantly, there's nothing in the world more dangerous than someone who knows just enough to get something working in C. C is a harder language to learn than Ruby, Python, or Javascript. Unlike any of those languages, to write in C you have to constantly keep object life cycles, memory hierarchy, and memory layout in mind. Most of the standard libraries of Python or Ruby --- even simple string manipulation --- are idiomatically written and rewritten by hand in C. Even experienced C programmers forget, routinely and to calamitous affect, how machine integers work. Writing something in C reduces the number of programmers who can contribute to a project. And that reduction is going to get worse and worse over time. When I started my career, if you were going to ship code to clients, C was the industry standard way to accomplish that. A giant chunk of all working programmers used C as their daily language. Today, only a small minority of programmers will ever once write a production C module.
cyphar 9y ago
The most "interesting" thing is that they decided to effectively combine all of the components of a mail server into a single binary. This is actually advertised as a feature ("packaging is hard"), which is quite worrying because having everything in one address space means a minor bug in one piece now affects the integrity of all of the server components.
elif 9y ago
Well, email is not that complicated, three letter agencies in the US have a history of hiding backdoors in technology, and one of the authors had his company destroyed by one of those agencies. If you're writing software for the tinfoil hat crowd, and you're using high level languages and existing libraries to do very basic things, you raise suspicion even if it isn't immediately justifiable... the suspicion that most major routers were routinely pwned wasn't justifiable in April, either. Speed is almost never the primary concern with email, which is a world full of rate limits, throttles, spam filters, transient failures, tarpitting, blacklisting, inbox placement delays, etc.
- doomrobo 9y ago
  >three letter agencies in the US have a history of hiding backdoors in technology So the solution is to use the simplest tech stack possible and put the vulnerabilities in there yourself? If you're using this software, you have to trust somebody. I think most people would sooner trust that Java isn't backdoored than trust that this new C project doesn't have any buffer overflows or memory leaks.
  - ladarlevison 9y ago
    I disagree. The more layers of abstraction you have, the more places there are for a backdoor to hide. Also, unless you're using OpenJDK, Java means downloading a binary blob which could be hiding any number of surprises, intentional, or accidental. With magma I've taken the approach of trying to limit my dependencies to the kernel, and libc. Anything else I use is bundled and thus gets tested extensively for leaks, overflows, etc. That doesn't mean bugs don't exist, but it does mean if they exist, then the source is there for you to inspect and fix.
  - elif 9y ago
    Different strokes for different folks. I think intentional bugs are far worse than unintended, and scanning a c project for buffer overflows is relatively trivial compared to scanning tons of libraries for flaws so obfuscated they made it through review. In my mind technical analysis is incomplete at finding 100% of intended or unintended flaws in either system, but the DIY approach allows for trust in the authors whereas a deep stack makes attribution muddy.
cbuq 9y ago
Your concern of memory management and undefined behavior is not unfounded. But in my opinion all languages have these problems, in the C the blame is shifted to the software engineer instead of the language authors.

djsumdog 9y ago

I see references to ClamAV, and references to soon coming official docker images. So is this a complete MTA and delivery (IMAP/POP/etc) servers? I was digging around on github and the about page, but there doesn't seem to be info on how it does spam detection, or DKIM. Does it support signing messages? I wish they had a more detailed about page and feature list, but I am still excited about this.

mark242 9y ago
The ClamAV dependency concerns me; messages can only be passed to clamd via a plaintext stream. This seems to be at odds with the end-to-end encryption.
- ladarlevison 9y ago
  The ClamAV library is built directly into magma so plain text messages can be scanned without leaving the magma process space. I'd like to eventually have magma fork and scan messages in an isolated process space that doesn't have access to anything else, but that's a long term goal. To clarify. We do not use clamd, or transfer the messages to another daemon.
- matt_wulfeck 9y ago
  Does the "encryption in flight" include interprocess communication? If so there's not much safety in the world!
  - mark242 9y ago
    There's a difference-- point-to-point encryption is what you're talking about. Anytime a message is transferred, it's done via TLS, but you can't be guaranteed that at any one point the message is kept encrypted. You can have this today -- just about any MTA has a configuration to only allow TLS connections, and then you can set up an IMAP server to only allow encrypted connections. End-to-end encryption is the system whereby the moment the message leaves the sender's hands (phone, desktop, whatever) it's encrypted until it gets into the receiver's hands. This is your OpenPGP/etc. You could think of that as "Signal for email". This is apparently not what Magma is doing. Here's the relevant line from Github: https://github.com/lavabit/magma/blob/develop/src/providers/checkers/clamav.c#L387 https://github.com/lavabit/magma/blob/develop/src/providers/... You can see that Magma is writing the email message to a file and instructing ClamAV to scan the file. This is actually worse than writing to a clamd socket, since (imho) it would be easier for someone to monitor a directory in a filesystem on a compromised Magma server. Compare this to how SpamAssassin communicates with ClamAV-- it streams to either the port or socket: https://wiki.apache.org/spamassassin/ClamAVPlugin https://wiki.apache.org/spamassassin/ClamAVPlugin

SwellJoe 9y ago

One of the interesting things about this is the work on DMTP/DMAP, which are "Dark Mail" protocols that have security built in from the ground up (developed by Zimmerman of PGP fame, some Silent Circle folks, and Ladar Levison of Lavabit). It's a pretty neat thing that I hope sees widespread adoption. Fixing email has proven very difficult, due to the incredible inertia of existing systems...but, at some point we really do need to move forward on secure email communication. So, this is really cool stuff.

deleted 9y ago
[deleted]
ape4 9y ago
This seems to be a paper about it https://www.net.in.tum.de/fileadmin/TUM/NET/NET-2015-09-1/NET-2015-09-1_03.pdf https://www.net.in.tum.de/fileadmin/TUM/NET/NET-2015-09-1/NE...

robert_foss 9y ago

I was hoping to see JMAP support. Is there a reason to prefer DMAP over JMAP?

ladarlevison 9y ago
DMAP is designed to work optimally with encrypted message objects. I looked into making it a JSON protocol but JMAP didn't adapt well and was even more complicated than IMAP (if you can believe it).
ycmbntrthrwaway 9y ago
JMAP has no E2E encryption: http://jmap.io/#end-to-end-encryption http://jmap.io/#end-to-end-encryption
SwellJoe 9y ago
As far as I understand it (which is quite limited, I haven't been able to find a white paper or whatever about DMAP thus far), there's a fundamental difference in what layer of the stack security resides in. JMAP is a vaguely IMAP compatible protocol, in that the mail storage is probably just Maildirs or whatever (with some indexes to make it fast). It is not a complete re-design of how email works. Encryption is just in the network layer between client and server; the server, if compromised, would compromise the mail (unless other encryption features are layered on top, as in the current email standards). That's a perfectly reasonable design decision, given how email has always worked, and the problems inherent in pushing encryption to the client...particularly if the client is a web browser. DMAP is not merely "IMAP with JSON over HTTP" (JMAP isn't just that, but that's a reasonable enough short description). It's a fundamental redesign of a mail client protocol. Again, I haven't read the formal spec or anything, and there is no source for DMAP yet that I can find in the repo (DMTP is there, but no DMAP), but given what it's promising, it can't possibly just be layered on top of IMAP without significant changes. DMAP seems like a very long-term project; maybe years away. I don't know if there are any implementations, yet. And, JMAP is so new as a public thing that I would guess it wasn't on Levison's radar until recently. It recently began the formal standards process, which is cool. But, it's also not really got any complete implementation (just a PoC proxy and an incomplete implementation in Cyrus with plans for Dovecot to get support at some point, too). So, I think it boils down to a few things: JMAP doesn't solve the problem DMAP sets out to solve (which is end-to-end encryption with at-rest encryption on both ends), JMAP is also very new and has no production-ready implementations on client or server, and these are both open sourcing of projects developed as commercial tools (JMAP comes out of Fastmail development, DMAP comes out of Lavabit and Silent Circle). Also, this does appear to have an HTTP server...so, probably something quite similar to JMAP, though certainly not talking the same protocol (since it pre-dates the publication of JMAP details). It could probably be made to support JMAP without a huge amount of work, since it already supports IMAP and HTTP.

HurrdurrHodor 9y ago

Can somebody tell me what this actually does? All I see is an MTA and some hand waving.

myhf 9y ago
Some history here: https://en.wikipedia.org/wiki/Lavabit https://en.wikipedia.org/wiki/Lavabit
- HurrdurrHodor 9y ago
  I know of lavabit but I want to know what this software does.
  - ladarlevison 9y ago
    It's a monolithic mail daemon that speaks SMTP, POP, IMAP and HTTP, and uses DIME to encrypt messages when they arrive unprotected. It has preliminary support for DMTP and DMAP. Basically it's a replacement for your MTA and access server, which you'd run if you wanted a mail server that protected messages using an individuals password.
  - liopleurodon 9y ago
    Here's the talk Levison gave on it at Defcon a few years ago https://www.youtube.com/watch?v=TWzvXaxR6us https://www.youtube.com/watch?v=TWzvXaxR6us

ianopolous 9y ago

At first glance it seems like this will still expose who is emailing who to the network (and potentially other metadata), as well as relying on dns to function. Is that correct?

kijin 9y ago

If the MTA supports regular SMTP, POP and IMAP, then it cannot be said to perform end-to-end encryption. When a client accesses mail through any of these legacy protocols, the server must decrypt all messages before pushing them down the last hop. This used to be Lavabit's way of offering "encrypted" email. If the MTA also supports some other protocol that affords end-to-end encryption, that's great, but now you need to use special client software that is either proprietary or seriously lacking in features. This is ProtonMail's way of offering encrypted email. ProtonMail is currently working on a proxy called ProtonMail Bridge that runs on the client and talks to legacy MUAs. All communication between Bridge and the outside world is still encrypted. That sounds like a good idea to me, since I have no intention to stop using Thunderbird anytime soon. It's also probably the only way to bring end-to-end encryption to a large number of people, since it embraces and accommodates the inertia around email instead of arrogantly telling people to switch and wondering why they don't. When it comes to email, backward compatibility is everything. I'm not sure if Lavavit's Magma can fill the same need; the documentation is rather sparse. But I would gladly run a local MTA if it means I can have my cake (end-to-end encryption) and eat it too (compatibility with legacy MUAs). DIME also looks like a much more open protocol than the vendor-specific stuff that ProtonMail is using.

ladarlevison 9y ago
You should look at the DIME spec: darkmail.info/spec specifically where it describes account modes. What we have working now is trustful mode, where the server acts a proxy to allow legacy clients to continue working. We're building DMAP clients which will allow cautious and paranoid mode, but it's taking awhile to get things built properly.
saurik 9y ago
While you lose a lot of the benefits of IMAP if the server does not have access to the content of messages, this is absolutely not necessary for POP and the only information which must be transferred in the clear to use SMTP is the TO address (and even then, only in the envelope, not in the actual message). Claiming that these protocols inherently are incompatible with transferring encrypted content is like claiming that any protocol that sits on top of TCP or HTTP is incapable of end-to-end encryption. What I think you are actually trying to say is "compatibility with existing clients without any additional software or plugins", but that is unrelated to the usage of SMTP, POP, and IMAP.

eliaskg 9y ago

Could someone ELI5 what that actually does and how one would use it?

thephyber 9y ago
Magma is an email server, supporting many of the common email protocols, designed with security in mind. It was designed and used by the guys at Lavabit (of Edward Snowden fame). It combines outgoing (SMTP) and incoming (POP, *MAP) functionality, so it would be like using Dovecot and Postfix (rival mail server products) together. It requires that you have a relational database (the documents describe MySql) and a key-value store (the documents describe Memcache) to support it and the documentation suggests it's designed for Linux (specifically with instructions for CentOS, which is a Red Hat distro). A mail server is generally installed on an "always-on" computer connected to the internet-connected network. Note that even though it's "designed with security in mind", I don't see any mention of a security audit on either the linked page or the GitHub repo.

konschubert 9y ago

How does it handle Spam?

nwmcsween 9y ago

mm_set in memory.c won't actually do what you want, the asm statement needs fixing and it can be implemented in terms of memset.

ladarlevison 9y ago
mm_set is being used. As I recall that pattern was inspired by looking at a libc implementation for memset. The combination of using volatile and the asm("") statements keep the compiler from optimizing away what it thinks are unnecessary memset (like a wipe operation). It's possible that solution doesn't work on every system though. So, how would you improve it?
- nwmcsween 9y ago
  http://www.5starstandard.ca/projects/myosin/myosin/string/memsset.c http://www.5starstandard.ca/projects/myosin/myosin/string/me... Basically just the right inline asm. But the project as a whole is a bit of a mess, multiple 'memssets' will not do anything to make things more secure than just one.
sillysaurus3 9y ago
Is it used anywhere? Whenever I find an obvious bug in a large codebase, it's usually in a section that turns out not to be used. Or used in a section that isn't itself used anywhere, etc.

mike-cardwell 9y ago

I blogged up some security issues with the Lavabit.com service which uses this Magma server. Some of you may be interested - https://www.grepular.com/Lavabit_Security_Misconfiguration https://www.grepular.com/Lavabit_Security_Misconfiguration

j88439h84 9y ago

Is it reasonable to run a secure personal email server with this?

__jal 9y ago
I don't mean to be a pain, but before anyone can try to answer that for you, you need to define 'reasonable' and 'secure' in terms of your environment and needs. Are you Just Some Person(tm) with some technical chops, who otherwise isn't going to attract a lot of unwanted attention? Are you someone who does things that might annoy others enough to make them want to mess with you? Who are they and how annoyed might they become? Are you an aspiring Ed Snowden? Do you have confidence in your skill set to 'know' that you've configured this and everything else on the machine securely? If so, do you watch the disclosure lists for vulnerabilities, enforce configuration, audit and monitor, and generally stay on top of these things? Etc. Again, I'm really not trying to be overly pedantic, but there's just no such thing as 'secure' without qualifying it with a description of 'against what'. Similarly, what's 'reasonable' for me - someone who's worked with this stuff my entire career - is absolutely bonkers to suggest to a non-technical person. Anyway, all that said, this doesn't look ready for prime time to me. I'm certainly interested in it, and might play some when I get a chance, but even after it looks ready for use, I'd likely wait for a while to see what other people find before deciding to use it in production. My mail system runs Postfix and Courier with various support systems. Mortals can indeed run this sort of setup, if they're willing to learn a bit. It isn't rocket surgery, but it takes a bit of work.
- drdaeman 9y ago
  I don't know about the parent comment's requirements, but I'm: 1. Someone who has some servers and a little tiny bit of technical knowledge. Or, to be exactly, I happen to already run Postfix+Dovecot+rspamd for my email. 2. Someone who is interested in personal email security/privacy, in particular in terms of both in-transit and on-storage encryption. I can't realistically watch for every CVE and patch all the holes on the day 1, so I would always love something that would let me trust my servers and network less. I don't have any specific threat model or a designated adversary. I just see "hey, there's a some new stuff that looks promising" (this submission) and wonder if it could improve my life. However, the documentation is sparse, or, I'd better say, nearly non-existent (or I'm looking at a wrong place). So, asking if someone had already spent time to evaluate or had actually used that and can share their experience is a logical choice. Thus, I just wonder - is Magma for me, can I run a secure personal email server with it? Like, use it to replace my Postfix+Dovecot system? Can I expect it to be no less reliable than my current setup (which is just your average run-off-the-mill configuration, documented in every other "how to set up your own mail server" article)? Is it correct to expect it to add some value in terms of security? What the exact those improvements would be?