8 ms·

NeoPG – an opiniated fork of GnuPG 2

112 points by sphinxc0re 9y ago

deleted 9y ago
[deleted]
pecg 9y ago
Writing a competent, simple replacement for GnuPG, yet PGP fully compatible, is one of the things I plan to do on the long road, thus I think neopg is a good idea, except the fact that they decided to use C++. I know that type safety is one of the considerations, but OOP always results in ambiguous unnecessarily complex code, and for me that defeats the purpose of simplicity. Nevertheless, the GnuPG codebase is a mess.
- nerdponx 9y ago
  They explicitly address the "Why C++?" question here: https://neopg.io/blog/cplusplus/ https://neopg.io/blog/cplusplus/ Choice quote: There are many programming languages, and I believe in picking the right tool for the job. In the case of NeoPG, the priorities were: - Support for strong cryptography. - Compatibility with C application developers. - Convert legacy code quickly. - Tool support for QA. Everything else is, at this point, a secondary concern. The Sequoia Project uses Rust, and I envy that. But the first thing they had to do was to wrap an existing C crypto library (they choose libnettle), because there is no high quality crypto library for Rust yet. That is their challenge. My challenge will be to stay focussed on the parts of C++ that are actually helpful, and not get bogged down by the rest.
  - nickpsecurity 9y ago
    Is using a C library or crypto library that hard in Rust? That's first Ive heard of it being a problem. In high-assurance, it was standard to limit unsafety to the stuff behind interfaces of unsafe modules (incl FFI) in otherwise memory-safe, systems language. We didn't see this as a detriment esp if was C with its wide support.
  - vvanders 9y ago
    > wrap an existing C crypto library I know it's mostly semantics but calling C from C++ still requires some wrapping(extern "C", integrating build system, etc). I've found with bindgen unless there's some crazy macro shenanigans going on it's actually quicker for me to integrate C libraries into Rust then wrangling CMake/Make/etc. The C FFI is very much a first-class citizen in Rust. Even doubly so if we're talking about a cross-platform library.
    - lambdafu 9y ago
      NeoPG uses Botan, which is a crypto library written in C++. I recommend reading its source code and comparing it with, for example, libgcrypt.
      - adultSwim 9y ago
        Is most of the removed code functionality that is now in the crypto library used, Botan? A blog post on the decision to use an external library might be good. Is there a reason to think that the library will be better maintained than code with similar functionality in GPG? Great work by the way! Thank you!
- zerokernel 9y ago
  C++ has a long and successful history of application development, regardless of project size (from 0 to >100 MLOC; from 1 guy sitting in his attic to thousands of engineers). Much like Java it isn't "hip". Instead people like to use "hip" languages with an underdeveloped ecosystem. Typical example: analysis tools, e.g. code analysers, performance and profiling tools. Comments like "couldn't they have used $niche-lang.org instead of [C++/Java/...]" bore me, to be honest.
  - wyager 9y ago
    The fact that C++ is so widely used reflects the fact that it’s both old and acceptably designed. It does not even remotely indicate that it’s universally a good tool choice. I think C++ is not a great choice for complex crypto software because it doesn’t have very good safety-by-construction properties and it has to deal with (extremely) untrusted and probably malicious input. Parsers written in C or C++ are historically one of the biggest attack vectors out there.
    - hh3k0 9y ago
      Okay, I'll bite. What language would be "a great choice" for complex cryptography applications, in your opinion?
      - xiphias 9y ago
        At this point Rust is the clear winner, as it's providing higher safety for similar run-time speed.
      - wyager 9y ago
        Depends on the application. I would personally use Rust for embedded applications (since it’s low-overhead and ADTs are basically a requirement for safe parsers) and Haskell for non-embedded. But if neither of those float your boat, there are other safe-by-construction parser-friendly languages available, mostly from the ML family.
      - nickpsecurity 9y ago
- b5 9y ago
  What $LANG would you use if you had the time to dedicate to the project and were rewriting it starting today? Why would you choose $LANG? (Not an attack -- a genuine question because I'm interested in your choice and reasoning.)
  - qubex 9y ago
    I’m not faulting you at all, but... different people know different tools, place premiums on different features, and of reach different conclusions as to what they wish to use. It's just an opinion, and comparing opinions is such a sterile exercise. Whatever value you assign to $LANG, you will always encounter a denizen on HackerNews that disparages $LANG and would've used $LANG-prime, but by not actually moving first to start a project to achieve that aim with the tool of their choice, they kind-of ceded their right to criticise. That’s my point of view, at least: don't criticise the artist's choice of tools. It's rude.
- emmelaich 9y ago
  Using C++ by no means implies using OOP. C++ has an excellent C FFI :-)
vesinisa 9y ago
Sounds great. I always wondered why GPG was so complex.
- sigsergv 9y ago
  GPG is not that complex, but PGP concept, keys structures and workflows - they are really hard. And this project has to implement all those hard elements too.
  - Shoothe 9y ago
    OpenPGP is kind of like git - if you understand the underlying concepts (in case of OpenPGP this is RFC 4880) then it is simple. GPG is very old and created in different times so it has many quirks at the implementation and UI level but I find the encoding and structures quite simple (there are some weird choices of course). I wonder what do you think is complex? Trust model maybe? Different kinds of signatures? Negotiating algorithms to use?
    - heavenlyblue 9y ago
      I think while I probably understand the underlying concepts better (cryptography-wise), there isn't much _practical_ infromation about GPG in the wild. Just as much as knowing how git works under the hood doesn't necessarily make you great at managing branches in git. More often than not it's way easier to just go with a sort of ready-made recipe, so that your workflow would be easily accepted by the industry; rather than read the doc.
      - Promarged 9y ago
        I think except the cryptography there is just trust calculations and encoding. These two articles describe trust in detail: https://www.linux.com/learn/pgp-web-trust-core-concepts-behind-trusted-communication https://www.linux.com/learn/pgp-web-trust-core-concepts-behi... https://www.linuxfoundation.org/blog/pgp-web-of-trust-delegated-trust-and-keyservers/ https://www.linuxfoundation.org/blog/pgp-web-of-trust-delega... GPG esoteric options is also a good read: https://www.gnupg.org/documentation/manuals/gnupg/GPG-Esoteric-Options.html https://www.gnupg.org/documentation/manuals/gnupg/GPG-Esoter... Besides that... the RFC itself I suppose: https://tools.ietf.org/html/rfc4880 https://tools.ietf.org/html/rfc4880
techwizrd 9y ago
I'm glad that folks are starting to take an interest into improving GnuPG (and PGP in general). I've been working on an SKS replacement[0] in Rust and I've had to dive into the OpenPGP message format, pgpdump, GnuPG, etc. and it's definitely a bit of a mess. 0: https://github.com/srct/sks-rs/ https://github.com/srct/sks-rs/
agrinman 9y ago
Glad someone is working on this! One of the reasons we built PGP signing capabilities into Krypton [0] is to make it easy for anyone to sign their Git commits/tags. Almost nobody uses this awesome feature of Git. We even ended up implementing parts of the OpenPGP spec in Swift [1]. 0: https://krypt.co/docs/start/code-signing.html https://krypt.co/docs/start/code-signing.html 1: https://github.com/kryptco/swift-pgp https://github.com/kryptco/swift-pgp
exabrial 9y ago
Nice! I like the idea from the descriptions on the first page
Sir_Cmpwn 9y ago
The switch to C++ is incredibly poorly justified. I don't care how much safer you think it is (and it really isn't), the language is way more complicated and error-prone and rewriting the code is ALWAYS going to introduce many, many bugs than would be found by simply maintaining the code. This guy also switched on -fpermissive as part of the process, which disables warnings which are there for a reason! This guy has also done a lot of lovely things like committing half-assed changes to master (the build breaks more often than it passes!) which have clearly undergone no review. Anyone this person convinces to use and contribute his fork are going to take time and money (yes, money - the guy put up a Patreon campaign and BountySource page) away from GnuPG, which sorely needs it, as we all hopefully remember. Please do not use this, and I hope the authors give it up.
- lambdafu 9y ago
  Hi, I'm "this guy". Thanks for pointing out -fpermissive, which was needed during the conversion for the legacy code. Since then I fixed all the warnings, so I can remove the flag (https://github.com/das-labor/neopg/commit/5359aab7885628554cd1c4418f574ed5b2d27e71 https://github.com/das-labor/neopg/commit/5359aab7885628554c...). I would love to do code review, but I need a second developer for that! Thank you for your interest and taking the time to write down your criticism.
outworlder 9y ago
> C++ is the better C Oh no it isn't. It does have more features, and for some fields, sure it is better (user interfaces, simulations). However, this is security software we are talking about. C++ introduces more complexity as the spec is an order of magnitude bigger than, say, C99. More complex language spec, more complex tooling, lots of places for bugs to hide. It can help with classic buffer overflows and the like, sure. But it introduces its own set of issues. >Many of these issues are well-known or can be easily researched. They have been documented many times, and can be avoided. Right. The issues can be avoided. You just need to not make mistakes, right? Also, this is worrying: > Converting the legacy code base (490,000 lines of code) to C++ was a straightforward, mechanical task that took only a couple of hours So we are good? The fact that it compiles (and maybe even works) doesn't automatically validates it as a secure rewrite (and it is a rewrite, no matter how similar the common language set is). The stakes are just much higher for something like GPG. I do wish this project good luck, though.
- beojan 9y ago
  > Converting the legacy code base (490,000 lines of code) to C++ was a straightforward, mechanical task that took only a couple of hours That sounds like they haven't actually converted it to C++ so much as compiled it as C++. That means they're probably still using raw owning pointers and no RAII, which means you get none of the safety benefits of C++ (which do exist, so long as you use modern C++ practices like smart pointers, and containers instead of C-style arrays).
  - lambdafu 9y ago
    This is true. But it is a necessary first step and some refactoring has already been done. It's a long way. Many comments here are missing the main inspiration of using C++, which is the Botan crypto library.
jancsika 9y ago
Should the NeoPG project mention why GPGME wasn't a suitable compromise? I understand the author wants to clean up the core and the API, but if there's already a project that cleans up the API isn't that good enough? Especially considering the standard it implements-- PGP-- is still quite complicated.
- jopsen 9y ago
  > Especially considering the standard it implements-- PGP-- is still quite complicated. A standard without multiple independent implementations is a bit unhealthy..
- aparadja 9y ago
  Not exactly related to NeoPG, but GPGME in general: Doesn't GPGME assume that the user already has GnuPG 2.x installed and running on their system? This might just be a misconception I have, but I always thought you couldn't make a batteries-included, self-contained, works-out-of-the-box app if you used GPGME. You'd first have to tell the user to go get a GnuPG implementation from somewhere. I'd be super happy to be wrong on this.
- lambdafu 9y ago
  Yes, I should document that. GPGME only exposes a high-level API, and application developers often want more control. For example, you can't inspect key material before importing it, but importing a key is not a reversible operation - so applications sometimes use a temporary HOMEDIR for GPGME/GnuPG to import the key there and inspect it with a keylisting. It can be very cumbersome.
MrBingley 9y ago
Hmmm, I don't think converting a 500k codebase in C to C++ has a meaningful cost/benefit ratio, especially for security critical code like this. It might be justified to do in something like Rust, but at that point I think it's better to move on from GPG entirely. The OpenPGP standard itself is showing its age (eg. no perfect forward secrecy) and is extremely complicated. I think GPG should be left as-is, and the community gradually move to something simpler, like OpenBSD's signify[0]. [0] https://www.openbsd.org/papers/bsdcan-signify.html https://www.openbsd.org/papers/bsdcan-signify.html
- lambdafu 9y ago
  You might want to check out https://sequoia-pgp.org https://sequoia-pgp.org - a new OpenPGP implementation written from scratch in Rust within the PEP project.
etu 9y ago
I did see this during day 4 of 32C3's Lightning talks. Found the link with time-index here: https://media.ccc.de/v/34c3-9258-lightning_talks_day_4#t=2845 https://media.ccc.de/v/34c3-9258-lightning_talks_day_4#t=284... Starts about 47:25 in if the time-index doesn't work for you.
rcdwealth 9y ago
https://github.com/das-labor/neopg/blob/master/license.txt https://github.com/das-labor/neopg/blob/master/license.txt It may be opinionated but not well thought of a fork. How is possible to take GPL 3.0 code, say how it is "more restricted", miss to mention the point of what would happen if it is not, and to claim that any additional code to the project shall be licensed under the BSD type of the license. The author has some serious misunderstands about licensing issues.

112 points by sphinxc0re 9y ago

deleted 9y ago

[deleted]

pecg 9y ago

Writing a competent, simple replacement for GnuPG, yet PGP fully compatible, is one of the things I plan to do on the long road, thus I think neopg is a good idea, except the fact that they decided to use C++. I know that type safety is one of the considerations, but OOP always results in ambiguous unnecessarily complex code, and for me that defeats the purpose of simplicity. Nevertheless, the GnuPG codebase is a mess.

nerdponx 9y ago
They explicitly address the "Why C++?" question here: https://neopg.io/blog/cplusplus/ https://neopg.io/blog/cplusplus/ Choice quote: There are many programming languages, and I believe in picking the right tool for the job. In the case of NeoPG, the priorities were: - Support for strong cryptography. - Compatibility with C application developers. - Convert legacy code quickly. - Tool support for QA. Everything else is, at this point, a secondary concern. The Sequoia Project uses Rust, and I envy that. But the first thing they had to do was to wrap an existing C crypto library (they choose libnettle), because there is no high quality crypto library for Rust yet. That is their challenge. My challenge will be to stay focussed on the parts of C++ that are actually helpful, and not get bogged down by the rest.
- nickpsecurity 9y ago
  Is using a C library or crypto library that hard in Rust? That's first Ive heard of it being a problem. In high-assurance, it was standard to limit unsafety to the stuff behind interfaces of unsafe modules (incl FFI) in otherwise memory-safe, systems language. We didn't see this as a detriment esp if was C with its wide support.
- vvanders 9y ago
  > wrap an existing C crypto library I know it's mostly semantics but calling C from C++ still requires some wrapping(extern "C", integrating build system, etc). I've found with bindgen unless there's some crazy macro shenanigans going on it's actually quicker for me to integrate C libraries into Rust then wrangling CMake/Make/etc. The C FFI is very much a first-class citizen in Rust. Even doubly so if we're talking about a cross-platform library.
  - lambdafu 9y ago
    NeoPG uses Botan, which is a crypto library written in C++. I recommend reading its source code and comparing it with, for example, libgcrypt.
    - adultSwim 9y ago
      Is most of the removed code functionality that is now in the crypto library used, Botan? A blog post on the decision to use an external library might be good. Is there a reason to think that the library will be better maintained than code with similar functionality in GPG? Great work by the way! Thank you!
zerokernel 9y ago
C++ has a long and successful history of application development, regardless of project size (from 0 to >100 MLOC; from 1 guy sitting in his attic to thousands of engineers). Much like Java it isn't "hip". Instead people like to use "hip" languages with an underdeveloped ecosystem. Typical example: analysis tools, e.g. code analysers, performance and profiling tools. Comments like "couldn't they have used $niche-lang.org instead of [C++/Java/...]" bore me, to be honest.
- wyager 9y ago
  The fact that C++ is so widely used reflects the fact that it’s both old and acceptably designed. It does not even remotely indicate that it’s universally a good tool choice. I think C++ is not a great choice for complex crypto software because it doesn’t have very good safety-by-construction properties and it has to deal with (extremely) untrusted and probably malicious input. Parsers written in C or C++ are historically one of the biggest attack vectors out there.
  - hh3k0 9y ago
    Okay, I'll bite. What language would be "a great choice" for complex cryptography applications, in your opinion?
    - xiphias 9y ago
      At this point Rust is the clear winner, as it's providing higher safety for similar run-time speed.
    - wyager 9y ago
      Depends on the application. I would personally use Rust for embedded applications (since it’s low-overhead and ADTs are basically a requirement for safe parsers) and Haskell for non-embedded. But if neither of those float your boat, there are other safe-by-construction parser-friendly languages available, mostly from the ML family.
    - nickpsecurity 9y ago
b5 9y ago
What $LANG would you use if you had the time to dedicate to the project and were rewriting it starting today? Why would you choose $LANG? (Not an attack -- a genuine question because I'm interested in your choice and reasoning.)
- qubex 9y ago
  I’m not faulting you at all, but... different people know different tools, place premiums on different features, and of reach different conclusions as to what they wish to use. It's just an opinion, and comparing opinions is such a sterile exercise. Whatever value you assign to $LANG, you will always encounter a denizen on HackerNews that disparages $LANG and would've used $LANG-prime, but by not actually moving first to start a project to achieve that aim with the tool of their choice, they kind-of ceded their right to criticise. That’s my point of view, at least: don't criticise the artist's choice of tools. It's rude.
emmelaich 9y ago
Using C++ by no means implies using OOP. C++ has an excellent C FFI :-)

vesinisa 9y ago

Sounds great. I always wondered why GPG was so complex.

sigsergv 9y ago
GPG is not that complex, but PGP concept, keys structures and workflows - they are really hard. And this project has to implement all those hard elements too.
- Shoothe 9y ago
  OpenPGP is kind of like git - if you understand the underlying concepts (in case of OpenPGP this is RFC 4880) then it is simple. GPG is very old and created in different times so it has many quirks at the implementation and UI level but I find the encoding and structures quite simple (there are some weird choices of course). I wonder what do you think is complex? Trust model maybe? Different kinds of signatures? Negotiating algorithms to use?
  - heavenlyblue 9y ago
    I think while I probably understand the underlying concepts better (cryptography-wise), there isn't much _practical_ infromation about GPG in the wild. Just as much as knowing how git works under the hood doesn't necessarily make you great at managing branches in git. More often than not it's way easier to just go with a sort of ready-made recipe, so that your workflow would be easily accepted by the industry; rather than read the doc.
    - Promarged 9y ago
      I think except the cryptography there is just trust calculations and encoding. These two articles describe trust in detail: https://www.linux.com/learn/pgp-web-trust-core-concepts-behind-trusted-communication https://www.linux.com/learn/pgp-web-trust-core-concepts-behi... https://www.linuxfoundation.org/blog/pgp-web-of-trust-delegated-trust-and-keyservers/ https://www.linuxfoundation.org/blog/pgp-web-of-trust-delega... GPG esoteric options is also a good read: https://www.gnupg.org/documentation/manuals/gnupg/GPG-Esoteric-Options.html https://www.gnupg.org/documentation/manuals/gnupg/GPG-Esoter... Besides that... the RFC itself I suppose: https://tools.ietf.org/html/rfc4880 https://tools.ietf.org/html/rfc4880

techwizrd 9y ago

I'm glad that folks are starting to take an interest into improving GnuPG (and PGP in general). I've been working on an SKS replacement[0] in Rust and I've had to dive into the OpenPGP message format, pgpdump, GnuPG, etc. and it's definitely a bit of a mess. 0: https://github.com/srct/sks-rs/ https://github.com/srct/sks-rs/

agrinman 9y ago

Glad someone is working on this! One of the reasons we built PGP signing capabilities into Krypton [0] is to make it easy for anyone to sign their Git commits/tags. Almost nobody uses this awesome feature of Git. We even ended up implementing parts of the OpenPGP spec in Swift [1]. 0: https://krypt.co/docs/start/code-signing.html https://krypt.co/docs/start/code-signing.html 1: https://github.com/kryptco/swift-pgp https://github.com/kryptco/swift-pgp

exabrial 9y ago

Nice! I like the idea from the descriptions on the first page

Sir_Cmpwn 9y ago

The switch to C++ is incredibly poorly justified. I don't care how much safer you think it is (and it really isn't), the language is way more complicated and error-prone and rewriting the code is ALWAYS going to introduce many, many bugs than would be found by simply maintaining the code. This guy also switched on -fpermissive as part of the process, which disables warnings which are there for a reason! This guy has also done a lot of lovely things like committing half-assed changes to master (the build breaks more often than it passes!) which have clearly undergone no review. Anyone this person convinces to use and contribute his fork are going to take time and money (yes, money - the guy put up a Patreon campaign and BountySource page) away from GnuPG, which sorely needs it, as we all hopefully remember. Please do not use this, and I hope the authors give it up.

lambdafu 9y ago
Hi, I'm "this guy". Thanks for pointing out -fpermissive, which was needed during the conversion for the legacy code. Since then I fixed all the warnings, so I can remove the flag (https://github.com/das-labor/neopg/commit/5359aab7885628554cd1c4418f574ed5b2d27e71 https://github.com/das-labor/neopg/commit/5359aab7885628554c...). I would love to do code review, but I need a second developer for that! Thank you for your interest and taking the time to write down your criticism.

outworlder 9y ago

> C++ is the better C Oh no it isn't. It does have more features, and for some fields, sure it is better (user interfaces, simulations). However, this is security software we are talking about. C++ introduces more complexity as the spec is an order of magnitude bigger than, say, C99. More complex language spec, more complex tooling, lots of places for bugs to hide. It can help with classic buffer overflows and the like, sure. But it introduces its own set of issues. >Many of these issues are well-known or can be easily researched. They have been documented many times, and can be avoided. Right. The issues can be avoided. You just need to not make mistakes, right? Also, this is worrying: > Converting the legacy code base (490,000 lines of code) to C++ was a straightforward, mechanical task that took only a couple of hours So we are good? The fact that it compiles (and maybe even works) doesn't automatically validates it as a secure rewrite (and it is a rewrite, no matter how similar the common language set is). The stakes are just much higher for something like GPG. I do wish this project good luck, though.

beojan 9y ago
> Converting the legacy code base (490,000 lines of code) to C++ was a straightforward, mechanical task that took only a couple of hours That sounds like they haven't actually converted it to C++ so much as compiled it as C++. That means they're probably still using raw owning pointers and no RAII, which means you get none of the safety benefits of C++ (which do exist, so long as you use modern C++ practices like smart pointers, and containers instead of C-style arrays).
- lambdafu 9y ago
  This is true. But it is a necessary first step and some refactoring has already been done. It's a long way. Many comments here are missing the main inspiration of using C++, which is the Botan crypto library.

jancsika 9y ago

Should the NeoPG project mention why GPGME wasn't a suitable compromise? I understand the author wants to clean up the core and the API, but if there's already a project that cleans up the API isn't that good enough? Especially considering the standard it implements-- PGP-- is still quite complicated.

jopsen 9y ago
> Especially considering the standard it implements-- PGP-- is still quite complicated. A standard without multiple independent implementations is a bit unhealthy..
aparadja 9y ago
Not exactly related to NeoPG, but GPGME in general: Doesn't GPGME assume that the user already has GnuPG 2.x installed and running on their system? This might just be a misconception I have, but I always thought you couldn't make a batteries-included, self-contained, works-out-of-the-box app if you used GPGME. You'd first have to tell the user to go get a GnuPG implementation from somewhere. I'd be super happy to be wrong on this.
lambdafu 9y ago
Yes, I should document that. GPGME only exposes a high-level API, and application developers often want more control. For example, you can't inspect key material before importing it, but importing a key is not a reversible operation - so applications sometimes use a temporary HOMEDIR for GPGME/GnuPG to import the key there and inspect it with a keylisting. It can be very cumbersome.

MrBingley 9y ago

Hmmm, I don't think converting a 500k codebase in C to C++ has a meaningful cost/benefit ratio, especially for security critical code like this. It might be justified to do in something like Rust, but at that point I think it's better to move on from GPG entirely. The OpenPGP standard itself is showing its age (eg. no perfect forward secrecy) and is extremely complicated. I think GPG should be left as-is, and the community gradually move to something simpler, like OpenBSD's signify[0]. [0] https://www.openbsd.org/papers/bsdcan-signify.html https://www.openbsd.org/papers/bsdcan-signify.html

lambdafu 9y ago
You might want to check out https://sequoia-pgp.org https://sequoia-pgp.org - a new OpenPGP implementation written from scratch in Rust within the PEP project.

etu 9y ago

I did see this during day 4 of 32C3's Lightning talks. Found the link with time-index here: https://media.ccc.de/v/34c3-9258-lightning_talks_day_4#t=2845 https://media.ccc.de/v/34c3-9258-lightning_talks_day_4#t=284... Starts about 47:25 in if the time-index doesn't work for you.

rcdwealth 9y ago

https://github.com/das-labor/neopg/blob/master/license.txt https://github.com/das-labor/neopg/blob/master/license.txt It may be opinionated but not well thought of a fork. How is possible to take GPL 3.0 code, say how it is "more restricted", miss to mention the point of what would happen if it is not, and to claim that any additional code to the project shall be licensed under the BSD type of the license. The author has some serious misunderstands about licensing issues.