11 ms·

ZeroMQ – Relicense from LGPL3 and exceptions to MPL 2.0

247 points by excerionsforte 3y ago

excerionsforte 3y ago
Also 4.3.5 is out today with the new license: https://github.com/zeromq/libzmq/releases/tag/v4.3.5 https://github.com/zeromq/libzmq/releases/tag/v4.3.5
- isoprophlex 3y ago
  O wow! It's been a while since the last release, in the back of my mind I had the idea that the project was slowly dying because of Pieter Hintjes' untimely demise... good to see it's still kicking. Very solid software, an absolute joy to work with.
  - jen20 3y ago
    Related: The ZeroMQ guide (published by O'Reilly, but available online at [1]) is one of the finest pieces of technical writing in existence. [1]: https://zguide.zeromq.org https://zguide.zeromq.org
    - sebosp 3y ago
      This. I have read the guide a few times through the years, it's such an interesting dive, specially if one follows the programming exercises... I must admit that any time I work with Tokio or other message passing libraries I end up doing some sort of majordomo, I do not know if this is food, but MDP got stuck in my mind for better or for worse... For example with Tokio mpsc I end up creating majordomo coordinator, clients and services in separate threads, I don't know if this good tbh but it's just so easy to split (and so easy to shoot myself in the foot with synchronize shutdown across threads). Do you know of other guides like this?
- oarfish 3y ago
  Remarkable, up until recently, requests for a new release were somewhat brusquely rejected and marked as spam. e.g. https://github.com/zeromq/libzmq/issues/4315 https://github.com/zeromq/libzmq/issues/4315 and https://github.com/zeromq/libzmq/issues/4455 https://github.com/zeromq/libzmq/issues/4455 I wonder what made the maintainer change his mind.
  - wngr 3y ago
    Issues have been deleted in the meantime ..
vitiral 3y ago
That is damn impressive considering the age of the project. Kudos for getting signoff from all relevant parties!
sdfghswe 3y ago
Could someone who understands these licenses ELI5?
- jacquesm 3y ago
  Improved interop with other open source programs because they used to have a 'one off' license. That makes using ZeroMQ harder than it should be because there is always the question of whether or not their license is more or less permissive than the one that you've picked for your project and risks future unintended side effects. This move should lay that all to rest.
  - bluGill 3y ago
    Lawyers have mostly looked at the common open source licenses long enough to know what they say: what is compatible with each other, what is compatible with their [corporate/clients] other needs, and otherwise how it works. While mostly they have not been tested in court, there general consensus they will hold up somehow (nobody knows for sure and there are a couple hundred countries that could each decide their own thing). When you do your own license though lawyers need to figure everything out for just your project.
- brylie 3y ago
  TLDR LGPL3: https://www.tldrlegal.com/license/gnu-lesser-general-public-license-v3-lgpl-3 https://www.tldrlegal.com/license/gnu-lesser-general-public-... TLDR MPL v2: https://www.tldrlegal.com/license/mozilla-public-license-2-0-mpl-2 https://www.tldrlegal.com/license/mozilla-public-license-2-0...
- CJefferson 3y ago
  The main difference is the LGPL works at the “library” level, and requires the source for the library is open source. It requires in particular you can replace the library, which means you can’t really static link to it from a closed source program (or open source but LGPL incompatible license). MPL works on the “source file” level. You have to release any changes you make to MPL licensed files, but you can link those files into a closed source program any way you like.
CharlieDigital 3y ago
Back in 2014, I was rebuilding the core of an event processing engine. At the time, the decision was between Apache Kafka and rolling our own. After investigating Zookeeper, we decided to roll our own and chose ZeroMQ as the messaging layer as our on-prem customers probably didn't want to own and manage Zookeeper. ZeroMQ was absolutely solid and stable; incredibly trouble free and the only issues we ran into were when IT teams didn't open the ports we documented in the configuration procedure. (The resultant architecture actually looks a lot like Flink) In any case, ZeroMQ is a fantastic piece of technology that I feel like I don't see out in the wild quite enough. It's dead simple and incredibly stable from my experience.
- vxxzy 3y ago
  If you look close at the neuralink demo you’ll notice they use zeromq. I suspect the flexibility and the reduced latency makes a difference.
- perbu 3y ago
  It's also quite fast. I remember we had a similar use-case. This was for collecting live statistics from a sizable Varnish Cache cluster. We wrote a in memory database to store the data. It's been chugging away for about 10 years now and last I heard the zmq traffic alone was about 3Gbps with zero issues.
  - formerly_proven 3y ago
    > It's also quite fast. IIRC it does all I/O on separate background threads which means every operation actually goes through an inter-thread queue. Which can be good and bad.
  - ksimukka 3y ago
    Aside from some scaling issues, it still is a great solution for having real-time insight into the performance of Varnish and the cache. We even went as far as writing a Golang version of the VCS server to better handle some of the scaling challenges. I don’t remember the exact library that we are using to call into ZeroMQ, but the CGO overhead was minimal.
F-W-M 3y ago
Worked on a team that used zeromq for a trading system. Worked well, but if you do not read the guide and get a good grip on what guarantees are provided by zero mq.
- jacquesm 3y ago
  > Worked well, but if you do not read the guide and get a good grip on what guarantees are provided by zero mq. Was that sentence finished?
  - F-W-M 3y ago
    No. I wanted to say that it is very easy to shoot yourself in the foot with zeromq if you put no thought into designing your protocols. It's a task that you have to do, since the basic guarantees of 0MQ are almost never enough and is very well explained in the guide. But you have to read it carefully.
  - slau 3y ago
    It was delivered by a ZeroMQ socket, the \0 byte arrived out of order.
    - sdfghswe 3y ago
      The \0 byte can't arrive out of order in a ZeroMQ socket.
      - pc86 3y ago
        It's a joke dude
        sdfghswe 3y ago
        Is it? It would've been a great joke if it were true that \0 can arrive out of order in a ZeroMQ socket. But as it can't, it's just confusing.
    - 3y ago
richardfontana 3y ago
The LGPLv3 exception had an atypical feature (for *GPL exceptions) and was possibly misdrafted, see: https://github.com/spdx/license-list-XML/issues/1672 https://github.com/spdx/license-list-XML/issues/1672
cryotopippto 3y ago
Poor Pieter Hintjens, he probably did this as soon as he learned about his cancer: -CEO, iMatix Corporation sprl -23 April 2016 By October 2016 he was dead. This PR basically merges his final commit. I find it very sad.
- brodouevencode 3y ago
  I talked with Pieter at some length about his cancer, which he was very open and forthcoming about, and his ultimate decision for euthanasia. He was very stoic about the whole ordeal. He was somewhat depressed about it to start, but eventually came around to realizing the accomplishments and contributions he made throughout his life in both the technical realm and outside. I say all that to mean that anyone being sad about his death is something that Pieter would not have wanted.
  - jwildeboer 3y ago
    I was at his wake party in Brussels. He did that before he died, so he was there. I only have the most positive memories, even though it was complicated. To give you all a bit of insight into his wonderful way of reasoning with his end: there was a T-Shirt we all could sign. It read „I was at my own wake party and all I got was this lousy T-Shirt“. That was Pieter. I can never NOT end with a smile on my face whenever I think about him.
- a_c 3y ago
  Pieter teaches me a lot through his writing. The books ZeroMQ and Confessions of a Necromancer still sit on my bookshelf.
- shadowtree 3y ago
  I can recommend his books, listed here: http://hintjens.com/books http://hintjens.com/books Confessions of a Necromancer is his work biography, I was totally caught off-guard how relatable his stories turned out to be.
  - Jonovono 3y ago
goodpoint 3y ago
What a pity. LGPL provides better protection.
- cmrdporcupine 3y ago
  It depends at this point on what the maintainers want/need from it.
abofh 3y ago
I need to look, but I think one of my clients might owe me 6 figures for this one... I'll never get paid, but I'm definitely going to write the invoice.
cobertos 3y ago
Is it very common to contact _every_ maintainer and back out changes from ones who don't respond for license changes? I feel like I've heard of many larger companies doing relicenses on their open source without this kind of effort.
- diggan 3y ago
  People contacted were people who made major modifications, not every single contributor. And yeah, if the contributor don't agree to the new license, you can either don't do the license change, or remove the code they wrote as they don't agree to the new license. There is more background here: https://github.com/zeromq/libzmq/issues/2376 https://github.com/zeromq/libzmq/issues/2376 Sometimes companies who do "FOSS" make you sign some sort of agreement that they own the code you produce and you won't have any say about re-licensing, so maybe the projects you're thinking about have done that?
- taway1237 3y ago
  They legally have to ask every contributor. The exception/loophole used by big companies (but also FSF for example) is that every contributor had to sign a CLA where they legally reassign ownership of their code to the project owner.
  - sneak 3y ago
    This is why you should never sign a CLA; it allows later relicensing of your work to nonfree licenses. Linux doesn’t have a CLA, and it’s the most popular operating system in the world.
    - loeg 3y ago
      Conversely, as an organization, it makes sense to never accept changes without a CLA.
      - sneak 3y ago
        * an organization that doesn’t believe in software freedoms If you believe in software freedoms, then there will never be any reason to need to relicense, nor would you want to. Free software is an ideology, like human rights. You can’t use it only sometimes and be said to support it.
chromatin 3y ago
Fun ZeroMQ fact: bitcoind has used zeromq since 2015. This is used by client software including lightning node daemons to get notification of incoming transactions and blocks in a highly reliable and rapid way. https://github.com/bitcoin/bitcoin/blob/master/doc/zmq.md https://github.com/bitcoin/bitcoin/blob/master/doc/zmq.md
alfalfasprout 3y ago
ZeroMQ is a fantastic piece of software that will still be in use decades from now. Also an amazing example of a very intuitive C API.
- Galanwe 3y ago
  Did you actually work seriously with / inside 0MQ? It's indeed an intuitive library, but it's far, far from perfect, thus why it has been rewritten countless times by Sustrik (crossroads), then by D'Amore (nanomsg). The state machine in 0MQ is a nightmare of maintenance and a constant source of tricky bugs. The overall "all sockets in one" design makes things such as reconnection and peer identification pretty much impossible. Don't get me wrong, there's a lot of good in 0MQ. But calling it a jewel that will be in use for decades is far from reality.
dsterry 3y ago
Found Pieter and his books a couple years ago and it make a big impact. The Collective Code Construction Contract is a brilliant piece of community engineering and should be adopted/adapted by every FOSS project on the planet. We can accomplish so much more together if we have C4 and similar methods of coordination. Next time you hear of a burnt-out maintainer, have them read https://rfc.zeromq.org/spec/42/ https://rfc.zeromq.org/spec/42/
sebosp 3y ago
How does ZMQ compare to recent message passing libraries? Does it have a place in rust's tokio ecosystem? Are there projects that implement the HA transaction store to disk protocol "out of the box" ? Forgot the name... Binary star something? I always think of working with ZMQ as I really enjoyed it, but I can't see it as easy no deploy as Kafka with so much ACL tooling, easy Auth and debugging, dashboards, metrics... Maybe one more question... What are recent projects that use ZMQ behind the scene that you find a good place to learn more message passing techniques and their tradeoffs?

Also 4.3.5 is out today with the new license: https://github.com/zeromq/libzmq/releases/tag/v4.3.5 https://github.com/zeromq/libzmq/releases/tag/v4.3.5

isoprophlex 3y ago
O wow! It's been a while since the last release, in the back of my mind I had the idea that the project was slowly dying because of Pieter Hintjes' untimely demise... good to see it's still kicking. Very solid software, an absolute joy to work with.
- jen20 3y ago
  Related: The ZeroMQ guide (published by O'Reilly, but available online at [1]) is one of the finest pieces of technical writing in existence. [1]: https://zguide.zeromq.org https://zguide.zeromq.org
  - sebosp 3y ago
    This. I have read the guide a few times through the years, it's such an interesting dive, specially if one follows the programming exercises... I must admit that any time I work with Tokio or other message passing libraries I end up doing some sort of majordomo, I do not know if this is food, but MDP got stuck in my mind for better or for worse... For example with Tokio mpsc I end up creating majordomo coordinator, clients and services in separate threads, I don't know if this good tbh but it's just so easy to split (and so easy to shoot myself in the foot with synchronize shutdown across threads). Do you know of other guides like this?
oarfish 3y ago
Remarkable, up until recently, requests for a new release were somewhat brusquely rejected and marked as spam. e.g. https://github.com/zeromq/libzmq/issues/4315 https://github.com/zeromq/libzmq/issues/4315 and https://github.com/zeromq/libzmq/issues/4455 https://github.com/zeromq/libzmq/issues/4455 I wonder what made the maintainer change his mind.
- wngr 3y ago
  Issues have been deleted in the meantime ..

vitiral 3y ago

That is damn impressive considering the age of the project. Kudos for getting signoff from all relevant parties!

sdfghswe 3y ago

Could someone who understands these licenses ELI5?

jacquesm 3y ago
Improved interop with other open source programs because they used to have a 'one off' license. That makes using ZeroMQ harder than it should be because there is always the question of whether or not their license is more or less permissive than the one that you've picked for your project and risks future unintended side effects. This move should lay that all to rest.
- bluGill 3y ago
  Lawyers have mostly looked at the common open source licenses long enough to know what they say: what is compatible with each other, what is compatible with their [corporate/clients] other needs, and otherwise how it works. While mostly they have not been tested in court, there general consensus they will hold up somehow (nobody knows for sure and there are a couple hundred countries that could each decide their own thing). When you do your own license though lawyers need to figure everything out for just your project.
brylie 3y ago
TLDR LGPL3: https://www.tldrlegal.com/license/gnu-lesser-general-public-license-v3-lgpl-3 https://www.tldrlegal.com/license/gnu-lesser-general-public-... TLDR MPL v2: https://www.tldrlegal.com/license/mozilla-public-license-2-0-mpl-2 https://www.tldrlegal.com/license/mozilla-public-license-2-0...
CJefferson 3y ago
The main difference is the LGPL works at the “library” level, and requires the source for the library is open source. It requires in particular you can replace the library, which means you can’t really static link to it from a closed source program (or open source but LGPL incompatible license). MPL works on the “source file” level. You have to release any changes you make to MPL licensed files, but you can link those files into a closed source program any way you like.

CharlieDigital 3y ago

Back in 2014, I was rebuilding the core of an event processing engine. At the time, the decision was between Apache Kafka and rolling our own. After investigating Zookeeper, we decided to roll our own and chose ZeroMQ as the messaging layer as our on-prem customers probably didn't want to own and manage Zookeeper. ZeroMQ was absolutely solid and stable; incredibly trouble free and the only issues we ran into were when IT teams didn't open the ports we documented in the configuration procedure. (The resultant architecture actually looks a lot like Flink) In any case, ZeroMQ is a fantastic piece of technology that I feel like I don't see out in the wild quite enough. It's dead simple and incredibly stable from my experience.

vxxzy 3y ago
If you look close at the neuralink demo you’ll notice they use zeromq. I suspect the flexibility and the reduced latency makes a difference.
perbu 3y ago
It's also quite fast. I remember we had a similar use-case. This was for collecting live statistics from a sizable Varnish Cache cluster. We wrote a in memory database to store the data. It's been chugging away for about 10 years now and last I heard the zmq traffic alone was about 3Gbps with zero issues.
- formerly_proven 3y ago
  > It's also quite fast. IIRC it does all I/O on separate background threads which means every operation actually goes through an inter-thread queue. Which can be good and bad.
- ksimukka 3y ago
  Aside from some scaling issues, it still is a great solution for having real-time insight into the performance of Varnish and the cache. We even went as far as writing a Golang version of the VCS server to better handle some of the scaling challenges. I don’t remember the exact library that we are using to call into ZeroMQ, but the CGO overhead was minimal.

F-W-M 3y ago

Worked on a team that used zeromq for a trading system. Worked well, but if you do not read the guide and get a good grip on what guarantees are provided by zero mq.

jacquesm 3y ago
> Worked well, but if you do not read the guide and get a good grip on what guarantees are provided by zero mq. Was that sentence finished?
- F-W-M 3y ago
  No. I wanted to say that it is very easy to shoot yourself in the foot with zeromq if you put no thought into designing your protocols. It's a task that you have to do, since the basic guarantees of 0MQ are almost never enough and is very well explained in the guide. But you have to read it carefully.
- slau 3y ago
  It was delivered by a ZeroMQ socket, the \0 byte arrived out of order.
  - sdfghswe 3y ago
    The \0 byte can't arrive out of order in a ZeroMQ socket.
    - pc86 3y ago
      It's a joke dude
      - sdfghswe 3y ago
        Is it? It would've been a great joke if it were true that \0 can arrive out of order in a ZeroMQ socket. But as it can't, it's just confusing.
  - 3y ago

richardfontana 3y ago

The LGPLv3 exception had an atypical feature (for *GPL exceptions) and was possibly misdrafted, see: https://github.com/spdx/license-list-XML/issues/1672 https://github.com/spdx/license-list-XML/issues/1672

cryotopippto 3y ago

Poor Pieter Hintjens, he probably did this as soon as he learned about his cancer: -CEO, iMatix Corporation sprl -23 April 2016 By October 2016 he was dead. This PR basically merges his final commit. I find it very sad.

brodouevencode 3y ago
I talked with Pieter at some length about his cancer, which he was very open and forthcoming about, and his ultimate decision for euthanasia. He was very stoic about the whole ordeal. He was somewhat depressed about it to start, but eventually came around to realizing the accomplishments and contributions he made throughout his life in both the technical realm and outside. I say all that to mean that anyone being sad about his death is something that Pieter would not have wanted.
- jwildeboer 3y ago
  I was at his wake party in Brussels. He did that before he died, so he was there. I only have the most positive memories, even though it was complicated. To give you all a bit of insight into his wonderful way of reasoning with his end: there was a T-Shirt we all could sign. It read „I was at my own wake party and all I got was this lousy T-Shirt“. That was Pieter. I can never NOT end with a smile on my face whenever I think about him.
a_c 3y ago
Pieter teaches me a lot through his writing. The books ZeroMQ and Confessions of a Necromancer still sit on my bookshelf.
shadowtree 3y ago
I can recommend his books, listed here: http://hintjens.com/books http://hintjens.com/books Confessions of a Necromancer is his work biography, I was totally caught off-guard how relatable his stories turned out to be.
- Jonovono 3y ago

goodpoint 3y ago

What a pity. LGPL provides better protection.

cmrdporcupine 3y ago
It depends at this point on what the maintainers want/need from it.

abofh 3y ago

I need to look, but I think one of my clients might owe me 6 figures for this one... I'll never get paid, but I'm definitely going to write the invoice.

cobertos 3y ago

Is it very common to contact _every_ maintainer and back out changes from ones who don't respond for license changes? I feel like I've heard of many larger companies doing relicenses on their open source without this kind of effort.

diggan 3y ago
People contacted were people who made major modifications, not every single contributor. And yeah, if the contributor don't agree to the new license, you can either don't do the license change, or remove the code they wrote as they don't agree to the new license. There is more background here: https://github.com/zeromq/libzmq/issues/2376 https://github.com/zeromq/libzmq/issues/2376 Sometimes companies who do "FOSS" make you sign some sort of agreement that they own the code you produce and you won't have any say about re-licensing, so maybe the projects you're thinking about have done that?
taway1237 3y ago
They legally have to ask every contributor. The exception/loophole used by big companies (but also FSF for example) is that every contributor had to sign a CLA where they legally reassign ownership of their code to the project owner.
- sneak 3y ago
  This is why you should never sign a CLA; it allows later relicensing of your work to nonfree licenses. Linux doesn’t have a CLA, and it’s the most popular operating system in the world.
  - loeg 3y ago
    Conversely, as an organization, it makes sense to never accept changes without a CLA.
    - sneak 3y ago
      * an organization that doesn’t believe in software freedoms If you believe in software freedoms, then there will never be any reason to need to relicense, nor would you want to. Free software is an ideology, like human rights. You can’t use it only sometimes and be said to support it.

chromatin 3y ago

Fun ZeroMQ fact: bitcoind has used zeromq since 2015. This is used by client software including lightning node daemons to get notification of incoming transactions and blocks in a highly reliable and rapid way. https://github.com/bitcoin/bitcoin/blob/master/doc/zmq.md https://github.com/bitcoin/bitcoin/blob/master/doc/zmq.md

alfalfasprout 3y ago

ZeroMQ is a fantastic piece of software that will still be in use decades from now. Also an amazing example of a very intuitive C API.

Galanwe 3y ago
Did you actually work seriously with / inside 0MQ? It's indeed an intuitive library, but it's far, far from perfect, thus why it has been rewritten countless times by Sustrik (crossroads), then by D'Amore (nanomsg). The state machine in 0MQ is a nightmare of maintenance and a constant source of tricky bugs. The overall "all sockets in one" design makes things such as reconnection and peer identification pretty much impossible. Don't get me wrong, there's a lot of good in 0MQ. But calling it a jewel that will be in use for decades is far from reality.

dsterry 3y ago

Found Pieter and his books a couple years ago and it make a big impact. The Collective Code Construction Contract is a brilliant piece of community engineering and should be adopted/adapted by every FOSS project on the planet. We can accomplish so much more together if we have C4 and similar methods of coordination. Next time you hear of a burnt-out maintainer, have them read https://rfc.zeromq.org/spec/42/ https://rfc.zeromq.org/spec/42/

sebosp 3y ago

How does ZMQ compare to recent message passing libraries? Does it have a place in rust's tokio ecosystem? Are there projects that implement the HA transaction store to disk protocol "out of the box" ? Forgot the name... Binary star something? I always think of working with ZMQ as I really enjoyed it, but I can't see it as easy no deploy as Kafka with so much ACL tooling, easy Auth and debugging, dashboards, metrics... Maybe one more question... What are recent projects that use ZMQ behind the scene that you find a good place to learn more message passing techniques and their tradeoffs?