4 ms·

Imagine being ok with trillion dollar companies using opensource code you wrote, for free. Imagine if the people who work at those companies aren't your enemy.

Imagine being ok with trillion dollar companies using opensource code you wrote, for free. Imagine if the people who work at those companies aren't your enemy. But instead are potential customers and potential collaborators, who see value in your work and want it to succeed. Imagine if there was a good implementation of some idea, expressed as code. And we can all share and contribute to that single good implementation to avoid pointless work and avoid fragmentation. Imagine people from all across the world contributing both financially and via bug reports and pull requests to making that implementation succeed. This fear of being exploited by big companies seems to motivate a lot of people in the free software world, and personally I find it pretty odious. Why shouldn't big businesses use my code? The better opensource companies (Google, new Microsoft, Facebook, etc) contribute back to the community in all sorts of ways; and I consider that a net positive on the world. I've seen this fear show up for decades in all sorts of contexts, and I think I disagree with it every time. For example: - A few years ago I was involved in a big argument over the licence for OpenStreetMaps. There were essentially two camps: the "Oh no what if (big evil corp) uses our community mapping data to make their map better" camp, who wanted a restrictive anti-company license. And people like me who just wanted to give a gift of mapping data to the world. - This is the gap between GNU GPL (especially GPL3) and BSD/MIT licenses. As an opensource maintainer, I do have a problem with people who work at big companies expecting me to volunteer my time responding to issues on github. I'd like big companies who file issues to contribute financially in exchange for my attention. But if they don't pay, I'm more than happy for them to use my code. I made it for everyone. I'd really like it if fewer people needed to reinvent the wheel for silly licensing reasons.

anonymoushn 4y ago
PRs to improve zstd's embarrassingly slow decoder require you to assign your firstborn to Zuck and new Microsoft is selling me a subscription to use my laundered open source code to generate autocomplete suggestions in their closed-source IDE that pegs 2 cores and needs 4 gigs of memory (on the remote machine only) to edit 50k text files over ssh
- mappu 4y ago
  Zstd's decoder is very very fast for what it does, what ideas do you have to improve it? It's unlikely to beat lzo family on decompression speed, but that's a different class of compressor entirely. And probably there are still some more tricks zstd can learn from closed-source Oodle leviathan, but i don't think much more can be learned.
  - anonymoushn 4y ago
    I expect any "perform a conversion of some sort on a byte stream" implementation that uses 0 SIMD instructions and is not memory bound is leaving a lot of performance on the table, especially if one is permitted to mess with the design and layout of the input to make it more amenable to vectorized consumption. I cannot confidently claim that we're missing a a >2x speedup in this case though, it may be as low as ~1.3x or something.
    - mappu 4y ago
      I checked the zstd source and i'm surprised to see you're right - There's a little x86_64 assembly but I don't see a single SIMD instruction anywhere, and no intrinsics neither. Seems like brotli is the same. I assume zstd still gains something from SIMD autovectorization in the compiler, that might be interesting to benchmark with- and without- such a flag. Since the zstd bytestream got frozen in RFC8478, messing with the layout too much will require a zstd2 and moving the whole world again to use it (linux kernel compression, rpm binary format, etc)
- bscphil 4y ago
  Ah yes, the embarrassingly slow decoder that just happens to be on the Pareto frontier for decompression speed: http://www.mattmahoney.net/dc/text.html http://www.mattmahoney.net/dc/text.html
  - anonymoushn 4y ago
    If you exclude oodle from your benchmarks?
thrown_22 4y ago
>Imagine if the people who work at those companies aren't your enemy. But instead are potential customers and potential collaborators, who see value in your work and want it to succeed. Yes, but we'd have to imagine it isn't microsoft doing it. You'd think people would learn the dozenth time it happens, but no, here we are again with copilot and VS Studio Code.
- josephg 4y ago
  Can you be more specific? What, exactly, are you mad about microsoft doing with vscode? Copilot is a whole other beast. I don't think there's any simple answers there.
  - happymellon 4y ago
    Probably presenting a code editor with IDE like extensions that are all Open Source, and then gradually killing the Open Source extensions and switching them out for proprietary ones.
    - MonkeyClub 4y ago
      Which is exactly the EEE technique, “patented” by Microsoft: https://en.m.wikipedia.org/wiki/Embrace,_extend,_and_extinguish https://en.m.wikipedia.org/wiki/Embrace,_extend,_and_extingu... If GitHub, VS code, an MS Linux, and perhaps somehow LinkedIn, prove to be a joint front to EEE the F/OSS community, what can I say... Seems somewhat unlikely though, given the number of allowances that must be made for that to hold true.
      - happymellon 4y ago
        Well Github is struggling to not abuse its Free Software users, VS Code has failed to remain Open Source, I'm not sure about LinkedIn because that never pretended to heart Linux so that gets a pass, and MS Linux isn't really a thing. Although if you are to believe the terrible kernel commits they were attempting to make to allow them to do an Nvidia closed source shim then it wouldn't shock me.
LeFantome 4y ago
I agree. Being ok with big company participation gets you Linux and / or FreeBSD. Not being ok with it gets you HURD. I am ok with it.
- dTal 4y ago
  Erm, Linux is GPL. Apple built on BSD. Which is doing better? Participation != appropriation.
  - josephg 4y ago
    > Participation != appropriation. As someone who contributes to BSD licensed code, please don’t tell me my users are “appropriating” my work. I’m happy my code is being used, and the more the merrier.
kasabali 4y ago
> This is the gap between GNU GPL (especially GPL3) and BSD/MIT licenses This is completely false. See https://www.gnu.org/licenses/gpl-faq.html#NoMilitary https://www.gnu.org/licenses/gpl-faq.html#NoMilitary > I'd like to license my code under the GPL, but I'd also like to make it clear that it can't be used for military and/or commercial uses. Can I do this? > No, because those two goals contradict each other. The GNU GPL is designed specifically to prevent the addition of further restrictions. GPLv3 allows a very limited set of them, in section 7, but any other added restriction can be removed by the user. > More generally, a license that limits who can use a program, or for what, is not a free software license.
- MattPalmer1086 4y ago
  I think the point is only that companies will often not use GPL licensed code in their own products, as that forces them to GPL their own code as well. So BSD/MIT is less restrictive in that sense. There is no obligation to change the license of any code that uses it.
dTal 4y ago
>Why shouldn't big businesses use my code? The better opensource companies (Google, new Microsoft, Facebook, etc) contribute back to the community in all sorts of ways; and I consider that a net positive on the world. Because free software is in a constant battle for relevance, and they will use your code to make their proprietary offerings strictly superior to the free ones. What do you think would happen if Autodesk were permitted to fork Blender into a paid product, extend it in all sorts of proprietary ways, and push it heavily through advertising campaigns and educational subsidies? Blender would die, is what.
- josephg 4y ago
  > Blender would die, is what. You know, I don’t think it would? If autodesk depended on blender, and the core of autodesk was blender, I think autodesk would throw gobs of money at making sure blender flourished. They’d probably try and hire lots of blender devs too - just to get them to work on core blender features so they could make autodesk better. There’s plenty of examples of this happening in the wild. Is FreeBSD dead because macos is built on top of a lot of its code? Is redis dead because of redislabs’ proprietary offerings? No. Is SQLite hurt by all the projects built on top of it? No. The opposite - it’s strengthened by being used.

anonymoushn 4y ago

PRs to improve zstd's embarrassingly slow decoder require you to assign your firstborn to Zuck and new Microsoft is selling me a subscription to use my laundered open source code to generate autocomplete suggestions in their closed-source IDE that pegs 2 cores and needs 4 gigs of memory (on the remote machine only) to edit 50k text files over ssh

mappu 4y ago
Zstd's decoder is very very fast for what it does, what ideas do you have to improve it? It's unlikely to beat lzo family on decompression speed, but that's a different class of compressor entirely. And probably there are still some more tricks zstd can learn from closed-source Oodle leviathan, but i don't think much more can be learned.
- anonymoushn 4y ago
  I expect any "perform a conversion of some sort on a byte stream" implementation that uses 0 SIMD instructions and is not memory bound is leaving a lot of performance on the table, especially if one is permitted to mess with the design and layout of the input to make it more amenable to vectorized consumption. I cannot confidently claim that we're missing a a >2x speedup in this case though, it may be as low as ~1.3x or something.
  - mappu 4y ago
    I checked the zstd source and i'm surprised to see you're right - There's a little x86_64 assembly but I don't see a single SIMD instruction anywhere, and no intrinsics neither. Seems like brotli is the same. I assume zstd still gains something from SIMD autovectorization in the compiler, that might be interesting to benchmark with- and without- such a flag. Since the zstd bytestream got frozen in RFC8478, messing with the layout too much will require a zstd2 and moving the whole world again to use it (linux kernel compression, rpm binary format, etc)
bscphil 4y ago
Ah yes, the embarrassingly slow decoder that just happens to be on the Pareto frontier for decompression speed: http://www.mattmahoney.net/dc/text.html http://www.mattmahoney.net/dc/text.html
- anonymoushn 4y ago
  If you exclude oodle from your benchmarks?

thrown_22 4y ago

>Imagine if the people who work at those companies aren't your enemy. But instead are potential customers and potential collaborators, who see value in your work and want it to succeed. Yes, but we'd have to imagine it isn't microsoft doing it. You'd think people would learn the dozenth time it happens, but no, here we are again with copilot and VS Studio Code.

josephg 4y ago
Can you be more specific? What, exactly, are you mad about microsoft doing with vscode? Copilot is a whole other beast. I don't think there's any simple answers there.
- happymellon 4y ago
  Probably presenting a code editor with IDE like extensions that are all Open Source, and then gradually killing the Open Source extensions and switching them out for proprietary ones.
  - MonkeyClub 4y ago
    Which is exactly the EEE technique, “patented” by Microsoft: https://en.m.wikipedia.org/wiki/Embrace,_extend,_and_extinguish https://en.m.wikipedia.org/wiki/Embrace,_extend,_and_extingu... If GitHub, VS code, an MS Linux, and perhaps somehow LinkedIn, prove to be a joint front to EEE the F/OSS community, what can I say... Seems somewhat unlikely though, given the number of allowances that must be made for that to hold true.
    - happymellon 4y ago
      Well Github is struggling to not abuse its Free Software users, VS Code has failed to remain Open Source, I'm not sure about LinkedIn because that never pretended to heart Linux so that gets a pass, and MS Linux isn't really a thing. Although if you are to believe the terrible kernel commits they were attempting to make to allow them to do an Nvidia closed source shim then it wouldn't shock me.

LeFantome 4y ago

I agree. Being ok with big company participation gets you Linux and / or FreeBSD. Not being ok with it gets you HURD. I am ok with it.

dTal 4y ago
Erm, Linux is GPL. Apple built on BSD. Which is doing better? Participation != appropriation.
- josephg 4y ago
  > Participation != appropriation. As someone who contributes to BSD licensed code, please don’t tell me my users are “appropriating” my work. I’m happy my code is being used, and the more the merrier.

kasabali 4y ago

> This is the gap between GNU GPL (especially GPL3) and BSD/MIT licenses This is completely false. See https://www.gnu.org/licenses/gpl-faq.html#NoMilitary https://www.gnu.org/licenses/gpl-faq.html#NoMilitary > I'd like to license my code under the GPL, but I'd also like to make it clear that it can't be used for military and/or commercial uses. Can I do this? > No, because those two goals contradict each other. The GNU GPL is designed specifically to prevent the addition of further restrictions. GPLv3 allows a very limited set of them, in section 7, but any other added restriction can be removed by the user. > More generally, a license that limits who can use a program, or for what, is not a free software license.

MattPalmer1086 4y ago
I think the point is only that companies will often not use GPL licensed code in their own products, as that forces them to GPL their own code as well. So BSD/MIT is less restrictive in that sense. There is no obligation to change the license of any code that uses it.

dTal 4y ago

>Why shouldn't big businesses use my code? The better opensource companies (Google, new Microsoft, Facebook, etc) contribute back to the community in all sorts of ways; and I consider that a net positive on the world. Because free software is in a constant battle for relevance, and they will use your code to make their proprietary offerings strictly superior to the free ones. What do you think would happen if Autodesk were permitted to fork Blender into a paid product, extend it in all sorts of proprietary ways, and push it heavily through advertising campaigns and educational subsidies? Blender would die, is what.

josephg 4y ago
> Blender would die, is what. You know, I don’t think it would? If autodesk depended on blender, and the core of autodesk was blender, I think autodesk would throw gobs of money at making sure blender flourished. They’d probably try and hire lots of blender devs too - just to get them to work on core blender features so they could make autodesk better. There’s plenty of examples of this happening in the wild. Is FreeBSD dead because macos is built on top of a lot of its code? Is redis dead because of redislabs’ proprietary offerings? No. Is SQLite hurt by all the projects built on top of it? No. The opposite - it’s strengthened by being used.