4 ms·

I agree. Innovation in open source is all on commodity computer stuff that developers understand, like operating systems, compilers, web browsers, and database

I agree. Innovation in open source is all on commodity computer stuff that developers understand, like operating systems, compilers, web browsers, and databases. For almost everything that requires other expertise the innovation is still done in closed source software. Stuff like photo editing (LightRoom, Photoshop, Capture One), video editing (Premiere, Flame, DS), high end rendering (Renderman, 3Delight), CAD (Catia, Solidworks, NX), embedded systems (almost anything in cars, airplanes, appliances), and a million other areas are all still as closed as can be, and open source isn't even competing, much less innovating.

nickpsecurity 11y ago
Those are good examples. I'm not sure what the actual split is. Don't forget, though, where the real majority of innovation is: academic (mainly) and corporate labs. I went into detail a little while ago: https://news.ycombinator.com/item?id=10269245 https://news.ycombinator.com/item?id=10269245
- jlarocco 11y ago
  I think you're right for computer related stuff, like networking, databases, operating systems, etc. And it may even be technically true for the examples I mentioned. But in those areas, for all practical purposes, research from academic and corporate research doesn't actually get used until it's implemented in closed source software. At the annual Siggraph conference a good portion of the papers come from companies like Pixar and Autodesk, and get implemented in their products. Outside of that, it's pretty rare for the academic stuff to go much beyond a cool demo, unless one of the big graphics companies thinks they can use it, at which time they buy up the rights to it.
  - nickpsecurity 11y ago
    That's the exact problem I'm talking about. Yet, there's so much great stuff that they make! Stuff I'd love to be using right now in something else I have. Gotta figure out how to solve this problem. It's why I keep thinking of creating a new resource that's an alternative to IEEE/ACM w/out their expensive paywall, links to Wiki's on the subject, and good search functionality. Get enough in there on key topics and on sites like HN might get it going.
    - jlarocco 11y ago
      I didn't mean to imply that the problem was an expensive paywall. Using graphics as an example again, imagine a research paper explaining a new rendering algorithm that can render in half the time of the next best algorithm. As great as it may be, it's not very useful without the rest of the software infrastructure that comes along with a production rendering system, like Renderman, 3Delight, or Maya. So even if the academic researchers release an implementation as open source, it's unlikely to be used in production until it's incorporated into one of the big closed source systems. I suppose pay walls could be a contributing factor, but in a lot of areas the open source solutions are so far behind the closed source systems that inaccessible bleeding edge research isn't biggest problem.
      - nickpsecurity 11y ago
        I implied that. Most programmers today use Google. A paywall prevents them from getting these answers. So, most are blocked out. Far as your example, I agree that algorithm support can't help without good software going along with it. That's probably true for many use cases with examples you cited being areas where proprietary tools and ecosystem dominate. Leads to some potential counters: 1. It might help in the many things that OSS is doing. That's a huge list that includes desktops, phones, servers, and a few major apps. 2. It might help in commercial sector if their developers see the cutting edge approaches. It might be obvious where rendering advances are but most stuff is scattered: finding it took lots of searching with my keyword, kung-fu. Might be affecting discovery in proprietary side, too. I know many of them keep inventing solutions weaker than even well-cited academic stuff. Aside from patents, gotta wonder why they didn't go with better approach if it seemed like a good fit. Likely explanation: didn't know it existed. For instance, it's commonly believed that signal processing can only use a commercial DSP or graphics card with nothing delivered aside from that. People wanting an open alternative don't even know where to start that's worth anything. I heard and believed that too till I landed on this bad boy: https://www.wikiwand.com/en/Asynchronous_array_of_simple_processors#/AsAP_2_chip:_167_processors https://www.wikiwand.com/en/Asynchronous_array_of_simple_pro... The architecture is described in enough detail to be clonable. It might even be licensable with open-source allowed. It should handle plenty of workloads and supports custom accelerators for common things. After the mask costs, it would have great price vs performance vs energy usage. Yet, with 8 years of research, I just now came across this despite the first deliverable being in 2005. Might be a dead end and might turn into something if pursued. Yet, I'm sure it had potential to benefit commercial and OSS projects whose design approach accomplished far less had they known about this. So, more full and legacy systems certainly have lots of momentum. You can't just have a piece of a system. I think even they might be affected by the problem I describe, though, given they're as sub-optimal vs cutting edge as OSS often are. So, easier movement of these ideas might benefit both.
teddyh 11y ago
You list things which proprietary software does way better than free software. I remember when the list was the other way around – i.e. people would list things that free software could now do, which were not very many at the time. It’s interesting that this relationship has now been reversed, and people, like yourself, now instead list the things which proprietary software can do better.
- nickpsecurity 11y ago
  That's interesting if true. It's going in that direction, too. However, I think the list is still the same: most tasks have proprietary applications that do it better with OSS mostly trying to clone them and more rarely innovating past them. I mean, MS Office vs OpenOffice? VMS or NonStop clustering vs whatever's on Linux? Windows or Mac OS vs GNOME? Most OSS just sucks compared to best proprietary stuff. It still can't do better. So, proprietary tools are the ones that OSS is still judged by. Certain categories, esp in infrastructure, have OSS kicking serious butt. The dynamic works the opposite way. Even stronger are the hybrids of proprietary and OSS that make the argument less meaningful. Many of us have predicted that hybrids would smash both with some evidence of this in cloud sector. OSS still isn't in that dominant position in general, though.
- jlarocco 11y ago
  I don't think the situation has reversed at all, and in fact open source has grown tremendously. There's more open source and free software, of higher quality, than any other time in the past. Commodity software like operating systems and web browsers are now dominated by open source. My point is that despite the growth, there are still many, many areas where the open source offerings are pretty lame and noncompetitive (if they even exist). Open source has never been very competitive in any of the areas I mentioned.
  - teddyh 11y ago
    I think you completely misread my comment.

nickpsecurity 11y ago

Those are good examples. I'm not sure what the actual split is. Don't forget, though, where the real majority of innovation is: academic (mainly) and corporate labs. I went into detail a little while ago: https://news.ycombinator.com/item?id=10269245 https://news.ycombinator.com/item?id=10269245

jlarocco 11y ago
I think you're right for computer related stuff, like networking, databases, operating systems, etc. And it may even be technically true for the examples I mentioned. But in those areas, for all practical purposes, research from academic and corporate research doesn't actually get used until it's implemented in closed source software. At the annual Siggraph conference a good portion of the papers come from companies like Pixar and Autodesk, and get implemented in their products. Outside of that, it's pretty rare for the academic stuff to go much beyond a cool demo, unless one of the big graphics companies thinks they can use it, at which time they buy up the rights to it.
- nickpsecurity 11y ago
  That's the exact problem I'm talking about. Yet, there's so much great stuff that they make! Stuff I'd love to be using right now in something else I have. Gotta figure out how to solve this problem. It's why I keep thinking of creating a new resource that's an alternative to IEEE/ACM w/out their expensive paywall, links to Wiki's on the subject, and good search functionality. Get enough in there on key topics and on sites like HN might get it going.
  - jlarocco 11y ago
    I didn't mean to imply that the problem was an expensive paywall. Using graphics as an example again, imagine a research paper explaining a new rendering algorithm that can render in half the time of the next best algorithm. As great as it may be, it's not very useful without the rest of the software infrastructure that comes along with a production rendering system, like Renderman, 3Delight, or Maya. So even if the academic researchers release an implementation as open source, it's unlikely to be used in production until it's incorporated into one of the big closed source systems. I suppose pay walls could be a contributing factor, but in a lot of areas the open source solutions are so far behind the closed source systems that inaccessible bleeding edge research isn't biggest problem.
    - nickpsecurity 11y ago
      I implied that. Most programmers today use Google. A paywall prevents them from getting these answers. So, most are blocked out. Far as your example, I agree that algorithm support can't help without good software going along with it. That's probably true for many use cases with examples you cited being areas where proprietary tools and ecosystem dominate. Leads to some potential counters: 1. It might help in the many things that OSS is doing. That's a huge list that includes desktops, phones, servers, and a few major apps. 2. It might help in commercial sector if their developers see the cutting edge approaches. It might be obvious where rendering advances are but most stuff is scattered: finding it took lots of searching with my keyword, kung-fu. Might be affecting discovery in proprietary side, too. I know many of them keep inventing solutions weaker than even well-cited academic stuff. Aside from patents, gotta wonder why they didn't go with better approach if it seemed like a good fit. Likely explanation: didn't know it existed. For instance, it's commonly believed that signal processing can only use a commercial DSP or graphics card with nothing delivered aside from that. People wanting an open alternative don't even know where to start that's worth anything. I heard and believed that too till I landed on this bad boy: https://www.wikiwand.com/en/Asynchronous_array_of_simple_processors#/AsAP_2_chip:_167_processors https://www.wikiwand.com/en/Asynchronous_array_of_simple_pro... The architecture is described in enough detail to be clonable. It might even be licensable with open-source allowed. It should handle plenty of workloads and supports custom accelerators for common things. After the mask costs, it would have great price vs performance vs energy usage. Yet, with 8 years of research, I just now came across this despite the first deliverable being in 2005. Might be a dead end and might turn into something if pursued. Yet, I'm sure it had potential to benefit commercial and OSS projects whose design approach accomplished far less had they known about this. So, more full and legacy systems certainly have lots of momentum. You can't just have a piece of a system. I think even they might be affected by the problem I describe, though, given they're as sub-optimal vs cutting edge as OSS often are. So, easier movement of these ideas might benefit both.

teddyh 11y ago

You list things which proprietary software does way better than free software. I remember when the list was the other way around – i.e. people would list things that free software could now do, which were not very many at the time. It’s interesting that this relationship has now been reversed, and people, like yourself, now instead list the things which proprietary software can do better.

nickpsecurity 11y ago
That's interesting if true. It's going in that direction, too. However, I think the list is still the same: most tasks have proprietary applications that do it better with OSS mostly trying to clone them and more rarely innovating past them. I mean, MS Office vs OpenOffice? VMS or NonStop clustering vs whatever's on Linux? Windows or Mac OS vs GNOME? Most OSS just sucks compared to best proprietary stuff. It still can't do better. So, proprietary tools are the ones that OSS is still judged by. Certain categories, esp in infrastructure, have OSS kicking serious butt. The dynamic works the opposite way. Even stronger are the hybrids of proprietary and OSS that make the argument less meaningful. Many of us have predicted that hybrids would smash both with some evidence of this in cloud sector. OSS still isn't in that dominant position in general, though.
jlarocco 11y ago
I don't think the situation has reversed at all, and in fact open source has grown tremendously. There's more open source and free software, of higher quality, than any other time in the past. Commodity software like operating systems and web browsers are now dominated by open source. My point is that despite the growth, there are still many, many areas where the open source offerings are pretty lame and noncompetitive (if they even exist). Open source has never been very competitive in any of the areas I mentioned.
- teddyh 11y ago
  I think you completely misread my comment.