6 ms·

I’m open to being wrong, but my domain is not and has not been web apps. It’s mostly been things like satellite antenna control software, quantum computers, sci

I’m open to being wrong, but my domain is not and has not been web apps. It’s mostly been things like satellite antenna control software, quantum computers, scientific computing, and AI. Each of those domains, in my experience, has been firmly grounded in open source. (It does not mean, of course, that the end product is open source.) I recognize there are some domains that are steeped in closed source solutions with closed source developer tooling with no viable alternatives, but IME—as limited as it may be—it’s uncommon. What programming language stack do you (and a lot of other developers) use that’s closed source with no realistic open source alternative? It’s not C, Python, Java, Ruby, JavaScript, Lisp, Fortran, C++, PHP, Perl, Go, SQL, Rust, Objective-C, or Swift? (All of these, of course, not only have industrial strength FOSS implementations, but the first or second most popular implementation is the FOSS one.)

rch 7y ago
An exhaustive list would get tedious pretty quickly, but I'm imagining that using this engine would be roughly equivalent to building around Rogue Wave IMSL libraries or Mapbox Atlas. It seems like using the SDK in your language of choice would be a reasonable way to leverage the engine where it makes sense.
gricardo99 7y ago
I'm not disagreeing with you, just answering your honest question: >What programming language stack do you use that’s closed source? Verilog/System Verilog (generally HDL/FPGA/ASIC design/verification). Yes, it's a niche (some may even say it's not software, it's hardware, but it is indeed a large code base that requires a software engineering environment/toolchain/practices/etc...). Again, just answering your question for the benefit of the discussion. It would be interesting to hear of others such closed source areas.
- reikonomusha 7y ago
  There’s also Q [0] and kdb+ [1] as another example. [0] https://en.m.wikipedia.org/wiki/Q_(programming_language_from_Kx_Systems) https://en.m.wikipedia.org/wiki/Q_(programming_language_from... [1] https://en.m.wikipedia.org/wiki/Kdb%2B https://en.m.wikipedia.org/wiki/Kdb%2B
  - gravypod 7y ago
    Kdb is a very niche product that has open source alternatives that are made use of by the rest of the software industry outside of FinTech. It's not bad, it's just not what the market wants to use because it's a closed box. Wikipedia has an article on Time series databases and they contain a short list of popular TSDBs [0]. On that list 11 are libre-software to some degree and 4 are commercial. Relational TSDBs are new to the open source space but they're still being built despite there being the existing kdb+. This shows that people will spend considerably more funds to engineer a replacement (and give that replacement away for free) to avoid using a closed source product for this use case. [0] - https://en.wikipedia.org/wiki/Time_series_database#List_of_time_series_databases https://en.wikipedia.org/wiki/Time_series_database#List_of_t...
    - atemerev 7y ago
      Unfortunately, there are no open source alternatives to kdb+. Effortlessly handling hundreds of terabytes of data on a single machine, while consisting of only a 300 kB (yes, kilobytes)... Nobody even came close. Market adoption could be better, but the license costs around $100,000/year (probably the most expensive software per kilobyte). Fintech can afford it, other industries can’t.
      - JackFr 7y ago
        You'll also end up spending at least that much on consultants. While KDB is remarkable in many ways -- the first time I saw it I thought the demo was fake, the performance so great -- it is very difficult to develop in. They have a an SQLish interface, that allows non-specialists to get work done, but anything serious needs to move beyond that. I haven't used it in a few years, but queries had to be carefully optimized -- swapping the order in a where clause could cause order of magnitude differences in performance. Also it is append-optimized. If you need to update or insert data, it is a nontrivial exercise.
        atemerev 7y ago
        Regarding being append-only — these days, where reactive programming, immutability and FP are all in vogue, this is a feature. As for the rest, yes — K/Q are idiosyncratic, but they are simpler than it looks at first sight (and second.. and fifth). :)
- equalunique 7y ago
  You wouldn't happen to be a commentor/author for this Reddit post[0], would you? Your sentiment here seems quite similar. I have no experience at all with hardware design, but it seems like that domain is especially geared towards closed-source for specific reasons. [0] https://www.reddit.com/r/FPGA/comments/a5pzs5/prediction_open_source_fpga_tools_will_not/ https://www.reddit.com/r/FPGA/comments/a5pzs5/prediction_ope...
  - gricardo99 7y ago
    No, that's not me. Interesting points though. For my comment I was thinking about HDL simulators, which are very expensive/proprietary tools (vendors like Synopsys, Cadence, Mentor Graphics), with a lot of vendor lock-in, but yet are based on standard/open languages: Verilog/System Verilog. I know there are some open-source simulators that have been around for a while, but they aren't widely used, and the big 3 EDA vendors still seem to have a strangle hold on the HDL simulator market. Simulation is generally more important for ASIC hardware than FPGA hardware, since you can quickly iterate on an FPGA design with logic/design fixes, whereas with ASICs design iteration can be prohibitively expensive and time consuming.
  - daveguy 7y ago
    That is an interesting post. I think there are two other potential ways FPGA tools could become open source: 1) If there are some widely used open hardware devices or open standards. The fact that FPGAs are proprietary designs and can't be copied is a significant barrier. If there was an analogue of ARM in the FPGA world that could open up open source dev tools too. 2) DIY FPGA. This one is almost certainly much further down the road than any of the other options. But if fab and design tools of the chips themselves become a commodity then there will be open source dev tools. It used to be pretty difficult to make a custom PCB, but now with PCB as a service open tools are seeing more use. I think the reason this is so far down the line is because fpga offers cutting edge performance for specialized applications. If you wait several generations or are ok with reduced performance then you can stick with generic CPU/GPU. With Moore's law running down there may be more and more things that aren't possible without fpga. And I don't think it would be until then that open source dev tools follow more diy fpga.
- sweeneyrod 7y ago
  I think hardware stuff is an expected exception because there are basically no open-source architectures.
  - gricardo99 7y ago
    I agree in terms of the equivalent of standard libraries. Ex: if you want a DMA engine, a PCIE interface, Ethernet, etc... you either have to build that from scratch, or buy IP from a vendor, there's not a lot of open source. There is opencores.org, but when a company is investing a lot in their own hardware design, then tend to want higher quality, or more recent/updated designs. But the IC/hardware toolchain is a separate issue from the design itself. It's largely proprietary, probably due to the niche market and large R&D costs involved in some of the tools.
  - pabs3 7y ago
    There are at least RISC-V, OpenRISC, MIPS and maybe SPARC: https://riscv.org/ https://riscv.org/ https://openrisc.io/ https://openrisc.io/ https://www.mips.com/mipsopen/ https://www.mips.com/mipsopen/ https://en.wikipedia.org/wiki/SPARC#Open_source_implementations https://en.wikipedia.org/wiki/SPARC#Open_source_implementati...
    - earthboundkid 7y ago
      RISC-V was explicitly created to solve the problem of their being basically no open source architectures, and it's still fairly new, so it really is the exception that proves the rule.
    - sweeneyrod 7y ago
      Yep, but AFAIK they don't see much use in industry. There are people trying to change that, but they haven't succeeded yet.
      - pabs3 7y ago
        MIPS gets used a lot in routers, it only went open relatively recently though.
- pabs3 7y ago
  There are open hardware design tools, some examples: http://iverilog.icarus.com/ http://iverilog.icarus.com/ https://www.veripool.org/wiki/verilator https://www.veripool.org/wiki/verilator http://www.clifford.at/yosys/ http://www.clifford.at/yosys/ https://symbiflow.github.io/ https://symbiflow.github.io/ https://github.com/Qucs/ADMS https://github.com/Qucs/ADMS http://www.myhdl.org/ http://www.myhdl.org/ https://chisel.eecs.berkeley.edu/ https://chisel.eecs.berkeley.edu/ https://wiki.debian.org/FPGA https://wiki.debian.org/FPGA
  - gricardo99 7y ago
    If you know of any chips taped-out, and shipping in volume, that used any of these as their primary tool, I’d be very interested to know more about that. I’m aware of chisel used to verify some RISC-V cores that were fabricated, but that’s a research POC, not a volume/production ASIC.
    - orbifold 7y ago
      Chisel was used to do Google‘s Coral TPU, although most likely they used commercial simulators as backend.
    - palmerdabbelt 7y ago
      All of SiFive's chips use Chisel, it's our primary design language. We use commercial tools post-Verilog (ie, synthesis and place+route). We use Verilator for some simulation, but also use commercial Verilog simulators. We have chips shipping in volume.
garmaine 7y ago
Those are still mostly scientific domains, and science has been rooted in open source. Most mechanical and civil engineering, on the other hand, is done in Matlab scripts and proprietary simulation frameworks. Matlab is Wolfram’s chief competitor, not Julia, R, or Sage.
- new4thaccount 7y ago
  Yea, AutoCAD is huge for Civil Engineers and Matlab is also pretty big for many engineering students, academics, and research professionals. I think most of the heavier users in industry used it all through grad school and then brought it over into industry. Most of us engineers that just have a bachelor's degree weren't as firmly rooted in Matlab and just picked up Python as our first post academic language. Both are good. Matlab has some advantages like bundling suitesparse (world class linear algebra subroutines)and handling the license of that so you don't have to get one yourself (assuming you want to sell your product). I think Python definitely wins as a scripting language lingua franca though. It is free as in beer, open-source, available on any OS I care about, and most of the software I use has a built-in Python API. Mathematica is big in academic math and physics programs, but I haven't seen it in industry as much as Matlab. Believe it or not, my company had a major accounting application written in Mathematica, so it does pop up in weird places.
Rebelgecko 7y ago
>It’s mostly been things like satellite antenna control software IME even getting binaries from an antenna vendor can be like pulling teeth, are there some good open source solutions out there?
- reikonomusha 7y ago
  I was involved in the software stack for developing antenna software for the antennas themselves, which used a Linux stack with C (and even Lisp!).
astura 7y ago
>What programming language stack do you (and a lot of other developers) use that’s closed source with no realistic open source alternative? Matlab?
- danaur 7y ago
  There are open source alternatives to Matlab like gnu octave
  - Double_a_92 7y ago
    Matlab lives from all the other toolboxes around it. It's not just the "coding" part.
  - Insanity 7y ago
    And also Scilab I believe. Though last time I used them, Scilab felt quite inferior to Matlab. (6-7ish years ago)
rusticpenn 7y ago
Cadence, Synopsys, mentor, code-meter, muneda and many more
r_singh 7y ago
>What programming language stack do you use that’s closed source? MATLAB
billfruit 7y ago
Even when proprietary software is used in these types of fields, like vworks,the source is available for you to build. I wish more proprietary software would include the source when you purchase a license.

rch 7y ago

An exhaustive list would get tedious pretty quickly, but I'm imagining that using this engine would be roughly equivalent to building around Rogue Wave IMSL libraries or Mapbox Atlas. It seems like using the SDK in your language of choice would be a reasonable way to leverage the engine where it makes sense.

gricardo99 7y ago

I'm not disagreeing with you, just answering your honest question: >What programming language stack do you use that’s closed source? Verilog/System Verilog (generally HDL/FPGA/ASIC design/verification). Yes, it's a niche (some may even say it's not software, it's hardware, but it is indeed a large code base that requires a software engineering environment/toolchain/practices/etc...). Again, just answering your question for the benefit of the discussion. It would be interesting to hear of others such closed source areas.

reikonomusha 7y ago
There’s also Q [0] and kdb+ [1] as another example. [0] https://en.m.wikipedia.org/wiki/Q_(programming_language_from_Kx_Systems) https://en.m.wikipedia.org/wiki/Q_(programming_language_from... [1] https://en.m.wikipedia.org/wiki/Kdb%2B https://en.m.wikipedia.org/wiki/Kdb%2B
- gravypod 7y ago
  Kdb is a very niche product that has open source alternatives that are made use of by the rest of the software industry outside of FinTech. It's not bad, it's just not what the market wants to use because it's a closed box. Wikipedia has an article on Time series databases and they contain a short list of popular TSDBs [0]. On that list 11 are libre-software to some degree and 4 are commercial. Relational TSDBs are new to the open source space but they're still being built despite there being the existing kdb+. This shows that people will spend considerably more funds to engineer a replacement (and give that replacement away for free) to avoid using a closed source product for this use case. [0] - https://en.wikipedia.org/wiki/Time_series_database#List_of_time_series_databases https://en.wikipedia.org/wiki/Time_series_database#List_of_t...
  - atemerev 7y ago
    Unfortunately, there are no open source alternatives to kdb+. Effortlessly handling hundreds of terabytes of data on a single machine, while consisting of only a 300 kB (yes, kilobytes)... Nobody even came close. Market adoption could be better, but the license costs around $100,000/year (probably the most expensive software per kilobyte). Fintech can afford it, other industries can’t.
    - JackFr 7y ago
      You'll also end up spending at least that much on consultants. While KDB is remarkable in many ways -- the first time I saw it I thought the demo was fake, the performance so great -- it is very difficult to develop in. They have a an SQLish interface, that allows non-specialists to get work done, but anything serious needs to move beyond that. I haven't used it in a few years, but queries had to be carefully optimized -- swapping the order in a where clause could cause order of magnitude differences in performance. Also it is append-optimized. If you need to update or insert data, it is a nontrivial exercise.
      - atemerev 7y ago
        Regarding being append-only — these days, where reactive programming, immutability and FP are all in vogue, this is a feature. As for the rest, yes — K/Q are idiosyncratic, but they are simpler than it looks at first sight (and second.. and fifth). :)
equalunique 7y ago
You wouldn't happen to be a commentor/author for this Reddit post[0], would you? Your sentiment here seems quite similar. I have no experience at all with hardware design, but it seems like that domain is especially geared towards closed-source for specific reasons. [0] https://www.reddit.com/r/FPGA/comments/a5pzs5/prediction_open_source_fpga_tools_will_not/ https://www.reddit.com/r/FPGA/comments/a5pzs5/prediction_ope...
- gricardo99 7y ago
  No, that's not me. Interesting points though. For my comment I was thinking about HDL simulators, which are very expensive/proprietary tools (vendors like Synopsys, Cadence, Mentor Graphics), with a lot of vendor lock-in, but yet are based on standard/open languages: Verilog/System Verilog. I know there are some open-source simulators that have been around for a while, but they aren't widely used, and the big 3 EDA vendors still seem to have a strangle hold on the HDL simulator market. Simulation is generally more important for ASIC hardware than FPGA hardware, since you can quickly iterate on an FPGA design with logic/design fixes, whereas with ASICs design iteration can be prohibitively expensive and time consuming.
- daveguy 7y ago
  That is an interesting post. I think there are two other potential ways FPGA tools could become open source: 1) If there are some widely used open hardware devices or open standards. The fact that FPGAs are proprietary designs and can't be copied is a significant barrier. If there was an analogue of ARM in the FPGA world that could open up open source dev tools too. 2) DIY FPGA. This one is almost certainly much further down the road than any of the other options. But if fab and design tools of the chips themselves become a commodity then there will be open source dev tools. It used to be pretty difficult to make a custom PCB, but now with PCB as a service open tools are seeing more use. I think the reason this is so far down the line is because fpga offers cutting edge performance for specialized applications. If you wait several generations or are ok with reduced performance then you can stick with generic CPU/GPU. With Moore's law running down there may be more and more things that aren't possible without fpga. And I don't think it would be until then that open source dev tools follow more diy fpga.
sweeneyrod 7y ago
I think hardware stuff is an expected exception because there are basically no open-source architectures.
- gricardo99 7y ago
  I agree in terms of the equivalent of standard libraries. Ex: if you want a DMA engine, a PCIE interface, Ethernet, etc... you either have to build that from scratch, or buy IP from a vendor, there's not a lot of open source. There is opencores.org, but when a company is investing a lot in their own hardware design, then tend to want higher quality, or more recent/updated designs. But the IC/hardware toolchain is a separate issue from the design itself. It's largely proprietary, probably due to the niche market and large R&D costs involved in some of the tools.
- pabs3 7y ago
  There are at least RISC-V, OpenRISC, MIPS and maybe SPARC: https://riscv.org/ https://riscv.org/ https://openrisc.io/ https://openrisc.io/ https://www.mips.com/mipsopen/ https://www.mips.com/mipsopen/ https://en.wikipedia.org/wiki/SPARC#Open_source_implementations https://en.wikipedia.org/wiki/SPARC#Open_source_implementati...
  - earthboundkid 7y ago
    RISC-V was explicitly created to solve the problem of their being basically no open source architectures, and it's still fairly new, so it really is the exception that proves the rule.
  - sweeneyrod 7y ago
    Yep, but AFAIK they don't see much use in industry. There are people trying to change that, but they haven't succeeded yet.
    - pabs3 7y ago
      MIPS gets used a lot in routers, it only went open relatively recently though.
pabs3 7y ago
There are open hardware design tools, some examples: http://iverilog.icarus.com/ http://iverilog.icarus.com/ https://www.veripool.org/wiki/verilator https://www.veripool.org/wiki/verilator http://www.clifford.at/yosys/ http://www.clifford.at/yosys/ https://symbiflow.github.io/ https://symbiflow.github.io/ https://github.com/Qucs/ADMS https://github.com/Qucs/ADMS http://www.myhdl.org/ http://www.myhdl.org/ https://chisel.eecs.berkeley.edu/ https://chisel.eecs.berkeley.edu/ https://wiki.debian.org/FPGA https://wiki.debian.org/FPGA
- gricardo99 7y ago
  If you know of any chips taped-out, and shipping in volume, that used any of these as their primary tool, I’d be very interested to know more about that. I’m aware of chisel used to verify some RISC-V cores that were fabricated, but that’s a research POC, not a volume/production ASIC.
  - orbifold 7y ago
    Chisel was used to do Google‘s Coral TPU, although most likely they used commercial simulators as backend.
  - palmerdabbelt 7y ago
    All of SiFive's chips use Chisel, it's our primary design language. We use commercial tools post-Verilog (ie, synthesis and place+route). We use Verilator for some simulation, but also use commercial Verilog simulators. We have chips shipping in volume.

garmaine 7y ago

Those are still mostly scientific domains, and science has been rooted in open source. Most mechanical and civil engineering, on the other hand, is done in Matlab scripts and proprietary simulation frameworks. Matlab is Wolfram’s chief competitor, not Julia, R, or Sage.

new4thaccount 7y ago
Yea, AutoCAD is huge for Civil Engineers and Matlab is also pretty big for many engineering students, academics, and research professionals. I think most of the heavier users in industry used it all through grad school and then brought it over into industry. Most of us engineers that just have a bachelor's degree weren't as firmly rooted in Matlab and just picked up Python as our first post academic language. Both are good. Matlab has some advantages like bundling suitesparse (world class linear algebra subroutines)and handling the license of that so you don't have to get one yourself (assuming you want to sell your product). I think Python definitely wins as a scripting language lingua franca though. It is free as in beer, open-source, available on any OS I care about, and most of the software I use has a built-in Python API. Mathematica is big in academic math and physics programs, but I haven't seen it in industry as much as Matlab. Believe it or not, my company had a major accounting application written in Mathematica, so it does pop up in weird places.

Rebelgecko 7y ago

>It’s mostly been things like satellite antenna control software IME even getting binaries from an antenna vendor can be like pulling teeth, are there some good open source solutions out there?

reikonomusha 7y ago
I was involved in the software stack for developing antenna software for the antennas themselves, which used a Linux stack with C (and even Lisp!).

astura 7y ago

>What programming language stack do you (and a lot of other developers) use that’s closed source with no realistic open source alternative? Matlab?

danaur 7y ago
There are open source alternatives to Matlab like gnu octave
- Double_a_92 7y ago
  Matlab lives from all the other toolboxes around it. It's not just the "coding" part.
- Insanity 7y ago
  And also Scilab I believe. Though last time I used them, Scilab felt quite inferior to Matlab. (6-7ish years ago)

rusticpenn 7y ago

Cadence, Synopsys, mentor, code-meter, muneda and many more

r_singh 7y ago

>What programming language stack do you use that’s closed source? MATLAB

billfruit 7y ago

Even when proprietary software is used in these types of fields, like vworks,the source is available for you to build. I wish more proprietary software would include the source when you purchase a license.