6 ms·

I can imagine that a fork of GNU APL, re-written in FORTRAN with multi-threaded and maybe distributed computing support, would be an amazing mathematical tool.

by ColanR 8y ago

I can imagine that a fork of GNU APL, re-written in FORTRAN with multi-threaded and maybe distributed computing support, would be an amazing mathematical tool. Something that's super fast and scales really well, and makes mathematical operations really easy.

patrickg_zill 8y ago
Gnu APL has multi core support for certain computations and you can fork APL child processes and interact with them. So not quite multi threaded but still very useful for heavy duty work where you want to have a great deal of concurrency.
- ColanR 8y ago
  Yes! I'm imagining a value proposition where APL is as fast as C or Fortran, on distributed systems, because at that point it would probably evaporate most of Matlab, Mathematica, and probably TensorFlow's users. Can you imagine? APL is an amazingly powerful language. It's easy to write nearly any mathematical function in it. What if it was faster than every other mathematical language? It would probably dominate the market overnight.
  - bladecatcher 8y ago
    As mruts pointed out above, kdb+/q provides what you're imagining - it provides fantastic distributed computing support via its IPC protocol and also has multi-core support.
    - ColanR 8y ago
      Multi-core is priced for big financial institutions, though. Only thing you or I could afford is the 32 bit single-core.
      - bladecatcher 8y ago
        You can start the 32 bit instance with the -s switch (for q to be started with multiple slaves) for parallel execution of a function over data using the "peach" command [0]. But yes, I do agree that the 32 bit instances have limited use, as they can't be used in for-profit projects. [0] https://code.kx.com/wiki/Reference/peach https://code.kx.com/wiki/Reference/peach
      - pinewurst 8y ago
        There is a free 64-bit version available with limits that don't seem harsh at all: The 64-bit kdb+ On-Demand Personal Edition is free for personal, non-commercial use. Currently it may be used on up to 2 computers, and up to a maximum of 16 cores per computer, but is not licensed for use on any cloud – only personal computers. It requires an always-on internet connection to operate, and a license key file, obtainable from ondemand.kx.com
      - patrickg_zill 8y ago
        Is the pricing published? I haven't been able to find it on the website.
        mruts 8y ago
        It’s definitely one of those “if you have to ask, you don’t need it” kind of things
  - vmchale 8y ago
    Supposedly numpy was inspired in part by J. I don't think it will ever dominate the market - some things such as FFI remain weak points compared to C. Plus a lot of what makes Mathematica good is that they have specific algorithms that they've optimized quite a bit, for things like computer algebra &c.
  - arcfide 8y ago
    You mean Dyalog APL? It already regularly outperforms normal, handwritten C code. It also has support for distributed computing, multi-threading, and the Co-dfns compiler can be used to compile your APL code to the GPU. For example, consider the following talk, which discusses sub-nanosecond lookups/search using the Dyalog APL interpreter. https://dyalog.tv/Dyalog18/?v=paxIkKBzqBU https://dyalog.tv/Dyalog18/?v=paxIkKBzqBU
mruts 8y ago
I think KDB might be what you’re looking for. It’s very efficient and widely used in finance. It’s pretty expensive, but I believe you can use it for personal use for free (there might be a single core restriction I think). As an aside, KDB devs make bank, like salaries >500k
- pmoriarty 8y ago
  There seem to be a couple of free versions, a 64-bit and a 32-bit version, with the latter being less restricted than the former.[1] I was curious about it, but they really don't make it easy to get started. There are no links to any documentation anywhere on their site. The download is hidden behind a licensing agreement and a form asking for the user's personal information. They mention GitHub (in the context of "community support") on their download page, but there are no links to any GitHub repos. [1] - https://kx.com/connect-with-us/download/ https://kx.com/connect-with-us/download/
  - bladecatcher 8y ago
    https://code.kx.com/q/ https://code.kx.com/q/ has learning material and documentation . Hope it helps
  - nyankosensei 8y ago
    I highly recommend the “Q for Mortals” book as an excellent introduction to using K and it’s available online: https://code.kx.com/q4m3/ https://code.kx.com/q4m3/
- chc4 8y ago
  I've always been curious how people get into this. I've been interested in KDB ever since I read http://archive.vector.org.uk/art10501320 http://archive.vector.org.uk/art10501320, but it's not exactly a language they teach in college or you'd pick up in independent study. Do people just apply to fintech places with relevant experience and then get training for kdb?
  - bladecatcher 8y ago
snapster 8y ago
somewhat related: https://github.com/Co-dfns/Co-dfns https://github.com/Co-dfns/Co-dfns

by ColanR 8y ago

patrickg_zill 8y ago

Gnu APL has multi core support for certain computations and you can fork APL child processes and interact with them. So not quite multi threaded but still very useful for heavy duty work where you want to have a great deal of concurrency.

ColanR 8y ago
Yes! I'm imagining a value proposition where APL is as fast as C or Fortran, on distributed systems, because at that point it would probably evaporate most of Matlab, Mathematica, and probably TensorFlow's users. Can you imagine? APL is an amazingly powerful language. It's easy to write nearly any mathematical function in it. What if it was faster than every other mathematical language? It would probably dominate the market overnight.
- bladecatcher 8y ago
  As mruts pointed out above, kdb+/q provides what you're imagining - it provides fantastic distributed computing support via its IPC protocol and also has multi-core support.
  - ColanR 8y ago
    Multi-core is priced for big financial institutions, though. Only thing you or I could afford is the 32 bit single-core.
    - bladecatcher 8y ago
      You can start the 32 bit instance with the -s switch (for q to be started with multiple slaves) for parallel execution of a function over data using the "peach" command [0]. But yes, I do agree that the 32 bit instances have limited use, as they can't be used in for-profit projects. [0] https://code.kx.com/wiki/Reference/peach https://code.kx.com/wiki/Reference/peach
    - pinewurst 8y ago
      There is a free 64-bit version available with limits that don't seem harsh at all: The 64-bit kdb+ On-Demand Personal Edition is free for personal, non-commercial use. Currently it may be used on up to 2 computers, and up to a maximum of 16 cores per computer, but is not licensed for use on any cloud – only personal computers. It requires an always-on internet connection to operate, and a license key file, obtainable from ondemand.kx.com
    - patrickg_zill 8y ago
      Is the pricing published? I haven't been able to find it on the website.
      - mruts 8y ago
        It’s definitely one of those “if you have to ask, you don’t need it” kind of things
- vmchale 8y ago
  Supposedly numpy was inspired in part by J. I don't think it will ever dominate the market - some things such as FFI remain weak points compared to C. Plus a lot of what makes Mathematica good is that they have specific algorithms that they've optimized quite a bit, for things like computer algebra &c.
- arcfide 8y ago
  You mean Dyalog APL? It already regularly outperforms normal, handwritten C code. It also has support for distributed computing, multi-threading, and the Co-dfns compiler can be used to compile your APL code to the GPU. For example, consider the following talk, which discusses sub-nanosecond lookups/search using the Dyalog APL interpreter. https://dyalog.tv/Dyalog18/?v=paxIkKBzqBU https://dyalog.tv/Dyalog18/?v=paxIkKBzqBU

mruts 8y ago

I think KDB might be what you’re looking for. It’s very efficient and widely used in finance. It’s pretty expensive, but I believe you can use it for personal use for free (there might be a single core restriction I think). As an aside, KDB devs make bank, like salaries >500k

pmoriarty 8y ago
There seem to be a couple of free versions, a 64-bit and a 32-bit version, with the latter being less restricted than the former.[1] I was curious about it, but they really don't make it easy to get started. There are no links to any documentation anywhere on their site. The download is hidden behind a licensing agreement and a form asking for the user's personal information. They mention GitHub (in the context of "community support") on their download page, but there are no links to any GitHub repos. [1] - https://kx.com/connect-with-us/download/ https://kx.com/connect-with-us/download/
- bladecatcher 8y ago
  https://code.kx.com/q/ https://code.kx.com/q/ has learning material and documentation . Hope it helps
- nyankosensei 8y ago
  I highly recommend the “Q for Mortals” book as an excellent introduction to using K and it’s available online: https://code.kx.com/q4m3/ https://code.kx.com/q4m3/
chc4 8y ago
I've always been curious how people get into this. I've been interested in KDB ever since I read http://archive.vector.org.uk/art10501320 http://archive.vector.org.uk/art10501320, but it's not exactly a language they teach in college or you'd pick up in independent study. Do people just apply to fintech places with relevant experience and then get training for kdb?
- bladecatcher 8y ago

snapster 8y ago

somewhat related: https://github.com/Co-dfns/Co-dfns https://github.com/Co-dfns/Co-dfns