4 ms·
APL is an option. You'll have to fork for Dyalog (or APLX or whatever else is out there) if you want to use it, there is no free IDE or anything available. J i
by beagle3 16y ago
APL is an option. You'll have to fork for Dyalog (or APLX or whatever else is out there) if you want to use it, there is no free IDE or anything available.
J is also an option. It's APL's cousin; it's free, and can handle lots of data with ease and elegance.
Both languages take time to learn, and are probably not similar to anything else you've used, but are very rewarding -- even if you end up not using them, your style in any other language will improve.
Other alternatives are Matlab, Numpy and R (which you've already mentioned). Numpy is great if you feel good with Python.
Basically, the magic elements are "64-bit" and "memory mapping". These two together, if used properly, will make it look like any data set fits in memory. Actually indexing it so that things don't slow down too much is up to you. Don't skimp on physical memory.