4 ms·
I think the confluence of new technologies, and the re-emergence / rediscovery of older technologies is going to be the best combination. Whether it goes that w
by eggy 10y ago
I think the confluence of new technologies, and the re-emergence / rediscovery of older technologies is going to be the best combination. Whether it goes that way is not certain, since the best technology doesn't always win out. Here, though, the money should, since all would greatly reduce time and energy in mining and validating:
* Vector processing computers - not von Neumann machines [1].
* Array languages new, or like J, K, or Q in the APL family [2,3]
* The replacement of floating point units with unum processors [4]
Neural networks are inherently arrays or matrices, and would do better on a designed vector array machine, not a re-purposed GPU, or even a TPU in the article in a standard von Neumann machine.
Maybe non-von Neumann architectire like the old Lisp Machines, but for arrays, not lists (and no, this is not a modern GPU. The data has to stay on the processor, not offloaded to external memory).
I started with neural networks in late 80s early 1990s, and I was mainly programming in C. matrices and FOR loops. I found J, the array language many years later, unfortunately.
Businesses have been making enough money off of the advantage of the array processing language A+, then K, that the per-seat cost of KDB+/Q (database/language) is easily justifiable. Other software like RiakTS are looking to get in the game using Spark/shark and other pieces of kit, but a K4 query is 230 times faster than Spark/shark, and uses 0.2GB of memory vs. 50GB. The similar technologies just don't fit the problem space as good as a vector language.
I am partial to J being a more mathematically pure array language in that it is based on arrays. K4 (soon to be K5/K6) is list-based at the lower level, and is honed for tick-data or time series data. J is a bit more general purpose or academic in my opinion.
Unums are theoretically more energy efficient and compact than floating point, and take away the error-guessing game. They are being tested with several different language implementations to validate their creator's claims, and practicality. The Mathematica notebook that John Gustafson modeled his work on is available free to download from the book publisher's site.
People have already done some type of explorator investigations in Python, Julia and even J already. I believe the J one is a 4-bit implementation of enums based on unums 1.0. John Gustafson just presented unums 2.0 in February 2016.
[1] http://conceptualorigami.blogspot.co.id/2010/12/vector-processing-languages-future-of.html http://conceptualorigami.blogspot.co.id/2010/12/vector-proce...
[2] jsoftware.com
[3] http://kxcommunity.com/an-introduction-to-neural-networks-with-kdb.php http://kxcommunity.com/an-introduction-to-neural-networks-wi...
[4] https://www.crcpress.com/The-End-of-Error-Unum-Computing/Gustafson/p/book/9781482239867 https://www.crcpress.com/The-End-of-Error-Unum-Computing/Gus...
- pwang 10y agoAt this point in time, I think that the Python/Numpy stack offers the best performance, productivity, and expressiveness trade-off. With the [Numba](http://numba.pydata.org http://numba.pydata.org) just-in-time compiler, you can now easily bounce between numeric SIMD codes that leverage tuned BLAS/MKL, then go into more explicit loop-oriented constructs that perform equivalently to hand-coded C, while still being Python. If I were starting anew, it would be hard to justify investing in a big J/K/Q code base or team, despite the potential performance benefits. I agree with your overall point that we're seeing a confluence of factors. The advances in compiler technology, combined with the vectorial nature of the problems that are interesting to solve in an era of big data, mean that we can achieve a great deal of productivity by using high-level vector-capable languages.
- eggy 10y agoYou may be right. I can't argue with Python's ubiquity; I have even steered my son in that direction, but with a hitch: I still had him learn some J. The creator of Pandas, Wes McKinney, had a link up a few years back mentioning he was looking for people who were familiar with APL, J or K. It seems he was working on a new project/startup I think (could this have been the shuttered DataPad?). The links are dead now, but I will double check. If the creator of Pandas is/was eyeing the older APL, and its newer brethren, I'd say it's a safe bet to keep J or K or Q on your radar because they fit. They're vector/array based; they are fast and iterative with a REPL; there is a lot of mathematical formalism in their origins and usage throughout the years, yet they are more beginner-friendly than say Haskell IMHO. I like Haskell too!