6 ms·
I am doing Andrew Ng’s ML Coursera course with Matlab. I’ve now got the crazy desire to see matrices and vectors built into every language in a clean and succi
by melling 6y ago
I am doing Andrew Ng’s ML Coursera course with Matlab.
I’ve now got the crazy desire to see matrices and vectors built into every language in a clean and succinct way.
- taxemeEvasion 6y agoIt's a main reason why Fortran still has such a hold on the SciComp/HPC community.
- gowld 6y agoFortran is also optimized and debugged over decades of testing.
- karma_fountain 6y agoYeah those algorithms are numerically tested, haven't changed in years, and really fast. You can certainly get to convenient matrix manipulation in C++, but then your compile times will increase significantly.
- Zardoz84 6y agoYou can have a fast implementation and fast compilation time with DLang
- andi999 6y agoPerformance might be worse in c++
- DesiLurker 6y agoCan the fortran implementations do auto-vectorization on simd hardware? or map to gpu using cuda-like library? not rhetorical, genuine question.
- cygx 6y agoYes. See e.g. https://gcc.gnu.org/projects/tree-ssa/vectorization.html#example6 https://gcc.gnu.org/projects/tree-ssa/vectorization.html#exa... https://developer.nvidia.com/cuda-fortran https://developer.nvidia.com/cuda-fortran
- jhayward 6y agoYes, in fact all the original research for those things was done in Fortran compilers.
- TheRealKing 6y agoFortran is a fully-vectorized and parallelized language at all levels, from instructions to shared and distributed parallelism (corresponding to one-sided MPI communications). This has been the case for at least 3 decades if not more. There are mature GPU support for Fortran. NVIDIA and PGI are currently implementing the standard parallel features of Fortran via GPUs.
- tasty_freeze 6y agoOne of the unexpected things is Dartmouth BASIC from 1964 had matrix primitives, despite being an otherwise primitive language. MS never took up the feature, but I know Wang BASIC did on their 2200 family of machines. For example: 10 DIM A(3,3),B(3,3),C(3,3) 20 MAT READ A 30 DATA 1,-5,7, .45,23,9, -5,-1,0 40 MAT B=A:REM assignment 50 MAT B=ZER:REM zero array 60 MAT B=CON:REM all 1s 70 MAT B=IDN:REM identity 80 MAT B=TRN(A):REM transpose 90 MAT B=(2)*A:REM scale 100 MAT B=INV(A),D:REM invert array, determinant in scalar D 110 MAT C=A*B:REM matrix multiplication 120 MAT PRINT C:REM prints something close to an IDN array
- coliveira 6y agoBelieve it or not, engineers were heavy users of BASIC in the first few years. And FORTRAN was created with the notion of matrix multiplication, so it was just natural that BASIC implementations supported matrix multiplication.
- pavlov 6y agoMy grandfather was a construction engineer. I remember visiting his office around 1990 just before he retired. His only computer was a Commodore 64 and he had written all the software himself in BASIC over the years, just translating a lifetime of professional experience into these little interactive programs on floppies. Maybe the modern equivalent would be to use Excel.
- Someone 6y agoThe first version of Dartmouth basic ran on a system with a whopping 20kB of memory (8k words at 20 bits each. See https://en.)wikipedia.org/wiki/GE-200_series https://en.)wikipedia.org/wiki/GE-200_series) The GE-635 (https://en.wikipedia.org/wiki/GE-600_series https://en.wikipedia.org/wiki/GE-600_series) that it ran on most of its life could have 4 CPUs, had protected memory and floating point hardware, and supported at least 200 kilowords of memory (http://www.bitsavers.org/pdf/ge/GE-6xx/CPB-371A_GE-635_System_Man_Jul64.pdf http://www.bitsavers.org/pdf/ge/GE-6xx/CPB-371A_GE-635_Syste...) At 36 bits/word, that’s 900 kilobytes. The first version of Microsoft basic (https://en.wikipedia.org/wiki/Altair_BASIC https://en.wikipedia.org/wiki/Altair_BASIC) ran in 4 kilobytes (leaving about ¾ kilobytes for programs and data), so it’s understandable that it lacked some features (it didn’t even have else clauses in if statements, had SIN, but not COS or TAN, didn’t support integer or string variables, etc))
- montalbano 6y agoConvenient matrix algebra is one of the selling points of Julia.
- nerdponx 6y agoI'm really hopeful that Julia will continue to gain adoption. That said, I would love to have good data science library support (bindings to stuff like BLAS/LAPACK, Torch, Stan, etc) in Idris. Imagine vector and matrix dimensions checked at compile time. Probabilities guaranteed to be between 0 and 1, checked at compile time. Compile-time enforcement that you are handling "nil/null" data inputs correctly when reading data from a database or CSV file. Writing a web scraper with linear types to help catch bugs & performance leaks. Maybe even proving theorems about the math code you're writing ("is this a valid way to express X"?). Maybe not ideal for rapid exploratory model development, but it'd be pretty damn cool for writing data science tooling/libraries and "production" ML code in my opinion.
- deleted 6y ago[deleted]
- Ericson2314 6y agoLooking forward to Dependent Haskell for linear algebra, and seeing what existing stuff can be retrofit.
- kristjansson 6y agoNot that it has bindings to other tools, but it sounds like Dex[1] would be relevant to your interests! [1]: https://github.com/google-research/dex-lang https://github.com/google-research/dex-lang
- centimeter 6y agoBeyond matrices and vectors, it would be nice to have generalized (typed) tensors. What you can do with matrices alone is really somewhat limited.
- Robotbeat 6y agoFrom a data structure perspective: You mean like an n-dimensional array? That’s what Matlab does. Matrices (2D arrays) are just a special case of array in Matlab.
- whimsicalism 6y agoI suspect by the usage of the phrase "typed" they mean something like that, but where the axes/dimensions/whatever you call them are typed - ie. you can't naively take a product along the time dimension of A with the batch dimension of B.
- jpeloquin 6y agoxarray seems to fit the bill. As long as you're ok with runtime axis/dimension/type checking. https://xarray.pydata.org/en/stable/ https://xarray.pydata.org/en/stable/
- whimsicalism 6y agoI've explored this space a bit, and I don't think the perfect solution is here yet. In my view, static checking should be the goal. It's harder to iterate quickly when your network dynamically fails 30 layers later due to some edge case where you forgot to take a transpose. Definitely much better than it silently succeeding if the dimensions happen to be the correct size though! https://github.com/deepmind/tensor_annotations https://github.com/deepmind/tensor_annotations seems promising. I've developed a mypy plugin for pytorch that does similarly off of the "Named Tensor" dynamic feature (which isn't well supported yet), but haven't released it yet. I'm also excited by the ways in which including this in some sort of "first-class" way could make tensor semantics simpler, which I'm assuming xarray allows? Many of the operations enabled by http://nlp.seas.harvard.edu/NamedTensor http://nlp.seas.harvard.edu/NamedTensor are quite nice and similar ideas can let you write more generic code (ie. code that works automatically for on tensors with and without a batch dimension)
- bobbybabylon 6y agoHave you looked at languages like J or APL?
- make3 6y agoYes, have you tried Julia? It is pretty much how you describe. There are other things that I don't like about that language, but the matrix part is really great
- mkaic 6y agoJust finished the course and couldn't agree more. Everything seemingly is faster with matrices!
- teruakohatu 6y agoThe base type in R is a vector. A number such as 3.14 is simply a vector of length 1. So functions are vectorized by default (as long as you don't do any operations that are explicitly not vectorizable). Once you get your head around everything being a vector, R can be a lot of fun. I love Julia but I wish I didnt need to explicitly broadcast functions across a vector.
- cbkeller 6y agoI was annoyed by that at first, but eventually came to appreciate how Julia's semantics for when broadcasting is or is not required for operations on scalars, vectors, and matrices map to the underlying math. I.e., for an N x N matrix `A` 5 * A does not require broadcasting because multiplication of a matrix by a scalar is mathematically well-defined, whereas 5 + A (assuming you want element-wise addition) does require broadcasting, i.e., 5 .+ A because addition of a vector and a matrix is not mathematically well-defined.
- teruakohatu 6y agoI do understand why they do it, and numpy can play a bit fast and loose which can be confusing, but I miss reading elegent vector-by-default R code. The Julia approach also has some drawbacks, I was writing some code the other day and forgot to broadcast exp() across a matrix. exp(A) and exp.(A) gives very different results but it's not immediately obvious looking at code, especially when the code is a mixture of broadcasted and native matrix function calls which dots peppered all over the place.
- eigenspace 6y agoWell, again I'd say that's just like the above examples. The exponential of a (square) matrix is indeed a very well defined object [1] with semantics perfectly consistent with the exponential of a scalar. On the other hand, broadcasting exp over a matrix is a very different operation, which doesn't really have any meaning in a linear algebra sense. I understand it's an easy mistake to make when you're used to R or Numpy, but I'd say it's their fault you're making this mistake, not Julia's. [1] https://en.wikipedia.org/wiki/Matrix_exponential https://en.wikipedia.org/wiki/Matrix_exponential