13 ms·

Python 3.5 to Ship with Matrix Multiplication Operator

268 points by madisonmay 13y ago

madisonmay 13y ago
Guido has accepted the proposal and work has now begun to add support for the new operator: http://bugs.python.org/issue21176 http://bugs.python.org/issue21176
BoppreH 13y ago
I think "Matrix Multiplication" is a little too specific for a syntax change, but I welcome the addition of a new operator to be overloaded. Am I missing some useful applications of matrix multiplication on a general purpose language?
- heron87 13y ago
  Ehm, Python is not a "scripting language". It is a — as you said — general purpose programming language. And it is becoming more and more popular for scientific applications, where matrix multiplication plays a key role.
  - BoppreH 13y ago
    I honestly didn't expect a downvote for calling Python a "scripting language". I use the term loosely and always thought Python fit spot on. I still think it applies here, but I've edited the parent post to correct this possibly incorrect usage of the word. Just for the record, these are the characteristics of Python that made me call it "scripting": - Interpreted - Highly expressive - No boilerplate - Vast majority of programs are short - Lots of libraries for gluing stuff together - Used for turing-complete customization of software in other languages (e.g.: Sublime Text) - Recommended as replacement for Bash in some cases Anyway, I'm happy to see the language evolving to meet its users' needs.
    - heron87 13y ago
      No offense intended. I just have a slightly different understanding of a "scripting language": a — often domain-specific — language that is merely used to write small scripts/macros. But anyway, I did not want to start a discussion about that. Sorry for that.
    - PeterisP 13y ago
      I believe that the common usage of python has switched away from "scripting usage" - it's now generally used to implement stuff, not glue stuff together or replace bash, and the vast majority of programs isn't short anymore. It can be used as a scripting language, but that doesn't seem to be the main use-case anymore.
    - wisty 13y ago
      Stolen from https://www.python.org/doc/humor/ https://www.python.org/doc/humor/ (in response to a claim that it's not a true OO language): > Indeed, and because it doesn't support closures, it's not a true functional programming language either. And because you have to import all sorts of modules to do the simplest things (e.g., regular expressions), neither is it a true scripting language. Indeed, because it doesn't support labeled break or continue statements, it's not even a true structured programming language. I think the point is, trying to classify programming languages (except domain specific ones like SQL or maybe PHP) is kind of pointless. But to answer your real question, Python is one of the top languages in number crunching. After web work, that's Python's second biggest niche.
      - TillE 13y ago
        > it doesn't support closures It does now, and thank goodness, because even C++ has closures these days.
        Chronic26 13y ago
        Python is a scripting language. People who write python are "scripting," not "programming."
    - 13y ago
  - adamors 13y ago
    I'm sorry, but how is Python not a scripting language?
    - mbq 13y ago
      It is just that "scripting" is now called "managed" because it hurts ego to be a scripting programmer.
- dtf 13y ago
  I, for one, welcome our new operator overloads. Partly a recognition of just how massive Python has become within the scientific community.
- paddyoloughlin 13y ago
  The beginning of the PEP (linked document) has a rationale for it, including an argument that matrix multiplication has a combination of features which distinguish it from other binary operations. Check it out. Like most Python PEPs it is quite readable. There are even alternative proposals and links to where the discussion of this issue has occurred. It seems that the most fundamental reason is that matrix and memberwise multiplication are two distinct operations which use the same datatypes as operands and both have a very good claim to use of the * operator. Also, just because a language is general purpose does not mean that it cannot support features which might be restricted in their use. Two of the python's most popular uses are in scientific and numeric fields (see SciPy, NumPy and ipython). This operator seems like it would be very useful for improving readability in those areas which is probably reason enough to use it. However, please resist the urge to just overload this new operator for something unrelated that you want. The point of infix operators is to improve the readability of code and lessen the cognitive load on the reader. Overloading an operator for a purpose which is not conventional goes exactly against this for... what benefit exactly?
  - BoppreH 13y ago
    Thanks. I understand its importance to scientific computing and I welcome the change. I was just wondering if matrix multiplication had more flexible uses, like `sum` does. And the benefit of overloading is still readability, when carefully used and in a different domain. And sparingly. For the `@` operator I will be trying it as an "at" operator for accessing objects from a game map, or some sort of translation. If it performs well I may start using it in my pet projects. If it changes my life and enlightens everyone who gazes upon it, I may start using in production. I do understand your worry and I promise to be extra careful.
    - dragonwriter 13y ago
      > For the `@` operator I will be trying it as an "at" operator for accessing objects from a game map I suppose you could use it for that that, but wouldn't that be a canonical case for indexing with []?
- CamperBob2 13y ago
  Here's my uneducated guess: the Python maintainers see NumPy as a bit of a killer app, and the NumPy maintainers would like to be a MATLAB killer. The entire MATLAB language is designed to make life easier when dealing with matrices (people think it stands for MATh LAB but it actually stands for MATrix LAB.) MATLAB went down the matrix rabbit hole so far, they made ordinary scalars second-class types. If you want to multiply scalars or otherwise do element-wise operations, you have to prefix the '*' operator with a '.'. If Python can make their language more linear-algebra friendly while avoiding that sort of silliness, I'm all for it.
  - GFK_of_xmaspast 13y ago
    """If you want to multiply scalars or otherwise do element-wise operations, you have to prefix the '*' operator with a '.'. """ what???
    - quux 13y ago
      You don't have to use .* and friends when operating in 2 scalers, 2 * 3 works just fine. But if you have 2 matrices A and B, A * B does a matrix multiplication, And A .* B is needed for element wise multiplication. I'm still new to matlab, but the place where this has bitten me is if I'm trying to write code that works on either a scalar or vector of scAlars, if you aren't specific, the code my run fine against a single scalar, but blow up or do the wrong thing with a vector.
- arnarbi 13y ago
  It's addressed in the PEP, even with some convincing empirical data. Python is becoming hugely popular as a scientific data processing language. This paragraph from the motivation summarizes: "We all live in our own little sub-communities, so some Python users may be surprised to realize the sheer extent to which Python is used for number crunching -- especially since much of this particular sub-community's activity occurs outside of traditional Python/FOSS channels. So, to give some rough idea of just how many numerical Python programmers are actually out there, here are two numbers: In 2013, there were 7 international conferences organized specifically on numerical Python [3] [4]. At PyCon 2014, ~20% of the tutorials appear to involve the use of matrices [6]."
segphault 13y ago
There are a few handy things that I'd like to see added to Python's syntax. A dedicated operator for matrix multiplication isn't one of them.
- aberrant 13y ago
  What would you like to see?
  - shadowmint 13y ago
    multi-line lambdas.
    - Noelkd 13y ago
      Wouldn't you just define a function in that case? Guido has mentioned[1] that he isn't a fan of lambda functions. [1] http://www.linuxjournal.com/article/2959 http://www.linuxjournal.com/article/2959
    - jdpage 13y ago
      Not happening, because it would mean multi-line function arguments, which have been declared "ugly". http://legacy.python.org/dev/peps/pep-3099/ http://legacy.python.org/dev/peps/pep-3099/
    - EliAndrewC 13y ago
      Guido explains in an old blog post (http://www.artima.com/weblogs/viewpost.jsp?thread=147358 http://www.artima.com/weblogs/viewpost.jsp?thread=147358) that he considers all of the possible implementations of multi-line lambdas to be un-Pythonic: > But the complexity of any proposed solution for this puzzle is immense, to me: it requires the parser (or more precisely, the lexer) to be able to switch back and forth between indent-sensitive and indent-insensitive modes, keeping a stack of previous modes and indentation level. Technically that can all be solved (there's already a stack of indentation levels that could be generalized). But none of that takes away my gut feeling that it is all an elaborate Rube Goldberg contraption.
  - 13y ago
beefsack 13y ago
You've got to wonder, does matrix multiplication happen so often and is it so inconvenient to call as a function that it's worth complicating the syntax? I'm really not sure about this one.
- c0g 13y ago
  Yes! Numpy is hugely popular in (at least) physics and machine learning and the main negative comment I get is "Wow that's horrible syntax". I'm still not sure they'll be convinced, however: trans_coef.T.dot(solve(H.dot(V).dot(H.T)), trans_coef) becomes trans_coef.T @ solve(H @ V @ H.T, trans_coef) but in MATLAB or Julia might be: trans_coef' * ((H * V * H') \ trans_coef) ....I want a backslash operator now!
- jonemo 13y ago
  A few fields that use matrix computations extensively that come to mind: robotics, graphics, control, machine learning. I'd go as far as saying that matrix computation plays an important role in more or less every kind of programming except for web/app development. edit: As another comment points out, the PEP actually answers this question too and contains a slightly more comprehensive list of disciplines where matrices find frequent use: http://legacy.python.org/dev/peps/pep-0465/#but-isn-t-matrix-multiplication-a-pretty-niche-requirement http://legacy.python.org/dev/peps/pep-0465/#but-isn-t-matrix...
- yen223 13y ago
  I'd say matrix multiplication is far more common than complex numbers, and Python natively supports those.
  - baq 13y ago
    yeah but complex numbers don't introduce new punctuation aka symbolic noise.
- stiff 13y ago
  When you have a multiplication of 5-10 matrices that has to be done in the correct order, the expression gets horribly unreadable when written as nested function applications. And those kinds of expressions are ubiquitous in numerical computing.
fest 13y ago
I hope this is not a first step towards transforming python syntax into perl.
- jdpage 13y ago
  It won't be. The Python parser has to remain LL(1) by BDFL fiat.[1] 1: http://legacy.python.org/dev/peps/pep-3099/ http://legacy.python.org/dev/peps/pep-3099/
  - lamby 13y ago
    OP was clearly referring to style and aesthetics, not the grammar's level in the Chomksy hierarchy.
willvarfar 13y ago
The nice thing is that it makes a nice symbol `@` available to objects that aren't matrices, to do with as they want ... The (ab)use seems endless :)
- scrollaway 13y ago
  @ is already used by decorators. I hope decorators will be reimplemented as prefix operators on functions with this change, so that @<foo> becomes available as well.
  - ecma 13y ago
    With @ as a grammatical infix matrix mult operator, there doesn't seem to be any real need beyond exclusivity to redefine the decorator grammar. By the same argument, we shouldn't use * for unpacking.
  - mcherm 13y ago
    But the use as an operator and the use as a decorator is easily distinguished by where they appear in the grammar. Neither the parser nor human readers (as far as I can tell) have any difficulty determining which use is which. So let's not confuse the two: they are two different uses that happen to utilize the same symbol.
    - scrollaway 13y ago
      My point is that if @<func> is re-implemented as, for example, the function.__dec__, then it can be overridden and some cool DSL uses can pop up. As it stands, the use of @ as a (class-)decorator is built in to the grammar, rather than simply being a prefix operator.
      - ecma 13y ago
        How might that implementation be used in code? Would coders be directly accessing and overriding the __dec__ attribute? Since decorators are definition time anyway, I don't really see much potential that decorated function factories can't already provide.
zurn 13y ago
To everyone arguing against special-casing matrix multiplication, please base your arguments on the PEP's "Motivation" section to avoid too much of rehashing the obvious. It even has a subsection "But isn't matrix multiplication a pretty niche requirement?"
- cabinpark 13y ago
  I would argue that almost every single linear algebra routine can be interpreted in some form as a matrix-matrix or matrix-vector multiplication. Considering that matrix-matrix and matrix-vector are fundamental numerical operations (they are part of BLAS which are the basic API functions called by every single linear algebra routine I know of) having a concise notation is key. And since formulas can get complicated quickly, having a closer 1-1 correspondence with the mathematics is critical for understanding the meaning of the code. The PEP contains a nice example from statistics. And trust me, when you are writing numerical code, being able to read the mathematical formula clearly is essential, especially when you have a lot of formulas and need to figure out why your code is giving you numerical non-sense.
  - omaranto 13y ago
    What you say about almost all linear algebra routines being multiplication sounds crazy to me. How are the following supposed to be multiplication: (1) computing the transpose, (2) finding a basis for the kernel of a matrix, (3) factorizations, such as QR, LU or SVD (3) computing the Jordan normal form of a matrix, etc.?
    - j2kun 13y ago
      It's not as crazy as it sounds, though I wouldn't make such a blanket statement. Things can be phrased in terms of matrix multiplication (maybe not a single matrix multiplication), it's just not the most efficient way to go about it. 1. Transpose is a linear map on matrices (vectors in R^nm), so in a very concrete sense it is precisely a matrix multiplication. And it's not hard to figure out the matrix, because it's analogous to the matrix that swaps entries in vectors. 2. Finding the first eigenvalue can be approximated via matrix multiplication, and for sparse matrices with good spectral gap this extends to all of them. 3. Row reduction can be phrased as matrix multiplication, and hence finding the basis for the kernel of a matrix is a byproduct of it, as is computing eigenvectors when given eigenvalues. 4. Computing orthonormal basis is a sequence of projection operators (and all linear maps are matrix multiplications) 5. I'm pretty sure nobody computes the Jordan canonical form on computers. The point is that the connection between matrices and linear maps is the spirit of linear algebra.
deleted 13y ago
[deleted]
- deleted 13y ago
  [deleted]
lmm 13y ago
After a few years in Scala, where all operators are methods and vice versa (2 + 3 is just sugar for 2.+(3)), the method-operator distinction just seems weird.
- RodericDay 13y ago
  this is really cool. it makes a lot of sense to me (never touched Scala)
  - draegtun 13y ago
    Smalltalk, Self, Ruby, Io and (i think) Perl6 are examples of other languages that also do this.
    - hesselink 13y ago
      Haskell also has first-class operators. Operators are infix, surrounding them with parentheses makes them prefix: (+) 1 2. Functions are prefix, surrounding them with backticks makes them infix: 1 `plus` 2. You can define your own, and define precedence and left/right associativity.
      - TazeTSchnitzel 13y ago
        Also, all methods are curried by default, meaning you can do this: map (*2) [1,2,3,4,5] :)
      - draegtun 13y ago
        Rebol is similar where it turns its (pre-defined) infix operators into function calls. For eg. 1 + 1 is transformed into... add 1 1
- seertaak 13y ago
  I don't think it's really that different -- either Scala does some magic to desugar the operators into method calls taking into account operator preference, or you're going to get results that don't match expectations. If latter, then like Rebol operators don't match (mathematical) expectation. If former, then there is actually a distinction between methods and operators in Scala, so Scala is just like Python. Or am I missing something?
Aardwolf 13y ago
They gave a rationale for @ compared to other not yet used ascii characters. But they didn't consider a combination of multiple characters, like e.g. 3 * s? (NOTE: Hacker news does not allow typing 3 * in a row)
- ecma 13y ago
  I suspect this was to distance matrix mult from traditional scalar mult operators. Remember that ** is just shorthand for n *s. whereas matrix mult is a different beast entirely. There is also added clutter with 3* where typos may be non-obvious (already a potential issue with ** I guess). Nice, using verbatim to get around markup...
deleted 13y ago
[deleted]
- ZirconCode 13y ago
  Why not just a * As in how ruby does it: http://www.ruby-doc.org/stdlib-2.0/libdoc/matrix/rdoc/Matrix.html#method-i-2A http://www.ruby-doc.org/stdlib-2.0/libdoc/matrix/rdoc/Matrix...
  - hoov 13y ago
    The * operator is already used for elementwise multiplication.
  - dagw 13y ago
    Because * is already used for element-wise matrix multiplication.
  - dragonwriter 13y ago
    Because the whole problem it is solving is that both matrix multiplication and elementwise multiplication are frequently used enough that there is API fragmentation in popular matrix-using Python libraries between places where * is used for matrix multiplication and places where it is used for elementwise multiplication. Read the linked PEP, this is addressed, at length.
- NamTaf 13y ago
  I agree with |* | (sans space, for HN foramtting purposes) over @. It isn't without precident, considering * * is also Python syntax.
bsaul 13y ago
from the pep as to why not using a type that defines _mul_ as matrix multiplication : " The result is a strong consensus among both numpy developers and developers of downstream packages that numpy.matrix should essentially never be used, because of the problems caused by having conflicting duck types for arrays " looks to me like an issue with python lack of static type checking. if you need to define an operator every time two types conflict, you're in for a long list...
waitingkuo 13y ago
Currently, we can use * to multiply two numpy matrix. We'll have two choices, * and @, to do the matrix multiply things in numpy. Is it pythonic? Or should we deprecate * ?
- apl 13y ago
  No, we should deprecate numpy.matrix which is the culprit here. If there's only numpy.ndarray, the semantics are absolutely clear: * for element-wise multiplication and @ for "proper" multiplication of two matrices.
- danohuiginn 13y ago
  The PEP explains this: "In numerical code, there are two important operations which compete for use of Python's * operator: elementwise multiplication, and matrix multiplication" The idea is to keep using * for elementwise multiplication, and use @ for matrix multiplication.
- dragonwriter 13y ago
  Adding @ for matrix multiplication is to enable a consistent matrix API for python where * is always elementwise multiplication. Numpy -- and other major projects that use * for matrix multiplication in APIs, as reported in the PEP -- have committed to migrate to the new standard approach once the PEP is implemented.
NamTaf 13y ago
Matlab's approach to using * as the matrix multiplier makes sense, because every numerical variable is an array with integers/floats simply being a 1x1 array. Using .* then as the elementwise version works. I'd have personally preferred to see Python do type testing on the variables - it already does. For example: 'quick' + 'fox' = 'quickfox' 3 + 5 = 8 [3] + [5] = [3,5] So why not make it a case where * on an int or float does the 'standard' multiplication that already exists whereas * on an array does matrix multiplication? You arrive at the problem of then not having an element-wise version of the multiplication but it's not as if this solves that problem anyway. What am I missing to make that a problem?
- tomp 13y ago
  Python's way of treating * as element-wise multiplication when operands are arrays makes it easier to vectorize algorithms - if f(a, b) = a * b, then you can treat a and b as numbers, a number and an array, or as two (multi-dimensional) arrays. In Matlab, you'd have to use *. for that.
  - Serow225 13y ago
    FYI, in MATLAB you'd only need to use .* in the last of your three cases.
- apl 13y ago
  > You arrive at the problem of then not having an element- > wise version of the multiplication but it's not as if this > solves that problem anyway. Please reread the text. It's exactly what it does, and an exhaustive rationale is given for why both operators are useful and necessary. # Matrix multiplication: a = b @ c # Element-wise multiplication: a = b * c
  - NamTaf 13y ago
    My apologies, I misremembered that from when I first read it. I'm still on python 2.7 - in 3, does: [3] * [5] = [15] ? Because really it needs to reduce down to that if it's going to make sense from a continuity point of view. edit: To clarify: Is that already part of 3.x, or shall it become part of 3.x once this is implemented?
riffraff 13y ago
one question I could not see from the PEP page: why not use a spelling "dot" instead of a symbol? i.e. "a dot b"
- ovis 13y ago
  That might lead to a backwards incompatible symbol clash with `numpy.dot`, `scipy.sparse.csr_matrix.dot`, user code, etc.
- alexchamberlain 13y ago
  When dealing with vectors, the dot product has a specific definition, so it would be very confusing.
mckoss 13y ago
Why not use the existing convention for names of infix operators like __mmul__ but allow then to be infix as well.. So, we could write A __mmul__ B. I think this is more clear than @ and is already familiar to programmers that deal with operator overloading methods. It would also enable the addition of arbitrary infix operators to python.
- atlbeer 13y ago
  __ methods are private and not meant to be called outside of the class The @ decorator vs @ MMUL seems pretty syntactically clear though. @decorator() def function(): variable @ variable
iskander 13y ago
This is surprisingly detailed design rationale which weighs many alternatives and gives careful consideration to possible tradeoffs. Great job by Nathaniel Smith and the numerical Python community.
pekk 13y ago
Python's syntax is already pretty ponderous, this kind of thing pushes it rapidly toward bash-like incomprehensibility. I've been a big fan of the language for years but this kind of thing makes me consider jumping to Lua or something which is more sparing with new complicated sigils.
- Borogove 13y ago
  How does this reduce comprehensibility? In cases where you're doing matrix muls, "a @ b" is way clearer than "a.dot(b)". In other cases, infix "@" won't appear.
- bqe 13y ago
  I actually find Python's syntax to be extremely clear, even to someone who hasn't written much Python. Can you give some examples of where it's particularly incomprehensible?
lutorm 13y ago
Since this is just a special case of a multiply-and-add indexing loop, maybe they should just introduce some form of tensor notation, so that A[i,j]*B[j,k] is the matrix product of A and B? That would extend to so many more use cases than just a 2d matrix product.
- gaius 13y ago
  U+2297 CIRCLED TIMES is already the logarithm operator in APL, so you'd want to avoid that confusion. Phew!
JadeNB 13y ago
My issue is not with the matrix-multiplication operator—I'm a mathematician before I'm a programmer, and so am all for it—but with the vector-to-matrix promotions: why not just consistently promote vectors to columns (or rows, if BDFL swings that way)? This would achieve almost the same effect as promoting to rows or columns as needed, and would avoid the non-associativity problem that the PEP mentions. This PEP seems to imply that the cost would be a profusion of superfluous `newaxis`s, but I can't see that: it seems to me that you would need only to remember which kind of promotion works by default, and sprinkle in a `.T` whenever you need the other kind. (Anyone who's uncomfortable with lots of `.T`s in matrix-crunching code is not, I submit, someone who writes or reads lots of matrix-crunching code.)
JoshTriplett 13y ago
Personally, I'd really like to see an equivalent in Python to Haskell's `binop` , to turn an arbitrary two-argument function into a binary operator.
- JadeNB 13y ago
  I think that you're not alone, but this is specifically considered and rejected (in the section on "Rejected alternatives …"): > Add lots of new operators, or add a new generic syntax for defining infix operators: In addition to being generally un-Pythonic and repeatedly rejected by BDFL fiat, this would be using a sledgehammer to smash a fly.
  - JoshTriplett 13y ago
    Anyone have a reference for where that in particular was rejected by BDFL fiat?
    - takluyver 13y ago
      PEP 225 was an attempt to add a load of new operators that was rejected (well, deferred 14 years ago). I don't know of a proposal to define arbitrary binary operators.
JadeNB 13y ago
The PEP claims that: > Matrix multiplication is more of a special case. It's only defined on 2d arrays (also known as "matrices") but this is not true. Matrix multiplication is just a special case of contraction of indices in a tensor (https://en.wikipedia.org/wiki/Tensor_contraction)—probably https://en.wikipedia.org/wiki/Tensor_contraction)—probably the most frequently used case, but not the only one. I'm certainly not arguing for the inclusion of general tensor-manipulating operators in Python, but it does seem to suggest a sensible alternative to: > For inputs with more than 2 dimensions, we treat the last two dimensions as being the dimensions of the matrices to multiply, and 'broadcast' across the other dimensions. namely, just contract on the inner indices. That is, arr(n1, ..., nk, m) @ arr(m, p1, ... pl) = arr(n1, ..., nk, p1, ..., pl). EDIT: scythe (https://news.ycombinator.com/item?id=7554013 https://news.ycombinator.com/item?id=7554013) already pointed this out in passing.
- onedognight 13y ago
  > namely, just contract on the inner indices. That is, arr(n1, ..., nk, m) @ arr(m, p1, ... pl) = arr(n1, ..., nk, p1, ..., pl). No. This is what a mathematician might assume the PEP proposes without actually reading it. It instead proposes an entirely non-obvious definition which not equivalent to what you wrote. In particular consider this example from the PEP. arr(10, 2, 3) @ arr(10, 3, 4) = arr(10, 2, 4)
  - JadeNB 13y ago
    I know it is not equivalent—that is why I proposed it as a sensible alternative to what the PEP proposes (which I quoted). The punctuation may have made it unclear, but what I was trying to say was: > … a sensible alternative to [PEP proposal]; namely, just contract on the inner indices. and not > … a sensible alternative to contracting on the inner indices. My argument for why it's sensible is precisely what you mentioned, namely, that it is what a mathematician would expect.
0xdeadbeefbabe 13y ago
Dear Guido, You lost me at 3.
pattern 13y ago
I really like this snippet from the justification for the symbol chosen: "APL apparently used +.×, which by combining a multi-character token, confusing attribute-access-like . syntax, and a unicode character, ranks somewhere below U+2603 SNOWMAN on our candidate list". One of the reasons I have been such a fan of Python for so long is the relatively no-nonsense approach to design decisions that many others would have rushed through.

268 points by madisonmay 13y ago

Guido has accepted the proposal and work has now begun to add support for the new operator: http://bugs.python.org/issue21176 http://bugs.python.org/issue21176

BoppreH 13y ago

I think "Matrix Multiplication" is a little too specific for a syntax change, but I welcome the addition of a new operator to be overloaded. Am I missing some useful applications of matrix multiplication on a general purpose language?

heron87 13y ago
Ehm, Python is not a "scripting language". It is a — as you said — general purpose programming language. And it is becoming more and more popular for scientific applications, where matrix multiplication plays a key role.
- BoppreH 13y ago
  I honestly didn't expect a downvote for calling Python a "scripting language". I use the term loosely and always thought Python fit spot on. I still think it applies here, but I've edited the parent post to correct this possibly incorrect usage of the word. Just for the record, these are the characteristics of Python that made me call it "scripting": - Interpreted - Highly expressive - No boilerplate - Vast majority of programs are short - Lots of libraries for gluing stuff together - Used for turing-complete customization of software in other languages (e.g.: Sublime Text) - Recommended as replacement for Bash in some cases Anyway, I'm happy to see the language evolving to meet its users' needs.
  - heron87 13y ago
    No offense intended. I just have a slightly different understanding of a "scripting language": a — often domain-specific — language that is merely used to write small scripts/macros. But anyway, I did not want to start a discussion about that. Sorry for that.
  - PeterisP 13y ago
    I believe that the common usage of python has switched away from "scripting usage" - it's now generally used to implement stuff, not glue stuff together or replace bash, and the vast majority of programs isn't short anymore. It can be used as a scripting language, but that doesn't seem to be the main use-case anymore.
  - wisty 13y ago
    Stolen from https://www.python.org/doc/humor/ https://www.python.org/doc/humor/ (in response to a claim that it's not a true OO language): > Indeed, and because it doesn't support closures, it's not a true functional programming language either. And because you have to import all sorts of modules to do the simplest things (e.g., regular expressions), neither is it a true scripting language. Indeed, because it doesn't support labeled break or continue statements, it's not even a true structured programming language. I think the point is, trying to classify programming languages (except domain specific ones like SQL or maybe PHP) is kind of pointless. But to answer your real question, Python is one of the top languages in number crunching. After web work, that's Python's second biggest niche.
    - TillE 13y ago
      > it doesn't support closures It does now, and thank goodness, because even C++ has closures these days.
      - Chronic26 13y ago
        Python is a scripting language. People who write python are "scripting," not "programming."
  - 13y ago
- adamors 13y ago
  I'm sorry, but how is Python not a scripting language?
  - mbq 13y ago
    It is just that "scripting" is now called "managed" because it hurts ego to be a scripting programmer.
dtf 13y ago
I, for one, welcome our new operator overloads. Partly a recognition of just how massive Python has become within the scientific community.
paddyoloughlin 13y ago
The beginning of the PEP (linked document) has a rationale for it, including an argument that matrix multiplication has a combination of features which distinguish it from other binary operations. Check it out. Like most Python PEPs it is quite readable. There are even alternative proposals and links to where the discussion of this issue has occurred. It seems that the most fundamental reason is that matrix and memberwise multiplication are two distinct operations which use the same datatypes as operands and both have a very good claim to use of the * operator. Also, just because a language is general purpose does not mean that it cannot support features which might be restricted in their use. Two of the python's most popular uses are in scientific and numeric fields (see SciPy, NumPy and ipython). This operator seems like it would be very useful for improving readability in those areas which is probably reason enough to use it. However, please resist the urge to just overload this new operator for something unrelated that you want. The point of infix operators is to improve the readability of code and lessen the cognitive load on the reader. Overloading an operator for a purpose which is not conventional goes exactly against this for... what benefit exactly?
- BoppreH 13y ago
  Thanks. I understand its importance to scientific computing and I welcome the change. I was just wondering if matrix multiplication had more flexible uses, like `sum` does. And the benefit of overloading is still readability, when carefully used and in a different domain. And sparingly. For the `@` operator I will be trying it as an "at" operator for accessing objects from a game map, or some sort of translation. If it performs well I may start using it in my pet projects. If it changes my life and enlightens everyone who gazes upon it, I may start using in production. I do understand your worry and I promise to be extra careful.
  - dragonwriter 13y ago
    > For the `@` operator I will be trying it as an "at" operator for accessing objects from a game map I suppose you could use it for that that, but wouldn't that be a canonical case for indexing with []?
CamperBob2 13y ago
Here's my uneducated guess: the Python maintainers see NumPy as a bit of a killer app, and the NumPy maintainers would like to be a MATLAB killer. The entire MATLAB language is designed to make life easier when dealing with matrices (people think it stands for MATh LAB but it actually stands for MATrix LAB.) MATLAB went down the matrix rabbit hole so far, they made ordinary scalars second-class types. If you want to multiply scalars or otherwise do element-wise operations, you have to prefix the '*' operator with a '.'. If Python can make their language more linear-algebra friendly while avoiding that sort of silliness, I'm all for it.
- GFK_of_xmaspast 13y ago
  """If you want to multiply scalars or otherwise do element-wise operations, you have to prefix the '*' operator with a '.'. """ what???
  - quux 13y ago
    You don't have to use .* and friends when operating in 2 scalers, 2 * 3 works just fine. But if you have 2 matrices A and B, A * B does a matrix multiplication, And A .* B is needed for element wise multiplication. I'm still new to matlab, but the place where this has bitten me is if I'm trying to write code that works on either a scalar or vector of scAlars, if you aren't specific, the code my run fine against a single scalar, but blow up or do the wrong thing with a vector.
arnarbi 13y ago
It's addressed in the PEP, even with some convincing empirical data. Python is becoming hugely popular as a scientific data processing language. This paragraph from the motivation summarizes: "We all live in our own little sub-communities, so some Python users may be surprised to realize the sheer extent to which Python is used for number crunching -- especially since much of this particular sub-community's activity occurs outside of traditional Python/FOSS channels. So, to give some rough idea of just how many numerical Python programmers are actually out there, here are two numbers: In 2013, there were 7 international conferences organized specifically on numerical Python [3] [4]. At PyCon 2014, ~20% of the tutorials appear to involve the use of matrices [6]."

segphault 13y ago

There are a few handy things that I'd like to see added to Python's syntax. A dedicated operator for matrix multiplication isn't one of them.

aberrant 13y ago
What would you like to see?
- shadowmint 13y ago
  multi-line lambdas.
  - Noelkd 13y ago
    Wouldn't you just define a function in that case? Guido has mentioned[1] that he isn't a fan of lambda functions. [1] http://www.linuxjournal.com/article/2959 http://www.linuxjournal.com/article/2959
  - jdpage 13y ago
    Not happening, because it would mean multi-line function arguments, which have been declared "ugly". http://legacy.python.org/dev/peps/pep-3099/ http://legacy.python.org/dev/peps/pep-3099/
  - EliAndrewC 13y ago
    Guido explains in an old blog post (http://www.artima.com/weblogs/viewpost.jsp?thread=147358 http://www.artima.com/weblogs/viewpost.jsp?thread=147358) that he considers all of the possible implementations of multi-line lambdas to be un-Pythonic: > But the complexity of any proposed solution for this puzzle is immense, to me: it requires the parser (or more precisely, the lexer) to be able to switch back and forth between indent-sensitive and indent-insensitive modes, keeping a stack of previous modes and indentation level. Technically that can all be solved (there's already a stack of indentation levels that could be generalized). But none of that takes away my gut feeling that it is all an elaborate Rube Goldberg contraption.
- 13y ago

beefsack 13y ago

You've got to wonder, does matrix multiplication happen so often and is it so inconvenient to call as a function that it's worth complicating the syntax? I'm really not sure about this one.

c0g 13y ago
Yes! Numpy is hugely popular in (at least) physics and machine learning and the main negative comment I get is "Wow that's horrible syntax". I'm still not sure they'll be convinced, however: trans_coef.T.dot(solve(H.dot(V).dot(H.T)), trans_coef) becomes trans_coef.T @ solve(H @ V @ H.T, trans_coef) but in MATLAB or Julia might be: trans_coef' * ((H * V * H') \ trans_coef) ....I want a backslash operator now!
jonemo 13y ago
A few fields that use matrix computations extensively that come to mind: robotics, graphics, control, machine learning. I'd go as far as saying that matrix computation plays an important role in more or less every kind of programming except for web/app development. edit: As another comment points out, the PEP actually answers this question too and contains a slightly more comprehensive list of disciplines where matrices find frequent use: http://legacy.python.org/dev/peps/pep-0465/#but-isn-t-matrix-multiplication-a-pretty-niche-requirement http://legacy.python.org/dev/peps/pep-0465/#but-isn-t-matrix...
yen223 13y ago
I'd say matrix multiplication is far more common than complex numbers, and Python natively supports those.
- baq 13y ago
  yeah but complex numbers don't introduce new punctuation aka symbolic noise.
stiff 13y ago
When you have a multiplication of 5-10 matrices that has to be done in the correct order, the expression gets horribly unreadable when written as nested function applications. And those kinds of expressions are ubiquitous in numerical computing.

fest 13y ago

I hope this is not a first step towards transforming python syntax into perl.

jdpage 13y ago
It won't be. The Python parser has to remain LL(1) by BDFL fiat.[1] 1: http://legacy.python.org/dev/peps/pep-3099/ http://legacy.python.org/dev/peps/pep-3099/
- lamby 13y ago
  OP was clearly referring to style and aesthetics, not the grammar's level in the Chomksy hierarchy.

willvarfar 13y ago

The nice thing is that it makes a nice symbol `@` available to objects that aren't matrices, to do with as they want ... The (ab)use seems endless :)

scrollaway 13y ago
@ is already used by decorators. I hope decorators will be reimplemented as prefix operators on functions with this change, so that @<foo> becomes available as well.
- ecma 13y ago
  With @ as a grammatical infix matrix mult operator, there doesn't seem to be any real need beyond exclusivity to redefine the decorator grammar. By the same argument, we shouldn't use * for unpacking.
- mcherm 13y ago
  But the use as an operator and the use as a decorator is easily distinguished by where they appear in the grammar. Neither the parser nor human readers (as far as I can tell) have any difficulty determining which use is which. So let's not confuse the two: they are two different uses that happen to utilize the same symbol.
  - scrollaway 13y ago
    My point is that if @<func> is re-implemented as, for example, the function.__dec__, then it can be overridden and some cool DSL uses can pop up. As it stands, the use of @ as a (class-)decorator is built in to the grammar, rather than simply being a prefix operator.
    - ecma 13y ago
      How might that implementation be used in code? Would coders be directly accessing and overriding the __dec__ attribute? Since decorators are definition time anyway, I don't really see much potential that decorated function factories can't already provide.

zurn 13y ago

To everyone arguing against special-casing matrix multiplication, please base your arguments on the PEP's "Motivation" section to avoid too much of rehashing the obvious. It even has a subsection "But isn't matrix multiplication a pretty niche requirement?"

cabinpark 13y ago
I would argue that almost every single linear algebra routine can be interpreted in some form as a matrix-matrix or matrix-vector multiplication. Considering that matrix-matrix and matrix-vector are fundamental numerical operations (they are part of BLAS which are the basic API functions called by every single linear algebra routine I know of) having a concise notation is key. And since formulas can get complicated quickly, having a closer 1-1 correspondence with the mathematics is critical for understanding the meaning of the code. The PEP contains a nice example from statistics. And trust me, when you are writing numerical code, being able to read the mathematical formula clearly is essential, especially when you have a lot of formulas and need to figure out why your code is giving you numerical non-sense.
- omaranto 13y ago
  What you say about almost all linear algebra routines being multiplication sounds crazy to me. How are the following supposed to be multiplication: (1) computing the transpose, (2) finding a basis for the kernel of a matrix, (3) factorizations, such as QR, LU or SVD (3) computing the Jordan normal form of a matrix, etc.?
  - j2kun 13y ago
    It's not as crazy as it sounds, though I wouldn't make such a blanket statement. Things can be phrased in terms of matrix multiplication (maybe not a single matrix multiplication), it's just not the most efficient way to go about it. 1. Transpose is a linear map on matrices (vectors in R^nm), so in a very concrete sense it is precisely a matrix multiplication. And it's not hard to figure out the matrix, because it's analogous to the matrix that swaps entries in vectors. 2. Finding the first eigenvalue can be approximated via matrix multiplication, and for sparse matrices with good spectral gap this extends to all of them. 3. Row reduction can be phrased as matrix multiplication, and hence finding the basis for the kernel of a matrix is a byproduct of it, as is computing eigenvectors when given eigenvalues. 4. Computing orthonormal basis is a sequence of projection operators (and all linear maps are matrix multiplications) 5. I'm pretty sure nobody computes the Jordan canonical form on computers. The point is that the connection between matrices and linear maps is the spirit of linear algebra.

deleted 13y ago

[deleted]

deleted 13y ago
[deleted]

lmm 13y ago

After a few years in Scala, where all operators are methods and vice versa (2 + 3 is just sugar for 2.+(3)), the method-operator distinction just seems weird.

RodericDay 13y ago
this is really cool. it makes a lot of sense to me (never touched Scala)
- draegtun 13y ago
  Smalltalk, Self, Ruby, Io and (i think) Perl6 are examples of other languages that also do this.
  - hesselink 13y ago
    Haskell also has first-class operators. Operators are infix, surrounding them with parentheses makes them prefix: (+) 1 2. Functions are prefix, surrounding them with backticks makes them infix: 1 `plus` 2. You can define your own, and define precedence and left/right associativity.
    - TazeTSchnitzel 13y ago
      Also, all methods are curried by default, meaning you can do this: map (*2) [1,2,3,4,5] :)
    - draegtun 13y ago
      Rebol is similar where it turns its (pre-defined) infix operators into function calls. For eg. 1 + 1 is transformed into... add 1 1
seertaak 13y ago
I don't think it's really that different -- either Scala does some magic to desugar the operators into method calls taking into account operator preference, or you're going to get results that don't match expectations. If latter, then like Rebol operators don't match (mathematical) expectation. If former, then there is actually a distinction between methods and operators in Scala, so Scala is just like Python. Or am I missing something?

Aardwolf 13y ago

They gave a rationale for @ compared to other not yet used ascii characters. But they didn't consider a combination of multiple characters, like e.g. 3 * s? (NOTE: Hacker news does not allow typing 3 * in a row)

ecma 13y ago
I suspect this was to distance matrix mult from traditional scalar mult operators. Remember that ** is just shorthand for n *s. whereas matrix mult is a different beast entirely. There is also added clutter with 3* where typos may be non-obvious (already a potential issue with ** I guess). Nice, using verbatim to get around markup...

deleted 13y ago

[deleted]

ZirconCode 13y ago
Why not just a * As in how ruby does it: http://www.ruby-doc.org/stdlib-2.0/libdoc/matrix/rdoc/Matrix.html#method-i-2A http://www.ruby-doc.org/stdlib-2.0/libdoc/matrix/rdoc/Matrix...
- hoov 13y ago
  The * operator is already used for elementwise multiplication.
- dagw 13y ago
  Because * is already used for element-wise matrix multiplication.
- dragonwriter 13y ago
  Because the whole problem it is solving is that both matrix multiplication and elementwise multiplication are frequently used enough that there is API fragmentation in popular matrix-using Python libraries between places where * is used for matrix multiplication and places where it is used for elementwise multiplication. Read the linked PEP, this is addressed, at length.
NamTaf 13y ago
I agree with |* | (sans space, for HN foramtting purposes) over @. It isn't without precident, considering * * is also Python syntax.

bsaul 13y ago

from the pep as to why not using a type that defines _mul_ as matrix multiplication : " The result is a strong consensus among both numpy developers and developers of downstream packages that numpy.matrix should essentially never be used, because of the problems caused by having conflicting duck types for arrays " looks to me like an issue with python lack of static type checking. if you need to define an operator every time two types conflict, you're in for a long list...

waitingkuo 13y ago

Currently, we can use * to multiply two numpy matrix. We'll have two choices, * and @, to do the matrix multiply things in numpy. Is it pythonic? Or should we deprecate * ?

apl 13y ago
No, we should deprecate numpy.matrix which is the culprit here. If there's only numpy.ndarray, the semantics are absolutely clear: * for element-wise multiplication and @ for "proper" multiplication of two matrices.
danohuiginn 13y ago
The PEP explains this: "In numerical code, there are two important operations which compete for use of Python's * operator: elementwise multiplication, and matrix multiplication" The idea is to keep using * for elementwise multiplication, and use @ for matrix multiplication.
dragonwriter 13y ago
Adding @ for matrix multiplication is to enable a consistent matrix API for python where * is always elementwise multiplication. Numpy -- and other major projects that use * for matrix multiplication in APIs, as reported in the PEP -- have committed to migrate to the new standard approach once the PEP is implemented.

NamTaf 13y ago

Matlab's approach to using * as the matrix multiplier makes sense, because every numerical variable is an array with integers/floats simply being a 1x1 array. Using .* then as the elementwise version works. I'd have personally preferred to see Python do type testing on the variables - it already does. For example: 'quick' + 'fox' = 'quickfox' 3 + 5 = 8 [3] + [5] = [3,5] So why not make it a case where * on an int or float does the 'standard' multiplication that already exists whereas * on an array does matrix multiplication? You arrive at the problem of then not having an element-wise version of the multiplication but it's not as if this solves that problem anyway. What am I missing to make that a problem?

tomp 13y ago
Python's way of treating * as element-wise multiplication when operands are arrays makes it easier to vectorize algorithms - if f(a, b) = a * b, then you can treat a and b as numbers, a number and an array, or as two (multi-dimensional) arrays. In Matlab, you'd have to use *. for that.
- Serow225 13y ago
  FYI, in MATLAB you'd only need to use .* in the last of your three cases.
apl 13y ago
> You arrive at the problem of then not having an element- > wise version of the multiplication but it's not as if this > solves that problem anyway. Please reread the text. It's exactly what it does, and an exhaustive rationale is given for why both operators are useful and necessary. # Matrix multiplication: a = b @ c # Element-wise multiplication: a = b * c
- NamTaf 13y ago
  My apologies, I misremembered that from when I first read it. I'm still on python 2.7 - in 3, does: [3] * [5] = [15] ? Because really it needs to reduce down to that if it's going to make sense from a continuity point of view. edit: To clarify: Is that already part of 3.x, or shall it become part of 3.x once this is implemented?

riffraff 13y ago

one question I could not see from the PEP page: why not use a spelling "dot" instead of a symbol? i.e. "a dot b"

ovis 13y ago
That might lead to a backwards incompatible symbol clash with `numpy.dot`, `scipy.sparse.csr_matrix.dot`, user code, etc.
alexchamberlain 13y ago
When dealing with vectors, the dot product has a specific definition, so it would be very confusing.

mckoss 13y ago

Why not use the existing convention for names of infix operators like __mmul__ but allow then to be infix as well.. So, we could write A __mmul__ B. I think this is more clear than @ and is already familiar to programmers that deal with operator overloading methods. It would also enable the addition of arbitrary infix operators to python.

atlbeer 13y ago
__ methods are private and not meant to be called outside of the class The @ decorator vs @ MMUL seems pretty syntactically clear though. @decorator() def function(): variable @ variable

iskander 13y ago

This is surprisingly detailed design rationale which weighs many alternatives and gives careful consideration to possible tradeoffs. Great job by Nathaniel Smith and the numerical Python community.

pekk 13y ago

Python's syntax is already pretty ponderous, this kind of thing pushes it rapidly toward bash-like incomprehensibility. I've been a big fan of the language for years but this kind of thing makes me consider jumping to Lua or something which is more sparing with new complicated sigils.

Borogove 13y ago
How does this reduce comprehensibility? In cases where you're doing matrix muls, "a @ b" is way clearer than "a.dot(b)". In other cases, infix "@" won't appear.
bqe 13y ago
I actually find Python's syntax to be extremely clear, even to someone who hasn't written much Python. Can you give some examples of where it's particularly incomprehensible?

lutorm 13y ago

Since this is just a special case of a multiply-and-add indexing loop, maybe they should just introduce some form of tensor notation, so that A[i,j]*B[j,k] is the matrix product of A and B? That would extend to so many more use cases than just a 2d matrix product.

gaius 13y ago
U+2297 CIRCLED TIMES is already the logarithm operator in APL, so you'd want to avoid that confusion. Phew!

JadeNB 13y ago

My issue is not with the matrix-multiplication operator—I'm a mathematician before I'm a programmer, and so am all for it—but with the vector-to-matrix promotions: why not just consistently promote vectors to columns (or rows, if BDFL swings that way)? This would achieve almost the same effect as promoting to rows or columns as needed, and would avoid the non-associativity problem that the PEP mentions. This PEP seems to imply that the cost would be a profusion of superfluous `newaxis`s, but I can't see that: it seems to me that you would need only to remember which kind of promotion works by default, and sprinkle in a `.T` whenever you need the other kind. (Anyone who's uncomfortable with lots of `.T`s in matrix-crunching code is not, I submit, someone who writes or reads lots of matrix-crunching code.)

JoshTriplett 13y ago

Personally, I'd really like to see an equivalent in Python to Haskell's `binop` , to turn an arbitrary two-argument function into a binary operator.

JadeNB 13y ago
I think that you're not alone, but this is specifically considered and rejected (in the section on "Rejected alternatives …"): > Add lots of new operators, or add a new generic syntax for defining infix operators: In addition to being generally un-Pythonic and repeatedly rejected by BDFL fiat, this would be using a sledgehammer to smash a fly.
- JoshTriplett 13y ago
  Anyone have a reference for where that in particular was rejected by BDFL fiat?
  - takluyver 13y ago
    PEP 225 was an attempt to add a load of new operators that was rejected (well, deferred 14 years ago). I don't know of a proposal to define arbitrary binary operators.

JadeNB 13y ago

The PEP claims that: > Matrix multiplication is more of a special case. It's only defined on 2d arrays (also known as "matrices") but this is not true. Matrix multiplication is just a special case of contraction of indices in a tensor (https://en.wikipedia.org/wiki/Tensor_contraction)—probably https://en.wikipedia.org/wiki/Tensor_contraction)—probably the most frequently used case, but not the only one. I'm certainly not arguing for the inclusion of general tensor-manipulating operators in Python, but it does seem to suggest a sensible alternative to: > For inputs with more than 2 dimensions, we treat the last two dimensions as being the dimensions of the matrices to multiply, and 'broadcast' across the other dimensions. namely, just contract on the inner indices. That is, arr(n1, ..., nk, m) @ arr(m, p1, ... pl) = arr(n1, ..., nk, p1, ..., pl). EDIT: scythe (https://news.ycombinator.com/item?id=7554013 https://news.ycombinator.com/item?id=7554013) already pointed this out in passing.

onedognight 13y ago
> namely, just contract on the inner indices. That is, arr(n1, ..., nk, m) @ arr(m, p1, ... pl) = arr(n1, ..., nk, p1, ..., pl). No. This is what a mathematician might assume the PEP proposes without actually reading it. It instead proposes an entirely non-obvious definition which not equivalent to what you wrote. In particular consider this example from the PEP. arr(10, 2, 3) @ arr(10, 3, 4) = arr(10, 2, 4)
- JadeNB 13y ago
  I know it is not equivalent—that is why I proposed it as a sensible alternative to what the PEP proposes (which I quoted). The punctuation may have made it unclear, but what I was trying to say was: > … a sensible alternative to [PEP proposal]; namely, just contract on the inner indices. and not > … a sensible alternative to contracting on the inner indices. My argument for why it's sensible is precisely what you mentioned, namely, that it is what a mathematician would expect.

0xdeadbeefbabe 13y ago

Dear Guido, You lost me at 3.

pattern 13y ago

I really like this snippet from the justification for the symbol chosen: "APL apparently used +.×, which by combining a multi-character token, confusing attribute-access-like . syntax, and a unicode character, ranks somewhere below U+2603 SNOWMAN on our candidate list". One of the reasons I have been such a fan of Python for so long is the relatively no-nonsense approach to design decisions that many others would have rushed through.