5 ms·

Does “90,000x speed up over pure python” with matrix multiplication mean vs using numpy arrays in python?

zellyn 3y ago
no
sirfz 3y ago
No It's vs a pure python implementation[1] [1] https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1e5860f79b31d15df116/examples/pymatmul.py#L41 https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1...
- icapybara 3y ago
  Seems a bit deceiving then, nobody does matrix multiplications in "pure python." They use numpy or some derivative of it.
elashri 3y ago
I think it is just pure python I don't know about mojo, but they seem to import python module for this benchmark. https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1e5860f79b31d15df116/examples/matmul.mojo#L23C26-L23C26 https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1... And for some reason they don't compare against NumPy https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1e5860f79b31d15df116/examples/pymatmul.py#L41 https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1... What would be more interesting is to use NumPy and do the vectorized opreations.
- almostdigital 3y ago
  I did just that, it's 120x faster than numpy. See my comment to OP.
viraptor 3y ago
No, plain python code: https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1e5860f79b31d15df116/examples/matmul.mojo#L74 https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1... I'd like to know how fast numpy is here, but they didn't compare... which is weird because that's what almost everyone would use.
- riscy 3y ago
  It’s not just weird, it’s an intentionally dishonest evaluation to build hype around their language. If you ask your average Python programmer what “pure Python” means they’d think numpy is included. Their Mojo code just does the same optimizations numpy certainly has.
  - theldnsn 3y ago
    [dead]
  - __mharrison__ 3y ago
    My experience is that most Python programmers would consider "pure Python" to basically be using the standard library. There's also the issue of doing complicated work with NumPy and you start looping and revert back to slow (pure) Python because you are crossing the NumPy/ Python interface. Tools like Cython, Numba, and probably Mojo help to solve this.
    - pseudalopex 3y ago
      Pure Python means written wholly in Python to most Python programmers in my experience. For some Python programmers this excludes the 3rd party libraries they care about.
    - gcr 3y ago
      For scientific users like they're targeting, `numpy` effectively _is_ part of the standard library.
    - ska 3y ago
woadwarrior01 3y ago
PyTorch recently added support for JIT compiling numpy code[0]. And then there are libraries like Numba[1]. I wonder how Mojo compares with exiting OSS Python JIT libraries like these? [0]: https://pytorch.org/blog/compiling-numpy-code/ https://pytorch.org/blog/compiling-numpy-code/ [1]: https://numba.pydata.org/ https://numba.pydata.org/
rdedev 3y ago
I really don't understand why they do this. I was really excited for the compilation and type checking features but this whole speedup thing is pretty dumb. And I know the people developing mojo knows that too. Kind of seems like some marketing team is pushing them to do all this but their core customers are people who knows what python and numpy is and this kind of talk puts just feels weird I think that mojo in it's current form is not on par with numpy performance (if it was they would be saying that). Even if the performance is same I would still give it a try. Their whole marketing though is making me reconsider
- mirekrusin 3y ago
  They do this because they want to showcase writing code in single language that can do things that other languages can't. Other langauges are either low level or slow, but they are high level and fast.
  - oddthink 3y ago
    Naive question, but how do they distinguish themselves from Julia, which is also in that space?
    - shwaj 3y ago
      Primarily by compatibility with Python, which is the de facto standard in deep learning.
    - NeuroCoder 3y ago
      I think there's a Julia forum thread where they are casually benchmarking comparisons between the two. Not sure if it is still active though.
    - mirekrusin 3y ago
      There is large overlap with julia, yes. Both are addressing two-language problem. Mojo is python syntax first where they want to be proper superset, which gives them access to wide python ecosystem and community. If executed well, this alone can absorb community similarly to how ie. typescript absorbed javascript community. Also similar thing happened with objective-c -> swift - also led by Chris, which gives a lot of credibility to the whole initiative. Julia is proper new language you need to learn, use new tooling around it, ecosystem is quite academia skewed, ie. writing web services is probably not the best idea etc. Additionally Julia suffers from "time to first plot" problem, which alienates a lot of newcomers who are not familiar or simply don't want to switch to programming mode where it becomes less of a problem (repl/notebook style where runtime is always active). Both are very interesting languages, but mojo's starting point and trajectory seem to be at different level, ie. adoption may be very sharp.
almostdigital 3y ago
Here's the same benchmark with numpy instead of native python (on M2 MBP) Python 4.216 GFLOPS Naive: 6.400 GFLOPS 1.52x faster than Python Vectorized: 22.232 GFLOPS 5.27x faster than Python Parallelized: 52.591 GFLOPS 12.47x faster than Python Tiled: 60.888 GFLOPS 14.44x faster than Python Unrolled: 62.514 GFLOPS 14.83x faster than Python Accumulated: 506.209 GFLOPS 120.07x faster than Python

No It's vs a pure python implementation[1] [1] https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1e5860f79b31d15df116/examples/pymatmul.py#L41 https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1...

icapybara 3y ago
Seems a bit deceiving then, nobody does matrix multiplications in "pure python." They use numpy or some derivative of it.

elashri 3y ago

I think it is just pure python I don't know about mojo, but they seem to import python module for this benchmark. https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1e5860f79b31d15df116/examples/matmul.mojo#L23C26-L23C26 https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1... And for some reason they don't compare against NumPy https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1e5860f79b31d15df116/examples/pymatmul.py#L41 https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1... What would be more interesting is to use NumPy and do the vectorized opreations.

almostdigital 3y ago
I did just that, it's 120x faster than numpy. See my comment to OP.

viraptor 3y ago

No, plain python code: https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1e5860f79b31d15df116/examples/matmul.mojo#L74 https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1... I'd like to know how fast numpy is here, but they didn't compare... which is weird because that's what almost everyone would use.

riscy 3y ago
It’s not just weird, it’s an intentionally dishonest evaluation to build hype around their language. If you ask your average Python programmer what “pure Python” means they’d think numpy is included. Their Mojo code just does the same optimizations numpy certainly has.
- theldnsn 3y ago
  [dead]
- __mharrison__ 3y ago
  My experience is that most Python programmers would consider "pure Python" to basically be using the standard library. There's also the issue of doing complicated work with NumPy and you start looping and revert back to slow (pure) Python because you are crossing the NumPy/ Python interface. Tools like Cython, Numba, and probably Mojo help to solve this.
  - pseudalopex 3y ago
    Pure Python means written wholly in Python to most Python programmers in my experience. For some Python programmers this excludes the 3rd party libraries they care about.
  - gcr 3y ago
    For scientific users like they're targeting, `numpy` effectively _is_ part of the standard library.
  - ska 3y ago

woadwarrior01 3y ago

PyTorch recently added support for JIT compiling numpy code[0]. And then there are libraries like Numba[1]. I wonder how Mojo compares with exiting OSS Python JIT libraries like these? [0]: https://pytorch.org/blog/compiling-numpy-code/ https://pytorch.org/blog/compiling-numpy-code/ [1]: https://numba.pydata.org/ https://numba.pydata.org/

rdedev 3y ago

I really don't understand why they do this. I was really excited for the compilation and type checking features but this whole speedup thing is pretty dumb. And I know the people developing mojo knows that too. Kind of seems like some marketing team is pushing them to do all this but their core customers are people who knows what python and numpy is and this kind of talk puts just feels weird I think that mojo in it's current form is not on par with numpy performance (if it was they would be saying that). Even if the performance is same I would still give it a try. Their whole marketing though is making me reconsider

mirekrusin 3y ago
They do this because they want to showcase writing code in single language that can do things that other languages can't. Other langauges are either low level or slow, but they are high level and fast.
- oddthink 3y ago
  Naive question, but how do they distinguish themselves from Julia, which is also in that space?
  - shwaj 3y ago
    Primarily by compatibility with Python, which is the de facto standard in deep learning.
  - NeuroCoder 3y ago
    I think there's a Julia forum thread where they are casually benchmarking comparisons between the two. Not sure if it is still active though.
  - mirekrusin 3y ago
    There is large overlap with julia, yes. Both are addressing two-language problem. Mojo is python syntax first where they want to be proper superset, which gives them access to wide python ecosystem and community. If executed well, this alone can absorb community similarly to how ie. typescript absorbed javascript community. Also similar thing happened with objective-c -> swift - also led by Chris, which gives a lot of credibility to the whole initiative. Julia is proper new language you need to learn, use new tooling around it, ecosystem is quite academia skewed, ie. writing web services is probably not the best idea etc. Additionally Julia suffers from "time to first plot" problem, which alienates a lot of newcomers who are not familiar or simply don't want to switch to programming mode where it becomes less of a problem (repl/notebook style where runtime is always active). Both are very interesting languages, but mojo's starting point and trajectory seem to be at different level, ie. adoption may be very sharp.

almostdigital 3y ago

Here's the same benchmark with numpy instead of native python (on M2 MBP) Python 4.216 GFLOPS Naive: 6.400 GFLOPS 1.52x faster than Python Vectorized: 22.232 GFLOPS 5.27x faster than Python Parallelized: 52.591 GFLOPS 12.47x faster than Python Tiled: 60.888 GFLOPS 14.44x faster than Python Unrolled: 62.514 GFLOPS 14.83x faster than Python Accumulated: 506.209 GFLOPS 120.07x faster than Python