8 ms·

Jeremy here. Thank you for sharing my article on HN! I've used more programming languages than I can count since I started coding 40 years ago, and it's not oft

by jph00 3y ago

Jeremy here. Thank you for sharing my article on HN! I've used more programming languages than I can count since I started coding 40 years ago, and it's not often I've gotten really excited about a new language. So the fact I've written this should tell you how special I think Mojo is. It's early days of course - Mojo is still a prerelease, which means it's only available in an online playground and it's not open source yet (although that's planned). But for such a small team to create something like this so quickly gives me a lot of confidence for what's coming over the next few months. Let me know if you have any questions or comments.

flufferstutter 3y ago
[flagged]
- rhogar 3y ago
  The broader context of the target audience of this language is very important here — it’s oriented towards the familiarity and needs of a community most comfortable with python
tysam_and 3y ago
Hi Jeremy, thanks for sharing this, hope it works out well. Also, thanks for doing the past work on Dawnbench&etc that y'all did. Really helped push an era of speedy deep learning to the forefront and helped stoke the flames of my passion for this particular subfield of research. FastAI really put out a ton of good work that inspired and helped me a lot. I'm generally stuck on the linear version of OneCycle, but I've referred a lot to y'all and the community over the years when building out my toolbox. In any case, you have more years under your belt than me in the software field, so I'm sure you have some of the standard skepticisms too, hopefully it works out really well. Will be keeping an eye on this software, it seems promising. Esp as I tend towards the more esoteric methods that tend to break the existing deep learning tooling out there.
- jph00 3y ago
  I really appreciate you saying that! I'm glad it helped you.
deleted 3y ago
[deleted]
hcrisp 3y ago
Thanks, Jeremy. It does sound exciting. It reminds me a bit of what Blaze hoped to offer (a unified interface but to diverse data storage instead of processing systems) but never came to fruition. And folks have been talking about needing an ML/AI-centric language that could feel more natural and expressive than the abstractions that Tensorflow, PyTorch, and Jax provide. Maybe Mojo is it. To get full performance though, you can't write just Python. As you show in your demo, you have to add verbose typedefs, structs, additional fn/defs, SIMD calls, vectorization hooks, loop-unrolling, autotune insertions, etc. While great, this adds mental overhead and clutters an otherwise concise and elegant syntax. Do you think this syntactic molasses will become second nature to developers? Will IDE tools make writing it easier?
- cafed00d 3y ago
  > To get full performance though, you can't write just Python. As you show in your demo, you have to add verbose typedefs, structs, additional fn/defs, SIMD calls, vectorization hooks, loop-unrolling, autotune insertions, etc. As an engineer, it feels like you can never escape this. Speaking for myself, nor do I want to! Zooming in & out of different syntaxes that are tuned to that local context seems to be the better developer experience. That’s what I felt when I saw mixed jsx (html & javascript) or mixed swift & objc & cpp code. Though, tbh, sometimes mixing stuff in does come with baggage; that’s hard to jettison. Regardless, as a software engineer who works with ml engineers, it is horrendously painful not having a unified systems language that helps build the model and deploy it into prod. Putting the ML scientists in charge of deploying to production by learning about SIMD calls, vectorization hooks, structs & typedefs is something I welcome.
- MattRix 3y ago
  Most of what you’re calling syntax isn’t syntax, it’s just extra code that allows you to be specific/concrete about how certain types and code should operate. When it comes to types, once you’ve written the optimized type definition, you don’t have to think about it as much when actually using it. It doesn’t add extra clutter either. As far as the other things, there isn’t really much extra to type other than being specific about what kind of optimization you want the code to use… vectorization, unrolling, etc. Also I think it’s worth pointing out that AIs will be writing more and more code for us in the future. So assuming we’re writing much code at all in the future, it will probably be in as simple of a form as possible (like python) and then we can ask the AI to write whatever performance annotations it thinks will be effective.
UncleOxidant 3y ago
It seems like Julia already had a pretty good solution to the two language problem as well as being designed from the ground up for numerical computation (Python has numpy, but it seems bolted on and clunky by comparison). Yes, there are issues with large executables (large runtime), non-optimal gpu kernels, etc, but it seems like many of Julia's nagging issues could have been fairly easily solved if it had received as much investment as some of the alternatives have received - Modular appears to have a lot of funding to develop Mojo, and frameworks like PyTorch and TensorFlow have the backing of mega tech corporations. At one point Swift for TensorFlow was going to be the next big ML language and was being funded and developed by Google. I'm not sure there's a question here... more of a frustration that Julia seems to be a very good solution already (and is more mature than Mojo) and yet it seems to get passed over because companies decide to fund some new(er) shiny thing instead. BTW: I think your matrix multiplication demo could achieve very similar performance (including the SIMD, parallelization and LoopVectorization) in Julia.
- Nimitz14 3y ago
  julia's syntax is weird.
  - pjmlp 3y ago
    As if that poses any problem to R, C++, Perl,...
    - nibles_and_bits 3y ago
      R -> dying, Perl -> dead, C++ -> also dying. Although C++'s existing size will make its death a long-drawn-out process.
      - pjmlp 3y ago
        A matter of opinion.
      - Y_Y 3y ago
        All the programmers are slowly dying too, so maybe it's a good fit.
v3ss0n 3y ago
This is very exciting , I had seen you had mentioned about Numba and Cython but any comments on Pypy ? How about Interfacing Mojo LLVM+MLIR With PyPy? Would it make it more compatible with python ecosystem since 90% of python ecosystem works on PyPy. I afraid i am not making any sense , but this is what i had been waiting for past 13 years since i touched python 2.5.
fanf2 3y ago
I am curious about a few syntactic details: Why is the & (reference) argument convention a suffix? That seems inconsistent with Python’s *args and **kwargs, and the keyword conventions. What is the difference between the “owned” and “consuming” conventions? And why is the ^ (consume) operator postfix? Python doesn’t have any postfix operators, and it seems awkward wrt the ^ infix xor operator.
- chrislattner 3y ago
  I'd love it if you could ask technical questions on the discord channel, as that's more scalable for us. Some quick answers: 1) We're likely to change the reference syntax to inout: https://github.com/modularml/mojo/issues/7 https://github.com/modularml/mojo/issues/7 2) they're the same thing. Consume is the word we're currently using for the operator, owned is the argument convention. We may need to iterate on terminology a bit more. 3) Because it composes properly with chained expressions: `x.foo().bar().baz^.do_thing()` vs something like `(move x.foo().bar().baz).do_thing()`
  - derbOac 3y ago
    I worry a bit that postfix use of ^ is going to create some headaches with people expecting exponentiation.
  - adgjlsfhk1 3y ago
    can you link the discord channel, I've seen it mentioned several times but haven't been able to find it. Edit: found it. https://discord.gg/modular https://discord.gg/modular

by jph00 3y ago

flufferstutter 3y ago

[flagged]

rhogar 3y ago
The broader context of the target audience of this language is very important here — it’s oriented towards the familiarity and needs of a community most comfortable with python

tysam_and 3y ago

Hi Jeremy, thanks for sharing this, hope it works out well. Also, thanks for doing the past work on Dawnbench&etc that y'all did. Really helped push an era of speedy deep learning to the forefront and helped stoke the flames of my passion for this particular subfield of research. FastAI really put out a ton of good work that inspired and helped me a lot. I'm generally stuck on the linear version of OneCycle, but I've referred a lot to y'all and the community over the years when building out my toolbox. In any case, you have more years under your belt than me in the software field, so I'm sure you have some of the standard skepticisms too, hopefully it works out really well. Will be keeping an eye on this software, it seems promising. Esp as I tend towards the more esoteric methods that tend to break the existing deep learning tooling out there.

jph00 3y ago
I really appreciate you saying that! I'm glad it helped you.

deleted 3y ago

[deleted]

hcrisp 3y ago

Thanks, Jeremy. It does sound exciting. It reminds me a bit of what Blaze hoped to offer (a unified interface but to diverse data storage instead of processing systems) but never came to fruition. And folks have been talking about needing an ML/AI-centric language that could feel more natural and expressive than the abstractions that Tensorflow, PyTorch, and Jax provide. Maybe Mojo is it. To get full performance though, you can't write just Python. As you show in your demo, you have to add verbose typedefs, structs, additional fn/defs, SIMD calls, vectorization hooks, loop-unrolling, autotune insertions, etc. While great, this adds mental overhead and clutters an otherwise concise and elegant syntax. Do you think this syntactic molasses will become second nature to developers? Will IDE tools make writing it easier?

cafed00d 3y ago
> To get full performance though, you can't write just Python. As you show in your demo, you have to add verbose typedefs, structs, additional fn/defs, SIMD calls, vectorization hooks, loop-unrolling, autotune insertions, etc. As an engineer, it feels like you can never escape this. Speaking for myself, nor do I want to! Zooming in & out of different syntaxes that are tuned to that local context seems to be the better developer experience. That’s what I felt when I saw mixed jsx (html & javascript) or mixed swift & objc & cpp code. Though, tbh, sometimes mixing stuff in does come with baggage; that’s hard to jettison. Regardless, as a software engineer who works with ml engineers, it is horrendously painful not having a unified systems language that helps build the model and deploy it into prod. Putting the ML scientists in charge of deploying to production by learning about SIMD calls, vectorization hooks, structs & typedefs is something I welcome.
MattRix 3y ago
Most of what you’re calling syntax isn’t syntax, it’s just extra code that allows you to be specific/concrete about how certain types and code should operate. When it comes to types, once you’ve written the optimized type definition, you don’t have to think about it as much when actually using it. It doesn’t add extra clutter either. As far as the other things, there isn’t really much extra to type other than being specific about what kind of optimization you want the code to use… vectorization, unrolling, etc. Also I think it’s worth pointing out that AIs will be writing more and more code for us in the future. So assuming we’re writing much code at all in the future, it will probably be in as simple of a form as possible (like python) and then we can ask the AI to write whatever performance annotations it thinks will be effective.

UncleOxidant 3y ago

It seems like Julia already had a pretty good solution to the two language problem as well as being designed from the ground up for numerical computation (Python has numpy, but it seems bolted on and clunky by comparison). Yes, there are issues with large executables (large runtime), non-optimal gpu kernels, etc, but it seems like many of Julia's nagging issues could have been fairly easily solved if it had received as much investment as some of the alternatives have received - Modular appears to have a lot of funding to develop Mojo, and frameworks like PyTorch and TensorFlow have the backing of mega tech corporations. At one point Swift for TensorFlow was going to be the next big ML language and was being funded and developed by Google. I'm not sure there's a question here... more of a frustration that Julia seems to be a very good solution already (and is more mature than Mojo) and yet it seems to get passed over because companies decide to fund some new(er) shiny thing instead. BTW: I think your matrix multiplication demo could achieve very similar performance (including the SIMD, parallelization and LoopVectorization) in Julia.

Nimitz14 3y ago
julia's syntax is weird.
- pjmlp 3y ago
  As if that poses any problem to R, C++, Perl,...
  - nibles_and_bits 3y ago
    R -> dying, Perl -> dead, C++ -> also dying. Although C++'s existing size will make its death a long-drawn-out process.
    - pjmlp 3y ago
      A matter of opinion.
    - Y_Y 3y ago
      All the programmers are slowly dying too, so maybe it's a good fit.

v3ss0n 3y ago

This is very exciting , I had seen you had mentioned about Numba and Cython but any comments on Pypy ? How about Interfacing Mojo LLVM+MLIR With PyPy? Would it make it more compatible with python ecosystem since 90% of python ecosystem works on PyPy. I afraid i am not making any sense , but this is what i had been waiting for past 13 years since i touched python 2.5.

fanf2 3y ago

I am curious about a few syntactic details: Why is the & (reference) argument convention a suffix? That seems inconsistent with Python’s *args and **kwargs, and the keyword conventions. What is the difference between the “owned” and “consuming” conventions? And why is the ^ (consume) operator postfix? Python doesn’t have any postfix operators, and it seems awkward wrt the ^ infix xor operator.

chrislattner 3y ago
I'd love it if you could ask technical questions on the discord channel, as that's more scalable for us. Some quick answers: 1) We're likely to change the reference syntax to inout: https://github.com/modularml/mojo/issues/7 https://github.com/modularml/mojo/issues/7 2) they're the same thing. Consume is the word we're currently using for the operator, owned is the argument convention. We may need to iterate on terminology a bit more. 3) Because it composes properly with chained expressions: `x.foo().bar().baz^.do_thing()` vs something like `(move x.foo().bar().baz).do_thing()`
- derbOac 3y ago
  I worry a bit that postfix use of ^ is going to create some headaches with people expecting exponentiation.
- adgjlsfhk1 3y ago
  can you link the discord channel, I've seen it mentioned several times but haven't been able to find it. Edit: found it. https://discord.gg/modular https://discord.gg/modular