4 ms·

C++ is pain, numpy is pain if you need a loop, numba and cython are pain if you need any more complex data structures. Sadly Julia is pain too if you don't do

C++ is pain, numpy is pain if you need a loop, numba and cython are pain if you need any more complex data structures. Sadly Julia is pain too if you don't do REPL/notebook (which you shouldn't). Julia has the design to solve the two language problem but not the implementation. And will probably never have because Julia community refuses to see this as a problem.

cosmic_quanta 3y ago
What's the two language problem? One language for a high level interface and one language for low level, high performance computations?
- adgjlsfhk1 3y ago
  exactly
qsort 3y ago
I'm starting to think that the two language problem is not really solvable at the language level. Maybe it is, but there's a significant friction. High level naturally pushes you towards abstract types, GC, not caring about allocation, do what I mean not what I said. Low level naturally pushes you towards concrete types, deterministic mallocs, do what I said not what I mean. e.g. do I want integers like Python or integers like C? Yes.
- markkitti 3y ago
  > do I want integers like Python or integers like C? Yes. Julia has BigInt and Cint. Maybe there could be a more Pythonic implementation that scales between machine types and BigInt. That would not be hard to implement in Julia. I just have not found a good use case. What I like about Julia is that I can do both the high level and low level in Julia. Abstract code becomes concrete due to late binding. You can access Libc.malloc and Libc.free if needed. See GPUCompiler.jl The effect system exists. If you can prove no side effects, eager finalization is also possible. That is the finalizer and deallocation will run deterministically. It's new though so effect analysis is a mostly manual affair at the moment.
- bmitc 3y ago
  I feel F# is a pretty good candidate. One can get pretty down and dirty with low-level CLR stuff or unmanaged code, but then you have a very nice high-level language.
  - soulbadguy 3y ago
    +1 on F#, one of the most underrated language around.
    - agumonkey 3y ago
      It's so odd how it's been probably a decade of "F# is underrated" but nothing makes it shine to the eyes of the market .. really strange
      - bmitc 3y ago
        It's really more of a result of human sentiment than anything. F# is underrated and underappreciated, and it just doesn't have a critical mass of people developing libraries for it. The issues to its adoption were being associated with .NET (Windows-only at the time) and Microsoft and being sold as a C# alternative rather than its own thing and an alternative to other languages like Python. An interesting thought experiment is to think if someone was to release F# today as a new language but calling it something else, like Flosure or something. I bet people would lose their minds over it.
      - blendo 3y ago
        I’ve always thought Microsoft should embed it into Excel as a coequal language to VBA. Finance programmers would go nuts!
        agumonkey 3y ago
        True, and it may even attract R users since it seems functional in its core. MS did include lambdas as a primitive but also decided to use python as new formula language.
        3y ago
- jampekka 3y ago
  Javascript is not that far from solving the two language problem. In fact v8/spidermonkey speed is very acceptable for most number crunching. And Javascript isn't even very friendly for high performance but The JITs are miraculous. Sadly Javascript as language nor its "ecosystem" are very suitable for data-analysis like use. I want integers like Python that are as fast as integers like C. I don't care how they are allocated or deallocated. Javascript comes very close even without having integers at all. Probably nobody expects to squeeze every last cycle with a higher level language. I'd say getting consistently within magnitude of C performance can be said to have solved the two-language problem. GC pauses etc are perfectly acceptable for e.g. data analysis.
  - markkitti 3y ago
    Via Pluto.jl notebooks, I have been able explore the Julia-JavaScript interface. It's quite fun using JavaScript and SVG to interactively render the accelerated BLAS and FFTW outputs from Julia.
    - jjoonathan 3y ago
      Ooh, interesting! Jupyter got rid of their direct javascript interface for security reasons that I don't agree with. Does Pluto.jl have a model where trusting display code is as easy as trusting the code being run?
  - staunton 3y ago
    Let's see how you do those high energy physics simulations with JS...
    - jampekka 3y ago
      Why would I do that? Although I wouldn't be surprised them being faster than ROOT with e.g. asm.js.
      - elashri 3y ago
        In some cases yes that will be true. But good luck having the inertia to replace ROOT in general (by anything). While many people hate ROOT (I do and feel the pain daily) it is highly tailored optimized piece that will take decades to try to implement something like it. And I doubt you would be able to handle low-level tasks like optimizing memory footprint needed for HEP applications when running stuff in online world (while collecting data). JS+ASM greatest advantage is cross-platform which is one of the least features requested in the field. i.e No one expect you to be able to run your analysis code on windows.
- th0ma5 3y ago
  I sincerely agree with your comments so much. People that don't live with code don't see this and have weird romantic ideas of what it is and what it does. Mixing levels of abstraction has never made much sense (ultimately) to me.
Rarebox 3y ago
https://yuri.is/not-julia/ https://yuri.is/not-julia/ is a good write-up of one person's opinion on the problems of Julia. I'm much less experienced with Julia but I somewhat agree. There's too much focus on "magic" instead of correctness for me to try building serious software using Julia. An amazing language in many aspects though.
- roflmaostc 3y ago
  Some of the response in the Julia community: https://discourse.julialang.org/t/discussion-on-why-i-no-longer-recommend-julia-by-yuri-vishnevsky/81151 https://discourse.julialang.org/t/discussion-on-why-i-no-lon...
roflmaostc 3y ago
>Sadly Julia is pain too if you don't do REPL/notebook (which you shouldn't) Can you be more precise on this? In my case, I do a lot work around visual outputs (such as images). So notebook based development (such as Pluto.jl) feels perfectly fine.
- jampekka 3y ago
  > Can you be more precise on this? Usage where you don't keep the Julia process running. E.g. "$ julia stuff.jl" form a shell.
  - amj7e 3y ago
    Yes, this has been discussed a lot recently in the Julia community and some efforts have been made to make it happen, there will be more about this in the 1.11 release. Standalone binaries are next.
    - jampekka 3y ago
      If this is fixed, I'll probably switch from Python to Julia. As a language Julia is vastly superior to Python. The complaint was dismissed for so long (and is still dismissed e.g. here in HN discussions) I gave up hope it will be fixed, but I'd be ecstatic to be proven wrong. Do standalone binaries here include shared libraries (with a C ABI)? That would be a dream.
      - amj7e 3y ago
        The one you need is this which is already merged but was after the 1.10 feature freeze so it has to wait till 1.11, though you can test it with nightly builds which is available on julialang site: https://github.com/JuliaLang/julia/pull/51435 https://github.com/JuliaLang/julia/pull/51435 Unfortunately, the core devs are not too chatty about standalone binaries, because of how Julia's internals are set there are going to be a lot of unforeseen challenges, so they are not trying to promise how things will be rather let's wait and see how things will turnout. Since packagecompiler.jl already has C ABI and one goal discussed about binaries being easily callable from other languages and vice versa, I would bet that it will have shared libraries.
markkitti 3y ago
I thought that notebook based development and package based development were diametrically opposed in the past, but Pluto.jl notebooks have changed my mind about this. A Pluto.jl notebook is a human readable Julia source file. The Pluto.jl package is itself developed via Pluto.jl notebooks. https://github.com/fonsp/Pluto.jl https://github.com/fonsp/Pluto.jl Also, the VSCode Julia plugin tooling has really expanded in functionality and usability for me in the past year. The integrated debugging took some work to setup, but is fast enough to drop into a local frame. https://code.visualstudio.com/docs/languages/julia https://code.visualstudio.com/docs/languages/julia Julia is the first language I have achieved full life cycle integration between exploratory code to sharable package. It even runs quite well on my Android. 2023 is the first year I was able to solve a differential equation or render a 3D surface from a calculated mesh with the hardware in my pocket.
- jampekka 3y ago
  Pluto.jl does solve the state problem of notebooks, which is the biggest problem in them. Probably should give it another go. How is the module reloading story nowadays? Last time I gave up the state of the art was Revise.jl, which was a world of pain.
  - staunton 3y ago
    > Pluto.jl does solve the state problem of notebooks Yes it does, unless you do crazy stuff (which you probably should put in a Julia file and let it precompile). The order of cells and execution order does not matter (unless you do crazy stuff).
    - markkitti 3y ago
      A Pluto.jl notebook is also a regular file. You need to use a package for precompilation caching though.
  - tamasgal 3y ago
    I have no idea what kind of problems you have ;) I use Revise.jl on a daily basis and it works flawlessly. I don't have any comparable workflow in any other language I work with on a daily basis. I wish I could do this style in those as well... Yes, you should give it a go (again)!
tempodox 3y ago
The maintainers seem to have a very narrow focus on what they think their target audience is and everything outside of that gets ignored.
- markkitti 3y ago
  I honestly do not understand comments like this with regard to an open source project. The language and its libraries are rather hackable as is with rather powerful packages. While I may not agree with every decision, I do not complain that someone is not implementing my favorite feature. I just do it myself. I frankly would be concerned if the main contributors were not focused.
- soulbadguy 3y ago
  I think the julia team is still relatively small. So it might be was they have to do to make meaningful progress?
  - jampekka 3y ago
    Julia's window has probably passed. Pytorch, JAX, Numba etc are eating its lunch. Which is sad because they are all pretty hacky stuff compared to what Julia could be.
    - soulbadguy 3y ago
      Sadly i think you maybe right. With support of one the big 3 (meta,google or MSFT),it will be hard for Julia to create and sustain significant mind share. Wich is really a shame because from technical design standpoint Julia is just brilliant. Adding to that modular mojo, and i think we might have a repeat of DART vs typescript...
    - uoaei 3y ago
      JAX is the least hacky of them and my personal favorite, but ultimately I have to agree. I'm a huge Julia stan but they simply don't have the institutional support that Meta and Google can provide.
    - 3y ago
staunton 3y ago
What's the problem when working without REPL/notebook? The debugger IDE support will be fixed eventually™ (I really think it will.. eventually...). What else is there?
- jampekka 3y ago
  Startup speed.
  - staunton 3y ago
    That's getting much better since they started caching compiled code. With more software (and hardware, and synergy-between-them) improvements coming, this will soon no longer be an issue.

cosmic_quanta 3y ago

What's the two language problem? One language for a high level interface and one language for low level, high performance computations?

adgjlsfhk1 3y ago
exactly

qsort 3y ago

I'm starting to think that the two language problem is not really solvable at the language level. Maybe it is, but there's a significant friction. High level naturally pushes you towards abstract types, GC, not caring about allocation, do what I mean not what I said. Low level naturally pushes you towards concrete types, deterministic mallocs, do what I said not what I mean. e.g. do I want integers like Python or integers like C? Yes.

markkitti 3y ago
> do I want integers like Python or integers like C? Yes. Julia has BigInt and Cint. Maybe there could be a more Pythonic implementation that scales between machine types and BigInt. That would not be hard to implement in Julia. I just have not found a good use case. What I like about Julia is that I can do both the high level and low level in Julia. Abstract code becomes concrete due to late binding. You can access Libc.malloc and Libc.free if needed. See GPUCompiler.jl The effect system exists. If you can prove no side effects, eager finalization is also possible. That is the finalizer and deallocation will run deterministically. It's new though so effect analysis is a mostly manual affair at the moment.
bmitc 3y ago
I feel F# is a pretty good candidate. One can get pretty down and dirty with low-level CLR stuff or unmanaged code, but then you have a very nice high-level language.
- soulbadguy 3y ago
  +1 on F#, one of the most underrated language around.
  - agumonkey 3y ago
    It's so odd how it's been probably a decade of "F# is underrated" but nothing makes it shine to the eyes of the market .. really strange
    - bmitc 3y ago
      It's really more of a result of human sentiment than anything. F# is underrated and underappreciated, and it just doesn't have a critical mass of people developing libraries for it. The issues to its adoption were being associated with .NET (Windows-only at the time) and Microsoft and being sold as a C# alternative rather than its own thing and an alternative to other languages like Python. An interesting thought experiment is to think if someone was to release F# today as a new language but calling it something else, like Flosure or something. I bet people would lose their minds over it.
    - blendo 3y ago
      I’ve always thought Microsoft should embed it into Excel as a coequal language to VBA. Finance programmers would go nuts!
      - agumonkey 3y ago
        True, and it may even attract R users since it seems functional in its core. MS did include lambdas as a primitive but also decided to use python as new formula language.
      - 3y ago
jampekka 3y ago
Javascript is not that far from solving the two language problem. In fact v8/spidermonkey speed is very acceptable for most number crunching. And Javascript isn't even very friendly for high performance but The JITs are miraculous. Sadly Javascript as language nor its "ecosystem" are very suitable for data-analysis like use. I want integers like Python that are as fast as integers like C. I don't care how they are allocated or deallocated. Javascript comes very close even without having integers at all. Probably nobody expects to squeeze every last cycle with a higher level language. I'd say getting consistently within magnitude of C performance can be said to have solved the two-language problem. GC pauses etc are perfectly acceptable for e.g. data analysis.
- markkitti 3y ago
  Via Pluto.jl notebooks, I have been able explore the Julia-JavaScript interface. It's quite fun using JavaScript and SVG to interactively render the accelerated BLAS and FFTW outputs from Julia.
  - jjoonathan 3y ago
    Ooh, interesting! Jupyter got rid of their direct javascript interface for security reasons that I don't agree with. Does Pluto.jl have a model where trusting display code is as easy as trusting the code being run?
- staunton 3y ago
  Let's see how you do those high energy physics simulations with JS...
  - jampekka 3y ago
    Why would I do that? Although I wouldn't be surprised them being faster than ROOT with e.g. asm.js.
    - elashri 3y ago
      In some cases yes that will be true. But good luck having the inertia to replace ROOT in general (by anything). While many people hate ROOT (I do and feel the pain daily) it is highly tailored optimized piece that will take decades to try to implement something like it. And I doubt you would be able to handle low-level tasks like optimizing memory footprint needed for HEP applications when running stuff in online world (while collecting data). JS+ASM greatest advantage is cross-platform which is one of the least features requested in the field. i.e No one expect you to be able to run your analysis code on windows.
th0ma5 3y ago
I sincerely agree with your comments so much. People that don't live with code don't see this and have weird romantic ideas of what it is and what it does. Mixing levels of abstraction has never made much sense (ultimately) to me.

Rarebox 3y ago

https://yuri.is/not-julia/ https://yuri.is/not-julia/ is a good write-up of one person's opinion on the problems of Julia. I'm much less experienced with Julia but I somewhat agree. There's too much focus on "magic" instead of correctness for me to try building serious software using Julia. An amazing language in many aspects though.

roflmaostc 3y ago
Some of the response in the Julia community: https://discourse.julialang.org/t/discussion-on-why-i-no-longer-recommend-julia-by-yuri-vishnevsky/81151 https://discourse.julialang.org/t/discussion-on-why-i-no-lon...

roflmaostc 3y ago

>Sadly Julia is pain too if you don't do REPL/notebook (which you shouldn't) Can you be more precise on this? In my case, I do a lot work around visual outputs (such as images). So notebook based development (such as Pluto.jl) feels perfectly fine.

jampekka 3y ago
> Can you be more precise on this? Usage where you don't keep the Julia process running. E.g. "$ julia stuff.jl" form a shell.
- amj7e 3y ago
  Yes, this has been discussed a lot recently in the Julia community and some efforts have been made to make it happen, there will be more about this in the 1.11 release. Standalone binaries are next.
  - jampekka 3y ago
    If this is fixed, I'll probably switch from Python to Julia. As a language Julia is vastly superior to Python. The complaint was dismissed for so long (and is still dismissed e.g. here in HN discussions) I gave up hope it will be fixed, but I'd be ecstatic to be proven wrong. Do standalone binaries here include shared libraries (with a C ABI)? That would be a dream.
    - amj7e 3y ago
      The one you need is this which is already merged but was after the 1.10 feature freeze so it has to wait till 1.11, though you can test it with nightly builds which is available on julialang site: https://github.com/JuliaLang/julia/pull/51435 https://github.com/JuliaLang/julia/pull/51435 Unfortunately, the core devs are not too chatty about standalone binaries, because of how Julia's internals are set there are going to be a lot of unforeseen challenges, so they are not trying to promise how things will be rather let's wait and see how things will turnout. Since packagecompiler.jl already has C ABI and one goal discussed about binaries being easily callable from other languages and vice versa, I would bet that it will have shared libraries.

markkitti 3y ago

I thought that notebook based development and package based development were diametrically opposed in the past, but Pluto.jl notebooks have changed my mind about this. A Pluto.jl notebook is a human readable Julia source file. The Pluto.jl package is itself developed via Pluto.jl notebooks. https://github.com/fonsp/Pluto.jl https://github.com/fonsp/Pluto.jl Also, the VSCode Julia plugin tooling has really expanded in functionality and usability for me in the past year. The integrated debugging took some work to setup, but is fast enough to drop into a local frame. https://code.visualstudio.com/docs/languages/julia https://code.visualstudio.com/docs/languages/julia Julia is the first language I have achieved full life cycle integration between exploratory code to sharable package. It even runs quite well on my Android. 2023 is the first year I was able to solve a differential equation or render a 3D surface from a calculated mesh with the hardware in my pocket.

jampekka 3y ago
Pluto.jl does solve the state problem of notebooks, which is the biggest problem in them. Probably should give it another go. How is the module reloading story nowadays? Last time I gave up the state of the art was Revise.jl, which was a world of pain.
- staunton 3y ago
  > Pluto.jl does solve the state problem of notebooks Yes it does, unless you do crazy stuff (which you probably should put in a Julia file and let it precompile). The order of cells and execution order does not matter (unless you do crazy stuff).
  - markkitti 3y ago
    A Pluto.jl notebook is also a regular file. You need to use a package for precompilation caching though.
- tamasgal 3y ago
  I have no idea what kind of problems you have ;) I use Revise.jl on a daily basis and it works flawlessly. I don't have any comparable workflow in any other language I work with on a daily basis. I wish I could do this style in those as well... Yes, you should give it a go (again)!

tempodox 3y ago

The maintainers seem to have a very narrow focus on what they think their target audience is and everything outside of that gets ignored.

markkitti 3y ago
I honestly do not understand comments like this with regard to an open source project. The language and its libraries are rather hackable as is with rather powerful packages. While I may not agree with every decision, I do not complain that someone is not implementing my favorite feature. I just do it myself. I frankly would be concerned if the main contributors were not focused.
soulbadguy 3y ago
I think the julia team is still relatively small. So it might be was they have to do to make meaningful progress?
- jampekka 3y ago
  Julia's window has probably passed. Pytorch, JAX, Numba etc are eating its lunch. Which is sad because they are all pretty hacky stuff compared to what Julia could be.
  - soulbadguy 3y ago
    Sadly i think you maybe right. With support of one the big 3 (meta,google or MSFT),it will be hard for Julia to create and sustain significant mind share. Wich is really a shame because from technical design standpoint Julia is just brilliant. Adding to that modular mojo, and i think we might have a repeat of DART vs typescript...
  - uoaei 3y ago
    JAX is the least hacky of them and my personal favorite, but ultimately I have to agree. I'm a huge Julia stan but they simply don't have the institutional support that Meta and Google can provide.
  - 3y ago

staunton 3y ago

What's the problem when working without REPL/notebook? The debugger IDE support will be fixed eventually™ (I really think it will.. eventually...). What else is there?

jampekka 3y ago
Startup speed.
- staunton 3y ago
  That's getting much better since they started caching compiled code. With more software (and hardware, and synergy-between-them) improvements coming, this will soon no longer be an issue.