6 ms·

Fair. But to play devils advocate: one stated goal of Julia is to solve the “two languages” problem where developers prototype in {scripting language} and then

Fair. But to play devils advocate: one stated goal of Julia is to solve the “two languages” problem where developers prototype in {scripting language} and then reimplement in {system language}. So while Julia is meant to be as easy as Python, it also needs to be as powerful and flexible as Rust (or C or whatever). It’s supposed to be the “best of both worlds” and Rust lives in one of those worlds. If developers are better off prototyping in Python jupyter notebooks and then reimplementing in Rust than they are implementing in Julia, then I’d say the comparison is relevant. Maybe it’s an unfair standard and Julia can be a “better Python” without needing to be a full replacement of a systems language, but then Julia isn’t fully solving the two language problem, right?

duped 5y ago
The two language problem exists for a bunch of reasons, some good and some not. The biggest reason is because some function of the high level language is incompatible with the application domain. Like garbage collection in hot or real-time code or proprietary compilers for processors. Julia does not solve these problems. Other reasons are practical, like portable executables in Go or Rust (portable in the sense they do not require dependencies on their target systems, usually). Julia does not solve this problem. Then there are the reasons to use scripting languages over the system languages, like expressive syntax with low cognitive overhead. Julia definitely helps here. But this comes at the cost of execution and startup time. Julia only kind of solves this problem. So if Julia is trying to make a more performance scripting language then that is admirable. But for most of the projects where I have needed to prototype in a script and implement in a systems language, Julia would not have worked. Even today I don't have a good reason to use it over MATLAB for day to day work, since I already have the license and their ecosystem is more mature.
- deleted 5y ago
  [deleted]
- eigenspace 5y ago
  > The biggest reason is because some function of the high level language is incompatible with the application domain. Like garbage collection in hot or real-time code or proprietary compilers for processors. Julia does not solve these problems. The presence of garbage collection in julia is not a problem at all for hot, high performance code. There's nothing stopping you from manually managing your memory in julia. The easiest way would be to just preallocate your buffers and hold onto them so they don't get collected. Octavian.jl is a BLAS library written in julia that's faster than OpenBLAS and MKL for small matrices and saturates to the same speed for very large matrices [1]. These are some of the hottest loops possible! For true, hard-real time, yes julia is not a good choice but it's perfectly fine for soft realtime. [1] https://github.com/JuliaLinearAlgebra/Octavian.jl/issues/24#issuecomment-766243445 https://github.com/JuliaLinearAlgebra/Octavian.jl/issues/24#...
  - duped 5y ago
    Doesn't Julia use a mark-and-sweep GC? How is it possible to use in a soft-real-time context without stopping the world?
    - eigenspace 5y ago
      Just because a GC exists doesn't mean you have to use it though. It's just another tool on your belt. There's various options depending on your problem. You could just hold firmly onto memory, preallocating all your buffers before the program starts and then don't allow them to be GC'd, you could also use stack allocated arrays like in StrideArrays.jl / StaticArrays.jl, or some combination of the above. You could also just manually use ccall to malloc and free memory as needed I guess. There's a nice talk here [1] about using julia in soft-real-time for robotics. I'd say the biggest impediment to real-time programming in julia currently is not the GC, but instead that the language has lots of optional optimizations that the compiler can choose to not perform if it thinks it'd be beneficial to do so, and those optimizations can change between minor versions. Hence, there's a lot of testing you'd need to do to make sure your code really is doing exactly what you want and you won't hit the GC or dynamic dispatch, and you'll need to redo that testing every time you update julia or any packages, which is obviously not ideal. [1] https://www.youtube.com/watch?v=dmWQtI3DFFo https://www.youtube.com/watch?v=dmWQtI3DFFo
hexane360 5y ago
Yes, this is exactly what I was getting at. Speed is one of the reasons to choose Rust/C/C++ over Python, but there's other compelling ones: static types, more control over execution, tight coupling to the OS, etc. To truly solve the two language problem, Julia should have APIs and coding styles which allow greater control, even if they're not used by everyone. Because it's so closely integrated with C, Python actually does a decent job at this, which IMO is one of the reasons it's been so successful as a general purpose language in addition to a scientific one. I really hope Julia can do the same. Off the top of my head, here are a few features I think would really improve Julia for general purpose programming: - Static type checking (JET.jl[1] looks promising) - Faster startup (again, great strides have been made) - Tagged, closed unions - Pattern matching [1]: https://github.com/aviatesk/JET.jl https://github.com/aviatesk/JET.jl
- eigenspace 5y ago
  > Pattern matching MLStyle.jl [1] is quite nice for this and has been around for a while. > Tagged, closed unions These are less general than 'real' unions and can be implemented using them. E.g. SumTypes.jl [2] has some macros to make it a bit more convenient to define them, it could use some other quality of life features though. [1] https://thautwarm.github.io/MLStyle.jl/latest/syntax/pattern.html https://thautwarm.github.io/MLStyle.jl/latest/syntax/pattern... [2] https://github.com/MasonProtter/SumTypes.jl https://github.com/MasonProtter/SumTypes.jl
  - hexane360 5y ago
    MLStyle.jl is nice, but I think Julia would really benefit from having it enshrined in the language. Multiple dispatch is great for a lot of problems, but sometimes it makes more sense to do your pattern matching inline. For tagged unions, the difference is in memory layout. A real tagged union type would eliminate indirection and allow more code to be type stable. For instance, mutable struct NotTypeStable o::Union{Some{Int}, Nothing} end function not_type_stable() o = returns_option() if isnothing(o) 0 else 1 end end versus: enum Option{T} Some(T) None end mutable struct TypeStable o::Option{Int} end function type_stable() o = returns_option() match o Some(_) => 1 None => 0 end end Both of these features aren't strictly necessary, but they act as a compliment to Julia's existing dynamic mechanisms (multiple dispatch & Union types)

duped 5y ago

The two language problem exists for a bunch of reasons, some good and some not. The biggest reason is because some function of the high level language is incompatible with the application domain. Like garbage collection in hot or real-time code or proprietary compilers for processors. Julia does not solve these problems. Other reasons are practical, like portable executables in Go or Rust (portable in the sense they do not require dependencies on their target systems, usually). Julia does not solve this problem. Then there are the reasons to use scripting languages over the system languages, like expressive syntax with low cognitive overhead. Julia definitely helps here. But this comes at the cost of execution and startup time. Julia only kind of solves this problem. So if Julia is trying to make a more performance scripting language then that is admirable. But for most of the projects where I have needed to prototype in a script and implement in a systems language, Julia would not have worked. Even today I don't have a good reason to use it over MATLAB for day to day work, since I already have the license and their ecosystem is more mature.

deleted 5y ago
[deleted]
eigenspace 5y ago
> The biggest reason is because some function of the high level language is incompatible with the application domain. Like garbage collection in hot or real-time code or proprietary compilers for processors. Julia does not solve these problems. The presence of garbage collection in julia is not a problem at all for hot, high performance code. There's nothing stopping you from manually managing your memory in julia. The easiest way would be to just preallocate your buffers and hold onto them so they don't get collected. Octavian.jl is a BLAS library written in julia that's faster than OpenBLAS and MKL for small matrices and saturates to the same speed for very large matrices [1]. These are some of the hottest loops possible! For true, hard-real time, yes julia is not a good choice but it's perfectly fine for soft realtime. [1] https://github.com/JuliaLinearAlgebra/Octavian.jl/issues/24#issuecomment-766243445 https://github.com/JuliaLinearAlgebra/Octavian.jl/issues/24#...
- duped 5y ago
  Doesn't Julia use a mark-and-sweep GC? How is it possible to use in a soft-real-time context without stopping the world?
  - eigenspace 5y ago
    Just because a GC exists doesn't mean you have to use it though. It's just another tool on your belt. There's various options depending on your problem. You could just hold firmly onto memory, preallocating all your buffers before the program starts and then don't allow them to be GC'd, you could also use stack allocated arrays like in StrideArrays.jl / StaticArrays.jl, or some combination of the above. You could also just manually use ccall to malloc and free memory as needed I guess. There's a nice talk here [1] about using julia in soft-real-time for robotics. I'd say the biggest impediment to real-time programming in julia currently is not the GC, but instead that the language has lots of optional optimizations that the compiler can choose to not perform if it thinks it'd be beneficial to do so, and those optimizations can change between minor versions. Hence, there's a lot of testing you'd need to do to make sure your code really is doing exactly what you want and you won't hit the GC or dynamic dispatch, and you'll need to redo that testing every time you update julia or any packages, which is obviously not ideal. [1] https://www.youtube.com/watch?v=dmWQtI3DFFo https://www.youtube.com/watch?v=dmWQtI3DFFo

hexane360 5y ago

Yes, this is exactly what I was getting at. Speed is one of the reasons to choose Rust/C/C++ over Python, but there's other compelling ones: static types, more control over execution, tight coupling to the OS, etc. To truly solve the two language problem, Julia should have APIs and coding styles which allow greater control, even if they're not used by everyone. Because it's so closely integrated with C, Python actually does a decent job at this, which IMO is one of the reasons it's been so successful as a general purpose language in addition to a scientific one. I really hope Julia can do the same. Off the top of my head, here are a few features I think would really improve Julia for general purpose programming: - Static type checking (JET.jl[1] looks promising) - Faster startup (again, great strides have been made) - Tagged, closed unions - Pattern matching [1]: https://github.com/aviatesk/JET.jl https://github.com/aviatesk/JET.jl

eigenspace 5y ago
> Pattern matching MLStyle.jl [1] is quite nice for this and has been around for a while. > Tagged, closed unions These are less general than 'real' unions and can be implemented using them. E.g. SumTypes.jl [2] has some macros to make it a bit more convenient to define them, it could use some other quality of life features though. [1] https://thautwarm.github.io/MLStyle.jl/latest/syntax/pattern.html https://thautwarm.github.io/MLStyle.jl/latest/syntax/pattern... [2] https://github.com/MasonProtter/SumTypes.jl https://github.com/MasonProtter/SumTypes.jl
- hexane360 5y ago
  MLStyle.jl is nice, but I think Julia would really benefit from having it enshrined in the language. Multiple dispatch is great for a lot of problems, but sometimes it makes more sense to do your pattern matching inline. For tagged unions, the difference is in memory layout. A real tagged union type would eliminate indirection and allow more code to be type stable. For instance, mutable struct NotTypeStable o::Union{Some{Int}, Nothing} end function not_type_stable() o = returns_option() if isnothing(o) 0 else 1 end end versus: enum Option{T} Some(T) None end mutable struct TypeStable o::Option{Int} end function type_stable() o = returns_option() match o Some(_) => 1 None => 0 end end Both of these features aren't strictly necessary, but they act as a compliment to Julia's existing dynamic mechanisms (multiple dispatch & Union types)