6 ms·

Startups are building with the Julia Programming Language

47 points by leephillips 4y ago

ubj 4y ago
If anyone from these startups is here on HN, I would love to hear your thoughts on the following: 1. How are you dealing with TTFX when deploying your code? If using PackageCompiler/StaticCompiler, is binary size an issue? 2. Are you using solely Julia, or a combination of Julia + other languages? 3. What unique value is Julia offering your startup that is difficult to find in other languages?
- logankilpatrick 4y ago
  When I worked at Beacon biosignals briefly in 2020, they had Simon Danish (the creator of Makie there full time deploying plots/makie in the browser). One point 2, you should check out Julius Tech which is bridging the multi language future using Julia through their graph computing tool. Other companies like RelationalAI are building on top of Julia but also have their own new language called Rel.
- Loic 4y ago
  TTFX - the time-to-first whatever a package does most[0]. [0]:https://discourse.julialang.org/t/taking-ttfx-seriously-can-we-make-common-packages-faster-to-load-and-use/74949 https://discourse.julialang.org/t/taking-ttfx-seriously-can-...
- thetwentyone 4y ago
  Jumping the gun slightly, but TTFX likely about to get a lot better (5x+ improvement in times) in the next version (1.9). Via cachine of the native compiled code. https://github.com/JuliaLang/julia/pull/47184#issuecomment-1337079281 https://github.com/JuliaLang/julia/pull/47184#issuecomment-1...
  - CyberDildonics 4y ago
    I've been hearing about improvements for the last 8 years. It has been a constant complaint and a constant broken promise.
    - adgjlsfhk1 4y ago
      Since 1.0 it has gotten way better. There's still work to do (obviously) but there has been continual improvement.
      - galangalalgol 4y ago
        My concern for TTFX is the case when X is something infrequent but sensitive to latency? This a ridiculous example but when it is time for the anti-collision feature on my car to kick in, I want it to run, not compile. I think the right answer is to run a bunch of tests to get all the paths compiled and then build a new system image. But I haven't found a ton of guidance on how to do that. Seems like a perfect pipeline task.
        leephillips 4y ago
        That’s the idea. Here’s an example: https://discourse.julialang.org/t/a-julia-dataanalysis-sysimage-from-packagecompiler-its-so-easy-you-should-do-it-too/68127 https://discourse.julialang.org/t/a-julia-dataanalysis-sysim...
      - lambdatronics 4y ago
        I think Jevon's paradox is also hitting to some extent, coupled with the ecosystem getting bigger, so a typical project may have more transitive dependencies behind it. The functionality is going up, and the speed of compilation is going up, but the total latency is staying similar or increasing, in my experience.
        adgjlsfhk1 4y ago
        that's definitely possible. I'm somewhat hopeful that some changes in 1.9 will let us cut back on transitive dependencies a lot, but it will take a few months to know for sure.
- etik 4y ago
  We (metalenz.com) use Julia extensively in our optical design/simulation/modeling workflows: 1. Design team keeps live sessions during interactive work, otherwise launches on virtualized servers (and things take sufficiently long to compute TTFX is a rounding error) 2. Above mentioned team is exclusively Julia. Julia shows up in other things we do (low-level computer vision), but not dominant nor exclusive 3. High performance & expressive programming and flexible autodiff system for scientific computation
philip1209 4y ago
My defunct startup Staffjoy operated from 2015 to 2017 and was an early production user of Julia. We open-sourced some of the code here: https://github.com/staffjoy https://github.com/staffjoy I haven't kept up with the ecosystem - I'm curious how things have improved in the last 5 year!
- leephillips 4y ago
  Probably the most obvious improvements for most users are the big growth in the package ecosystem, with some science and math packages becoming the best available in any language (DifferentialEquations.jl, for example) and the drastic improvement in interactivity: the REPL starts instantly now, and most packages compile much faster. This will get even better in the next major release.
  - Q6T46nT668w6i3m 4y ago
    I’m curious (I’m planning on designing an ODE/PDE package relatively soon), what makes DifferentialEquations.jl the best available?
    - logankilpatrick 4y ago
      The fact that Chris R and the whole SciML ecosystem are using / building on it. Lots of resources going into it and usage in real world places like Pfizer and Moderna.
    - leephillips 4y ago
      This lists some of its unique abilities: https://docs.sciml.ai/DiffEqDocs/stable/ https://docs.sciml.ai/DiffEqDocs/stable/ The routines are sufficiently generic, with regard to Julia’s type system, to allow the solvers to automatically compose with other packages and to seamlessly use types other than Numbers. For example, instead of handling just functions Number→Number, you can define your ODE in terms of quantities with physical dimensions, uncertainties, quaternions, etc., and it will just work (for example, propagating uncertainties correctly to the solution¹). Recent developments involve research into the automated selection of solution routines based on the properties of the ODE, something that seems really next-level to me. [1] https://lwn.net/Articles/834571/ https://lwn.net/Articles/834571/
open-source-ux 4y ago
Julia scientific apps and libraries continue to grow and see momentum. Is there interest in Julia outside the scientifc sphere? I'd be interested to hear of non-scientific projects implemented in Julia (for example, business apps or web-related projects.) Related: I think Julia is an excellent starting language (compared to Python). Is anyone teaching Julia to new programmers with success?
- logankilpatrick 4y ago
  Look at GenieCloud, they are essentially building the Django ecosystem in Julia, it will be huge if they are successful.
- cbkeller 4y ago
  I'm teaching an intro programming course (discipline-specific, for Earth sciences) in Julia at Dartmouth. So far so good I think -- but definitely on the sciences side of things.
vHMtsdf 4y ago
I heard a talk from a large cybersecurity company that was using Julia for research and is apparently in the process of dropping Julia due to: - difficulty in hiring, apparently it was too expensive/difficult to hire Julia programmers, - poorly performing garbage collector. If I recall correctly, they had issues when trying to process huge amounts of data. Seems like small teams will not run into these issues at least initially. I really like Julia and I hope it becomes huge, but stories like this make me wonder whether the whole Julia ecosystem will ever mature to a level of Python for example. And in turn, making it viable for large organizations that might be using e.g. Python + C++ combo that might be easier to hire for/more versatile/with better known drawbacks. Essentially, I am not sure whether the potential of Julia is that much greater than the realized potential of the incumbents to warrant the "Julia is the language of the future".
- logankilpatrick 4y ago
  Please put them in touch with me if possible. RelationalAI has similar GC issues which they are the core Julia developers are working hard to resolve. I have also seen zero postings for Julia roles in cyber security companies so it is likely that they aren't tapping into the community which is why they can't find people.
  - vHMtsdf 4y ago
    It was a talk by folks from this team: https://gradient.avast.io/people-behind/ https://gradient.avast.io/people-behind/
- europeanguy 4y ago
  If you care about performance your code shouldn't be allocating.
- warinukraine 4y ago
  My (more or less extensive) experience with Julia has been rather good. Step 1. Prototyping. Small data, don't care about performance. Julia is as easy as python, but faster. GC doesn't matter. Step 2. Production. Very large datasets. Performance is very important so I make sure that hot loops don't allocate, and so GC doesn't matter. Really, I struggle to see how someone can not like this. Regarding hiring. My experience has been that you can roughly divide people into two categories. Those who like using tools, and those who like solving problems. For the former, a different language might be an issue because they might not know and might not be in the mood for learning it. For the latter, they're happy to learn any tool that can help solve their problem. We want the latter, and for the latter learning Julia hasn't been an issue.
dartharva 4y ago
How good is Julia as a general-purpose language? Can it be a viable replacement for, say, Lua in game development?
- DougBTX 4y ago
  Julia is a very nice language, its core abstractions (mutli-dispatch and macros) are not tied to any particular use-case. My overriding feeling using it is that it is a language visiting from the future, it is very well put together. Having said that, I've not had the opportunity to use it in production, Python is very popular!
  - ModernMech 4y ago
    > My overriding feeling using it is that it is a language visiting from the future Either that, or most languages used today are stuck in the past!
- affinepplan 4y ago
  It is a very well-designed language for many (although of course not all) applications. Most of the ecosystem revolves around scientific computing so you may find there is a bit of a learning curve / sparse existing libraries if you want to use it for game development. Also, on the language side, Julia is optimized for total throughput, which means even though it is easy to write extremely fast code, it sometimes suffers latency issues during stop-the-world garbage collection or JIT compilation. Julia gives the experienced developer tools to avoid this, but especially for those less practiced with the language it could potentially be an obstacle for real-time games like FPS--of course if you are developing a turn-based game this is less of a concern.
- maria2 4y ago
  Isn't Lua's main benefit that it has a small runtime that can be executed inside the context of a large C/C++ executable? Julia has support for C FFI, but it looks like you'd have to manually wrap all of your native code in Julia wrappers. One library I've really liked for this use case is QuickJS. It's a super small ES2020 implementation written in C. It has a great API that lets you bind native C functions and types into JavaScript functions, objects, and classes. This lets you bind native C code that gets used naturally in JavaScript without needing to write wrappers.
eigenspace 4y ago
I'm a huge julia fan, but man is this ever an insubstantial and unconvincing listicle.
- sundarurfriend 4y ago
  Yeah, it doesn't really seem intended for a HN type audience. More of a high level overview of the language to show to a non-technical co-founder or senior executive.
  - markkitti 4y ago
    What is the HN type audience? HN is a news feed run by Ycombinator, a "startup accelerator". While many people around here have deep technical expertise, given the name "Hacker News", I always thought the main purpose was to make news and technical discussions more accessible to non-technical co-founders and startup executives.

47 points by leephillips 4y ago

ubj 4y ago

If anyone from these startups is here on HN, I would love to hear your thoughts on the following: 1. How are you dealing with TTFX when deploying your code? If using PackageCompiler/StaticCompiler, is binary size an issue? 2. Are you using solely Julia, or a combination of Julia + other languages? 3. What unique value is Julia offering your startup that is difficult to find in other languages?

logankilpatrick 4y ago
When I worked at Beacon biosignals briefly in 2020, they had Simon Danish (the creator of Makie there full time deploying plots/makie in the browser). One point 2, you should check out Julius Tech which is bridging the multi language future using Julia through their graph computing tool. Other companies like RelationalAI are building on top of Julia but also have their own new language called Rel.
Loic 4y ago
TTFX - the time-to-first whatever a package does most[0]. [0]:https://discourse.julialang.org/t/taking-ttfx-seriously-can-we-make-common-packages-faster-to-load-and-use/74949 https://discourse.julialang.org/t/taking-ttfx-seriously-can-...
thetwentyone 4y ago
Jumping the gun slightly, but TTFX likely about to get a lot better (5x+ improvement in times) in the next version (1.9). Via cachine of the native compiled code. https://github.com/JuliaLang/julia/pull/47184#issuecomment-1337079281 https://github.com/JuliaLang/julia/pull/47184#issuecomment-1...
- CyberDildonics 4y ago
  I've been hearing about improvements for the last 8 years. It has been a constant complaint and a constant broken promise.
  - adgjlsfhk1 4y ago
    Since 1.0 it has gotten way better. There's still work to do (obviously) but there has been continual improvement.
    - galangalalgol 4y ago
      My concern for TTFX is the case when X is something infrequent but sensitive to latency? This a ridiculous example but when it is time for the anti-collision feature on my car to kick in, I want it to run, not compile. I think the right answer is to run a bunch of tests to get all the paths compiled and then build a new system image. But I haven't found a ton of guidance on how to do that. Seems like a perfect pipeline task.
      - leephillips 4y ago
        That’s the idea. Here’s an example: https://discourse.julialang.org/t/a-julia-dataanalysis-sysimage-from-packagecompiler-its-so-easy-you-should-do-it-too/68127 https://discourse.julialang.org/t/a-julia-dataanalysis-sysim...
    - lambdatronics 4y ago
      I think Jevon's paradox is also hitting to some extent, coupled with the ecosystem getting bigger, so a typical project may have more transitive dependencies behind it. The functionality is going up, and the speed of compilation is going up, but the total latency is staying similar or increasing, in my experience.
      - adgjlsfhk1 4y ago
        that's definitely possible. I'm somewhat hopeful that some changes in 1.9 will let us cut back on transitive dependencies a lot, but it will take a few months to know for sure.
etik 4y ago
We (metalenz.com) use Julia extensively in our optical design/simulation/modeling workflows: 1. Design team keeps live sessions during interactive work, otherwise launches on virtualized servers (and things take sufficiently long to compute TTFX is a rounding error) 2. Above mentioned team is exclusively Julia. Julia shows up in other things we do (low-level computer vision), but not dominant nor exclusive 3. High performance & expressive programming and flexible autodiff system for scientific computation

philip1209 4y ago

My defunct startup Staffjoy operated from 2015 to 2017 and was an early production user of Julia. We open-sourced some of the code here: https://github.com/staffjoy https://github.com/staffjoy I haven't kept up with the ecosystem - I'm curious how things have improved in the last 5 year!

leephillips 4y ago
Probably the most obvious improvements for most users are the big growth in the package ecosystem, with some science and math packages becoming the best available in any language (DifferentialEquations.jl, for example) and the drastic improvement in interactivity: the REPL starts instantly now, and most packages compile much faster. This will get even better in the next major release.
- Q6T46nT668w6i3m 4y ago
  I’m curious (I’m planning on designing an ODE/PDE package relatively soon), what makes DifferentialEquations.jl the best available?
  - logankilpatrick 4y ago
    The fact that Chris R and the whole SciML ecosystem are using / building on it. Lots of resources going into it and usage in real world places like Pfizer and Moderna.
  - leephillips 4y ago
    This lists some of its unique abilities: https://docs.sciml.ai/DiffEqDocs/stable/ https://docs.sciml.ai/DiffEqDocs/stable/ The routines are sufficiently generic, with regard to Julia’s type system, to allow the solvers to automatically compose with other packages and to seamlessly use types other than Numbers. For example, instead of handling just functions Number→Number, you can define your ODE in terms of quantities with physical dimensions, uncertainties, quaternions, etc., and it will just work (for example, propagating uncertainties correctly to the solution¹). Recent developments involve research into the automated selection of solution routines based on the properties of the ODE, something that seems really next-level to me. [1] https://lwn.net/Articles/834571/ https://lwn.net/Articles/834571/

open-source-ux 4y ago

Julia scientific apps and libraries continue to grow and see momentum. Is there interest in Julia outside the scientifc sphere? I'd be interested to hear of non-scientific projects implemented in Julia (for example, business apps or web-related projects.) Related: I think Julia is an excellent starting language (compared to Python). Is anyone teaching Julia to new programmers with success?

logankilpatrick 4y ago
Look at GenieCloud, they are essentially building the Django ecosystem in Julia, it will be huge if they are successful.
cbkeller 4y ago
I'm teaching an intro programming course (discipline-specific, for Earth sciences) in Julia at Dartmouth. So far so good I think -- but definitely on the sciences side of things.

vHMtsdf 4y ago

I heard a talk from a large cybersecurity company that was using Julia for research and is apparently in the process of dropping Julia due to: - difficulty in hiring, apparently it was too expensive/difficult to hire Julia programmers, - poorly performing garbage collector. If I recall correctly, they had issues when trying to process huge amounts of data. Seems like small teams will not run into these issues at least initially. I really like Julia and I hope it becomes huge, but stories like this make me wonder whether the whole Julia ecosystem will ever mature to a level of Python for example. And in turn, making it viable for large organizations that might be using e.g. Python + C++ combo that might be easier to hire for/more versatile/with better known drawbacks. Essentially, I am not sure whether the potential of Julia is that much greater than the realized potential of the incumbents to warrant the "Julia is the language of the future".

logankilpatrick 4y ago
Please put them in touch with me if possible. RelationalAI has similar GC issues which they are the core Julia developers are working hard to resolve. I have also seen zero postings for Julia roles in cyber security companies so it is likely that they aren't tapping into the community which is why they can't find people.
- vHMtsdf 4y ago
  It was a talk by folks from this team: https://gradient.avast.io/people-behind/ https://gradient.avast.io/people-behind/
europeanguy 4y ago
If you care about performance your code shouldn't be allocating.
warinukraine 4y ago
My (more or less extensive) experience with Julia has been rather good. Step 1. Prototyping. Small data, don't care about performance. Julia is as easy as python, but faster. GC doesn't matter. Step 2. Production. Very large datasets. Performance is very important so I make sure that hot loops don't allocate, and so GC doesn't matter. Really, I struggle to see how someone can not like this. Regarding hiring. My experience has been that you can roughly divide people into two categories. Those who like using tools, and those who like solving problems. For the former, a different language might be an issue because they might not know and might not be in the mood for learning it. For the latter, they're happy to learn any tool that can help solve their problem. We want the latter, and for the latter learning Julia hasn't been an issue.

dartharva 4y ago

How good is Julia as a general-purpose language? Can it be a viable replacement for, say, Lua in game development?

DougBTX 4y ago
Julia is a very nice language, its core abstractions (mutli-dispatch and macros) are not tied to any particular use-case. My overriding feeling using it is that it is a language visiting from the future, it is very well put together. Having said that, I've not had the opportunity to use it in production, Python is very popular!
- ModernMech 4y ago
  > My overriding feeling using it is that it is a language visiting from the future Either that, or most languages used today are stuck in the past!
affinepplan 4y ago
It is a very well-designed language for many (although of course not all) applications. Most of the ecosystem revolves around scientific computing so you may find there is a bit of a learning curve / sparse existing libraries if you want to use it for game development. Also, on the language side, Julia is optimized for total throughput, which means even though it is easy to write extremely fast code, it sometimes suffers latency issues during stop-the-world garbage collection or JIT compilation. Julia gives the experienced developer tools to avoid this, but especially for those less practiced with the language it could potentially be an obstacle for real-time games like FPS--of course if you are developing a turn-based game this is less of a concern.
maria2 4y ago
Isn't Lua's main benefit that it has a small runtime that can be executed inside the context of a large C/C++ executable? Julia has support for C FFI, but it looks like you'd have to manually wrap all of your native code in Julia wrappers. One library I've really liked for this use case is QuickJS. It's a super small ES2020 implementation written in C. It has a great API that lets you bind native C functions and types into JavaScript functions, objects, and classes. This lets you bind native C code that gets used naturally in JavaScript without needing to write wrappers.

eigenspace 4y ago

I'm a huge julia fan, but man is this ever an insubstantial and unconvincing listicle.

sundarurfriend 4y ago
Yeah, it doesn't really seem intended for a HN type audience. More of a high level overview of the language to show to a non-technical co-founder or senior executive.
- markkitti 4y ago
  What is the HN type audience? HN is a news feed run by Ycombinator, a "startup accelerator". While many people around here have deep technical expertise, given the name "Hacker News", I always thought the main purpose was to make news and technical discussions more accessible to non-technical co-founders and startup executives.