4 ms·

I think Python is great for exploration and method development, while I would prefer something more efficient like Julia for production systems. On the other ha

I think Python is great for exploration and method development, while I would prefer something more efficient like Julia for production systems. On the other hand, Julia requires compilation while Python allows open-heart surgery on the production system (for the masochists among us). I personally don't like Python that much because every library does things differently and sometimes it feels like learning a completely new (sub) language. E.g. NumPy DataFrames allow to do the same thing in multiple ways (e.g. adding an index column, or removing a column). Often when I need to look up how to do a particular thing I end up finding many solutions that simply don't function with the version I am working with. Sometimes looking even into old code of mine doesn't work any more and requires either me using an older library of relearning how to do things. That being said, a friend of mine has been quite fond of Julia lately. Which put Julia on the top of my list of programming languages to do a deep dive.

longemen3000 5y ago
Just to be clear Julia is "AOT-JIT", that means that the methods are compiled when used the first time. With Revise.jl (a package for interactive development) you can make a fully interactive Julia process, rewriting things on the fly, while benefiting from fast code
- freemint 5y ago
  Running Revise in production sounds unwise (sorry for the pun). Also you can very much not redefine functions on the fly due to world age (assuming the caller doesn't pay for invoke-latest).
  - ChrisRackauckas 5y ago
    You're not doing interactive development on production. Production is the case where you just build a system image and deploy it without compile times. This is done all of the time, for example with Pumas.
    - freemint 5y ago
      In certain common lisp deployment you might do that. There are legends from Common Lisp users where this is very much done. > When one of the customer support people came to me with a report of a bug in the editor, I would load the code into the Lisp interpreter and log into the user's account. If I was able to reproduce the bug I'd get an actual break loop, telling me exactly what was going wrong. Often I could fix the code and release a fix right away. And when I say right away, I mean while the user was still on the phone. > Such fast turnaround on bug fixes put us into an impossibly tempting position. If we could catch and fix a bug while the user was still on the phone, it was very tempting for us to give the user the impression that they were imagining it. And so we sometimes (to their delight) had the customer support people tell the user to just try logging in again and see if they still had the problem. And of course when the user logged back in they'd get the newly released version of the software with the bug fixed, and everything would work fine. I realize this was a bit sneaky of us, but it was also a lot of fun. Taken from https://sep.yimg.com/ty/cdn/paulgraham/bbnexcerpts.txt?t=1637873414& https://sep.yimg.com/ty/cdn/paulgraham/bbnexcerpts.txt?t=163... I know that this is not possible in Julia like that. But just because it's not possible in Julia doesn't mean what they did was somehow wrong when it worked well for them.
      - ChrisRackauckas 5y ago
        Call me a cynic but I would hope almost no devs had the power to live update the production codebase. That sounds crazy to me. To me, that kind of thing should only be done with (approved/verified) versions which are cut and tested before deployment, and deployment should act like an unchangeable immutable compiled binary until the next binary is ready. Not only is this legally required in many scientific cases, but also production should have a lot more care. This kind of thing sounds fine for a dev server though, in which case Revise+RuntimeGeneratedFunctions will get it done in the cases I've seen.
        pm90 5y ago
        I understand where you’re coming from, it’s the production robustness paradigm of someone who has worked with such systems and knows what kind of controls are required to release code. eg live editing like that would almost certainly go against SOC2 controls. However, I’m also tempted very much by a system that allows for fixing things so quickly, because it provides for a delightful customer experience. Most customers don’t report most issues, only when something is persistently wrong or is something they can’t get around. So being able to quickly fix it this way seems genuinely amazing. If I were to take this idea further… maybe some sort of customer isolated canary deploy/ feature flag might be the way to express this customer experience in todays world. The problem is always that you might break other things with your fix. Maybe a battery of testing before canary deploys can raise confidence about the validity of the fix.

longemen3000 5y ago

Just to be clear Julia is "AOT-JIT", that means that the methods are compiled when used the first time. With Revise.jl (a package for interactive development) you can make a fully interactive Julia process, rewriting things on the fly, while benefiting from fast code

freemint 5y ago
Running Revise in production sounds unwise (sorry for the pun). Also you can very much not redefine functions on the fly due to world age (assuming the caller doesn't pay for invoke-latest).
- ChrisRackauckas 5y ago
  You're not doing interactive development on production. Production is the case where you just build a system image and deploy it without compile times. This is done all of the time, for example with Pumas.
  - freemint 5y ago
    In certain common lisp deployment you might do that. There are legends from Common Lisp users where this is very much done. > When one of the customer support people came to me with a report of a bug in the editor, I would load the code into the Lisp interpreter and log into the user's account. If I was able to reproduce the bug I'd get an actual break loop, telling me exactly what was going wrong. Often I could fix the code and release a fix right away. And when I say right away, I mean while the user was still on the phone. > Such fast turnaround on bug fixes put us into an impossibly tempting position. If we could catch and fix a bug while the user was still on the phone, it was very tempting for us to give the user the impression that they were imagining it. And so we sometimes (to their delight) had the customer support people tell the user to just try logging in again and see if they still had the problem. And of course when the user logged back in they'd get the newly released version of the software with the bug fixed, and everything would work fine. I realize this was a bit sneaky of us, but it was also a lot of fun. Taken from https://sep.yimg.com/ty/cdn/paulgraham/bbnexcerpts.txt?t=1637873414& https://sep.yimg.com/ty/cdn/paulgraham/bbnexcerpts.txt?t=163... I know that this is not possible in Julia like that. But just because it's not possible in Julia doesn't mean what they did was somehow wrong when it worked well for them.
    - ChrisRackauckas 5y ago
      Call me a cynic but I would hope almost no devs had the power to live update the production codebase. That sounds crazy to me. To me, that kind of thing should only be done with (approved/verified) versions which are cut and tested before deployment, and deployment should act like an unchangeable immutable compiled binary until the next binary is ready. Not only is this legally required in many scientific cases, but also production should have a lot more care. This kind of thing sounds fine for a dev server though, in which case Revise+RuntimeGeneratedFunctions will get it done in the cases I've seen.
      - pm90 5y ago
        I understand where you’re coming from, it’s the production robustness paradigm of someone who has worked with such systems and knows what kind of controls are required to release code. eg live editing like that would almost certainly go against SOC2 controls. However, I’m also tempted very much by a system that allows for fixing things so quickly, because it provides for a delightful customer experience. Most customers don’t report most issues, only when something is persistently wrong or is something they can’t get around. So being able to quickly fix it this way seems genuinely amazing. If I were to take this idea further… maybe some sort of customer isolated canary deploy/ feature flag might be the way to express this customer experience in todays world. The problem is always that you might break other things with your fix. Maybe a battery of testing before canary deploys can raise confidence about the validity of the fix.