7 ms·

What happens for that scientist when they have to dive into Julia’a stack to debug something weird? In Python and C, you have established debuggers, semantics

What happens for that scientist when they have to dive into Julia’a stack to debug something weird? In Python and C, you have established debuggers, semantics etc, which means that, yes, there are two languages instead of one, but neither is a moving target compared to a language which just had a 1.0 release. I get the issue with scientists writing poor code, but Numba has largely solved this problem, by packing an LLVM JIT into a decorator which can be applied to any numerical code to get same speed ups as Julia, except no language switch required. Citing slow code in the wild with a fast rewrite is a hilariously poor anecdote performance wise. I’ve rewritten Fortran code into Python and gotten speed ups. Regardless of the language, garbage in, garbage out. Stan is an example where the modeling is “just” a DSL implemented as C++ templates. Does that make that a good choice?

srean 8y ago
> What happens for that scientist when they have to dive into Julia’a stack to debug something weird? The same things that happened when we had this conversation about what happens to the Fortran writing scientist when Python and Numpy came along, even at that time it wasnt the first time. I am sure it would not have been a whole lot different when a COBOL alternative had come along.
- marmaduke 8y ago
  Not quite: the argument for Julia is that a casual user won’t drop down into C from Python for performance while in Julia it wouldn’t be necessary, thus easier.
celrod 8y ago
It'd probably be a lot easier to debug something somewhere in the Julia stack, than in the C/C++/Fortran code that many R libraries run through. My point with the rewrite was not garbage in, garbage out. It was that even though the original R code was using a library written in C, that library had to call a function he wrote in R millions of times. That R function being inherently slow is part of the problem. (The easiest fix for that is just writing the function you pass to that library in RCpp, but the overhead on that is still close to a microsecond -- not sure how it is in R. Numba is probably easier.) It is nicer to not have to worry about that. An alternative approach some libraries provide, like my Stan example in RStan or PyStan, is the DSL they implement to make it easier for end users in R or Python to write fast C++ code. But, now lets say you're working on an optimization problem. You want to use a gradient-based optimization method, while your code is heavily dependent on the FooBars and Widgets libraries. If these libraries are written in Julia, you can write code in Julia using these libraries, and automatic differentiation will just work as you pass it to Optim.jl. If FooBars and Widgets were Cython libraries, optionally wrapping C/C++ code, would this work? Could you write functions making use of these libraries, and get efficient gradients for optimization for use with an optimization library and have everything be fast? 'Stan is an example where the modeling is “just” a DSL implemented as C++ templates. Does that make that a good choice?' I gave Stan as an example of a less than ideal situation, because people normally use Stan from R or Python, not from within C++. Therefore there R and Python don't integrate well. If you're already working in C++, Stan seems ideal. You can use arbitrary templated C++ code with Stan, include external libraries, etc. It needs to be C++ because of their autodiff.
- marmaduke 8y ago
  Optimization (or any gradient based algorithm) is a good use case for AD, but I don’t see why Julia’s approaches are any better than Python’s, eg autograd, theano, pytorch etc. And sure that wouldn’t work with arbitrary Cython modules because Cython was designed as a Pythonic syntax over the Python C-API, and it just happened to become popular for numerical work. I don’t think that’s a strong argument, though, because anything small enough to be usable with AD can be rewritten without too much time lost, whereas those massive Fortran routines with iterative algorithms wouldn’t produce useful gradients in any language.
  - ChrisRackauckas 8y ago
    >And sure that wouldn’t work with arbitrary Cython modules because Cython was designed as a Pythonic syntax over the Python C-API, and it just happened to become popular for numerical work. >I don’t think that’s a strong argument, though, because anything small enough to be usable with AD can be rewritten without too much time lost, whereas those massive Fortran routines with iterative algorithms wouldn’t produce useful gradients in any language. No. In Julia you can just stick the entire delay differential equation solver into the AD functions and get a gradient for parameter estimation. Saying you cannot use an arbitrary Cython code is a limitation, and saying you cannot put a random large code into AD is a limitation. It wouldn't be an issue if these weren't already solved problems, but having a performant software with simple and available AD is not something that's unreasonable anymore. If you use Julia, it's just something you can expect to work.
    - marmaduke 8y ago
      No you can’t, at least, not if you want it work. This has nothing to do with Julia though and I didn’t say you can’t run AD over the solver but that that generally will not produce good gradient estimates unless the solver is written with AD in mind. DDEs are a mixed case where I’d expect some parts to be workable but not in general. Another example is the FFT, totally worthless to use AD.
  - Certhas 8y ago
ChrisFoster 8y ago
Sometimes low level debugging is a surprisingly pleasant experience as the julia JIT generates proper DWARF debug info. So for instance, you can break in gdb and see the julia source code for any julia generated stack frames, neatly intertwined with the frames of the C runtime. To be clear, I don't remember needing to do this as a regular user. As an occasional compiler hacker it's been quite nice though.
- marmaduke 8y ago
  That’s cool, I wasn’t aware. I think that’s most useful when working with external libs.
- ScottPJones 8y ago
  As someone who's spent decades programming in C/C++, and diving into assembly code (and writing a fair share when the compiler just couldn't do what I needed), I love being able to directly inspect the output code at many levels, including all the way down to "bare metal". Yes, there's a lot of work to be done in the area of debuggers for Julia, but there are already useful debugging tools (like Rebugger) that I haven't seen for any other language.
Certhas 8y ago
Anecdote time: I hit a non-deterministic bug in one of the C based Python packages we were using. Most of the time it worked, but we were running MonteCarlo on it and saw many errors. I guessed correctly that it was using uninitialized memory, and errored when that wasn't zeroed out, but my C wasn't good enough to find where. Had this been Julia the whole C code would have been Julia code and I would have had a chance to dive in and debug. I ended up having to get a colleague who's fluent in C to help.
- marmaduke 8y ago
  Let me guess, this can’t happen in Julia because memory is always initialized? Sounds like a performance hit if you know what you’re doing, so maybe you can use uninitialized memory in Julia and run into the same bug. Perhaps Julia makes it easy to use LLVM sanitizers? But you could’ve done this with your C code as well.
  - Certhas 8y ago
    The bug could totally happen in Julia. The point was your question: "What happens for that scientist when they have to dive into Julia’a stack to debug something weird?" And then claimed that this was somehow better in Python + C, which is not my experience. I expect this to be easier in Julia than in Python and C. I totally agree that the tooling is not where it needs to be, btw, but now that the target has stopped moving I expect it to get there soon.

srean 8y ago

> What happens for that scientist when they have to dive into Julia’a stack to debug something weird? The same things that happened when we had this conversation about what happens to the Fortran writing scientist when Python and Numpy came along, even at that time it wasnt the first time. I am sure it would not have been a whole lot different when a COBOL alternative had come along.

marmaduke 8y ago
Not quite: the argument for Julia is that a casual user won’t drop down into C from Python for performance while in Julia it wouldn’t be necessary, thus easier.

celrod 8y ago

It'd probably be a lot easier to debug something somewhere in the Julia stack, than in the C/C++/Fortran code that many R libraries run through. My point with the rewrite was not garbage in, garbage out. It was that even though the original R code was using a library written in C, that library had to call a function he wrote in R millions of times. That R function being inherently slow is part of the problem. (The easiest fix for that is just writing the function you pass to that library in RCpp, but the overhead on that is still close to a microsecond -- not sure how it is in R. Numba is probably easier.) It is nicer to not have to worry about that. An alternative approach some libraries provide, like my Stan example in RStan or PyStan, is the DSL they implement to make it easier for end users in R or Python to write fast C++ code. But, now lets say you're working on an optimization problem. You want to use a gradient-based optimization method, while your code is heavily dependent on the FooBars and Widgets libraries. If these libraries are written in Julia, you can write code in Julia using these libraries, and automatic differentiation will just work as you pass it to Optim.jl. If FooBars and Widgets were Cython libraries, optionally wrapping C/C++ code, would this work? Could you write functions making use of these libraries, and get efficient gradients for optimization for use with an optimization library and have everything be fast? 'Stan is an example where the modeling is “just” a DSL implemented as C++ templates. Does that make that a good choice?' I gave Stan as an example of a less than ideal situation, because people normally use Stan from R or Python, not from within C++. Therefore there R and Python don't integrate well. If you're already working in C++, Stan seems ideal. You can use arbitrary templated C++ code with Stan, include external libraries, etc. It needs to be C++ because of their autodiff.

marmaduke 8y ago
Optimization (or any gradient based algorithm) is a good use case for AD, but I don’t see why Julia’s approaches are any better than Python’s, eg autograd, theano, pytorch etc. And sure that wouldn’t work with arbitrary Cython modules because Cython was designed as a Pythonic syntax over the Python C-API, and it just happened to become popular for numerical work. I don’t think that’s a strong argument, though, because anything small enough to be usable with AD can be rewritten without too much time lost, whereas those massive Fortran routines with iterative algorithms wouldn’t produce useful gradients in any language.
- ChrisRackauckas 8y ago
  >And sure that wouldn’t work with arbitrary Cython modules because Cython was designed as a Pythonic syntax over the Python C-API, and it just happened to become popular for numerical work. >I don’t think that’s a strong argument, though, because anything small enough to be usable with AD can be rewritten without too much time lost, whereas those massive Fortran routines with iterative algorithms wouldn’t produce useful gradients in any language. No. In Julia you can just stick the entire delay differential equation solver into the AD functions and get a gradient for parameter estimation. Saying you cannot use an arbitrary Cython code is a limitation, and saying you cannot put a random large code into AD is a limitation. It wouldn't be an issue if these weren't already solved problems, but having a performant software with simple and available AD is not something that's unreasonable anymore. If you use Julia, it's just something you can expect to work.
  - marmaduke 8y ago
    No you can’t, at least, not if you want it work. This has nothing to do with Julia though and I didn’t say you can’t run AD over the solver but that that generally will not produce good gradient estimates unless the solver is written with AD in mind. DDEs are a mixed case where I’d expect some parts to be workable but not in general. Another example is the FFT, totally worthless to use AD.
- Certhas 8y ago

ChrisFoster 8y ago

Sometimes low level debugging is a surprisingly pleasant experience as the julia JIT generates proper DWARF debug info. So for instance, you can break in gdb and see the julia source code for any julia generated stack frames, neatly intertwined with the frames of the C runtime. To be clear, I don't remember needing to do this as a regular user. As an occasional compiler hacker it's been quite nice though.

marmaduke 8y ago
That’s cool, I wasn’t aware. I think that’s most useful when working with external libs.
ScottPJones 8y ago
As someone who's spent decades programming in C/C++, and diving into assembly code (and writing a fair share when the compiler just couldn't do what I needed), I love being able to directly inspect the output code at many levels, including all the way down to "bare metal". Yes, there's a lot of work to be done in the area of debuggers for Julia, but there are already useful debugging tools (like Rebugger) that I haven't seen for any other language.

Certhas 8y ago

Anecdote time: I hit a non-deterministic bug in one of the C based Python packages we were using. Most of the time it worked, but we were running MonteCarlo on it and saw many errors. I guessed correctly that it was using uninitialized memory, and errored when that wasn't zeroed out, but my C wasn't good enough to find where. Had this been Julia the whole C code would have been Julia code and I would have had a chance to dive in and debug. I ended up having to get a colleague who's fluent in C to help.

marmaduke 8y ago
Let me guess, this can’t happen in Julia because memory is always initialized? Sounds like a performance hit if you know what you’re doing, so maybe you can use uninitialized memory in Julia and run into the same bug. Perhaps Julia makes it easy to use LLVM sanitizers? But you could’ve done this with your C code as well.
- Certhas 8y ago
  The bug could totally happen in Julia. The point was your question: "What happens for that scientist when they have to dive into Julia’a stack to debug something weird?" And then claimed that this was somehow better in Python + C, which is not my experience. I expect this to be easier in Julia than in Python and C. I totally agree that the tooling is not where it needs to be, btw, but now that the target has stopped moving I expect it to get there soon.