3 ms·

With RCpp writing a function in C++ that you can then use from R is easy. So you do as much as you can in R, then if you reach a bottleneck, use a lil CPP, the

With RCpp writing a function in C++ that you can then use from R is easy. So you do as much as you can in R, then if you reach a bottleneck, use a lil CPP, then R. It's still nice.

krastanov 4y ago
Same with python. But then you have the following problem: How do you use tools that were designed to exploit the introspection of these languages when you have C++ submodules? This is called "the two language problem", and it is not just about ergonomics (e.g. you do not seem to have issues with the ergonomics of this approach). Rather the issue is that now I can not have a library that introspects my R code and performs automatic differentiation or some other advanced compile pass over my code, because not all of the code is written in R. This is also why most of the advanced SciPy functionality can not be used with Tensorflow, or use Jax's autodiff feature with Pandas dataframes or Pillow images. In many cases this might not matter, but when creating sophisticated large computational packages, the level of interoperability that Julia provides is both incredibly empowering and as of yet unsurpassed.
- kgwgk 4y ago
  > when creating sophisticated large computational packages, the level of interoperability that Julia provides is both incredibly empowering and as of yet unsurpassed It's difficult for outsiders to know what to think when confronted with so divergent opinions: "I would hit bugs of this severity often enough to make me question the correctness of any moderately complex computation in Julia. This was particularly true when trying a novel combination of packages or functions — composing together functionality from multiple sources was a significant source of bugs."
  - krastanov 4y ago
    That is a very reasonable complaint, and I think the core devs (of the language and of the flagship ecosystem projects) take it fairly seriously. For me personally, it has been incredibly empowering to be able to mix and match very diverse packages, but maybe I am just used to defensive coding: I start with tests, including randomized tests and contract tests. On the other hand, though, I have not encountered more bugs in the Julia ecosystem than in the Sympy or Theano (deprecated now) or Tensorflow or Pytorch systems when I was using them to build "sophisticated large computational packages". Maybe the happy users are just not as vocal, but that is a silly excuse.
  - sundarurfriend 4y ago
    Both opinions go hand in hand, in fact. Julia allows packages to interoperate in a way most other mainstream languages simply don't, due to the combination of its multiple dispatch and the type hierarchy. The fact that the language allows this interoperability to such level is very useful and empowering. But making proper use of this interoperablity means writing careful generic code that doesn't make narrow assumptions based only on currently known uses of it. Different parts of the ecosystem do this with different degress of care and success, with newer ones being better at it as lessons are learned and implemented. Yuri happened to work on parts of the ecosystem that were not-so-new, and made combinations that hadn't been commonly attempted. That doesn't mean that these weren't issues that need fixing and preventing in the future, for sure. But hitting bugs of this severity often isn't the typical experience for the average user - if it was, Julia would hardly have as many users, much less ones like NASA.

With RCpp writing a function in C++ that you can then use from R is easy. So you do as much as you can in R, then if you reach a bottleneck, use a lil CPP, then R. It's still nice.

krastanov 4y ago

Same with python. But then you have the following problem: How do you use tools that were designed to exploit the introspection of these languages when you have C++ submodules? This is called "the two language problem", and it is not just about ergonomics (e.g. you do not seem to have issues with the ergonomics of this approach). Rather the issue is that now I can not have a library that introspects my R code and performs automatic differentiation or some other advanced compile pass over my code, because not all of the code is written in R. This is also why most of the advanced SciPy functionality can not be used with Tensorflow, or use Jax's autodiff feature with Pandas dataframes or Pillow images. In many cases this might not matter, but when creating sophisticated large computational packages, the level of interoperability that Julia provides is both incredibly empowering and as of yet unsurpassed.

kgwgk 4y ago
> when creating sophisticated large computational packages, the level of interoperability that Julia provides is both incredibly empowering and as of yet unsurpassed It's difficult for outsiders to know what to think when confronted with so divergent opinions: "I would hit bugs of this severity often enough to make me question the correctness of any moderately complex computation in Julia. This was particularly true when trying a novel combination of packages or functions — composing together functionality from multiple sources was a significant source of bugs."
- krastanov 4y ago
  That is a very reasonable complaint, and I think the core devs (of the language and of the flagship ecosystem projects) take it fairly seriously. For me personally, it has been incredibly empowering to be able to mix and match very diverse packages, but maybe I am just used to defensive coding: I start with tests, including randomized tests and contract tests. On the other hand, though, I have not encountered more bugs in the Julia ecosystem than in the Sympy or Theano (deprecated now) or Tensorflow or Pytorch systems when I was using them to build "sophisticated large computational packages". Maybe the happy users are just not as vocal, but that is a silly excuse.
- sundarurfriend 4y ago
  Both opinions go hand in hand, in fact. Julia allows packages to interoperate in a way most other mainstream languages simply don't, due to the combination of its multiple dispatch and the type hierarchy. The fact that the language allows this interoperability to such level is very useful and empowering. But making proper use of this interoperablity means writing careful generic code that doesn't make narrow assumptions based only on currently known uses of it. Different parts of the ecosystem do this with different degress of care and success, with newer ones being better at it as lessons are learned and implemented. Yuri happened to work on parts of the ecosystem that were not-so-new, and made combinations that hadn't been commonly attempted. That doesn't mean that these weren't issues that need fixing and preventing in the future, for sure. But hitting bugs of this severity often isn't the typical experience for the average user - if it was, Julia would hardly have as many users, much less ones like NASA.