8 ms·

Weave.jl – Scientific Reports Using Julia

127 points by phillc73 6y ago

mark_l_watson 6y ago
Weave looks great! Integrating dev and publishing.
likelybear 6y ago
I'm a big fan of Weave/knitr/Rmd style notebooks over Jupyter for reproducibility. I've had a great experience with parameterized Weave.jl reports. My one complaint is that a streamlined dev workflow depends on good caching. knitr nailed that part, checking code hashes and checking dependencies between chunks. Weave.jl caching has been... finicky. For a completely different style of notebook, folks should check out Pluto.jl https://github.com/fonsp/Pluto.jl https://github.com/fonsp/Pluto.jl
- tfehring 6y ago
  Genuinely curious, what makes Weave and Pluto "completely different" styles of notebook? Is there some reason that Pluto features like interactivity and reactivity couldn't be incorporated in Weave, at least in theory? I get that they focus on different problems, but based on how R Markdown is often used in practice, it seems like there's a lot of overlap between the two problem spaces.
  - MisterBiggs 6y ago
    Plutos killer feature to me is that it is not mutable. It can be challenging to adjust initially, mainly because it forces you to write extremely DRY code. The benefit is it makes your code much more stable/reproducible than jupyter and interactivity stronger because anytime something changes in any cell, all the dependent cells, regardless of the order in the doc, are updated.
    - tfehring 6y ago
      I completely agree! But I'm wondering whether those benefits are incompatible with the benefits of Weave.jl. E.g., can an arbitrary Pluto notebook be woven to a PDF? If so, why are they separate packages - and if not, is it because of a fundamental difference, or just a difference in conventions or currently supported features?
- sp33der89 6y ago
  Why would one chose Pluto over say Weave or the other way around?
junippor 6y ago
I use Julia and used to use Python. I've used jupyter lab for both (although for Julia I prefer Atom for IDE) What is the advantage of Weave.jl over jupyter lab? Is this just a way of integrating something "like jupyter" into Atom?
- phillc73 6y ago
  It's a way to write scientific reports, or other documents integrating Julia code chunks, from inside your current IDE (Juno, VSCodium etc). There's no need for an external server, like Jupyter. The code execution relies on the IDE's REPL and the locally installed instance of Julia. The reports can be exported to various formats - HTML, PDF etc. In the R ecosystem it's similar to using RMarkdown.
- fn-mote 6y ago
  Generic advantages: - no binaries in your source code make git diff more useful - continue to use your favorite editor especially if you don't like the notebook interface and keybindings - batch mode can be good or bad, people might not like it as much for exploratory analysis but prefer it when they have a lot of writing to do and want to polish "later"
  - junippor 6y ago
    > no binaries in your source code make git diff more useful Oh damn that's a good one. Thanks for pointing it out.
aseerdbnarng 6y ago
Genuinely nice. If you do need to mix and match languages in a single document knitr/Rmarkdown has support for the most popular languages including Julia https://bookdown.org/yihui/rmarkdown/language-engines.html https://bookdown.org/yihui/rmarkdown/language-engines.html and that’s working nicely for me.
- snicker7 6y ago
  Org-mode can also do the trick.
- tvb12 6y ago
  Can weave do other languages? It looks like all of the blocks begin with the language name, but I only see julia in the screenshots.
mhh__ 6y ago
On the subjects of reports, it would be fun to have a system that parses the LaTeX and forcibly stops you deleting a plot from what generated it. I sometimes want to scream when watching the practices we get taught for doing "programming" in academia, there's just no standard of trying to do thing right.
TobySKT 6y ago
Great article!
cbkeller 6y ago
I use this to format all the assignment handouts for my intro-to-programming-in-Earth-sciences class. Has worked great so far.
arnavs 6y ago
I use this stack. One extension I wrote is https://github.com/quantecon/instantiatefromurl.jl https://github.com/quantecon/instantiatefromurl.jl, to bind generated notebooks to Julia TOML (i.e. Julia's `requirements.txt`) that lives in a git repo. This means they don't depend on local machine state, so can move and run freely.
3JPLW 6y ago
Lots of the sciml.ai packages and documentation use Weave — you can look there for some examples. Here's one: https://diffeq.sciml.ai/stable/tutorials/sde_example/ https://diffeq.sciml.ai/stable/tutorials/sde_example/

127 points by phillc73 6y ago

Weave looks great! Integrating dev and publishing.

I'm a big fan of Weave/knitr/Rmd style notebooks over Jupyter for reproducibility. I've had a great experience with parameterized Weave.jl reports. My one complaint is that a streamlined dev workflow depends on good caching. knitr nailed that part, checking code hashes and checking dependencies between chunks. Weave.jl caching has been... finicky. For a completely different style of notebook, folks should check out Pluto.jl https://github.com/fonsp/Pluto.jl https://github.com/fonsp/Pluto.jl

tfehring 6y ago
Genuinely curious, what makes Weave and Pluto "completely different" styles of notebook? Is there some reason that Pluto features like interactivity and reactivity couldn't be incorporated in Weave, at least in theory? I get that they focus on different problems, but based on how R Markdown is often used in practice, it seems like there's a lot of overlap between the two problem spaces.
- MisterBiggs 6y ago
  Plutos killer feature to me is that it is not mutable. It can be challenging to adjust initially, mainly because it forces you to write extremely DRY code. The benefit is it makes your code much more stable/reproducible than jupyter and interactivity stronger because anytime something changes in any cell, all the dependent cells, regardless of the order in the doc, are updated.
  - tfehring 6y ago
    I completely agree! But I'm wondering whether those benefits are incompatible with the benefits of Weave.jl. E.g., can an arbitrary Pluto notebook be woven to a PDF? If so, why are they separate packages - and if not, is it because of a fundamental difference, or just a difference in conventions or currently supported features?
sp33der89 6y ago
Why would one chose Pluto over say Weave or the other way around?

junippor 6y ago

I use Julia and used to use Python. I've used jupyter lab for both (although for Julia I prefer Atom for IDE) What is the advantage of Weave.jl over jupyter lab? Is this just a way of integrating something "like jupyter" into Atom?

phillc73 6y ago
It's a way to write scientific reports, or other documents integrating Julia code chunks, from inside your current IDE (Juno, VSCodium etc). There's no need for an external server, like Jupyter. The code execution relies on the IDE's REPL and the locally installed instance of Julia. The reports can be exported to various formats - HTML, PDF etc. In the R ecosystem it's similar to using RMarkdown.
fn-mote 6y ago
Generic advantages: - no binaries in your source code make git diff more useful - continue to use your favorite editor especially if you don't like the notebook interface and keybindings - batch mode can be good or bad, people might not like it as much for exploratory analysis but prefer it when they have a lot of writing to do and want to polish "later"
- junippor 6y ago
  > no binaries in your source code make git diff more useful Oh damn that's a good one. Thanks for pointing it out.

aseerdbnarng 6y ago

Genuinely nice. If you do need to mix and match languages in a single document knitr/Rmarkdown has support for the most popular languages including Julia https://bookdown.org/yihui/rmarkdown/language-engines.html https://bookdown.org/yihui/rmarkdown/language-engines.html and that’s working nicely for me.

snicker7 6y ago
Org-mode can also do the trick.
tvb12 6y ago
Can weave do other languages? It looks like all of the blocks begin with the language name, but I only see julia in the screenshots.

mhh__ 6y ago

On the subjects of reports, it would be fun to have a system that parses the LaTeX and forcibly stops you deleting a plot from what generated it. I sometimes want to scream when watching the practices we get taught for doing "programming" in academia, there's just no standard of trying to do thing right.

TobySKT 6y ago

Great article!

cbkeller 6y ago

I use this to format all the assignment handouts for my intro-to-programming-in-Earth-sciences class. Has worked great so far.

arnavs 6y ago

I use this stack. One extension I wrote is https://github.com/quantecon/instantiatefromurl.jl https://github.com/quantecon/instantiatefromurl.jl, to bind generated notebooks to Julia TOML (i.e. Julia's `requirements.txt`) that lives in a git repo. This means they don't depend on local machine state, so can move and run freely.

3JPLW 6y ago

Lots of the sciml.ai packages and documentation use Weave — you can look there for some examples. Here's one: https://diffeq.sciml.ai/stable/tutorials/sde_example/ https://diffeq.sciml.ai/stable/tutorials/sde_example/