9 ms·

I would say, just start! It's indispensable for exploratory data analysis for sure.

by kghose 13y ago

gknoy 13y ago
Good point! I will look more closely at the Notebook site's library of samples. So far it seems like the main reason is to have documented analysis or good plots / calculations for data. Usually I am looking less at analyzing reams of data, though, and more at the relations between objects. ("What kind of object is User.things? It's a query set, but do we ever have more than one? Do we have any users with no Thing associated?") While notebooks can surely express that, I rarely feel like I need to re-create the asking of the question. Perhaps it is that I have not yet graduated far enough in my explorations of data that a Notebook is the best tool. I'm going to look more closely at them this week, though. Thank you for the encouragement to do so.
- kghose 13y ago
  Well, for what you are looking for I would use an IPython session, or perhaps an embedded IPython session. Look at http://ipython.org/ipython-doc/dev/interactive/reference.html#embedding-ipython http://ipython.org/ipython-doc/dev/interactive/reference.htm...
- oddthink 13y ago
  I use the ipython notebook sometimes, but I almost always use something similar. (In my case, I often use emacs org-mode files to make basic literate program-style documents.) I keep these things in a dated "Notebook" file, like "Notebook/2013-07", just to hold all my one-offs. Why do I keep them? Well, often it takes a while for me to set up the particular environment I need. If there's a problem with the calculation for X, I need to load the right data, set up the context for X, run through the individual bits until I see what the issue is. I save the ipython notebook where I go through all that, and then if I need to go back to the calculation, for example to demonstrate to someone else my example that something has gone wrong, then I can just load up the notebook. If something is supposed to have been fixed, I can load up the notebook and repeat the calculation and see if it works now. Unlike emacs org-mode, I can easily save the graphs as part of this. ("Look, if I plot the distribution, there's this big extra lump over there. Where's that coming from?") If I go back to a file, it typically takes me a bit of thought to remember exactly what I was doing there. Once I figure it out again, I add sentence or two of markdown. It's a useful thing to have around.
  - mjn 13y ago
    > Unlike emacs org-mode, I can easily save the graphs as part of this. I might be misunderstanding what you're wanting here, but are you looking for something like (R example): #+begin_src R :exports both :results graphics :file graph.png ... #+end_src

I would say, just start! It's indispensable for exploratory data analysis for sure.

by kghose 13y ago

I would say, just start! It's indispensable for exploratory data analysis for sure.

gknoy 13y ago

Good point! I will look more closely at the Notebook site's library of samples. So far it seems like the main reason is to have documented analysis or good plots / calculations for data. Usually I am looking less at analyzing reams of data, though, and more at the relations between objects. ("What kind of object is User.things? It's a query set, but do we ever have more than one? Do we have any users with no Thing associated?") While notebooks can surely express that, I rarely feel like I need to re-create the asking of the question. Perhaps it is that I have not yet graduated far enough in my explorations of data that a Notebook is the best tool. I'm going to look more closely at them this week, though. Thank you for the encouragement to do so.

kghose 13y ago
Well, for what you are looking for I would use an IPython session, or perhaps an embedded IPython session. Look at http://ipython.org/ipython-doc/dev/interactive/reference.html#embedding-ipython http://ipython.org/ipython-doc/dev/interactive/reference.htm...
oddthink 13y ago
I use the ipython notebook sometimes, but I almost always use something similar. (In my case, I often use emacs org-mode files to make basic literate program-style documents.) I keep these things in a dated "Notebook" file, like "Notebook/2013-07", just to hold all my one-offs. Why do I keep them? Well, often it takes a while for me to set up the particular environment I need. If there's a problem with the calculation for X, I need to load the right data, set up the context for X, run through the individual bits until I see what the issue is. I save the ipython notebook where I go through all that, and then if I need to go back to the calculation, for example to demonstrate to someone else my example that something has gone wrong, then I can just load up the notebook. If something is supposed to have been fixed, I can load up the notebook and repeat the calculation and see if it works now. Unlike emacs org-mode, I can easily save the graphs as part of this. ("Look, if I plot the distribution, there's this big extra lump over there. Where's that coming from?") If I go back to a file, it typically takes me a bit of thought to remember exactly what I was doing there. Once I figure it out again, I add sentence or two of markdown. It's a useful thing to have around.
- mjn 13y ago
  > Unlike emacs org-mode, I can easily save the graphs as part of this. I might be misunderstanding what you're wanting here, but are you looking for something like (R example): #+begin_src R :exports both :results graphics :file graph.png ... #+end_src