4 ms·

For exploratory data visualization I can recommend Matplotlib: http://matplotlib.org/ http://matplotlib.org/ It's flexible, easy to use and can -with some twe

For exploratory data visualization I can recommend Matplotlib: http://matplotlib.org/ http://matplotlib.org/ It's flexible, easy to use and can -with some tweaking- produce publication-quality output. It supports a wide range of backends such as PDF, SVG, PNG and a range of GUIs like Qt, so it is easy to use both for static and interactive graphs. More importantly, it plays nicely with the IPython notebook (http://ipython.org/notebook.html http://ipython.org/notebook.html) and with libraries such as numpy, pandas and scikit-learn. The last point is very important because when visualizing data half of the problem is getting the data in shape, and Python offers some excellent tools for this (much better than what you would get in Javascript). The choice of the right tool depends of course on where you want to show your plots and what kind of plots you want to generate. If you intend showing your visualizations in a web browser then D3 or a similar JS library would probably be a good choice (also have a look at Bokeh, which is becoming a very good alternative to Matplotlib). For iPhone applications there are probably native toolkits to do this, but unfortunately I have no experience there and cannot make any recommendations. Concerning the scalability of your visualizations: To my knowledge there are not many tools that can handle more than a few 10.000 data points without becoming slow. So you will probably always have to reduce the number of data points before plotting them. In any case, showing all of your data points without prior aggregation will probably not be a good idea in most cases, performance aside. If you really need to work with a massive amounts of data points have a look at Processing, which to my understanding can handle pretty large data sets (but is not a data visualization toolkit per se). OpenGL-based approaches can also be interesting for visualizing large amounts of data points, especially but not only in 3D. Concerning your examples: - Gephi is a nice tool for visualizing graphs (http://gephi.github.io/ http://gephi.github.io/), but a bit clunky to use at time. - R has the excellent GGPlot module that can work with pretty large data sets and has specialized graphs for e.g. time series data.

pvnick 11y ago
Agreed, matplotlib is great. We use it all the time to make publication-quality graphs (output to pdf), specifically with time series.
lasryaric 11y ago
It was not clear in my question but I was mostly talking about how do people visualise there data while they are working on it? Lets say you are a developer working on the well known Shazam application. You want to visualize your signals and frequencies for debugging purpose, how do you quickly visualize them? In my case, while working on signal processing using C and objective C, I had no easy solution to visualise my data in a iterative way (code, visualize, repeat). I was looking for a simple objective-c library to include, in order to visualise my data in a web browser. by the way, thanks for your input, this is very valuable.
- ThePhysicist 11y ago
  If you can work with Python/Julia/R then I would still recommend the IPython notebook, since you will not get anything more interactive and better suited to data analysis and visualization in one package. It basically allows you to run arbitrary snippets of code and interactively work with your data (code, visualize repeat). In general, I would not try to develop your algorithms directly within your app but instead generate some example data, export that data and then work with it in IPython (or another tool) to iteratively build the algorithm. As soon as you have a working solution you can then integrate/translate the code back into your application.
  - lasryaric 11y ago
    Good point!
    - pvnick 11y ago
      Ipython notebook completely changed the way I do data science, specifically because its iterative/exploratory capabilities are so powerful. You can also hook it up to large scale service like hadoop/spark. Embedded Matplotlib charts have spoiled me. I cringe at the thought that I used to write scripts to output csv, run them on the command line, then copy the output to excel to graph the data. Never again!
  - jlees 11y ago
    Seconding the IPython Notebook. I've used matplotlib but also like Plot.ly for its prettier, more sharable charts. It's also not a huge amount of work to set up a JSON-Flask-D3 visualisation pipeline out of test data; that's exactly what I did for my current data project, using cubism.js to easily visualise a time series. Again, benefits are the ability to easily share. Also, sometimes, Google Spreadsheets/Excel can be the easiest solution to throw up quickly - or using built-in data explorations in RStudio. I've also thrown graph data into Neo4j because it gives you a nice visual interface for free.
elcritch 11y ago
The default themes / colors in matplotlib leaves something to be desired. I'd recommend checking into [seaborn(http://stanford.edu/~mwaskom/software/seaborn/ http://stanford.edu/~mwaskom/software/seaborn/) for setting nice default colors and handling many types of statistical data plots. It reduces a lot of the boiler plate. For visualizations that need to be shared IPython notebooks are great and work with Julia & R as well. The @interact widgets are fantastic for exploring data. Unfortunately interactive notebooks are difficult to post on a static website. To answer the OP's other questions: For that, I've been moving to a mix of html5/javascript/d3. While it's still a new-ish mix compared to matplotlib, there is something about being able to save all the data and computations to an html file which anyone with a decent browser can open and interact with is fantastic. I utilize this method to store my protocol records for experimental tests. One file can contain pretty self-describing data along with javascript code to nicely present it.
- bluusteel 11y ago
  I don't think I realized how important default colors/styles are for visualization software until using seaborn. Quickly producing sensible plots in terms of aesthetics should not require 10+ lines of extra code. Fortunately, it looks like the matplotlib folks are looking to change default colors/styles with 2.0 [1]. [1]http://matplotlib.org/style_changes.html http://matplotlib.org/style_changes.html

pvnick 11y ago

Agreed, matplotlib is great. We use it all the time to make publication-quality graphs (output to pdf), specifically with time series.

lasryaric 11y ago

It was not clear in my question but I was mostly talking about how do people visualise there data while they are working on it? Lets say you are a developer working on the well known Shazam application. You want to visualize your signals and frequencies for debugging purpose, how do you quickly visualize them? In my case, while working on signal processing using C and objective C, I had no easy solution to visualise my data in a iterative way (code, visualize, repeat). I was looking for a simple objective-c library to include, in order to visualise my data in a web browser. by the way, thanks for your input, this is very valuable.

ThePhysicist 11y ago
If you can work with Python/Julia/R then I would still recommend the IPython notebook, since you will not get anything more interactive and better suited to data analysis and visualization in one package. It basically allows you to run arbitrary snippets of code and interactively work with your data (code, visualize repeat). In general, I would not try to develop your algorithms directly within your app but instead generate some example data, export that data and then work with it in IPython (or another tool) to iteratively build the algorithm. As soon as you have a working solution you can then integrate/translate the code back into your application.
- lasryaric 11y ago
  Good point!
  - pvnick 11y ago
    Ipython notebook completely changed the way I do data science, specifically because its iterative/exploratory capabilities are so powerful. You can also hook it up to large scale service like hadoop/spark. Embedded Matplotlib charts have spoiled me. I cringe at the thought that I used to write scripts to output csv, run them on the command line, then copy the output to excel to graph the data. Never again!
- jlees 11y ago
  Seconding the IPython Notebook. I've used matplotlib but also like Plot.ly for its prettier, more sharable charts. It's also not a huge amount of work to set up a JSON-Flask-D3 visualisation pipeline out of test data; that's exactly what I did for my current data project, using cubism.js to easily visualise a time series. Again, benefits are the ability to easily share. Also, sometimes, Google Spreadsheets/Excel can be the easiest solution to throw up quickly - or using built-in data explorations in RStudio. I've also thrown graph data into Neo4j because it gives you a nice visual interface for free.

elcritch 11y ago

The default themes / colors in matplotlib leaves something to be desired. I'd recommend checking into [seaborn(http://stanford.edu/~mwaskom/software/seaborn/ http://stanford.edu/~mwaskom/software/seaborn/) for setting nice default colors and handling many types of statistical data plots. It reduces a lot of the boiler plate. For visualizations that need to be shared IPython notebooks are great and work with Julia & R as well. The @interact widgets are fantastic for exploring data. Unfortunately interactive notebooks are difficult to post on a static website. To answer the OP's other questions: For that, I've been moving to a mix of html5/javascript/d3. While it's still a new-ish mix compared to matplotlib, there is something about being able to save all the data and computations to an html file which anyone with a decent browser can open and interact with is fantastic. I utilize this method to store my protocol records for experimental tests. One file can contain pretty self-describing data along with javascript code to nicely present it.

bluusteel 11y ago
I don't think I realized how important default colors/styles are for visualization software until using seaborn. Quickly producing sensible plots in terms of aesthetics should not require 10+ lines of extra code. Fortunately, it looks like the matplotlib folks are looking to change default colors/styles with 2.0 [1]. [1]http://matplotlib.org/style_changes.html http://matplotlib.org/style_changes.html