5 ms·

Plotly graphing API for Python, Matlab, R, and Arduino

56 points by cparmer 13y ago

cparmer 13y ago
The idea behind Plotly is: to build a platform where people can collaboratively graph, analyze, and share data. Two months ago we launched Plotly on HN (https://news.ycombinator.com/item?id=5671831 https://news.ycombinator.com/item?id=5671831). We received extremely helpful feedback, and are grateful to everyone who commented and used the product. Hearing from readers and new users was exciting, helpful, and insightful. So, thanks very much. The #1 request was an API to generate plotly graphs from desktop environments. In response to that feedback, we are launching a new service so you can now send data from Python, MATLAB, R, and Arduino to your plotly account. Calls made with the API will return a link with a graph of your data, i.e. something like this: https://plot.ly/~chris/421/ https://plot.ly/~chris/421/. You can sign up through the API. The graphs are interactive: click-drag to zoom, double-click to autoscale, shift-drag to pan, hover to view data points. You can view the graphs inside plotly and style them with our GUI. For example, I sent data with the API to make this: http://plot.ly/~chris/554/ http://plot.ly/~chris/554/ and then styled it inside plotly to create this: https://plot.ly/~chris/653/ https://plot.ly/~chris/653/ HN: What do you think of the API? What else can we do to make Plotly a tool that you would realistically use every single day?
- Mithaldu 13y ago
  Why don't you have a Perl API?
  - zhemao 13y ago
    Probably because they didn't think many people would use it and had limited man-hours to spare. But it would be nice if they released some language-agnostic API like a REST interface so that people could write bindings for their own languages. They must be using something like that internally anyway.
    - cema 13y ago
      Yes, it would be helpful both for the customers (users, that is) and themselves too, as the potential number of users (and usages!) will grow.
    - cparmer 13y ago
      Yeah, that's a great idea. We'll definitely check that out.
      - zhemao 13y ago
        Hey, that's great! And good work so far, by the way. This looks pretty cool.
      - zhemao 13y ago
        Also, are your existing libraries open source? If not, it might be a good idea to, after finishing the REST interface, refactor your libraries to use the REST interface and then release them as open source. That way people could contribute to them and see how to write their own.
  - cparmer 13y ago
    We're focussing on the scientific-computation languages since those users are probably doing the most plotting on the day-to-day and are in most need of a better data collaboration tool. That being said, we'll add interfaces to other languages over time. Do you do a lot of plotting in Perl?
    - Mithaldu 13y ago
      Not right now, but in the past i've worked on applications for plotting health-related data, activity of financial markets and created visualizations for bio-engineering researchers. Perl is one of the go-to languages for converting raw data into manager-friendly representations, has many tools for that already, and i'm sure many would be interested in adding another tool to their repertoire. Frankly, i find myself baffled that you'd think Perl is not a scientific-computation language, given the existence of BioPerl, PDL, GD::Graph and many other widely-used tools that relate to your field. If you'd agree to putting it on your API page, i'll happily do the needful and make a release to CPAN and port the examples you have there.
      - cparmer 13y ago
        Cool, that's really great feedback. I'm certainly more unfamiliar with the Perl community than with the other 3 languages, so thanks for sharing and offering your help!
        Mithaldu 13y ago
        So, that doesn't answer the question. If i write the module and port the examples, will you put it on your website? (If you agree, giving me a github repo to make a pull req to, or an email to send the data to would help too.)
        cparmer 13y ago
        I'll definitely place a Perl (and other language modules) on the API page! I think the best way to go about this is to write a REST api as @zhemao suggested. Feel free to email me at chris[at]plot.ly :)
- drmr 13y ago
  Looks nice and I would definitely like to see support for other languages, like perl. But, what if I want to run my own infrastructure?
  - cparmer 13y ago
    Cool, that's super helpful feedback. We definitely support installing plotly on private servers -- there are certainly a lot of companies that won't let their data leave their building.
  - cparmer 13y ago
    Just curious, for what reasons would you like to run it on your own infrastructure (computational reasons, data privacy reasons, etc)? Thanks for the feedback!
- stephengillie 13y ago
  Looking at the Arduino code, this would be a cool way to visualize data from a temperature sensor project which I was working on. I notice it will require an always-on connection in order to transmit data back to your servers. Do you have any plans for making something slightly more modular which could deal with devices that can't always "phone home"? Or maybe that's on us makers -- we need to store the data to SD or something, then build routines that send data when connected. Either way, using Plotly means your project has to have either an Ethernet shield or a Wifi module.
  - cparmer 13y ago
    I see, thanks for the feedback. You could do that now by storing the data on your SD card and then dumping that data all at once whenever you want to connect. What kind of example do you have in mind when you say that your device can't "phone home"?
    - RaphiePS 13y ago
      I'd love to see a Plotly-Serial bridge, which could be as simple as a tiny CLI. I keep most of my Arduino projects tethered to my computer anyway, so this wouldn't require any extra hardware. All I'd have to do is send something like "PLOT <number>" over the usb-serial connection. In fact, it'd be completely device-agnostic--it shouldn't matter if the device is an Arduino or a Propeller or a custom board as long as it can talk over serial. I might be in the minority of users who tether, and it'd be pretty easy to roll my own solution using the Python API. Nevertheless, it'd sure be convenient.
      - cparmer 13y ago
        That's a really great idea. I don't think that you are in the minority, considering the number of scientific instruments that talk over serial. It would be cool to make plotting from these devices one (or several) step(s) easier. Thanks for the feedback!
    - stephengillie 13y ago
      I don't really have a scenario in mind, I'm just thinking that including this with a project means spending another $35 for a wifi or ethernet module (for either shield, it's about the same), the pinouts they'll take up, and the extra code to operate the modules. But there are many projects with those modules already integrated, and for them this looks like will be basically just a bolt-on of code.
- adiM 13y ago
  Any plans to add support for Julia as well. One of the things holding Julia back is a good plotting library.
  - cparmer 13y ago
    I've been hearing that more and more lately. We'll definitely roll out Julia soon :)
    - KenoFischer 13y ago
      If you guys need help with that, do email julia-dev.
      - cparmer 13y ago
        I will - thanks for tip!
schrodingersCat 13y ago
As someone who routinely uses python, R, and matlab for data analysis, I am going to have to give this a try. Matplotlib can be cumbersome and I am always looking for alternatives. This looks great! Can't wait to try it.
- deleted 13y ago
  [deleted]
- bernardom 13y ago
  Yeah, this looks pretty cool. I currently use RStudio server to do analysis from the RStudio IDE in my browser- while all the client-sensitive data and computation happens in a remote box. Allows me to play with graphs on my browser as well,. It looks like Plotly's advantage is in sharing the plots easily.
  - cparmer 13y ago
    That's cool. Does your company set up this server version or is this something that you manage yourself? I've been using RStudio in my desktop, what do you like about using it in your browser?
    - bernardom 13y ago
      My company set it up this way, at my request. I enjoy RStudio over using R over ssh (in a CLI). The advantage of using it in a browser: it's not on my local machine! So if I run some godawful super-intensive script, my machine doesn't slow to a crawl. Also, I work with client sensitive data and we try not to hold any on our local laptops. It's really a nice solution. Now if only they supported multiple monitors...
- tmarthal 13y ago
  python has a ton of in browser plotting mechanisms (including matplotlib inside iPython notebook files), and you can always use vincent which is a wrapper around vega to create d3 plots (svg html elements, which can be styled with CSS). Here is the repo/tutorial for plotting with vincent https://github.com/wrobstory/vincent https://github.com/wrobstory/vincent [Note that vincent works great if you are using pandas for your data analysis.]
seiferteric 13y ago
Oh interesting, I built something similar to this a while back https://graphit.io https://graphit.io
crntaylor 13y ago
This looks really neat. I produce a lot of plots in my day to day work (mainly in MATLAB) so I'm excited to see if there's an opportunity to use Plotly. Can you point me to some examples of how you've brought Plotly into your workflow? Do you generally send data via the API and style it manually, or do you do all the styling in code? How easy is it to grab plots from Plotly to be included in e.g. a PDF?
- cparmer 13y ago
  That's great to hear. Personally, I prefer to style the plots with the GUI... it's so much easier and faster than re-running some styling script. Of course, some stylings are too cumbersome to do by hand. For example, I adjusted the opacity of every trace with code to make this plot: https://plot.ly/~cparmer2/302/ https://plot.ly/~cparmer2/302/, which was easier than manually selecting every trace in the GUI. A couple workflow examples: - last semester, I used plotly to share results with my advisor. It was much easier to send short-links of my simulation graphs than to export images and send them as an attachment. My advisor could update the styling and view the underlying data behind the graph and he only needed to open up his browser. - Some R&D labs are interested in using plotly to centralize their data streams. They send data to plotly from their lab instruments (through the api) and share the graphs within plotly to all their team members (through a group feature in plotly) Plotly graphs can be exported to SVG, EPS, PDF, and PNG, so it's pretty easy to include your graphs in a PDF. Do you use MATLAB in a team environment?
  - adiM 13y ago
    Could you please add an example to show how to export graphs to SVG, EPS, PDF, etc. (Sorry if the example is included in the documentation, but I could not find it).
    - cparmer 13y ago
      Sure. You export graphs from within your plotly account (i.e. at https://plot.ly/plot https://plot.ly/plot). With a loaded graph, click "File >" on the toolbar and you should see PNG, PDF, SVG, EPS. You can view any linked graph (i.e. something like this: https://plot.ly/~jackp/683/ https://plot.ly/~jackp/683/) from within your plotly account by clicking the "View in Plotly" link in the bottom righthand corner.
computerality 13y ago
Is there going to be an offline version of this?
- cparmer 13y ago
  We're interested in providing plotly to companies that can't have their data leave their servers (for ip reasons) but otherwise we're keeping plotly online. Just curious, for what reasons would you prefer an offline version?
platz 13y ago
I tried to pip install plotly with python 3.3 and got some errors. Is it Py3 compatible?
- cparmer 13y ago
  To be honest, I haven't tested with Py3. It sounds like it isn't... I'll look into it. Thanks for reporting!

56 points by cparmer 13y ago

The idea behind Plotly is: to build a platform where people can collaboratively graph, analyze, and share data. Two months ago we launched Plotly on HN (https://news.ycombinator.com/item?id=5671831 https://news.ycombinator.com/item?id=5671831). We received extremely helpful feedback, and are grateful to everyone who commented and used the product. Hearing from readers and new users was exciting, helpful, and insightful. So, thanks very much. The #1 request was an API to generate plotly graphs from desktop environments. In response to that feedback, we are launching a new service so you can now send data from Python, MATLAB, R, and Arduino to your plotly account. Calls made with the API will return a link with a graph of your data, i.e. something like this: https://plot.ly/~chris/421/ https://plot.ly/~chris/421/. You can sign up through the API. The graphs are interactive: click-drag to zoom, double-click to autoscale, shift-drag to pan, hover to view data points. You can view the graphs inside plotly and style them with our GUI. For example, I sent data with the API to make this: http://plot.ly/~chris/554/ http://plot.ly/~chris/554/ and then styled it inside plotly to create this: https://plot.ly/~chris/653/ https://plot.ly/~chris/653/ HN: What do you think of the API? What else can we do to make Plotly a tool that you would realistically use every single day?

Mithaldu 13y ago
Why don't you have a Perl API?
- zhemao 13y ago
  Probably because they didn't think many people would use it and had limited man-hours to spare. But it would be nice if they released some language-agnostic API like a REST interface so that people could write bindings for their own languages. They must be using something like that internally anyway.
  - cema 13y ago
    Yes, it would be helpful both for the customers (users, that is) and themselves too, as the potential number of users (and usages!) will grow.
  - cparmer 13y ago
    Yeah, that's a great idea. We'll definitely check that out.
    - zhemao 13y ago
      Hey, that's great! And good work so far, by the way. This looks pretty cool.
    - zhemao 13y ago
      Also, are your existing libraries open source? If not, it might be a good idea to, after finishing the REST interface, refactor your libraries to use the REST interface and then release them as open source. That way people could contribute to them and see how to write their own.
- cparmer 13y ago
  We're focussing on the scientific-computation languages since those users are probably doing the most plotting on the day-to-day and are in most need of a better data collaboration tool. That being said, we'll add interfaces to other languages over time. Do you do a lot of plotting in Perl?
  - Mithaldu 13y ago
    Not right now, but in the past i've worked on applications for plotting health-related data, activity of financial markets and created visualizations for bio-engineering researchers. Perl is one of the go-to languages for converting raw data into manager-friendly representations, has many tools for that already, and i'm sure many would be interested in adding another tool to their repertoire. Frankly, i find myself baffled that you'd think Perl is not a scientific-computation language, given the existence of BioPerl, PDL, GD::Graph and many other widely-used tools that relate to your field. If you'd agree to putting it on your API page, i'll happily do the needful and make a release to CPAN and port the examples you have there.
    - cparmer 13y ago
      Cool, that's really great feedback. I'm certainly more unfamiliar with the Perl community than with the other 3 languages, so thanks for sharing and offering your help!
      - Mithaldu 13y ago
        So, that doesn't answer the question. If i write the module and port the examples, will you put it on your website? (If you agree, giving me a github repo to make a pull req to, or an email to send the data to would help too.)
        cparmer 13y ago
        I'll definitely place a Perl (and other language modules) on the API page! I think the best way to go about this is to write a REST api as @zhemao suggested. Feel free to email me at chris[at]plot.ly :)
drmr 13y ago
Looks nice and I would definitely like to see support for other languages, like perl. But, what if I want to run my own infrastructure?
- cparmer 13y ago
  Cool, that's super helpful feedback. We definitely support installing plotly on private servers -- there are certainly a lot of companies that won't let their data leave their building.
- cparmer 13y ago
  Just curious, for what reasons would you like to run it on your own infrastructure (computational reasons, data privacy reasons, etc)? Thanks for the feedback!
stephengillie 13y ago
Looking at the Arduino code, this would be a cool way to visualize data from a temperature sensor project which I was working on. I notice it will require an always-on connection in order to transmit data back to your servers. Do you have any plans for making something slightly more modular which could deal with devices that can't always "phone home"? Or maybe that's on us makers -- we need to store the data to SD or something, then build routines that send data when connected. Either way, using Plotly means your project has to have either an Ethernet shield or a Wifi module.
- cparmer 13y ago
  I see, thanks for the feedback. You could do that now by storing the data on your SD card and then dumping that data all at once whenever you want to connect. What kind of example do you have in mind when you say that your device can't "phone home"?
  - RaphiePS 13y ago
    I'd love to see a Plotly-Serial bridge, which could be as simple as a tiny CLI. I keep most of my Arduino projects tethered to my computer anyway, so this wouldn't require any extra hardware. All I'd have to do is send something like "PLOT <number>" over the usb-serial connection. In fact, it'd be completely device-agnostic--it shouldn't matter if the device is an Arduino or a Propeller or a custom board as long as it can talk over serial. I might be in the minority of users who tether, and it'd be pretty easy to roll my own solution using the Python API. Nevertheless, it'd sure be convenient.
    - cparmer 13y ago
      That's a really great idea. I don't think that you are in the minority, considering the number of scientific instruments that talk over serial. It would be cool to make plotting from these devices one (or several) step(s) easier. Thanks for the feedback!
  - stephengillie 13y ago
    I don't really have a scenario in mind, I'm just thinking that including this with a project means spending another $35 for a wifi or ethernet module (for either shield, it's about the same), the pinouts they'll take up, and the extra code to operate the modules. But there are many projects with those modules already integrated, and for them this looks like will be basically just a bolt-on of code.
adiM 13y ago
Any plans to add support for Julia as well. One of the things holding Julia back is a good plotting library.
- cparmer 13y ago
  I've been hearing that more and more lately. We'll definitely roll out Julia soon :)
  - KenoFischer 13y ago
    If you guys need help with that, do email julia-dev.
    - cparmer 13y ago
      I will - thanks for tip!

schrodingersCat 13y ago

As someone who routinely uses python, R, and matlab for data analysis, I am going to have to give this a try. Matplotlib can be cumbersome and I am always looking for alternatives. This looks great! Can't wait to try it.

deleted 13y ago
[deleted]
bernardom 13y ago
Yeah, this looks pretty cool. I currently use RStudio server to do analysis from the RStudio IDE in my browser- while all the client-sensitive data and computation happens in a remote box. Allows me to play with graphs on my browser as well,. It looks like Plotly's advantage is in sharing the plots easily.
- cparmer 13y ago
  That's cool. Does your company set up this server version or is this something that you manage yourself? I've been using RStudio in my desktop, what do you like about using it in your browser?
  - bernardom 13y ago
    My company set it up this way, at my request. I enjoy RStudio over using R over ssh (in a CLI). The advantage of using it in a browser: it's not on my local machine! So if I run some godawful super-intensive script, my machine doesn't slow to a crawl. Also, I work with client sensitive data and we try not to hold any on our local laptops. It's really a nice solution. Now if only they supported multiple monitors...
tmarthal 13y ago
python has a ton of in browser plotting mechanisms (including matplotlib inside iPython notebook files), and you can always use vincent which is a wrapper around vega to create d3 plots (svg html elements, which can be styled with CSS). Here is the repo/tutorial for plotting with vincent https://github.com/wrobstory/vincent https://github.com/wrobstory/vincent [Note that vincent works great if you are using pandas for your data analysis.]

seiferteric 13y ago

Oh interesting, I built something similar to this a while back https://graphit.io https://graphit.io

crntaylor 13y ago

This looks really neat. I produce a lot of plots in my day to day work (mainly in MATLAB) so I'm excited to see if there's an opportunity to use Plotly. Can you point me to some examples of how you've brought Plotly into your workflow? Do you generally send data via the API and style it manually, or do you do all the styling in code? How easy is it to grab plots from Plotly to be included in e.g. a PDF?

cparmer 13y ago
That's great to hear. Personally, I prefer to style the plots with the GUI... it's so much easier and faster than re-running some styling script. Of course, some stylings are too cumbersome to do by hand. For example, I adjusted the opacity of every trace with code to make this plot: https://plot.ly/~cparmer2/302/ https://plot.ly/~cparmer2/302/, which was easier than manually selecting every trace in the GUI. A couple workflow examples: - last semester, I used plotly to share results with my advisor. It was much easier to send short-links of my simulation graphs than to export images and send them as an attachment. My advisor could update the styling and view the underlying data behind the graph and he only needed to open up his browser. - Some R&D labs are interested in using plotly to centralize their data streams. They send data to plotly from their lab instruments (through the api) and share the graphs within plotly to all their team members (through a group feature in plotly) Plotly graphs can be exported to SVG, EPS, PDF, and PNG, so it's pretty easy to include your graphs in a PDF. Do you use MATLAB in a team environment?
- adiM 13y ago
  Could you please add an example to show how to export graphs to SVG, EPS, PDF, etc. (Sorry if the example is included in the documentation, but I could not find it).
  - cparmer 13y ago
    Sure. You export graphs from within your plotly account (i.e. at https://plot.ly/plot https://plot.ly/plot). With a loaded graph, click "File >" on the toolbar and you should see PNG, PDF, SVG, EPS. You can view any linked graph (i.e. something like this: https://plot.ly/~jackp/683/ https://plot.ly/~jackp/683/) from within your plotly account by clicking the "View in Plotly" link in the bottom righthand corner.

computerality 13y ago

Is there going to be an offline version of this?

cparmer 13y ago
We're interested in providing plotly to companies that can't have their data leave their servers (for ip reasons) but otherwise we're keeping plotly online. Just curious, for what reasons would you prefer an offline version?

platz 13y ago

I tried to pip install plotly with python 3.3 and got some errors. Is it Py3 compatible?

cparmer 13y ago
To be honest, I haven't tested with Py3. It sounds like it isn't... I'll look into it. Thanks for reporting!