5 ms·

Does anyone know of a good non-blocking plotting library for Python? I find that for some applications matplotlib really slows down my loops, but there are tim

Does anyone know of a good non-blocking plotting library for Python? I find that for some applications matplotlib really slows down my loops, but there are times that I really need to visualize the in-progress computations. While some other libraries might be faster, what I really want is simply to send instructions and data to a separate process that is performing the plotting. I'd like it to completely render a frame and simply skip instructions for following frames until it's done rendering, avoid buffering up every single frame that is sent to it, and have it plot the next available frame when it's done. I have yet to see a library that functions this way. Basically I want to be able to easily visualize an on-going computation without blocking it just for the rendering of temporary results.

malux85 9y ago
This is a cool idea. I would be open to working with you on this if you'd like to collaborate! (my contact in profile)
MattConfluence 9y ago
What I did when I had a similar problem was to have a completely separate visualization program and pass data messages to it (using Celery and RabbitMQ). But that was in a setting where I was already using Celery for other tasks. Setting up a message queue for just visualization feels a bit over-engineered. Maybe you can just write to a text file, or a Sqlite db, and have the visualization program poll for changes.
- oneweekwonder 9y ago
  Rabbitmq or redis will already be in apt or yum so a simple sudo apt-get install rabbitmq-server # or redis pip install celery You are ready to use their example[0] from celery import Celery app = Celery('tasks', broker='pyamqp://guest@localhost//') @app.task def add(x, y): return x + y The idea of a mq might feel over-engineered. But the deployment and use is actually straight forward. [0]: http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html#application http://docs.celeryproject.org/en/latest/getting-started/firs...
edraferi 9y ago
Bokeh supports streaming data to existing graphs. This lets you define a graph, display it in a browser, and then push updates from your code. This is a lot lighter than doing a full draw in matplotlib, plus works over the web. Here's an intro blog post from 2013 [0] and a current tutorial [1]. This functionality depends on Bokeh Server, docs here [2] [0] https://www.anaconda.com/developer-blog/painless-streaming-plots-bokeh/ https://www.anaconda.com/developer-blog/painless-streaming-p... [1] http://nbviewer.jupyter.org/github/bokeh/bokeh-notebooks/blob/master/tutorial/11%20-%20Running%20Bokeh%20Applictions.ipynb#Streaming-Data http://nbviewer.jupyter.org/github/bokeh/bokeh-notebooks/blo... [2] https://bokeh.pydata.org/en/latest/docs/user_guide/server.html https://bokeh.pydata.org/en/latest/docs/user_guide/server.ht...
- radarsat1 9y ago
  Oh that sounds excellent, I'll have to check it out.
- Dowwie 9y ago
  coincidentally: https://news.ycombinator.com/item?id=15483264 https://news.ycombinator.com/item?id=15483264
trextrex 9y ago
You should check out https://github.com/IGITUGraz/live-plotter https://github.com/IGITUGraz/live-plotter (I am the author of the package) It is designed exactly for this use-case -- visualizing results of an ongoing simulation. It does best effort plotting, but doesn't block computation for plotting. I would be happy to receive feedback on the package.
- radarsat1 9y ago
  Cool, thanks.
xioxox 9y ago
Veusz (https://veusz.github.io/ https://veusz.github.io/) does this. Data are sent to the plotting process using pickle over a socket.

malux85 9y ago

This is a cool idea. I would be open to working with you on this if you'd like to collaborate! (my contact in profile)

MattConfluence 9y ago

What I did when I had a similar problem was to have a completely separate visualization program and pass data messages to it (using Celery and RabbitMQ). But that was in a setting where I was already using Celery for other tasks. Setting up a message queue for just visualization feels a bit over-engineered. Maybe you can just write to a text file, or a Sqlite db, and have the visualization program poll for changes.

oneweekwonder 9y ago
Rabbitmq or redis will already be in apt or yum so a simple sudo apt-get install rabbitmq-server # or redis pip install celery You are ready to use their example[0] from celery import Celery app = Celery('tasks', broker='pyamqp://guest@localhost//') @app.task def add(x, y): return x + y The idea of a mq might feel over-engineered. But the deployment and use is actually straight forward. [0]: http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html#application http://docs.celeryproject.org/en/latest/getting-started/firs...

edraferi 9y ago

Bokeh supports streaming data to existing graphs. This lets you define a graph, display it in a browser, and then push updates from your code. This is a lot lighter than doing a full draw in matplotlib, plus works over the web. Here's an intro blog post from 2013 [0] and a current tutorial [1]. This functionality depends on Bokeh Server, docs here [2] [0] https://www.anaconda.com/developer-blog/painless-streaming-plots-bokeh/ https://www.anaconda.com/developer-blog/painless-streaming-p... [1] http://nbviewer.jupyter.org/github/bokeh/bokeh-notebooks/blob/master/tutorial/11%20-%20Running%20Bokeh%20Applictions.ipynb#Streaming-Data http://nbviewer.jupyter.org/github/bokeh/bokeh-notebooks/blo... [2] https://bokeh.pydata.org/en/latest/docs/user_guide/server.html https://bokeh.pydata.org/en/latest/docs/user_guide/server.ht...

radarsat1 9y ago
Oh that sounds excellent, I'll have to check it out.
Dowwie 9y ago
coincidentally: https://news.ycombinator.com/item?id=15483264 https://news.ycombinator.com/item?id=15483264

trextrex 9y ago

You should check out https://github.com/IGITUGraz/live-plotter https://github.com/IGITUGraz/live-plotter (I am the author of the package) It is designed exactly for this use-case -- visualizing results of an ongoing simulation. It does best effort plotting, but doesn't block computation for plotting. I would be happy to receive feedback on the package.

radarsat1 9y ago
Cool, thanks.

xioxox 9y ago

Veusz (https://veusz.github.io/ https://veusz.github.io/) does this. Data are sent to the plotting process using pickle over a socket.