Generating interactive HTML charts from Python?
November 25, 2016 12:45 PM Subscribe
I need to generate a series of interactive charts and maps from data I have in Pandas dataframes.
The user should be able to filter by a category and have the chart update in realtime.
This has to be published to a website, which is where my doubts start.
Basically, I need to go Pandas > HTML + Javascript with select widgets so users can play around with different parameters.
Some of the charts will be bar, some scatter and some maps with georeferenced circles representing data.
I've been looking at and copying from this example in Bokeh, which is pretty close to what I need. However, as I have to publish this, I don't know how easy it is to get the Bokeh server up on commodity web hosts (in this case, Webfaction).
I've also been looking at Plotly, which seems to have interactive widgets as well (like here), but I remember reading that you couldn't publish the widgets to a static file, so again I'd have to figure out how to host plotly on a webserver?
I have plenty of experience with python, html, javascript, etc., but haven't done this specific thing before and need to get it done over the weekend, so help from people with actual experience would be very much appreciated.
Basically, I need to go Pandas > HTML + Javascript with select widgets so users can play around with different parameters.
Some of the charts will be bar, some scatter and some maps with georeferenced circles representing data.
I've been looking at and copying from this example in Bokeh, which is pretty close to what I need. However, as I have to publish this, I don't know how easy it is to get the Bokeh server up on commodity web hosts (in this case, Webfaction).
I've also been looking at Plotly, which seems to have interactive widgets as well (like here), but I remember reading that you couldn't publish the widgets to a static file, so again I'd have to figure out how to host plotly on a webserver?
I have plenty of experience with python, html, javascript, etc., but haven't done this specific thing before and need to get it done over the weekend, so help from people with actual experience would be very much appreciated.
D3 and javascript are really great for this. If you can find a similar example, and then build from there, the ability to quickly import data and display it to your liking is fantastic.
https://d3js.org/
You can develop locally in the browser, then push to some site where it is visible to all.
posted by nickggully at 2:04 PM on November 25, 2016
https://d3js.org/
You can develop locally in the browser, then push to some site where it is visible to all.
posted by nickggully at 2:04 PM on November 25, 2016
D3 is excellent for doing interactive web based charts. However, you're basically writing Javascript code that's charting data from JSON blobs. Ie: no matplotlib, no pandas plotting, no Seaborn. Basically your Python code will be emitting the JSON data, and then you write Javascript charts in D3. If you're willing to put in the work the result will be excellent.
If you want something closer to Pandas / Jupyter, give mpld3 a look. It lets you create charts in Python with a matplotlib API, but then the charts are rendered using D3 in the browser instead of as static raster images. It works reasonably well but I imagine has to have some limitations. And the project seems to have stalled out a couple of years ago. But it works, and you can try it out easily.
posted by Nelson at 2:14 PM on November 25, 2016
If you want something closer to Pandas / Jupyter, give mpld3 a look. It lets you create charts in Python with a matplotlib API, but then the charts are rendered using D3 in the browser instead of as static raster images. It works reasonably well but I imagine has to have some limitations. And the project seems to have stalled out a couple of years ago. But it works, and you can try it out easily.
posted by Nelson at 2:14 PM on November 25, 2016
Response by poster: I'm willing to put in the work, just don't know if I'll have the time to do so, so I'm looking for something higher level.
I know some javascript, but am much more proficient in Python, hence my preference for a Python library that will output the appropriate javascript.
Thanks!
posted by signal at 4:19 PM on November 25, 2016
I know some javascript, but am much more proficient in Python, hence my preference for a Python library that will output the appropriate javascript.
Thanks!
posted by signal at 4:19 PM on November 25, 2016
d3 is really the right way to do this. There's a library called c3, which is basically a wrapper meant to make d3 more friendly. I've never used it and it may not have the flexibility you require. Generally, the trick to getting things done with d3 is to find an example here and modify it. It'll be frustrating, but easily doable in a weekend.
You don't need to get a Bokeh server up. Check out the Embedding Plots and Apps part of the documentation. However, I will caution that when we did this at work, it was so bloated as to be unusable. (This was for use within our team, so it can be pretty bad before we call it unusable.) We ripped it out in favor of d3, which is what we were already using elsewhere. (Basically, bokeh was the new shiny thing and didn't live up to the hype.)
posted by hoyland at 5:36 PM on November 25, 2016 [1 favorite]
You don't need to get a Bokeh server up. Check out the Embedding Plots and Apps part of the documentation. However, I will caution that when we did this at work, it was so bloated as to be unusable. (This was for use within our team, so it can be pretty bad before we call it unusable.) We ripped it out in favor of d3, which is what we were already using elsewhere. (Basically, bokeh was the new shiny thing and didn't live up to the hype.)
posted by hoyland at 5:36 PM on November 25, 2016 [1 favorite]
Highcharts is free for non-commercial use, and extremely reasonable license terms for commercial use.
posted by rhizome at 6:40 PM on November 25, 2016
posted by rhizome at 6:40 PM on November 25, 2016
I have been trying to do this exact thing for work for a while and have cycled through bokeh, plotly, and the interactive widgets in Jupyter (ipython) notebooks. I haven't tried spinning up a tornado server to do interactive MPL plots outside of Jupyter, although that's supposedly possible now -- here's the Webagg recipe in the MPL docs.
The upshot is that everything I've tried has been enough of a hacked-together pain (bokeh in particular seems to just leak memory constantly, which is deeply irritating) that I'm pretty convinced at this point that learning D3 is genuinely the past of least resistance, despite having, like you, much more facility with Python than with JS.
posted by dorque at 6:47 PM on November 25, 2016
The upshot is that everything I've tried has been enough of a hacked-together pain (bokeh in particular seems to just leak memory constantly, which is deeply irritating) that I'm pretty convinced at this point that learning D3 is genuinely the past of least resistance, despite having, like you, much more facility with Python than with JS.
posted by dorque at 6:47 PM on November 25, 2016
D3 is absolutely amazing but the learning curve is a bit steep. Totally worth the effort to learn it in the long run, but it's not so useful if you want to get something done quickly.
I've used NVD3 successfully in the past - it's another high level library on top of D3. Much faster to get results than D3 on its own.
From your description it sounds like you should also check out crossfilter which, for the right use-cases, is phenomenal.
posted by simonw at 8:05 PM on November 25, 2016 [1 favorite]
I've used NVD3 successfully in the past - it's another high level library on top of D3. Much faster to get results than D3 on its own.
From your description it sounds like you should also check out crossfilter which, for the right use-cases, is phenomenal.
posted by simonw at 8:05 PM on November 25, 2016 [1 favorite]
I can second hoyland's recommendation of C3. I spent this week trying to put together some charts with D3, in the end I gave up and discovered C3. D3 is great if you want to make your own chart formats, but it sounds like you want to put together some basic stuff.
I eventually put together a site showing some C3 charts with live updates coming over a websocket from aiohttp. If you want to have a look at what I did the code is on my github here.
posted by adventureloop at 4:36 AM on November 26, 2016 [1 favorite]
I eventually put together a site showing some C3 charts with live updates coming over a websocket from aiohttp. If you want to have a look at what I did the code is on my github here.
posted by adventureloop at 4:36 AM on November 26, 2016 [1 favorite]
Response by poster: 1 month later: I did get it up and running with Pandas + Bokeh, but it's super unstable, sometimes the dots are in the wrong place on the map and then they sort of jump back into position on zoom, etc.
I currently rewriting it in straight D3 using data exported form python.
posted by signal at 12:43 PM on December 26, 2016 [1 favorite]
I currently rewriting it in straight D3 using data exported form python.
posted by signal at 12:43 PM on December 26, 2016 [1 favorite]
This thread is closed to new comments.
posted by the agents of KAOS at 1:43 PM on November 25, 2016