Help Me Make An Interactive Data Visualization
June 14, 2012 7:45 PM Subscribe
I want to make a map-based interactive data visualization (to be accessed on the Internet) and I have no idea where to start.
I have very limited programming experience but I thoroughly enjoy creating things with code and as a hobby project I'd like to make a web-based interactive data visualization using census data and crime statistics publicly available from my city. I like maps, I like data visualizations, and I love my city, so it seems like a good goal. However, because of my lack of technical knowledge, I'm not sure what to focus on or how to really go about making this happen.
My knowledge:
I recently took a very basic and utilitarian programming class in which I learned some Visual Basic and HTML/CSS. In Visual Basic I've created various programs for the lab I work in that we use to analyze tab delimited data. In HTML I created a very rudimentary personal portfolio using some very basic concepts (placing text and borders spatially on a page and altering their appearance somewhat). I'm currently taking the CS101 Udacity course.
What I want to achieve:
I'd like to take data from the 2010 Census (which I think I can gather from here) in addition to crime data from my city (which is tabulated by reporting district and looks like this) to create a map that color codes reporting districts based on assorted crimes committed per capita and provides more information (I'm thinking in a side panel of some sort) when a user hovers over each district. Also, considering the city provides historical data going back a few years, I'd like to include a time slider type thing that would allow a user to see this data based on the time span chosen.
I'm imagining something similar to this or this in style and form.
So I have the general interface down in my mind and the data is out there (although I'm thinking I'll have to try and get the crime data in non-pdf form, right?) but I just don't know how to actually make the thing.
I'm aware that I'll probably have to spend quite a bit of time learning the basics before creating something complex like this. That's completely fine and sounds like fun, but what should I go learn and how should I go about learning it? Should I pick some smaller goals before tackling this one? Are there any good communities out there that I should know about that would help me?
I just really need some focus and there is so much out there on data visualization that I don't know where to start.
Thank you so much in advance!
I have very limited programming experience but I thoroughly enjoy creating things with code and as a hobby project I'd like to make a web-based interactive data visualization using census data and crime statistics publicly available from my city. I like maps, I like data visualizations, and I love my city, so it seems like a good goal. However, because of my lack of technical knowledge, I'm not sure what to focus on or how to really go about making this happen.
My knowledge:
I recently took a very basic and utilitarian programming class in which I learned some Visual Basic and HTML/CSS. In Visual Basic I've created various programs for the lab I work in that we use to analyze tab delimited data. In HTML I created a very rudimentary personal portfolio using some very basic concepts (placing text and borders spatially on a page and altering their appearance somewhat). I'm currently taking the CS101 Udacity course.
What I want to achieve:
I'd like to take data from the 2010 Census (which I think I can gather from here) in addition to crime data from my city (which is tabulated by reporting district and looks like this) to create a map that color codes reporting districts based on assorted crimes committed per capita and provides more information (I'm thinking in a side panel of some sort) when a user hovers over each district. Also, considering the city provides historical data going back a few years, I'd like to include a time slider type thing that would allow a user to see this data based on the time span chosen.
I'm imagining something similar to this or this in style and form.
So I have the general interface down in my mind and the data is out there (although I'm thinking I'll have to try and get the crime data in non-pdf form, right?) but I just don't know how to actually make the thing.
I'm aware that I'll probably have to spend quite a bit of time learning the basics before creating something complex like this. That's completely fine and sounds like fun, but what should I go learn and how should I go about learning it? Should I pick some smaller goals before tackling this one? Are there any good communities out there that I should know about that would help me?
I just really need some focus and there is so much out there on data visualization that I don't know where to start.
Thank you so much in advance!
Google Fusion Tables are a great place to start with mapping stuff- once you get things in there, it is really easy to get them into a google map- and then you can pretty easily pull that map into a website.
Google Refine is THE tool to use to take the raw data you can get from publicly available sources and clean it up into a usable form that you can then stick in your fusion tables.
Good luck- this stuff is fun! The livehoods one looks like something you could put together pretty easily in fusiontables (and actually it might be, I'm not sure).
The NY times one is quite a bit harder...you aren't going to be doing that with a smattering of coding and some CSS.
posted by rockindata at 7:55 PM on June 14, 2012 [1 favorite]
Google Refine is THE tool to use to take the raw data you can get from publicly available sources and clean it up into a usable form that you can then stick in your fusion tables.
Good luck- this stuff is fun! The livehoods one looks like something you could put together pretty easily in fusiontables (and actually it might be, I'm not sure).
The NY times one is quite a bit harder...you aren't going to be doing that with a smattering of coding and some CSS.
posted by rockindata at 7:55 PM on June 14, 2012 [1 favorite]
Another option is d3.js, which is an amazing framework for web-based data visualization. You could definitely use that.
posted by Blazecock Pileon at 7:55 PM on June 14, 2012 [1 favorite]
posted by Blazecock Pileon at 7:55 PM on June 14, 2012 [1 favorite]
Response by poster: These are great suggestions so far.
rockindata, if I had all of the time in the world what would I learn to make the NY times one?
posted by Defenestrator at 8:04 PM on June 14, 2012
rockindata, if I had all of the time in the world what would I learn to make the NY times one?
posted by Defenestrator at 8:04 PM on June 14, 2012
If the data is public, meant to be consumed on a public website, check out what you can do with Tableau Public.
posted by tayknight at 8:21 PM on June 14, 2012 [1 favorite]
posted by tayknight at 8:21 PM on June 14, 2012 [1 favorite]
Seconding BP - you need to learn R. It's a long way from VB\HTML\CSS but it will do what you want.
posted by barely legal at 9:26 PM on June 14, 2012
posted by barely legal at 9:26 PM on June 14, 2012
If you've got server space and some SQL knowledge, install WEAVE for the interactive map and data panels you desire.
QuantumGIS is free, and is an excellent starting point for GIS. I'd recommend you learn to generate some static maps before plunging into interactivity. Learn your way around the Census Bureau's FTP server for direct access to data and shapefiles--you'll find all the basic boundaries you need.
posted by Hollywood Upstairs Medical College at 9:36 PM on June 14, 2012
QuantumGIS is free, and is an excellent starting point for GIS. I'd recommend you learn to generate some static maps before plunging into interactivity. Learn your way around the Census Bureau's FTP server for direct access to data and shapefiles--you'll find all the basic boundaries you need.
posted by Hollywood Upstairs Medical College at 9:36 PM on June 14, 2012
I'll recommend Tableau Public as well (caveat: I work at Tableau). You should be able to do all you want, without any programming required.
Here are examples similar to what you want to do: Crime in Vancouver, Seattle, DC (you can download the Tableau workbooks yourself and play with them to understand how they were created.)
posted by ShooBoo at 10:02 PM on June 14, 2012
Here are examples similar to what you want to do: Crime in Vancouver, Seattle, DC (you can download the Tableau workbooks yourself and play with them to understand how they were created.)
posted by ShooBoo at 10:02 PM on June 14, 2012
There are a few ways you can go about doing the thing that you want to do, and it's worthwhile to pause for a second and consider the suggestions given in the thread in the context of larger ecosystem. At the end of the day, you need to:
1. Find the right data
2. Find technology to render your data on/within a map
3. Find technology to provide interactivity features you seek (change over time, side panels)
For the first step, you need to get the data from census website, and ensure it's in the right format for the next step. Previously-mentioned Google Refine is a fine tool to do it, but if your source data is already pretty clean (and, if you get it from the right place, aggregate census data ought to be clean), it may need nothing more than a quick pass in e.g. Excel to get rid of extraneous columns. Just looking at the data will quickly tell you what processing/clean-up it needs, if any.
Now that you have the data, the next step is visualization - steps 2 and 3 above. Map-based visualizations are a reasonably common thing to do, and there are no shortage of different ways to do this. The real question is, what do you want to get out of it? Would you just like to get to the final result with minimum effort? To have a fun programming project? To have an excuse for learning a visualization package? The choices are many. Here are a few:
If you just want to get a visualization going quickly and to play with it ASAP, and if you don't care about your data being seen by other people (e.g. it's all public anyway), the above-mentioned Google Fusion Tables and (especially) Tableau Public should do exactly what you want, no programming required. You can have the whole thing working in 10 minutes, once you get all your data into one place and cleaned up (step 1)
If you want to have a programming project, and end up with a little web application, you should explore a javascript-based mapping library as likely lowest-barrier-to-entry option. For example, check out JQueryMap. This way, you would get someone else's technology to render a map for you, and then you can write some simple javascript (code) and css (styles) to add the interactive functionality you want, like the fly-outs or time control. Definitely more work, but maybe more fun?
A few folks above suggested R. I don't think that it's the best choice for your particular project. R has a number of excellent visualization libraries that can output gorgeous maps, but as far as I know there are no packages that output a fully-interactive visualization project. In other words, R will help you make pretty static images (e.g. crime by neighborhood), but bolting interactivity on top of these images will be difficult and probably not worth the effort, given the availability of data-driven mapping approaches like the HTML/Javascript/CSS route above.
Have fun!
posted by blindcarboncopy at 11:15 PM on June 14, 2012
1. Find the right data
2. Find technology to render your data on/within a map
3. Find technology to provide interactivity features you seek (change over time, side panels)
For the first step, you need to get the data from census website, and ensure it's in the right format for the next step. Previously-mentioned Google Refine is a fine tool to do it, but if your source data is already pretty clean (and, if you get it from the right place, aggregate census data ought to be clean), it may need nothing more than a quick pass in e.g. Excel to get rid of extraneous columns. Just looking at the data will quickly tell you what processing/clean-up it needs, if any.
Now that you have the data, the next step is visualization - steps 2 and 3 above. Map-based visualizations are a reasonably common thing to do, and there are no shortage of different ways to do this. The real question is, what do you want to get out of it? Would you just like to get to the final result with minimum effort? To have a fun programming project? To have an excuse for learning a visualization package? The choices are many. Here are a few:
If you just want to get a visualization going quickly and to play with it ASAP, and if you don't care about your data being seen by other people (e.g. it's all public anyway), the above-mentioned Google Fusion Tables and (especially) Tableau Public should do exactly what you want, no programming required. You can have the whole thing working in 10 minutes, once you get all your data into one place and cleaned up (step 1)
If you want to have a programming project, and end up with a little web application, you should explore a javascript-based mapping library as likely lowest-barrier-to-entry option. For example, check out JQueryMap. This way, you would get someone else's technology to render a map for you, and then you can write some simple javascript (code) and css (styles) to add the interactive functionality you want, like the fly-outs or time control. Definitely more work, but maybe more fun?
A few folks above suggested R. I don't think that it's the best choice for your particular project. R has a number of excellent visualization libraries that can output gorgeous maps, but as far as I know there are no packages that output a fully-interactive visualization project. In other words, R will help you make pretty static images (e.g. crime by neighborhood), but bolting interactivity on top of these images will be difficult and probably not worth the effort, given the availability of data-driven mapping approaches like the HTML/Javascript/CSS route above.
Have fun!
posted by blindcarboncopy at 11:15 PM on June 14, 2012
I've been working with Leaflet, and, assuming a little javascript knowledge, it's very easy to get started, particularly with regard to layering sets of geodata with mouseover properties.
posted by urbanwhaleshark at 6:55 AM on June 15, 2012
posted by urbanwhaleshark at 6:55 AM on June 15, 2012
Lots of tools here.
The Library of Congress recently released something that might fit the ill, but I can't remember the name or find it right now...
posted by LarryC at 9:41 AM on June 15, 2012
The Library of Congress recently released something that might fit the ill, but I can't remember the name or find it right now...
posted by LarryC at 9:41 AM on June 15, 2012
Came here to nth Tableau Public.
posted by kprincehouse at 2:36 PM on June 15, 2012
posted by kprincehouse at 2:36 PM on June 15, 2012
This thread is closed to new comments.
posted by Blazecock Pileon at 7:53 PM on June 14, 2012 [1 favorite]