Comments on: Help me find some numbers.

Question: Help me find some numbers.

jbickers — Thu, 23 Apr 2009 12:05:36 -0800

I'm interested in doing data-based mashups. What are some really cool, deep sources of publicly available data?

I realize I'm casting a wide net here, but I'm intrigued by the data mashup, and wonder (a) just how much, and how diverse, publicly available data is out there, and (b) where to find it.

So, just pulling some examples off the top of my head ...

* Obesity rates vs. shopping mall density
* Population vs. bankruptcies
* iPod sales over time vs. weather trends in a specific geographic area

... you get the idea. Anything goes. Is there a canonical list of publicly available data? Are there meta-indexes to indexes of the stuff? How does one go about keeping up to date with all of the numbers that are surely being published every day?

By: phil

phil — Thu, 23 Apr 2009 12:07:29 -0800

amazon's public data sets are worth checking out

By: odinsdream

odinsdream — Thu, 23 Apr 2009 12:10:39 -0800

Netflix Prize is a contest to design a better rating algorithm. There's a data-set available if you register as a team.

By: soma lkzx

soma lkzx — Thu, 23 Apr 2009 12:11:45 -0800

The US census bureau is a fine source, and freebase looks nice (though I've never used it).

By: mhum

mhum — Thu, 23 Apr 2009 12:23:39 -0800

The Bureau of Labor Statistics, the Federal Reserve, and the Bureau of Economic Analysis all make a bunch of data available.

By: bonehead

bonehead — Thu, 23 Apr 2009 12:39:31 -0800

The US Energy Information Administration is a great source for anything to do with energy or energy consumption (movement of crude oil, comparison of electricity prices, etc...).

The OECD has lots of good transnational and comparison stats. Also sourceOECD: Taxes, GDP, death rates, education, status of women, etc...

By: schyler523

schyler523 — Thu, 23 Apr 2009 12:41:36 -0800

There is a lot of Missouri data here. You're likely to find a lot of similar data if you search for state specific GIS data.

By: chrisamiller

chrisamiller — Thu, 23 Apr 2009 12:52:57 -0800

Probably not what you had in mind, but the amount of biological data available through the NCBI is staggering.

By: archofatlas

archofatlas — Thu, 23 Apr 2009 12:53:21 -0800

Case Western's Center on Urban Poverty & Community Development in Cleveland, OH has a great publicly available dataset culled from the following sources:

* Census
* Crime data from the Cleveland Police Department
* Vital statistics from the Ohio Department of Health
* Property characteristics and sales information from the Cuyahoga County Auditor and Recorder
* Public assistance data from Cuyahoga County Employment and Family Services
* Juvenile delinquency data from the Cuyahoga County Juvenile Court
* Child maltreatment data from the Cuyahoga County Department of Children and Family Services
* Mortgage lending data (HMDA) from the FFIEC
* Enrollment and attendance from the Cleveland Municipal School District

By: pwnguin

pwnguin — Thu, 23 Apr 2009 13:14:51 -0800

What you're asking for is the same thing experts in the field are asking for: publicly accessible raw data. TED has a few videos of experts explaining the situation and what it will result in:

Tim Berners Lee
Hans Rosling (sequel)

By: Alterscape

Alterscape — Thu, 23 Apr 2009 13:46:48 -0800

Have you looked at ManyEyes yet? There are a lot of visualizations with associated data available for further use.

By: motherly corn

motherly corn — Thu, 23 Apr 2009 14:42:50 -0800

Population Reference Bureau
Department of Energy Data Explorer
Numbrary
Datamob has an extensive list of datasets
Comprehensive Knowledge Archive Network
Infochimps.org
OpenStreetMap for geographic data

By: puckupdate

puckupdate — Thu, 23 Apr 2009 16:31:24 -0800

IPUMS. They link to a lot of good stuff, too.

By: the latin mouse

the latin mouse — Thu, 23 Apr 2009 16:32:55 -0800

The MySociety websites have lot of UK data involving national and local government.

By: zippy

zippy — Thu, 23 Apr 2009 17:35:25 -0800

Seconding Freebase (disclaimer: I worked there). I believe it's the broadest and most frequently updated open database.

By: tybstar

tybstar — Fri, 24 Apr 2009 10:05:07 -0800

Check out Swivel. It's an interface to upload data sets and generate various displays of the data, but you can usually go backwards and just download the data set.

By: davar

davar — Sat, 25 Apr 2009 03:05:03 -0800

The delicious datasets tag is also interesting.

By: thaths

thaths — Tue, 28 Apr 2009 16:31:24 -0800

Just saw Google's blog post about a search interface to public data. Might be useful to you. Also look at Google Visualization toolkit (esp. the pieces from Trendalyzer) for visualizing how data changes over time. The Hans Rosling talk mentioned by a previous commenter uses Trendalyzer visualizations.