Quantified self

December 15, 2015 9:29 PM

I collect a lot of data about myself. I want to aggregate all this stuff in one place for personal use.

I use RescueTime, Runkeeper, a Withings scale, a FitBit, and I make posts on Twitter and occasionally Facebook. I'll probably collect other data about myself as well at some point.

I am comfortable with computers, but not comfortable enough that I really know where to start. First, I would like to be able to get all this data in one place.

Then, I want to have a system to allow me to visually compare data from different sources over time, so that I can see, for example, that on days that I run on my treadmill for 1 hour I also work more productively. It would probably also be nice if I could integrate data from other places, like the weather, or even the top news headline.

I'm seeing this as sort of a 21st century diary, so I also the data to be archivable and formatted for long-term access and use.

I'm happy to spend a lot of time figuring this out, learning to code and hacking away at things. I know some very basic things in html, perl, javascript, and so on. I do not know Python or Ruby. I am definitely willing to learn, but I would like to learn things specifically in relation to this project. I imagine Python would be good for this, for example, but I want to learn the stuff I need to know to get data from other sources and aggregate and display it in one place. I find the generic "101" stuff to be not focused enough on my own goals to keep my interest.

So, where do I start? Does this already exist out there somewhere? If not, how do I build it?

posted by sockermom to Computers & Internet (5 answers total)

Response by poster: Oh! I also use Sleep Cycle on my phone to log sleep data, and would like to integrate this as well.
posted by sockermom at 9:31 PM on December 15, 2015

Old school: I think you can probably export the data to .csv files, make tables using the headings that appear*, use the date for each entry as the unique identifier for each table, and throw them into a database like Microsoft Access. Then, construct queries to relate information bound to the date from different tables. Last time I used Access, the Help tutorials were pretty useful, and I'm sure there are YouTube videos as well.

*If the names of some headings are repeated in different tables, change one of them slightly, to avoid possible issues.

Data export info:
Runkeeper
RescueTime
Withings
FitBit
Twitter (Quora page, not sure if current)
FB (haven't tried this, you may find something easier)
posted by cotton dress sock at 9:46 PM on December 15, 2015

(Here's one for SleepCycle.)

Looks like ZenoBase is a popular tool people use for the aggregation and analysis.
posted by cotton dress sock at 9:49 PM on December 15, 2015 [2 favorites]

You can use IFTTT to automatically transfer at least some of these data into Google drive spreadsheets and manipulate it from there. FitBit and WiThings for sure.
posted by deludingmyself at 5:23 AM on December 16, 2015 [1 favorite]

You should check https://gyrosco.pe/. They aggregate all the data you are collecting.
posted by cl3m at 12:19 AM on December 17, 2015