How to display 300+ GPX cycling routes over a Google Maps base layer

December 2, 2013 11:31 AM

GPX Visualization on Google Maps: I have 300+ GPX files of bike rides that I'd like to be able to link to somehow over a Google Maps base layer, visually approximating this. This is related to my previous AskMe about cycling every road in a town; I need to visualize what roads I have/haven't ridden, and link that progress to a blog. Would like it to be as low-weight as possible; the 300 (uncompressed) GPX files amount to 100+mb currently (route+track+waypoints) and is only getting larger -- vanilla GMaps doesn't play nice with that many files. Much more information inside...

[If there's a smarter way to approach any of this, I'm all ears. I have very little idea what it is I'm doing here, and if you can correct any wrong assumptions/bad concepts on my part you'd be doing me a huge kindness; so fire away.]

So there used to be a site ('Multi-Ride Mapper') that worked with the Strava API that would display all of a user's rides. It was made by a guy named Jonathan O'Keefe, and it would produce a map that looked like this (while retaining the full GMaps functionality). It was low-weight and loaded quickly, and pulled the data off of the Strava API. Here's the HTML source for the site in question

But Strava changed (and then re-changed) its API. Now I can't link to a map of my rides to someone who isn't a Strava member and signed in. This sucks, and I'd like to recreate that functionality.

SO, I have the GPX files (and, also, the Garmin FIT files, if it makes a difference). They're heavy; in total the 300 rides in GPX are around 100mb. They're hosted on my server; here's a link to the zip file of all rides (5mb, ZIP file) if it's of any use.

I'm a babe in the woods, and I'm not sure what I need to do to do this; but what I think I'd like done approximates:

1. Pointing google maps to a sub-directory on my website where all the GPXs live (eg .../strava/gpxs/)
2. Not sure what happens this step; possibly magic
3. Ride overlays of, say, 2px in an opaque color are overlayed on a Google Map base layer (visually approximating this)

Does this require use of the Google Maps API? What's the most hand-holdingest way to walk me through setting such a thing up? I've researched this for several hours now and actually feel dumber about this than when I started.

Other stray questions (I'm sure I'll have more):

1. How do I (and: 'do I need to') reduce the size of the (100+mb, total) GPX rides, and what data can be stripped while preserving the visualization? How is this best done?
• I know I don't need 'heart rate' or 'cadence' info, but I'm not sure if I can strip one or more of the following: route, track, waypoints. Not sure which would be most useful to strip, size-wise, or indeed how this would even be done.
2. How would I create a 0.4km "halo" of anonymity around my house

Again, J. O'Keefe's (amazing) Multiple Ride Mapper used to do this great. Through no fault of his, the Strava API neutered the functionality -- I can't link to non-strava members. His site gets/got the data from the Strava API. Looking to cut Strava out of the loop here, and use my data how I'd like.

I'm very sure I don't know enough about what I'm asking to have included all the pertinent information, and I welcome the chance to clarify anything I've left unclear. Many, many thanks for your time and your expertise.

posted by jjjjjjjijjjjjjj to Technology (14 answers total)

Does the author of the multi-map thing know it's broken? Seems like something that he could fix on his end.

Strava itself has a "heat map" thing that miiight be suitable. Just takes a second, try it out on their site.
posted by RustyBrooks at 11:40 AM on December 2, 2013

Response by poster:

> Does the author of the multi-map thing know it's broken?

The problem is is that it's not really 'broken' (at least in any fixible way); Strava API v2 and v3 has removed public access to that data. The author knows about the changes (and has worked several times to restore what functionality he can), but it's taken away, now, and the access to the data required to make it work like it did previously is gone.

A heel-turn quasi-shady move on Strava's part, to be sure; but they've thrown their lots in with denying their users access to their created data. The backlash was strong, but died down quickly. It ain't coming back.

> Strava itself has a "heat map" thing that miiight be suitable.

Yeah, I was real excited for a second when they rolled that out about 3 months ago, but it's a total flop. It looks like shit, and looks worse once you drill down to whatever zoom level they've capped it at. I can't tell much from that about which roads I've been on.

In summary, I guess it does passably well on the "heat" part, and terribly bad on the "map" part.
posted by jjjjjjjijjjjjjj at 11:55 AM on December 2, 2013

I understand the changes to the API. But this guy doesn't have to approach it as "a member of the public". He could have an account for his app and log in with it. I've done that kind of thing in lots of different web sites that added similar restrictions.

Sorry the heat map sucks. I'd never really checked it out.

I took one shot at making something with the google maps API, got kinda frustrated by it, and quit. Wish I knew an app that would work for you but I can't really think of one.
posted by RustyBrooks at 12:05 PM on December 2, 2013

Response by poster:

> But this guy doesn't have to approach it as "a member of the public". He could have an account for his app and log in with it.

I'm trying to display my data to the public, though. I have the GPX data and want to display it over a (GMaps or OpenStreetmap) basemap while placing no restrictions on who can see it.

I appreciate the suggestions and the effort, though; and I sympathize with your frustrations at using the gmaps API... it was a fish-out-of-water thing for sure. I feel a bit better knowing that it wasn't just me, so thanks for that.
posted by jjjjjjjijjjjjjj at 12:12 PM on December 2, 2013

I would pose this question to the guy who built VeloViewer.com because I remember him writing some stuff on his blog about possibly building a heatmap visualization. I've kicked some emails around with him and he is very cool and enthusiastic about this stuff.
posted by dgran at 12:27 PM on December 2, 2013

You should be able to do this with leaflet and a bit of work.
posted by ludwig_van at 12:49 PM on December 2, 2013 [1 favorite]

To your 0.4km "halo" of anonymity around your house, you need GPSBabel. As per these instructions, you could do (assuming you live at 38.61,-90.25):

 gpsbabel -i gpx -f infile.gpx  -x radius,distance=400m,lat=38.61,lon=-90.25,nosort,exclude -o gpx -F outfile.gpx 

You might wish to add a few random exclusion points near your house. Your tracks have enough waypoints that it would be easy to back-calculate where you live.

Leaving only tracks with time and elevation, I can get the files down to ⅓ of the original size. If you wanted smaller, there are ways of filtering and smoothing tracks to reduce points in gpsbabel, but these might lose some of the detail you needed to capture every turn in every road.

For web display, if tmcw shows up in this thread: what he said, by way of pre-emptive seconding ☺
posted by scruss at 1:19 PM on December 2, 2013 [2 favorites]

Experiment 1: Google Maps
1. Go to Strava activity, export as GPX
2. Convert GPX to KMZ via gpsvisualizer.com
4. Right-click on the output kmz file "Copy Link Location".
5. Open Google Maps. Paste the kmz file link in google maps search box.

Presumably the GPSVisualizer link won't live forever, so you need to move your KMZ files to some hosted server eventually.

I can't find a way to display multiple kmz files on google maps, but a kmz file can contain multiple tracks. See this example.

Here are my kmz output files (these links probably won't stay working long):
http://www.gpsvisualizer.com/display/20131202124837-21774-map.kmz
http://www.gpsvisualizer.com/display/20131202130515-23128-map.kmz

View them in google maps (probably won't stay working long):
- ex1, ride1
- ex1, ride2


Experiment 2: Google Earth
1. Download google earth
2. Translate GPX to KMZ as above
3. Save the output kmz files to your computer.
3. Load one or more kmz files into Google earth

This option is nice if you want to have all your data off-line. Google earth will display multiple kmz files (I tried it with my two files above).

There are probably many ways to translate from GPX to KMZ. I used GPSVisualizer.com because it was convenient. Could probably download a utility that will do them in bulk if you want...
posted by sarah_pdx at 1:27 PM on December 2, 2013 [1 favorite]

ludwig_van mentioned leaflet, which is the right answer. But if you aren't crazy about coding javascript and websites and all that, have a look at http://geojson.io/.

I only have two rides on strava, but here they are.
posted by kjell at 4:28 PM on December 2, 2013 [1 favorite]

And if you're at all OK with using command line tools:

https://github.com/mapbox/togeojson will convert the gpxes to geojson.
then https://github.com/mapbox/geojsonio-cli will upload it to geojson.io

example! It will look something like this: npm install -g togeojson geojsonio && togeojson 12_09_2012.gpx | geojsonio

(You need to have something called node.js installed for either of those to work.)
posted by kjell at 4:48 PM on December 2, 2013 [1 favorite]

I'm so glad you asked this question, I had wondered about doing this and you were my push to do it :)

I wrote up how I did it using GPSBabel and leaflet, both mentioned above. You can see my notes and a leaflet map here. I didn't use Google Maps. Leaflet uses OpenStreetMap.

I only included three runs in my map. I did load all 144 runs I did in 2013 into one GPX file. It took several seconds to load on my beefy computer. I think GPSBabel and maybe the GPX plugin for leaflet have the option to simplify map files, but that's beyond what I had time to figure out. I also think you could put each activity onto a layer so you could pick and choose what to show. But I didn't really have time to show that either.
posted by tayknight at 6:59 AM on December 3, 2013 [3 favorites]

All your rides.
posted by kjell at 7:07 AM on December 3, 2013 [1 favorite]

I saw something (http://quantifiedself.com/2013/11/qs-gallery-eric-jain/) this morning that looked like it might be useful along these lines. Someone did this for his outdoor hiking. If you set the map to use the regular view instead of terrain it would show the roads.
posted by dgran at 5:31 AM on December 4, 2013

Adding this late, I know, but I just wanted to sneak in ptrv/gpx2spatialite, which is a fast and powerful way of maintaining a database of track logs.
posted by scruss at 3:55 AM on August 30, 2014 [1 favorite]