Mapping a city's anti-space to pedestrians
October 15, 2021 4:46 PM Subscribe
Through the pandemic I have been recording GPS tracks of exercise walks, now adding up to 272 tracks and 2895 km.
I am curious what I can do with them in GIS or data processing software. It's easy enough to roughly overlap them all in Google Earth Pro, but I would like to see a heat map of the probability of any particular area being visited. It also occurred to me that it would be neat to see a representation of the inverse of the pedestrian-walked area: the places which are obstructed to a person on foot.
I'd love some help getting started. Is there already freeware software available to do this, or am I looking at data manipulation using my own code? What software should I be looking at?
I can export the GPS tracks individually or as a set, in GPX, KML, and GeoJSON formats — which are presently meaningless to me beyond knowing that KML will open in Google Earth Pro.
I'd love some help getting started. Is there already freeware software available to do this, or am I looking at data manipulation using my own code? What software should I be looking at?
I can export the GPS tracks individually or as a set, in GPX, KML, and GeoJSON formats — which are presently meaningless to me beyond knowing that KML will open in Google Earth Pro.
Response by poster: I'd like to be able to tailor the distance around each point that gets 'presence probability' score, and essentially use some kind of GUI to play around with the data and look for interesting patterns.
Bonus for freeware and Mac software.
posted by sindark at 5:09 PM on October 15, 2021
Bonus for freeware and Mac software.
posted by sindark at 5:09 PM on October 15, 2021
Response by poster: These give a sense of what the tracks look like as a set:
https://www.sindark.com/share/2021-10-15-pandemic-walks-1.png
https://www.sindark.com/share/2021-10-15-pandemic-walks-2.png
posted by sindark at 5:09 PM on October 15, 2021
https://www.sindark.com/share/2021-10-15-pandemic-walks-1.png
https://www.sindark.com/share/2021-10-15-pandemic-walks-2.png
posted by sindark at 5:09 PM on October 15, 2021
If your dataset is just the traces, your heat-map will just be frequency of visits, not probability.
There are models that predict the probability of a space being visited based on its geometry and visual structure (sightlines, basically).
One such model is called Isovists, and there's software to perform the analysis, it basically assumes that spaces that are more visible from other spaces will receive more visits.
Here's a paper by Turner and Penn about it.
This all relates to the general theory of urban space known as Space Syntax "a science-based, human-focused approach that investigates relationships between spatial layout and a range of social, economic and environmental phenomena."
posted by signal at 5:26 PM on October 15, 2021
There are models that predict the probability of a space being visited based on its geometry and visual structure (sightlines, basically).
One such model is called Isovists, and there's software to perform the analysis, it basically assumes that spaces that are more visible from other spaces will receive more visits.
Here's a paper by Turner and Penn about it.
This all relates to the general theory of urban space known as Space Syntax "a science-based, human-focused approach that investigates relationships between spatial layout and a range of social, economic and environmental phenomena."
posted by signal at 5:26 PM on October 15, 2021
Response by poster: To get started, is there a way to decompose the tracks into just a big set of coordinates?
posted by sindark at 5:33 PM on October 15, 2021
posted by sindark at 5:33 PM on October 15, 2021
Well, you could read the geojson into python (for example) and have it join all the point coordinates into one long list, I guess?
posted by signal at 5:41 PM on October 15, 2021
posted by signal at 5:41 PM on October 15, 2021
Response by poster: Some useful information / ideas are here: Creating polyline-based "heatmap" from GPS tracks?
posted by sindark at 6:55 PM on October 15, 2021
posted by sindark at 6:55 PM on October 15, 2021
My experience is with a 10-year-old Garmin GPS, but I doubt things have changed.
The data saved for a track consists of points defined by Lat/Long and time. Time interval between points for my GPS is about 30 seconds. If the unit fails to get a new reading in the time interval, it will project the track straight ahead which can lead to some oddities.
The Garmin software will convert the track from their proprietary format to any of several common formats including GPS, which seems to be the most common, and a plain text version (.csv, I think) that can be opened by Excel.
You don't say you are a programmer. This could be rather fun doing the work yourself rather than searching out existing apps. The first thing would be an aggregation program to collect like points. The usual sort of thing is to look at every pair of points coming from different tracks, and find the two that are closest. Then the average of those replaces the originals. Repeat until the least difference between points gets too big to sensibly represent the same place. (Lots of details omitted.) The number of points aggregated is your heat index.
posted by SemiSalt at 5:44 AM on October 16, 2021
The data saved for a track consists of points defined by Lat/Long and time. Time interval between points for my GPS is about 30 seconds. If the unit fails to get a new reading in the time interval, it will project the track straight ahead which can lead to some oddities.
The Garmin software will convert the track from their proprietary format to any of several common formats including GPS, which seems to be the most common, and a plain text version (.csv, I think) that can be opened by Excel.
You don't say you are a programmer. This could be rather fun doing the work yourself rather than searching out existing apps. The first thing would be an aggregation program to collect like points. The usual sort of thing is to look at every pair of points coming from different tracks, and find the two that are closest. Then the average of those replaces the originals. Repeat until the least difference between points gets too big to sensibly represent the same place. (Lots of details omitted.) The number of points aggregated is your heat index.
posted by SemiSalt at 5:44 AM on October 16, 2021
Response by poster: If anyone knows how to use QGIS, I am 90% of the way to getting a heat map from the total collection of points. I just don't understand what it means by radius and why it insists on inappropriate units like milimeters or entire degrees of latitude and longitude. I think the problem relates in part to the coordinate systems used by the GPS app and the base maps you can add in QGIS.
No clue whatsoever about how to look at the untrallevel space rather than travelled space, except perhaps reversing the heat map in graphics software once it's made.
posted by sindark at 1:43 PM on October 16, 2021
No clue whatsoever about how to look at the untrallevel space rather than travelled space, except perhaps reversing the heat map in graphics software once it's made.
posted by sindark at 1:43 PM on October 16, 2021
My first guess on "radius" is that it refers to how close two points need to be to each other in order to be considered the same place.
posted by SemiSalt at 4:44 PM on October 16, 2021
posted by SemiSalt at 4:44 PM on October 16, 2021
Response by poster: For any radius I put, it either shows tiny little circles around each data point or undifferentiated rectangular blocks the size of half the city. Much of the time, it just starts a progress bar that never seems to end and leaves me with a blank white screen.
posted by sindark at 5:07 PM on October 16, 2021
posted by sindark at 5:07 PM on October 16, 2021
I've made heat maps from gps files before. I'm away from home but I'll try to remember to post more when I'm back or tomorrow. If you don't hear from me it's because I forgot so please PM me. I've done some other interesting stuff with gps that you might like to try
posted by RustyBrooks at 5:52 PM on October 16, 2021
posted by RustyBrooks at 5:52 PM on October 16, 2021
OK so for heatmaps I use a highly modified version of this
http://sethoscope.net/heatmap/
https://github.com/sethoscope/heatmap
The stock version is probably fine for your purposes, mine is modified to integrate better into a webserver environment and to add some features that I wanted. You can get it to be either very precise, or more diffuse and blobby. You need python and some familiarity with the command line to run it.
As far as graphing "not visited" places, heatmaps are both. Like if you have a black-to-orange heatmap where more orange means more visits, then less orange means less and black means none. All the black regions will be places you didn't go. You could reverse the fore/background colors if you wanted to make orange mean "less visited" of course.
I also did another project that was really fun - unfortunately none of this stuff is online right now because I moved hosts and didn't stand everything back up, but I could probably get stuff back online if you want to see it. This other project I started because I was going on medium length bike rides that included significant off-road portions in suburban areas. These would be, for example, mountain bike trails (official and non-official), hike and bike trails, maintenance right of ways, maybe cutting through parking lots etc.
I wanted a way to show the parts that were not on roads, and also I wanted to create new maps that included these so that you could do route-planning that included those off-road portions. So essentially I used openstreetmaps, and looked for segments in my GPS files that were not suitable close to any existing segments in OSM. This is a little fuzzy because GPS isn't perfect but I was able to find most of the off road bits this way.
posted by RustyBrooks at 6:28 PM on October 16, 2021
http://sethoscope.net/heatmap/
https://github.com/sethoscope/heatmap
The stock version is probably fine for your purposes, mine is modified to integrate better into a webserver environment and to add some features that I wanted. You can get it to be either very precise, or more diffuse and blobby. You need python and some familiarity with the command line to run it.
As far as graphing "not visited" places, heatmaps are both. Like if you have a black-to-orange heatmap where more orange means more visits, then less orange means less and black means none. All the black regions will be places you didn't go. You could reverse the fore/background colors if you wanted to make orange mean "less visited" of course.
I also did another project that was really fun - unfortunately none of this stuff is online right now because I moved hosts and didn't stand everything back up, but I could probably get stuff back online if you want to see it. This other project I started because I was going on medium length bike rides that included significant off-road portions in suburban areas. These would be, for example, mountain bike trails (official and non-official), hike and bike trails, maintenance right of ways, maybe cutting through parking lots etc.
I wanted a way to show the parts that were not on roads, and also I wanted to create new maps that included these so that you could do route-planning that included those off-road portions. So essentially I used openstreetmaps, and looked for segments in my GPS files that were not suitable close to any existing segments in OSM. This is a little fuzzy because GPS isn't perfect but I was able to find most of the off road bits this way.
posted by RustyBrooks at 6:28 PM on October 16, 2021
If you're not familiar with installing/running python programs, you could PM me and we could arrange something, I could generate a half dozen examples and you could say "I like #1 but with more X" and I could generate you some examples. I could probably set up a simple web page to do it if I get a few minutes.
posted by RustyBrooks at 6:29 PM on October 16, 2021
posted by RustyBrooks at 6:29 PM on October 16, 2021
Response by poster: That's very generous, but most of the point is to mess around with it myself — learn a bit about GIS, and see if anything interesting can be pulled from this data
posted by sindark at 6:38 PM on October 16, 2021
posted by sindark at 6:38 PM on October 16, 2021
Response by poster: Thanks to some significant help from RustyBrooks I have produced a couple of heat maps that look OK.
posted by sindark at 8:13 PM on November 4, 2021 [1 favorite]
posted by sindark at 8:13 PM on November 4, 2021 [1 favorite]
Response by poster: Related:
"The city is not without its backcountry-like challenges. Liz discovered urban “cliffing out,” which in nature means following a route to a ledge or drop-off where the only alternative appears to be turning around and back-tracking. In a city, that can mean unmapped dead ends or re-routes due to constructions."
https://www.mountaineers.org/blog/urban-speed-hiking-with-liz-thomas
posted by sindark at 3:14 PM on November 7, 2021
"The city is not without its backcountry-like challenges. Liz discovered urban “cliffing out,” which in nature means following a route to a ledge or drop-off where the only alternative appears to be turning around and back-tracking. In a city, that can mean unmapped dead ends or re-routes due to constructions."
https://www.mountaineers.org/blog/urban-speed-hiking-with-liz-thomas
posted by sindark at 3:14 PM on November 7, 2021
« Older Tips for addressing pregnancy pain (symphysis... | Buying a home for/with an aging parent - USA Newer »
This thread is closed to new comments.
posted by dmd at 5:00 PM on October 15, 2021