How to get started with free GIS tools
December 6, 2017 6:18 AM Subscribe
I've been thinking about trying to dabble with GIS in an urban setting. I'm not going to fork out for expensive tools, so I'm looking for free ones. I'm asking this as a complete noob - so I probably don't even know the right questions to be asking here - but my initial requirements are inside. Thank you!
I live in a large city. I'd like initially to be able to attempt watershed modeling things like:
- model the underlying topology, for instance in terms of contours (maybe import USGS or LIDAR files?)
- map how things like rivers and riverbanks in the city - as portrayed in other documents, such as historic maps and aerial photos, for example - have changed over time
- for example, how have rivers have been channelized between embankments, and rivers and streams been covered over?
And also
- import and represent other data sets (??not sure how to phrase this??) - for instance street maps, and census data
- learn/adopt open data standards, convert from proprietary standards
General 'would be nice for me' stuff includes:
- open data standards
- user community
- documentation
- online tutorials/videos
Also - related - I was looking at data standards. I'm assuming that the more common standards (such as KML??) are supported by more tools. I took a quick look at the opengeospatial.org site, will look further, is there any key standards documentation that I should be looking at?
One aim of this is to write an article or two for local history and planning blogs, on these topics. I'm just interested in this. I did check this previous question.
Thanks again!
I live in a large city. I'd like initially to be able to attempt watershed modeling things like:
- model the underlying topology, for instance in terms of contours (maybe import USGS or LIDAR files?)
- map how things like rivers and riverbanks in the city - as portrayed in other documents, such as historic maps and aerial photos, for example - have changed over time
- for example, how have rivers have been channelized between embankments, and rivers and streams been covered over?
And also
- import and represent other data sets (??not sure how to phrase this??) - for instance street maps, and census data
- learn/adopt open data standards, convert from proprietary standards
General 'would be nice for me' stuff includes:
- open data standards
- user community
- documentation
- online tutorials/videos
Also - related - I was looking at data standards. I'm assuming that the more common standards (such as KML??) are supported by more tools. I took a quick look at the opengeospatial.org site, will look further, is there any key standards documentation that I should be looking at?
One aim of this is to write an article or two for local history and planning blogs, on these topics. I'm just interested in this. I did check this previous question.
Thanks again!
Best answer: QGIS is in a period of transition, about to jump from version 2 to 3. QGIS 3 will have an improved interface, and adds a bunch of neat stuff, but final features and documentation are still being nailed down.
posted by zamboni at 6:49 AM on December 6, 2017 [1 favorite]
posted by zamboni at 6:49 AM on December 6, 2017 [1 favorite]
Best answer: QGIS is an excellent way to get started, it's sort of the OpenOffice of GIS tools. (ESRI ArcGIS is the Microsoft Office commercial consensus tool). The UI is not awesome in QGIS 2 but it's serviceable. I wouldn't wait for QGIS 3, just get going.
Another angle to GIS, how I got started, is with Javascript maps. If you already know some basic web programming it may feel natural to you. I recommend starting with either Leaflet or Mapbox. Tangram is also worth a look.
The key thing with free GIS work is you need free data. This list of data sources is a good place to start. OpenStreetMap is indeed excellent, and MapZen's Metro Extracts are a way to download a manageable chunk of data in an easy-to-use format. Natural Earth is another excellent data source, particularly for large scale low-resolution data.
You asked about data formats. The most important formats are ESRI Shapefile and GeoJSON for vector data and GeoTIFF for raster data. Most tools will just load those with no problem. There's a zillion other vector and raster formats out there, like KML is popular for non-specialist data because Google Earth is such a nice tool. QGIS can handle most formats. One last thing you'll want to learn about is map tiles, used in web maps as a way to stream data while a user pans and zooms around a map. There's a lot of formats for what go in the tiles.
You also asked about watersheds; there is a lot of watershed data out there. I've done a lot of work with NHDPlus which is very nice for basics of streams. USGS is a good place to start too.
There's a huge amount of GIS work going on, it's somewhat overwhelming at first. My advice is to pick a very simple project of local interest to you and try to make a good map of it. "Map all the streams in my town from original data sources" is a good if ambitious example.
posted by Nelson at 7:04 AM on December 6, 2017 [7 favorites]
Another angle to GIS, how I got started, is with Javascript maps. If you already know some basic web programming it may feel natural to you. I recommend starting with either Leaflet or Mapbox. Tangram is also worth a look.
The key thing with free GIS work is you need free data. This list of data sources is a good place to start. OpenStreetMap is indeed excellent, and MapZen's Metro Extracts are a way to download a manageable chunk of data in an easy-to-use format. Natural Earth is another excellent data source, particularly for large scale low-resolution data.
You asked about data formats. The most important formats are ESRI Shapefile and GeoJSON for vector data and GeoTIFF for raster data. Most tools will just load those with no problem. There's a zillion other vector and raster formats out there, like KML is popular for non-specialist data because Google Earth is such a nice tool. QGIS can handle most formats. One last thing you'll want to learn about is map tiles, used in web maps as a way to stream data while a user pans and zooms around a map. There's a lot of formats for what go in the tiles.
You also asked about watersheds; there is a lot of watershed data out there. I've done a lot of work with NHDPlus which is very nice for basics of streams. USGS is a good place to start too.
There's a huge amount of GIS work going on, it's somewhat overwhelming at first. My advice is to pick a very simple project of local interest to you and try to make a good map of it. "Map all the streams in my town from original data sources" is a good if ambitious example.
posted by Nelson at 7:04 AM on December 6, 2017 [7 favorites]
Also, check what GIS resources your local or federal government provides. Sites like New York State's GIS catalog makes it pretty easy to drill down to local data of interest.
posted by zamboni at 7:14 AM on December 6, 2017 [1 favorite]
posted by zamboni at 7:14 AM on December 6, 2017 [1 favorite]
Response by poster: Thanks all so far - this is super useful! - I have no programming experience as such, but do have some basic dataset experience, e.g. with metadata, schemas, formatting, basic XML, etc. Also I'm on a Mac but this does not seem to make a difference these days.
Yes am probably not going to start with all the watersheds. I was thinking of looking at short stretch of a river downtown - maybe 200-300 meters - it used to be wharves, etc., and is now a highway and parks. Gives me an excuse to get down to the city archives!
posted by carter at 7:22 AM on December 6, 2017
Yes am probably not going to start with all the watersheds. I was thinking of looking at short stretch of a river downtown - maybe 200-300 meters - it used to be wharves, etc., and is now a highway and parks. Gives me an excuse to get down to the city archives!
posted by carter at 7:22 AM on December 6, 2017
I'd start with qGIS, I'm not entirely sure but you should be able to input shapefiles which you can find for free on government websites. Search for "whatever state you're in" geospatial clearing house, most states have some form of data repository.
posted by carnivoregiraffe at 7:37 AM on December 6, 2017 [1 favorite]
posted by carnivoregiraffe at 7:37 AM on December 6, 2017 [1 favorite]
Best answer: whatever you do for local work...
a) get an esri account and use arcgisonline to understand geo web services and clients
b) get a mapbox account for the same reason, but widely differing approach.
lots of samples and freely published data on both services.
posted by j_curiouser at 9:42 AM on December 6, 2017 [1 favorite]
a) get an esri account and use arcgisonline to understand geo web services and clients
b) get a mapbox account for the same reason, but widely differing approach.
lots of samples and freely published data on both services.
posted by j_curiouser at 9:42 AM on December 6, 2017 [1 favorite]
Best answer: I was looking at data standards. I'm assuming that the more common standards (such as KML??) are supported by more tools. I took a quick look at the opengeospatial.org site, will look further, is there any key standards documentation that I should be looking at?
Depends on what kind of data standards you're talking about - how files and data are formatted, or how the data is entered. Here's another forum (School of Data's "ask" subsite ;)) discussing KML vs shapefile. But if you're looking to know about how data should be collected and entered, that differs from project to project, agency to entity, and tends to get down to how much time people have for accuracy, as well as how the data will be used. For example, if you're creating data for an entire state, you might be happy to pin things down to the general vicinity for boundaries, where as if you're mapping sidewalks in a neighborhood, inches can matter.
And here's some more (generally) free data to chew on: GIS data sources, identified by the Federal Highway Administration (FHWA). Again, if the data source is a national dataset, regional accuracy can be rough, but free's free.
(Note: I'm a GIS noob who probably knows enough to get into trouble, but not enough to speak with real authority)
posted by filthy light thief at 11:13 AM on December 6, 2017 [1 favorite]
Depends on what kind of data standards you're talking about - how files and data are formatted, or how the data is entered. Here's another forum (School of Data's "ask" subsite ;)) discussing KML vs shapefile. But if you're looking to know about how data should be collected and entered, that differs from project to project, agency to entity, and tends to get down to how much time people have for accuracy, as well as how the data will be used. For example, if you're creating data for an entire state, you might be happy to pin things down to the general vicinity for boundaries, where as if you're mapping sidewalks in a neighborhood, inches can matter.
And here's some more (generally) free data to chew on: GIS data sources, identified by the Federal Highway Administration (FHWA). Again, if the data source is a national dataset, regional accuracy can be rough, but free's free.
(Note: I'm a GIS noob who probably knows enough to get into trouble, but not enough to speak with real authority)
posted by filthy light thief at 11:13 AM on December 6, 2017 [1 favorite]
No longer being updated, so I imagine some of the info and links will be starting to go off, but Free Geography Tools was a big help to me in my various mapping and GIS explorations.
posted by snoboy at 11:36 AM on December 6, 2017
posted by snoboy at 11:36 AM on December 6, 2017
A few years ago, I downloaded every Shapefile that my county offered, which included parcel maps and USGS hydrology (streams), took a thumb drive down to my city offices and got that loaded up with everything the city GIS person would give me, loaded those into PostGIS, used Mapnik to render PostGIS queries, and found all sorts of coolness. Now I work in the field, with those tools.
These days, QGIS can point to a PostGIS layer, which makes the layer rendering a heck of a lot easier. As everyone else is suggesting, get QGIS, learn the basics of how to load layers, adjust projections, that sort of thing.
Then start pulling the Shapefiles in through PostGIS and you'll find you can do the most amazing things with the data: I started with "show all the parcels within N meters of a stream", and went to town from there.
Also, GIS people often love to talk about mapmaking and data. Find those people in your area. They're using ESRI, not free tools, but at least in my area they love to share data sources.
posted by straw at 1:51 PM on December 6, 2017
These days, QGIS can point to a PostGIS layer, which makes the layer rendering a heck of a lot easier. As everyone else is suggesting, get QGIS, learn the basics of how to load layers, adjust projections, that sort of thing.
Then start pulling the Shapefiles in through PostGIS and you'll find you can do the most amazing things with the data: I started with "show all the parcels within N meters of a stream", and went to town from there.
Also, GIS people often love to talk about mapmaking and data. Find those people in your area. They're using ESRI, not free tools, but at least in my area they love to share data sources.
posted by straw at 1:51 PM on December 6, 2017
Can you program in Python at all? If you can, you can manipulate shape files/layers in just about any way you can conceive of programmatically, and mash up with other datasets like the Census, and do even more sophisticated stuff like geospatial stats, animation, etc. using the free library Geopandas https://automating-gis-processes.github.io/2016/Lesson2-geopandas-basics.html
This is definitely the hardest possible route unless you already program to some degree but affords possibilities that tools like QGIS don't offer
posted by slow graffiti at 2:27 PM on December 6, 2017 [1 favorite]
This is definitely the hardest possible route unless you already program to some degree but affords possibilities that tools like QGIS don't offer
posted by slow graffiti at 2:27 PM on December 6, 2017 [1 favorite]
Excellent answers here. I'll only underline the general awesomeness of the datasets provided by both the US Census and by your state/province/locality's own planning and GIS office, who are likely doing yeoman's work making this data available.
posted by Emperor SnooKloze at 6:55 PM on December 6, 2017
posted by Emperor SnooKloze at 6:55 PM on December 6, 2017
« Older Dog displayed food aggression for the first time | Like TED, but you can be in your pyjamas the whole... Newer »
This thread is closed to new comments.
One useful thing to know is that data in GIS tends to be either rasters or vectors. Rasters contain a regular grid of pixels representing some data value, like an image file. Orthophoto coverage or Digital Elevation Models (DEMs, where the value of each pixel is the elevation above some datum). Vectors contain points, lines, and shapes, and are used to describe things like roads, rivers, property lines, building positions, etc.
An example: you can download vectors representing streets, building footprints, etc from OpenStreetMap, though be aware that there are license restrictions as to how you can use/re-share the data.
I wish I knew more to help out -- in general my work deals with querying existing GIS databases and rendering the results on top of other data in tools my team builds, rather than creating new data or integrating data in the GIS tools.
posted by Alterscape at 6:33 AM on December 6, 2017 [1 favorite]