Scraping an online locator?
April 26, 2013 2:14 PM   Subscribe

I am trying to scrape online locators that use google maps. I only know HTML and CSS, so, I am looking for programs or fairly simple workarounds.

Here is something similar to what I'm talking about, though this is not one of the sites I'd like to scrape. The user inputs their zip, and perhaps selects a few options and a radius, and then some sort of list of dealers or people within that radius is generated.

I have an "outline" of what I could do, but I am no brainiac and would have to research further to learn how to execute my plan, so I would really love some feedback from the experts on ask mefi.

1. Acquire list of all zip codes in US and import into excel. Then, make macro to generate a URL for every possible search of zips on the locator site I am using (EXAMPLE: www.hermanmiller.com/DealerLocator/&postalcode=02133 ; EXAMPLE: www.hermanmiller.com/DealerLocator/&postalcode=09190).

2. Make a scraper that combs through the right fields in the search return (business name, phone #, etc)

3. Make an automation that runs the scraper for every single URL in my excel file.

Can this be done in OutWit Hub? I have managed to make the scraper in the lite, version, at least, but have yet to make a zip list or automation.
posted by Halogenhat to Computers & Internet (4 answers total) 4 users marked this as a favorite
 
Unfortunately you’re going to have to learn some code, but Scraperwiki is made for you. You can create a new scraper and pull data from pages by CSS selector, which should roughly match your existing skillset.

I’m not sure what they offer for the multi-zipcode thing.
posted by migurski at 4:08 PM on April 26, 2013 [2 favorites]


Also, a list of all US zip codes (based on 2010 census ZCTA’s).
posted by migurski at 4:10 PM on April 26, 2013


Response by poster: Thanks, scraperwiki looks great and hopefully a few hours of learning will lead me somewhere.
posted by Halogenhat at 4:13 PM on April 26, 2013


Nthing Scraperwiki (and hey @migurski), but depending on the result pages you might also be able to do something in Google Docs which has some scraping tools.
posted by drwelby at 10:30 PM on April 26, 2013 [1 favorite]


« Older Can germs travel from cheek to mouth?   |   Help me organize my teaching life with Evernote Newer »
This thread is closed to new comments.