How do I screen scrap a changing website?
August 18, 2012 3:09 PM Subscribe
How do I save a (that is constantly getting updated) webpage automatically every time it changes so that I can parse the page later with a program? Note I am trying to save someone else's website that is being updated. I want to extract their data. Screen Scrapers I have seen don't do what I want and I am not sure what this feature is called if it exists.
posted by santogold to Computers & Internet (4 answers total) 1 user marked this as a favorite
I understand that I can use a packet sniffer and intercept and decode the incoming traffic but it seems much simpler that i could just save the html and process it later.
Are their any open source packages that might make this simpler?