Best way to download several hundred Wikipedia articles?

January 10, 2011 10:03 PM

What's the best way to download several hundred articles (including images) from Wikipedia for offline reading?

I want a local copy of Wikipedia's articles on networking protocols so I can read them without Internet access (e.g. on a plane). This totals about 300 articles — too many to download manually with a browser and not nearly enough to warrant downloading the whole static archive. Besides, the static archive lacks pictures, and the diagrams in these articles are pretty important.

Using a tool like curl or wget would be perfect (and I know how to use them), but I want to respect their request not to do it that way.

Any ideas on the best approach to download a few hundred complete articles?

posted by tomwheeler to Computers & Internet (4 answers total)

Best answer: Use the book function on the left-hand menu. Add all articles you want, and then export them as a PDF or something.
posted by Harald74 at 10:09 PM on January 10, 2011 [2 favorites]

Their concern about crawlers seems to relate to the number of articles and the delay. If you use wget with a large delay between requests (e.g. --wait=45) and you limit the crawl depth so that you're really only getting those 300 pages (e.g. starting it on a category page and setting --level=1), then I don't think they'd have a problem with that.

I don't really see fundamentally how there can be any other options than downloading the entire dump or respectfully crawling a smallish number of pages.
posted by Rhomboid at 11:11 PM on January 10, 2011 [1 favorite]

I agree with Rhomboid. Just don't be aggressive about it & you won't make anybody unhappy. They don't really care if there's a person sitting at the console, they care about system load. You can be polite but still automate the process.
posted by scalefree at 12:01 AM on January 11, 2011

Response by poster: I didn't even know about Wikipedia's book function — very handy! It turns out that once you've started creating a book, you can mouse over a link to add that page to it. In the end, it was an easy task to just add the links this way.
posted by tomwheeler at 6:59 AM on January 11, 2011 [3 favorites]