Comments on: One-click download of websites

Question: One-click download of websites

Saucy Intruder — Thu, 08 Sep 2005 17:22:34 -0800

Help me get through long classes without Internet access - I want to click one button while I'm online and download all my newspapers, blogs, etc. into a flat file that I can view offline. Bloglines often doesn't give me the full article.

By: kdern

kdern — Thu, 08 Sep 2005 17:46:35 -0800

I've been playing with a new Firefox extension called Scrapbook - it allows you to save and organize web pages very quickly. But I think you'd have to open each page to do so. I plan to use it to save things to read for long flights.

By: Saucy Intruder

Saucy Intruder — Thu, 08 Sep 2005 18:28:36 -0800

As someone who has spent nineteen years in school, I would say that good teachers are unthreatened by technology and trust their students to do what helps them learn best and which isn't disruptive to other students.

By: Saucy Intruder

Saucy Intruder — Thu, 08 Sep 2005 18:43:36 -0800

I am now playing with the Scrapbook extension, but I wonder if there is a way to automatically mine a website - for example, to grab the Askme front page and then all the threads within it.

By: jessamyn

jessamyn — Thu, 08 Sep 2005 19:00:41 -0800

[a few comments removed, if you'd like to discuss the ethics of reading blogs during class, please take it to metatalk]

By: nicwolff

nicwolff — Thu, 08 Sep 2005 19:12:58 -0800

If you're using Windows, I'm pretty sure this is what HTTrack does.

By: nicwolff

nicwolff — Thu, 08 Sep 2005 19:13:58 -0800

Oh look, HTTrack is actually for Unix too, including OS X.

By: kdern

kdern — Thu, 08 Sep 2005 20:07:42 -0800

In Scrapbook, it looks like you can highlight a section of text that includes links and right click, select "capture as" and click on "save all pages sequentially". It saves more pages than I'd like, but it's worth experimenting with. I like programs that work within Firefox.

By: NucleophilicAttack

NucleophilicAttack — Thu, 08 Sep 2005 21:04:35 -0800

Can't curl or wget be made to grab stuff recursively? A bit of coaxing with shell or perl might be necessary, but should be quite doable. Take the HTML that results, and use any of the HTML tag stripping utilities ou there.

Stick the script in cron. Profit.

By: webmeta

webmeta — Thu, 08 Sep 2005 23:21:25 -0800

Maybe Newsgator ?

By: catachresoid

catachresoid — Fri, 09 Sep 2005 05:53:24 -0800

I would consider the Slogger extension to be preferable to Scrapbook for this purpose. Autologging, good indexing of pages, highly configurable.

By: AmbroseChapel

AmbroseChapel — Sun, 11 Sep 2005 03:09:15 -0800

>Can't curl or wget be made to grab stuff recursively?

wget definitely can, with no coaxing at all. I think it's as simple as putting -r as an option.