Comments on: I need to get part of a website onto a USB sick at will

Question: I need to get part of a website onto a USB sick at will

Chuck Cheeze — Wed, 24 Oct 2007 11:15:21 -0800

I need to get part of a website onto a USB sick at will

I have a client for my web development biz that has a website in ASP (classic) and SQL Server Express. The old version of the site had some method where certain customers of this client, who had been given a USB stick by my client, could hit a certain page on the site and it would prompt them for a download location, and they could choose the USB stick and then it would download a certian part of the website to the stick as a self contained little site that they could run on their computer from the stick.

I understand being able to open a webpage from a USB stick and browse the "local" site that is on the stick, no problem. But my question is how to make static, local, browseable pages out of a dynamic SQL server based site?

The site has a lot of products by a lot of manufacturers, and the pages that are downloaded are for a specific manufacturer. So its pulling a subset of info from the db and building out those pages and saving them as static ones.

By: derbs

derbs — Wed, 24 Oct 2007 11:42:06 -0800

I found a good solution to this problem, a free windows program called HTTrack. You give it a URL, and it downloads the entire site into a folder.

I've not actually used it for a few months, but as i remember, it even converts dynamic querystring links (eg. page.asp?p=312432) to proper html files, and fixes the navigation. It was a really useful program.

Although it's not an ideal solution for you, you could set up a scheduler on the server to run it every day, and then zip up the folder. Then you could have a link to the zip file on your website, saying something like "Click here to download an offline copy of this site".....

By: Kadin2048

Kadin2048 — Wed, 24 Oct 2007 11:58:33 -0800

This is trivial to do using the command-line program 'wget' which is available for pretty much every OS (I think there are even GUI versions for Windows/Mac, and I suspect that a lot of offline browsing tools are built around it).

It has a fairly complex syntax and can do a lot of stuff (including random waits between pages, spoofing client-types, etc.) but in its most basic forms it will pull down pages, turn them into static HTML, and rewrite their links to reflect the local hierarchy so you can browse them.

Particularly if you want to script something that will work as part of a bigger system, I think it's what you want.

By: tmcw

tmcw — Wed, 24 Oct 2007 13:04:50 -0800

Uh, it seems like the OP is looking for something web-based.

So, options....

- Just let them download normal, rendered web pages. Create a page in ASP, and change the HTTP headers ("application/octet-stream" might work) so that browsers will, by default, download instead of view the page.

- Make a special page with some AJAX so that it contains multiple pages, and do the same header trick

- Make a page which is a MHTML page so it'll be self-contained but will only work in the devil's browser.

Notes: It doesn't matter if it's a USB stick or anything else, it's just download pages, from what it looks like. You could also, if you were crazy, package a small web server in the download, but that probably doesn't make sense.

By: Chuck Cheeze

Chuck Cheeze — Wed, 24 Oct 2007 13:10:39 -0800

Does not need to be web-based actually. PC based is the need- and this could even be something that saves the pages as PDFs - I'm looking into wget now.

Thanks all so far

By: mathowie

mathowie — Wed, 24 Oct 2007 14:46:51 -0800

It seems like an overly complicated way to connect with customers instead of a simple web-based store site. Was the owner of the company selling access to these thumb drives as an exclusive thing? (were the thumb drives themselves part of the business income?)

By: Chuck Cheeze

Chuck Cheeze — Wed, 31 Oct 2007 10:46:16 -0800

Hey Matt nice to hear from you- this is simply a way for my client to give their regularly updated product catalog (the website is the most regularly updated catalog) to the customer's sales reps and knowing that the reps have the client's catalog easily at hand, whether they have an internet connection or not.

The client's customer accounts for 60% of their business so when the customer asks for something, the client usually tries to oblige. I am guessing the customer gets this sort of thing from other vendors.

By: Chuck Cheeze

Chuck Cheeze — Wed, 07 Nov 2007 14:13:44 -0800

I found HTTrack and its a perfect solution. Seems like a WGet type program with a nice GUI. It was able to download my site and make all links recursive, and it worked beautifully. Also has a cool update feature.