I need to get part of a website onto a USB sick at will

October 24, 2007 11:15 AM

I need to get part of a website onto a USB sick at will

I have a client for my web development biz that has a website in ASP (classic) and SQL Server Express. The old version of the site had some method where certain customers of this client, who had been given a USB stick by my client, could hit a certain page on the site and it would prompt them for a download location, and they could choose the USB stick and then it would download a certian part of the website to the stick as a self contained little site that they could run on their computer from the stick.

I understand being able to open a webpage from a USB stick and browse the "local" site that is on the stick, no problem. But my question is how to make static, local, browseable pages out of a dynamic SQL server based site?

The site has a lot of products by a lot of manufacturers, and the pages that are downloaded are for a specific manufacturer. So its pulling a subset of info from the db and building out those pages and saving them as static ones.

posted by Chuck Cheeze to computers & internet (7 comments total)

I found a good solution to this problem, a free windows program called HTTrack. You give it a URL, and it downloads the entire site into a folder.

I've not actually used it for a few months, but as i remember, it even converts dynamic querystring links (eg. page.asp?p=312432) to proper html files, and fixes the navigation. It was a really useful program.

Although it's not an ideal solution for you, you could set up a scheduler on the server to run it every day, and then zip up the folder. Then you could have a link to the zip file on your website, saying something like "Click here to download an offline copy of this site".....
posted by derbs at 11:42 AM on October 24, 2007

This is trivial to do using the command-line program 'wget' which is available for pretty much every OS (I think there are even GUI versions for Windows/Mac, and I suspect that a lot of offline browsing tools are built around it).

It has a fairly complex syntax and can do a lot of stuff (including random waits between pages, spoofing client-types, etc.) but in its most basic forms it will pull down pages, turn them into static HTML, and rewrite their links to reflect the local hierarchy so you can browse them.

Particularly if you want to script something that will work as part of a bigger system, I think it's what you want.
posted by Kadin2048 at 11:58 AM on October 24, 2007 [1 favorite has favorites]

Uh, it seems like the OP is looking for something web-based.

So, options....

- Just let them download normal, rendered web pages. Create a page in ASP, and change the HTTP headers ("application/octet-stream" might work) so that browsers will, by default, download instead of view the page.

- Make a special page with some AJAX so that it contains multiple pages, and do the same header trick

- Make a page which is a MHTML page so it'll be self-contained but will only work in the devil's browser.



Notes: It doesn't matter if it's a USB stick or anything else, it's just download pages, from what it looks like. You could also, if you were crazy, package a small web server in the download, but that probably doesn't make sense.
posted by tmcw at 1:04 PM on October 24, 2007

Does not need to be web-based actually. PC based is the need- and this could even be something that saves the pages as PDFs - I'm looking into wget now.

Thanks all so far
posted by Chuck Cheeze at 1:10 PM on October 24, 2007

It seems like an overly complicated way to connect with customers instead of a simple web-based store site. Was the owner of the company selling access to these thumb drives as an exclusive thing? (were the thumb drives themselves part of the business income?)
posted by mathowie at 2:46 PM on October 24, 2007

Hey Matt nice to hear from you- this is simply a way for my client to give their regularly updated product catalog (the website is the most regularly updated catalog) to the customer's sales reps and knowing that the reps have the client's catalog easily at hand, whether they have an internet connection or not.

The client's customer accounts for 60% of their business so when the customer asks for something, the client usually tries to oblige. I am guessing the customer gets this sort of thing from other vendors.
posted by Chuck Cheeze at 10:46 AM on October 31, 2007

I found HTTrack and its a perfect solution. Seems like a WGet type program with a nice GUI. It was able to download my site and make all links recursive, and it worked beautifully. Also has a cool update feature.
posted by Chuck Cheeze at 2:13 PM on November 7, 2007