Super Slow Internet!! Is there a way to automatically store/autosave webpages?

August 5, 2011 9:09 AM

Ok so I'm overseas for awhile, horrible internet; is there a way store webpages automatically so I don't have to reload them next time? I view page, what i see is saved?? Is it possible?

In short: overseas, terrible internet (56k modem sounds good sometimes, im not kidding) bandwidth limit of 10gb a month)

So when i look up stuff online, it takes awhile.. and if i want to look at it again, i need to reload it.. which takes forever. I know I can save pages individually on command, or use firefox's scrapbook extension or httrack to download linked pages; but what I'd like is something that saved each page as i view them.

I know it might take up a bit of space, but I have space. I don't have bandwidth (only 10gb a month.. 0.o); so if im loading the page once i'd like to have it save to the drive so i can get to it again. Is there a browser or extension that does this?

posted by rmathew1 to computers & internet (10 answers total)

In Firefox, "File->Save Page As" has a pulldown over on the lower right that lets me save "Web Page, Complete".
posted by straw at 9:11 AM on August 5, 2011

@straw tried that first :) but need something that works in the background, without user intervention. Basically an automatic Save As for each page viewed.
posted by rmathew1 at 9:17 AM on August 5, 2011

Google Desktop says that it saves webpages that you view automatically.
posted by oranger at 9:24 AM on August 5, 2011

I don't know where you are, but in parts of sub Saharan Africa Google is using Global Cache to help cut data and bandwidth costs.

The Google Global Cache (GGC) was announced in May at the African Network Operators Group (AFNOG) conference in Morocco. In lieu of data centers in Africa, Google has created a strategy that is housed at major exchange points to serve Africa at the edge of Google’s network. Internal tests suggested at least 20% performance increase in high latency links, like East Africa. (2008)

However today this means its dependent on your operator if they've installed this or not, so if you're in the region, ask around if there's an operator that's using this system, it may help.

Also, changing browsers may help and there should be tips and tricks on managing on limited bandwidth surfing available on many African techie blogs. Write them with your question - there may be apps or tweaks available that we are not aware of.
posted by infini at 9:28 AM on August 5, 2011

@oranger hmm interesting and ur right; i remember it used to do that; but if i recall correctly it just indexes the text of the pages for searching. Which imo would be workable but u can't access the page content yourself; it sends u to the site when it finds a match - which reloads the site again.

@infini I'm def going to check out those African techie blogs; ur right they probably have tips and tricks for working on limited bandwidth connections. In India now, you would think they would have decent internet connections near the cities, but unfortunately where I am doesn't seem to have them ( the local town people don't want to let the companies run proper lines :( according to my admittedly technically disinclined host. )

Even back home though I thought about this idea though; there should be an autosave of each webpage you visit, with the cache wiped every few weeks unless you wanted to store it longer. Like temp internet files only more accessible and comprehensive.. if I open a article in the US and close the tab, it shouldn't have to reload the whole or part of the content from the internet when we have multi-TB hard drives.

I know enterprises have dedicated appliances that do this for their local networks - universities especially - but im surprised there isn't an browser extension or app to do something similar locally. :(

Anyone have any ideas? Just seems like something that someone would have addressed before..
posted by rmathew1 at 9:51 AM on August 5, 2011

Well, this is what squid-cache or polipo is for. They sit in between you and the internet, on your computer, and it saves every page you visit, as well as all the assets on the page. Polipo can be setup to block all of the advertising as well, which should help with your bandwidth needs. Privoxy and Glimmerblocker (Mac only) are dedicated ad blocking proxies.

I'd also recommend tuning off images in your browser, as well as javascript and flash. In firefox you

additionally, I'd look into using a text only web browser such as lynx.

What operating system/browser are you using?

If I was you, (I use mac os x) i would use Polipo and glimmerblocker. the way it works is that the proxies and arranged in a chain, Using firefox or chrome, I would install the noscript, flashbock and adblock plugins, then I would install glimmerblocker and setup glimmerblocker to connect to polipo, which then goes out and gets the website, and keeps a copy of the website on your hard drive, as well as all the assets. Polipo uses a bunch of tricks that make the connecting to a website seem faster as well, without breaking complex sites like facebook and gmail, etc. I use it even on a high bandwidth connection.

All of this is a bit technical, so if you want a more thourough walkthrough, let me know, and I'll provide one.
posted by Freen at 10:17 AM on August 5, 2011 [3 favorites]

mathew1 - go and spend money on Reliance CDMA sim card modem if you are not already using that. If its not available for sale in your town, pick it up the next time you're int he closest city. Take all your paperwork for registration. Android also turns phones into pretty good wifi hotspots.
posted by infini at 10:23 AM on August 5, 2011

I haven't used it myself, but there's this.
posted by benito.strauss at 8:52 PM on August 5, 2011

I used Loband when I lived in West Africa. Not sure if it suits your purposes, but I found it incredibly useful.
posted by hasna at 11:00 AM on August 7, 2011

@Freen ty!!! that's exactly what I was looking for. I'll try to set it up but I'll prob take u up on ur offer and message you :)

@infini I actually did that, using it now; def better when it works :) but often enough it gets really slow, like 1-3KB/s :( I have a android phone too :D but it's CDMA from Sprint (US carrier) so no ruim slot and i have no idea how to get tata/reliance to add its ESN to their database or modify the phone to match the ESN/MIED of a local phone.

@benito.strauss that looks really interesting; ty! going to try that too

@hasna oh that's gonna help when I can't use my computer :) thanks!!

ty everyone for all the suggestions!! :)
posted by rmathew1 at 11:28 AM on August 7, 2011