Choking on search logs
August 16, 2006 2:25 PM Subscribe
Trying to open a 200MB .txt file on my Win2KPro machine, and all I get is hard drive spinning, CPU cycling, but nothing opening...
You may have already figured out that I'm trying to open those famous search log files that became such a big deal a few weeks ago. They're HUGE though-- so big that my machine just balks at them. I've tried WordPad, Notepad++, MS Word, and even ATLAS/ti. Nothing is up to the task.
Then again, perhaps it's my machine, but I don't think so. It's an adequate-enough PIII 1Ghz w/ 512MB of RAM, which ought to give it enough overhead to load the whole file (since most of the RAM is free & available).
What to do?
You may have already figured out that I'm trying to open those famous search log files that became such a big deal a few weeks ago. They're HUGE though-- so big that my machine just balks at them. I've tried WordPad, Notepad++, MS Word, and even ATLAS/ti. Nothing is up to the task.
Then again, perhaps it's my machine, but I don't think so. It's an adequate-enough PIII 1Ghz w/ 512MB of RAM, which ought to give it enough overhead to load the whole file (since most of the RAM is free & available).
What to do?
Visual Studio works okay at this kind of thing too.
It might be easiest to split it up into, say, 100 2 meg files using any number of googlable file splitters.
posted by aubilenon at 2:41 PM on August 16, 2006
It might be easiest to split it up into, say, 100 2 meg files using any number of googlable file splitters.
posted by aubilenon at 2:41 PM on August 16, 2006
it will likely take several minutes before it opens; might want to leave it running and come back and check later. I've had large files take up to 10 minutes to open, though they eventually did, I didnt think it was very good for my computer (sluggish behaviour after that). You could speed things up with more ram maybe.
Your best bet tho is to split the file.
posted by jak68 at 3:05 PM on August 16, 2006
Your best bet tho is to split the file.
posted by jak68 at 3:05 PM on August 16, 2006
Large Text File Viewer. Very good at this kind of thing, as you would expect, though please disable the god-awful background image.
posted by smackfu at 3:30 PM on August 16, 2006
posted by smackfu at 3:30 PM on August 16, 2006
The problem is that the tool(s) you were trying to use to look at the file was(were) "thrashing". That means it was trying to use so much memory that the memory was being swapped constantly to HD and back in again. Thrashing is a Bad Thing, not least of which because it can decrease performance by a factor of a hundred or even worse.
Your 512MB of memory isn't as large as you think, because the program may be storing the data it loads in a large datastructure which occupies much more room than the raw size of the text. That's very common.
posted by Steven C. Den Beste at 6:49 PM on August 16, 2006
Your 512MB of memory isn't as large as you think, because the program may be storing the data it loads in a large datastructure which occupies much more room than the raw size of the text. That's very common.
posted by Steven C. Den Beste at 6:49 PM on August 16, 2006
emacs
http://www.gnu.org/software/emacs/windows/ntemacs.html
I've found most of the basic Windows utilities chock on really large files. Emacs on my windows box still takes a while to open a file that size (blame the OS) but works better than notepad
posted by zaphod at 8:20 PM on August 16, 2006
http://www.gnu.org/software/emacs/windows/ntemacs.html
I've found most of the basic Windows utilities chock on really large files. Emacs on my windows box still takes a while to open a file that size (blame the OS) but works better than notepad
posted by zaphod at 8:20 PM on August 16, 2006
« Older Waiter! Waiter! There are dots in my spam. Could... | What do I do with this %&#@$ degree? Newer »
This thread is closed to new comments.
posted by bigmusic at 2:26 PM on August 16, 2006