Choking on search logs

August 16, 2006 2:25 PM

Trying to open a 200MB .txt file on my Win2KPro machine, and all I get is hard drive spinning, CPU cycling, but nothing opening...

You may have already figured out that I'm trying to open those famous search log files that became such a big deal a few weeks ago. They're HUGE though-- so big that my machine just balks at them. I've tried WordPad, Notepad++, MS Word, and even ATLAS/ti. Nothing is up to the task.

Then again, perhaps it's my machine, but I don't think so. It's an adequate-enough PIII 1Ghz w/ 512MB of RAM, which ought to give it enough overhead to load the whole file (since most of the RAM is free & available).

What to do?

posted by LGCNo6 to computers & internet (10 comments total)

Notetab can handle them.
posted by bigmusic at 2:26 PM on August 16, 2006

Visual Studio works okay at this kind of thing too.

It might be easiest to split it up into, say, 100 2 meg files using any number of googlable file splitters.
posted by aubilenon at 2:41 PM on August 16, 2006

it will likely take several minutes before it opens; might want to leave it running and come back and check later. I've had large files take up to 10 minutes to open, though they eventually did, I didnt think it was very good for my computer (sluggish behaviour after that). You could speed things up with more ram maybe.

Your best bet tho is to split the file.
posted by jak68 at 3:05 PM on August 16, 2006

ListXP can open it.
posted by misterbrandt at 3:06 PM on August 16, 2006

Large Text File Viewer. Very good at this kind of thing, as you would expect, though please disable the god-awful background image.
posted by smackfu at 3:30 PM on August 16, 2006

I use TextPad for my insanely-long-file needs
posted by JZig at 5:36 PM on August 16, 2006

The problem is that the tool(s) you were trying to use to look at the file was(were) "thrashing". That means it was trying to use so much memory that the memory was being swapped constantly to HD and back in again. Thrashing is a Bad Thing, not least of which because it can decrease performance by a factor of a hundred or even worse.

Your 512MB of memory isn't as large as you think, because the program may be storing the data it loads in a large datastructure which occupies much more room than the raw size of the text. That's very common.
posted by Steven C. Den Beste at 6:49 PM on August 16, 2006

emacs

http://www.gnu.org/software/emacs/windows/ntemacs.html


I've found most of the basic Windows utilities chock on really large files. Emacs on my windows box still takes a while to open a file that size (blame the OS) but works better than notepad
posted by zaphod at 8:20 PM on August 16, 2006

duh!

I meant to spell choke not chock.
posted by zaphod at 8:21 PM on August 16, 2006

www.vim.org
posted by baylink at 2:09 PM on August 17, 2006