Help me parse the logs, Jim!

October 9, 2006 2:53 AM

Hi, I'm hosted with TextDrive who don't provide stats or log analysis for their customers. I do have access to the raw Apache log files, though, and would dearly love to parse them and get some meaningful stats and hopefully graphical charts out of them, ala AWStats, Analog or Webalizer.

I've tried installing AWStats onto the host but permissions problems are causing that to fail. Surely there must be a local solution? I have access to Mac and Windows boxes - can anyone talk me through parsing my logs locally? For extra bonus love - would anyone be prepared to produce a oneoff document from a couple of log files while this is being sorted out?

posted by dance to computers & internet (9 comments total)

>I've tried installing AWStats onto the host but permissions problems are causing that to fail.

IMHO, there's your actual AskMe question. Can't we just fix that?

As AWStats is a perl program, there should be very little trouble installing it on a modern OSX Mac, because they all have Apache and perl already. The devil being in the details though, you might need to fiddle with modules etc. I'll check on that and post again.

What do you need produced from the logs exactly?
posted by AmbroseChapel at 3:13 AM on October 9, 2006

Just the usual:
details on visitors: browsers, hits per hour, day, week, month in graphical form, country by IP would be nice.

I have installed AWStats on the Windows machine but I've no clue how to feed it logs and get it to do anything.
posted by dance at 3:29 AM on October 9, 2006

Have you taken a look at Google Analytics? Really easy to install (just a script on each page, nothing to be installed on your server) and loads of stats, nicely laid out.
posted by twistedonion at 3:34 AM on October 9, 2006

For an app on your mac to parse the files try -

Webalizer
posted by twistedonion at 3:40 AM on October 9, 2006

I'd love to use Google Analytics but I want to view data for the last week - wouldn't GA only start analysing now?

Will try Webalizer...
posted by dance at 3:50 AM on October 9, 2006

Yeah, it would only start collecting once you add the scripts to your page.
posted by twistedonion at 4:54 AM on October 9, 2006

Parsing logs locally with Analog is really easy. You just need to download/install it on to your machine. It will install in a directory. Then you need to download your log files, rename the file as logfile.log (this is if you don't want to do any configration at all, that's what Analog is expecting to find in its directory) and put it in the directory with Analog. This will generate a report called report.html which you can open with a web browser. You can customise the hell out of this report using the analog.cfg file and if you look at the sample config file they include (called something like big.cfg) you can see what most of the config syntaxes arfe like.

So, I'm not sure how much you know, or how much account access you have with TextDrive (and you may want to ask in their forums as well since I'm sure many people do this with their log files) but a workflow would look like this

- Download/install Analog
- Concatenate your log files. If you have shell access you can just do something like this from within the log directory cat apache-log* > logfile.log. Otherwise you can download all the files to your mac (you don't need the error logs) and do it there. Your host may be zipping the files, so you may need to unzip them first. You'll be able to tell by looking at the suffix of the files.
- Download this file and stick it in the Analog directory
- Run Analog (there will be a default config file that you may want to change)
- Open report.html and look at your stats

Let me know if you need more detail than this, or catch me on chat. I use Analog on a Mac all the time.
posted by jessamyn at 5:55 AM on October 9, 2006

I'm on TextDrive as well. There is some good documentation on installing stats packages on your own.

Here's how to install peastat. Here's how to install AWStats.
posted by ewagoner at 7:13 AM on October 9, 2006

You can try out Weblog Expert Lite. It's just a standard Windows program that you can run locally.
posted by smackfu at 7:36 AM on October 9, 2006