Configured, Boiled Down Numbers Please
May 14, 2008 9:36 PM Subscribe
A good, modern, local website traffic stats solution?
I used to know this stuff, but it's been many years and technology changes, yadda yadda, and this has some special odd requirements... so I prefer some hive mind opinions...
I have a colleague looking for a good solution to traffic stats reporting. He is responsible for dozens of websites, with new ones being added/removed every couple of weeks -- promotions and projects from various departments. He has access to combined/extended Apache logs (regular + referrer) already, but since he controls the websites, he is also open to a javascript-triggered thingie on each page, and a custom server that logs things itself. Either is fine as long as it churns out the results.
For results, he'd need an assortment of the usual hits/views/referrers stuff (visitors from Chile, most popular pages this week, hits on this certain link), but boiled down nicely like so:
1) takes raw logs and turns them into something human-readable.
2) scheduled (weekly or monthly) reports automatically saved as fixed files (auditable, this not recreated each time it's run).
3) reports showing just the info he wants* e-mailed to him as attachments (PDF, ideally, but text files would work) so he can present to management.
4) the ability to request custom reports (the period from Sept 1 to Sept 14, users from China only) on demand, presumably from some kind of web interface -- an on-the-fly config run one-off.
5) the ability to combine or separate data from diff sites/logs by configuration. (that is: these two sites are the same, but these four are all to be tracked separately)
* I don't have a list, but he says it will be one or two pages and NOT change over time. A template of certain reports?
He is considering hiring a human being part-time to do this with some offline program as a secretarial task every week... but that struck me as crazy since it's automatable, right? Right? Hm.
It can't be Google Analytics, sadly for me (I know how to use that) and for privacy/security reasons he'd prefer something running on his own Un*x server rather than a web service.... whether it's the same server as the websites or another doesn't matter, as long as he can control access and keep the data private.
In the old days, I would say this all called for urchin or awstats, I think... plus some magic I don't know about to turn the results into pretty reports and mail them from a big old cron job.
From my perspective (helping find/source/make/install this), I'd be fine with something very complicated that I can tweak the configuration of to produce just what he's asking for -- and no more.
Or if something does all the processing, will run reports on a schedule and produce some kind of manageable (parsable?) output like plain flat tables, I can probably manage turning that into pretty documents and mailing them out with additional scripty code that I can stumble through.
Open source and hackable preferred, because of the configurability and tweakiness required... or guidance to existing things that can be cobbled together. If someone has built/integrated one of these before, or something similar, please mefi-mail me and maybe it's easy work, too. I may have overcommitted myself in trying to help.
I used to know this stuff, but it's been many years and technology changes, yadda yadda, and this has some special odd requirements... so I prefer some hive mind opinions...
I have a colleague looking for a good solution to traffic stats reporting. He is responsible for dozens of websites, with new ones being added/removed every couple of weeks -- promotions and projects from various departments. He has access to combined/extended Apache logs (regular + referrer) already, but since he controls the websites, he is also open to a javascript-triggered thingie on each page, and a custom server that logs things itself. Either is fine as long as it churns out the results.
For results, he'd need an assortment of the usual hits/views/referrers stuff (visitors from Chile, most popular pages this week, hits on this certain link), but boiled down nicely like so:
1) takes raw logs and turns them into something human-readable.
2) scheduled (weekly or monthly) reports automatically saved as fixed files (auditable, this not recreated each time it's run).
3) reports showing just the info he wants* e-mailed to him as attachments (PDF, ideally, but text files would work) so he can present to management.
4) the ability to request custom reports (the period from Sept 1 to Sept 14, users from China only) on demand, presumably from some kind of web interface -- an on-the-fly config run one-off.
5) the ability to combine or separate data from diff sites/logs by configuration. (that is: these two sites are the same, but these four are all to be tracked separately)
* I don't have a list, but he says it will be one or two pages and NOT change over time. A template of certain reports?
He is considering hiring a human being part-time to do this with some offline program as a secretarial task every week... but that struck me as crazy since it's automatable, right? Right? Hm.
It can't be Google Analytics, sadly for me (I know how to use that) and for privacy/security reasons he'd prefer something running on his own Un*x server rather than a web service.... whether it's the same server as the websites or another doesn't matter, as long as he can control access and keep the data private.
In the old days, I would say this all called for urchin or awstats, I think... plus some magic I don't know about to turn the results into pretty reports and mail them from a big old cron job.
From my perspective (helping find/source/make/install this), I'd be fine with something very complicated that I can tweak the configuration of to produce just what he's asking for -- and no more.
Or if something does all the processing, will run reports on a schedule and produce some kind of manageable (parsable?) output like plain flat tables, I can probably manage turning that into pretty documents and mailing them out with additional scripty code that I can stumble through.
Open source and hackable preferred, because of the configurability and tweakiness required... or guidance to existing things that can be cobbled together. If someone has built/integrated one of these before, or something similar, please mefi-mail me and maybe it's easy work, too. I may have overcommitted myself in trying to help.
I've used the free version of sitemeter, but I think you can customize it in a variety of ways with the pay version.
posted by amyms at 11:14 PM on May 14, 2008
posted by amyms at 11:14 PM on May 14, 2008
I still like Analog. It doesn't (to my knowledge) do the auto emails, and a nice interface is I think only available as an add-on, but I think it meets the other requirements, and some little python scripts could probably provide auto-email pretty easily.
posted by kristi at 10:33 AM on May 15, 2008
posted by kristi at 10:33 AM on May 15, 2008
Response by poster: Sitemeter seems to be a service only (no-go, see above), but I guess I'll look into Webalizer, Mint, and the latest Analog. What are these add-ons, Kristi?
I was hoping there were more new options I'd missed. Sounds like this is a project for sure. :(
posted by rokusan at 3:33 PM on May 15, 2008
I was hoping there were more new options I'd missed. Sounds like this is a project for sure. :(
posted by rokusan at 3:33 PM on May 15, 2008
This thread is closed to new comments.
Mint looks like it could do what he wants - it's extensible and I know that there are extensions for notifications, but I've never used it myself, so I can't say for sure.
posted by Galt at 10:31 PM on May 14, 2008