Big difference between Wordpress stats and server logs
November 20, 2017 8:38 PM   Subscribe

Hi, I volunteer as co-editor and main tech person for an academic collaborative blog. In our most recent full calendar month (October 2017), there was a 4-fold difference between the unique visitors reported by our server logs (Awstats) and what was reported by Why?

We host our blog on our own domain and those are the only two stats programs I have access to. The big difference last month is not unique - in most months, Awstats/server logs are showing 4 times more unique visitors to our blog. Whom do we trust - the webhost or Wordpress? And why the big discrepancy? I have searched the web and the AskMe archives as well as both Wordpress and server help files, with no success.

Feel free to also recommend any good resources for non-profit web management and analytics. Thanks in advance for your help!
posted by acridrabbit to Technology (7 answers total)
AWStats is probably counting all hits to the site, including those from bots, spammers, scrapers, browsers running ad- and tracking-blockers, and other programs. Not necessarily what you would call a "visitor", in other words. Wordpress is probably reporting stats based on tracking scripts which run in the browser. That's why the discrepancy. As for who you can trust: nobody, really. Web stats are an illusion. Pick one, and use it for relative numbers -- was this week higher than last week? -- but don't try to squeeze meaningful absolute numbers out of anything. You can use both for relative stats, but the important thing is to use them relative to themselves, not to each other.
posted by hades at 8:53 PM on November 20, 2017

It's been so long since I've touched Wordpress that I won't speculate on what's going on there, but Google Analytics is pretty much the standard in this stuff and it generally does a fairly good job of filtering out the garbage. But hades is correct that none of these are perfect, and you should think about things in relative terms.
posted by primethyme at 8:54 PM on November 20, 2017

Response by poster: I should have specified that the Awstats report I am looking at is specifically the "viewed traffic," which excludes spiders, bots, etc.

Also, there's nothing else on this domain except the blog, so there aren't any non-Wordpress areas or pages.
posted by acridrabbit at 9:26 AM on November 21, 2017

Best answer: Is it possible someone is hotlinking your content? That would show up in AWS but not Wordpress.
posted by justkevin at 10:01 AM on November 21, 2017

Best answer: "Not viewed traffic includes traffic generated by robots, worms, or replies with special HTTP status codes."

They can claim that they're excluding that traffic, but in reality they can only exclude it from well-behaved robots and spiders which identify themselves as such. Unless there's a lot more intelligence under the hood than there was the last time I looked at AWStats, it doesn't exclude traffic from automated agents which don't announce themselves as bots, because from the server's point of view there is no difference between a human visitor and a bot which mimics a human visitor for the purpose of spamming or scraping content.

No two web stats packages are going to produce the same numbers, unless they are both log analyzers doing no filtering whatsoever. Anything based on browser tracking (Google Analytics, Wordpress' built-in stats) is going to report lower raw numbers than something log-based, although a 4x discrepancy does seem kind of high. Hotlinking is another likely culprit, yeah, as there's no way for Wordpress to put a tracking script on an image or other non-html file.

If a high percentage of your visitors run privacy plugins that block tracking scripts, that could account for some of the discrepancy, too. Wordpress can only count people who allow themselves to be counted.
posted by hades at 12:09 PM on November 21, 2017 [1 favorite]

I browse with NoScript and RequestPolicy plugins, so if counting is done by something that those block only the logs will notice me. If your site doesn't display/work right and I actually try to enable some scripts and requests to make it work ... that forces the browser to reload the page. Depending on your site, it might take me 3-4 rounds of enabling various suspect looking scripts and such to make it work at all. More than 3-4 tries and I just give up on your site entirely.

Just from that, I can see myself showing up 3-4x in logs vs some smarter sort of tracking.
posted by zengargoyle at 3:50 PM on November 21, 2017

Two things to check out:

Have a look at the actual files that awstats is counting as pages via the full list. Are all those what you would consider pages, or can see you other types of files in there?

Have a look at pageviews for a few particular pages in the two systems. Do they all follow the same ratio or do some match more closely than others? Is there a common factor for outliers?
posted by Sparx at 7:16 PM on November 23, 2017 [1 favorite]

« Older Buying better   |   Dog breeds that are good with cats and with stairs Newer »
This thread is closed to new comments.