Is this what traffic-shaping looks like?

November 18, 2007 1:10 PM

What would cause web site traffic to rise and then hit a plateau for the middle of each day before falling again?

The image below is a graph of a typical day of web browser traffic from one of my web sites. See that plateau between the two yellow bars? That's what's bugging me. Unique visitor traffic doesn't look like that—or never has before—for any of my sites until I moved this one to a new provider.

For a site like this, with about 3000 unique web visitors a day, the site arc should normally be curved, more or less having a round hump in the middle of the day. (The darker green area is page views, which does tend to be more jagged and less humped.) The yellow dotted line is more or less what I would expect.

My theory is that my service provider is doing traffic-shaping for what I consider to be low levels of traffic and might be delaying or denying HTTP requests in order to preserve bandwidth. Do you think that's what is illustrated by this graph? If not, how would you explain it?

I thought that it might be that Mint, my traffic-measuring software, isn't getting all the uniques, but that plateau is more or less borne out by Google Analytics. I say "more or less" because no two traffic-measuring programs do it the same.

http://www.doubletongued.org/flattop_stats.gif

posted by Mo Nickels to computers & internet (16 comments total)

I can't see the picture, but I wonder if what you're seeing is curved traffic from four contiguous timezones added together. Given that there's a one-hour delay from EST to CST, to MST, to PST, then if each of them is a curve, the sum should have a plateau.
posted by Steven C. Den Beste at 1:25 PM on November 18, 2007

This is what the traffic on my server looks like per hour over the last ten weeks. And there is definitely no throttling going on in my case.
posted by Steven C. Den Beste at 1:30 PM on November 18, 2007

I would expect traffic shaping to affect page views and bytes transferred, but not unique visitors (the same people will still come to your site, but with a degraded experience they might leave sooner).
posted by hattifattener at 1:31 PM on November 18, 2007

Lunch time.
posted by dhammond at 1:46 PM on November 18, 2007 [1 favorite]

The rolling wave of traffic from across North America leaves a recognizable arc to the day's traffic for site whose visitor base is largely North American. It'd be different if it was a very local site. There's not a canceling effect that leaves a plateau, there's simply a longer arc to the curve, with the largest peak occurring earlier in the day because most of the American population centers are in the East.

It's not lunch time.

Also, you're forgetting that it didn't use to be that flat.

Stephen, your graph is not flat.
posted by Mo Nickels at 2:03 PM on November 18, 2007

On that image, the previous day's traffic was apparently not flattened off like this day's traffic (or it jumped up suddenly at the end of the cap). What does a multi-day (say, 1-week) graph look like?
posted by hattifattener at 2:06 PM on November 18, 2007

There's a long flat spot extending from 1300 to 1900 on my chart. That was the point I was trying to make.

My usage pattern is probably somewhat different than yours, of course.
posted by Steven C. Den Beste at 2:06 PM on November 18, 2007

Lunch time. Ever try to load MetaFilter between 12 and 1?
posted by KokuRyu at 2:06 PM on November 18, 2007

What I get for not previewing.

Can't say it looks that odd to me. Are you quite sure this isn't a case of Y-axis-not-as-stretched-as-before?
posted by genghis at 2:09 PM on November 18, 2007

To repeat, it's not lunch time. There would still be a curve, not a plateau.

Hattifattener, the previous day had several strange traffic features that make it not an ordinary day, but the plateau reoccurs on normal days. Also, I do not currently have a way to show a week's traffic with hourly datapoints. All the software I have will only do it with a single data point for each day.
posted by Mo Nickels at 2:14 PM on November 18, 2007

I see you're using Mint. Mine behaved weirdly a few weeks ago. Total page views remained as usual. Unique vistores dropped inexplicably for about fortnight. Without changing anything numbers came back to normal.

Mien needs upgrading but I don't like the upgrade process. Is your Mint up to date ?
posted by Baud at 2:26 PM on November 18, 2007

What timezone is your server set to? Is it possible the drop is not in the middle of the day, but is at the start/end of the day and the timezone offset is wrong? If you just moved to a new host, it possible some of the settings are not what you expect.
posted by nalyd at 2:32 PM on November 18, 2007

Just at a wild guess, if you're seeing points on your outbound traffic graph where you have a plateau and expect a rounded peak, I would be inclined to suspect a bandwidth ceiling (such as you might expect if your host was buying not enough capped bandwidth) or request throttling on the server rather than any of the more commonly discussed kinds of traffic shaping....
posted by nonliteral at 2:35 PM on November 18, 2007

mornings: coffee, danish, look at the internet.
12:00: lunchies
afternoons: gotta get some work done.
posted by bruce at 2:59 PM on November 18, 2007

I'd say that the new provider is running out of bandwidth at lunch time.

Your best way to check this is to put the website in question on a diet (optimise or remove images, simplify pages, etc) and see if the plateau moves up. If it does, it means your bandwidth is being throttled. If not, it could be a limitation on the number of simultaneous database queries?
posted by krisjohn at 3:18 PM on November 18, 2007

Do the graphs from your traffic monitoring service match your webserver logs?
posted by Good Brain at 3:24 PM on November 18, 2007