Comments on: Quantifying the Slashdot Effect

Question: Quantifying the Slashdot Effect

evariste — Fri, 24 Oct 2008 16:54:19 -0800

I'm trying to figure out if a server I'm working on can withstand being linked to by very high traffic websites. Can anyone tell me the basic traffic characteristics of, say, a front page link from: Drudge Report, Yahoo! Buzz, Huffington Post, Slashdot, Reddit, or Digg? I'm using ab (ApacheBench) and I'd like to have some idea of how many requests, and at what level of concurrency, I should be trying to make this server capable of pushing out. Thanks!

I already know what to do (log and observe, cache cache cache, tune and tweak db and web server configuration, etc) and am experienced in that process. I just need some typical numbers to help me figure out when I'm done!

By: evariste

evariste — Fri, 24 Oct 2008 17:01:11 -0800

The particular server I'm working on right now seems to handle ab -n 10000 -c 500 really well, which means ten thousand requests for a page, issued 500 at a time. Some of the page requests are slow, but 75% take under 2 seconds and 99% are under 17 seconds. The slowest one took 45 seconds, and one request couldn't be satisfied. Throughout this, the server's load average remained well below 1. A good start? Just average? I have no basis on which to judge this.

It would be cool if someone could tell me, e.g., "Slashdot's traffic comes in spurts of about 5000, maybe 20 concurrently, until you've seen about two hundred thousand pageloads" because that way I could write a script that simulated a Slashdotting (or whatever).

By: qxntpqbbbqxl

qxntpqbbbqxl — Fri, 24 Oct 2008 17:31:59 -0800

This seems to be a quantitative description of the Slashdot effect from ten years ago. So it's definitely out of date, but might still be useful.

By: evariste

evariste — Fri, 24 Oct 2008 17:33:31 -0800

A great start, thank you!

By: mandal

mandal — Fri, 24 Oct 2008 18:45:31 -0800

Kottke: digg vs Slashdot

By: tallus

tallus — Sat, 25 Oct 2008 10:40:57 -0800

I work for a company that among other things hosts the top search result for the google that was linked to from a 'Google Doodle' (one-off logo) on several of the Google regional sites.

These are the stats for the 24 hour period

Requests per second: up to ~200
Data transfer: up to ~1.8MB/second (6GB/hour)
Requests being processed by apache: up to ~600 simultaneous
Total data transferred: >100GB

this represents about 200,000 visits. We couldn't really get any indication of what to expect so we had moved the site to a dedicated box ahead of time.