How many servers per country are there?
July 16, 2008 4:06 PM Subscribe
How would you estimate the number of web servers that a country uses? According to Gartner and IDC, about 8-9 million servers are shipped in a year (2007) -- but how does that break down by country?
One of the ways I'm thinking about measuring this.. is to use Netcraft's data for estimating the number of servers by country. Is there a better data source for something like this?
Netcraft doesn't really break down server shipments.. it measures servers that are in use already. but that's okay.. I'm just looking for a ballpark method to estimate this kind of thing.
One of the ways I'm thinking about measuring this.. is to use Netcraft's data for estimating the number of servers by country. Is there a better data source for something like this?
Netcraft doesn't really break down server shipments.. it measures servers that are in use already. but that's okay.. I'm just looking for a ballpark method to estimate this kind of thing.
Best answer: Tomorrowful is right about it being hard to measure.
What you could do, though, is use something like Netcraft's data, to get the number of websites per country. This is only going halfway, as you'd then need to figure out how many physical servers there were, but it's at least getting you partway there.
You might be interested in a look at IP addresses allocated per country... This, too, is very rough: the US has about 2.1 billion IPs (roughly half the world's capacity), but that doesn't mean there are anywhere near 2.1 billion servers, or even (necessarily) 2.1 billion computers. By contrast, other countries have miniscule numbers—Tunisia has 20.
I guess it depends a bit on what you're doing—servers shipped to a country and servers used as webservers in a country could be radically different.
posted by fogster at 4:30 PM on July 16, 2008
What you could do, though, is use something like Netcraft's data, to get the number of websites per country. This is only going halfway, as you'd then need to figure out how many physical servers there were, but it's at least getting you partway there.
You might be interested in a look at IP addresses allocated per country... This, too, is very rough: the US has about 2.1 billion IPs (roughly half the world's capacity), but that doesn't mean there are anywhere near 2.1 billion servers, or even (necessarily) 2.1 billion computers. By contrast, other countries have miniscule numbers—Tunisia has 20.
I guess it depends a bit on what you're doing—servers shipped to a country and servers used as webservers in a country could be radically different.
posted by fogster at 4:30 PM on July 16, 2008
BTW: I neglected to cite where I got the statistics. Are self-links allowed for these type of things? Basically, I took a set of IP-to-country mappings, put them into an SQL database, and then sorted the results by country.
posted by fogster at 4:33 PM on July 16, 2008
posted by fogster at 4:33 PM on July 16, 2008
Not to mention the fact that what Gartner calls a "server" and what ends up actually serving web pages might have nothing in common. Google is famous for running on (lots and lots of) last-year's overstocked PC parts.
posted by Skorgu at 4:34 PM on July 16, 2008
posted by Skorgu at 4:34 PM on July 16, 2008
Knowing the number of websites wouldn't necessarily help either. As Skorgu just mentioned, plenty of websites run on desktop or workstation class machines, embedded devices, virtual servers, etc.
posted by JaredSeth at 4:40 PM on July 16, 2008
posted by JaredSeth at 4:40 PM on July 16, 2008
Response by poster: well.. I'm trying to get some ballpark figures.. and "servers" is admittedly a pretty vague unit.
But then, so is "IT spending" per nation.. which includes employee salaries and other network equipment that isn't relevant to me.
Where would I find IP addresses allocated per country? And what does that actually mean? IP addresses, as you point out, don't really correspond to any physical hardware...
Tomorrowful, you're also right... But Netcraft seems to be the only service that even tries to count up the number of web servers in the world.
----
on preview.. fogster.. if you want to PM me your self-link.. please do!
posted by mhh5 at 4:44 PM on July 16, 2008
But then, so is "IT spending" per nation.. which includes employee salaries and other network equipment that isn't relevant to me.
Where would I find IP addresses allocated per country? And what does that actually mean? IP addresses, as you point out, don't really correspond to any physical hardware...
Tomorrowful, you're also right... But Netcraft seems to be the only service that even tries to count up the number of web servers in the world.
----
on preview.. fogster.. if you want to PM me your self-link.. please do!
posted by mhh5 at 4:44 PM on July 16, 2008
Translating shipped servers into a breakdown of server role(s) is going to be very difficult. This data would need to be self-reported by those taking shipment and IT architecture is typically handled under trade secret and/or security considerations. You might want to rethink this.
posted by rhizome at 4:45 PM on July 16, 2008
posted by rhizome at 4:45 PM on July 16, 2008
Response by poster: hmm. well.. I guess I'm not looking for perfect accuracy.. maybe I'm looking for some upper limits and lower limits.. like the maximum (or minimum) number of web servers that a country could be using right now?
posted by mhh5 at 12:40 AM on July 17, 2008
posted by mhh5 at 12:40 AM on July 17, 2008
Best answer: Well, publicly available servers have an obvious maximum of the number of available IPs. You can do IP-sharing tricks of course but those are relatively rare. The only minimum number I can think of is how many netcraft servers are in the same national IP block.
This is skating right over the issues of private, internal webservers, webservers behind load balancers, firewalls, private-IP space, embedded webservers for things like print servers and other appliances and of course webservers not serving on port 80.
The only way I can think of to get an even reasonably accurate count of anything at all would be to limit yourself to publicly available webservers on port 80 and just portscan the entire national IP block. For the US this is obviously impractical unless a botnet owner owes you some favors or you want to drop coin on enough EC2 instances to make it take less than a year, but for smaller countries it ought to be somewhere between feasible and practical. Note that I didn't say legal, not a lawyer, etc.
posted by Skorgu at 8:08 AM on July 17, 2008
This is skating right over the issues of private, internal webservers, webservers behind load balancers, firewalls, private-IP space, embedded webservers for things like print servers and other appliances and of course webservers not serving on port 80.
The only way I can think of to get an even reasonably accurate count of anything at all would be to limit yourself to publicly available webservers on port 80 and just portscan the entire national IP block. For the US this is obviously impractical unless a botnet owner owes you some favors or you want to drop coin on enough EC2 instances to make it take less than a year, but for smaller countries it ought to be somewhere between feasible and practical. Note that I didn't say legal, not a lawyer, etc.
posted by Skorgu at 8:08 AM on July 17, 2008
Response by poster: Skorgu,
Isn't that sorta what Netcraft does? Netcraft portscans >170million websites and tries to determine how many servers are behind a site. But I guess you're saying to not crawl websites, but to go through each IP address assigned to a country... to be more thorough in the counting. yikes. I'm guessing Google has already done something like this in the process of indexing the visible web... and I assume the task is non-trivial.
Is there any service that monitors overall Web traffic? Is there no way to know that there are approximately "1.3 gazillion web transactions a month" (transferring 5.9 kazillion bytes) in the US? The data could probably collected from several ISPs.. but has that data been collected already? A data point like that would be proportional to web servers in a way... discounting that not every transaction is created equal...
any other ideas?
posted by mhh5 at 11:42 AM on July 17, 2008
Isn't that sorta what Netcraft does? Netcraft portscans >170million websites and tries to determine how many servers are behind a site. But I guess you're saying to not crawl websites, but to go through each IP address assigned to a country... to be more thorough in the counting. yikes. I'm guessing Google has already done something like this in the process of indexing the visible web... and I assume the task is non-trivial.
Is there any service that monitors overall Web traffic? Is there no way to know that there are approximately "1.3 gazillion web transactions a month" (transferring 5.9 kazillion bytes) in the US? The data could probably collected from several ISPs.. but has that data been collected already? A data point like that would be proportional to web servers in a way... discounting that not every transaction is created equal...
any other ideas?
posted by mhh5 at 11:42 AM on July 17, 2008
This thread is closed to new comments.
posted by Tomorrowful at 4:18 PM on July 16, 2008