Does Google still use primarily cheap hard drives/infrastructure?
August 20, 2013 7:13 PM
Since 2007, does Google still use the Buy Cheap method to grow infrastructure? Or have they moved on to more traditional hardware growth?
Most recently in 2007 and previous, one of Google's ways of being different was playing the numbers of hardware cost and using large arrays of cheap hard drives/servers rather than the more traditional system of large, high-premium enterprise-class servers with 10k+ RPM SAS drives. Apparently this kept costs down and made growing their services much easier on many fronts. They released a lot of data in 2007 about it, which is the last mention I can find of their philosphy. Practically every tech site on the planet wrote an article about it.
But none since, apparently. Every search I do points back to some coverage of that article (or the 2011 Thailand flood that globally slammed HDD production.) Does Google still use the consumer hardware in bulk philosophy? Can anyone find a source?
Most recently in 2007 and previous, one of Google's ways of being different was playing the numbers of hardware cost and using large arrays of cheap hard drives/servers rather than the more traditional system of large, high-premium enterprise-class servers with 10k+ RPM SAS drives. Apparently this kept costs down and made growing their services much easier on many fronts. They released a lot of data in 2007 about it, which is the last mention I can find of their philosphy. Practically every tech site on the planet wrote an article about it.
But none since, apparently. Every search I do points back to some coverage of that article (or the 2011 Thailand flood that globally slammed HDD production.) Does Google still use the consumer hardware in bulk philosophy? Can anyone find a source?
I believe making your own collection of servers from cheap components is catching on. Facebook does something similar these days.
posted by Wild_Eep at 7:56 PM on August 20, 2013
posted by Wild_Eep at 7:56 PM on August 20, 2013
How Google arranges their servers is now a trade secret (article from 2012), so it's not a surprise that information is harder to find.
posted by meowzilla at 9:10 PM on August 20, 2013
posted by meowzilla at 9:10 PM on August 20, 2013
Such information has long been a Google trade secret, but they also have been in the habit of providing information on the design of "obsolete" data centers which gives a sense of how their design philosophy has played out over multiple generations.
Overall though, most indications are that the same basic philosophy is at work at Google and has taken hold elsewhere (like Facebook), but the manifestations are different.
They aren't buying "enterprise-class" servers and components, they are instead buying what you might call "internet scale" class servers and components. Anything running at internet scale has to have an overall architecture that 1) scales over thousands of machines 2) is resilient in the face of the failure modes that crop up in infrastructures containing thousands of machines. Given that context, avoiding enterprise-class hardware still makes sense.
That doesn't mean that they are doing what they were doing the better part of a decade ago because the world has changed since then. More companies are operating large-scale infrastructures, meanwhile, the traditional desktop and enterprise-server markets have flattened out or even started shrinking. Together, its shifted the dynamics of the industry. Google and the like have more pull, and existing OEM and upstarts are doubtless clamoring for their business. One of those companies, Isilon, now owned by EMC, had a clustered-storage product that Apple reportedly is using a lot of in their new datacenter(s).
Some people now have their eye on components from much more vital areas of the commodity technology markets: Mobile, which has the advantage of being very sensitive to energy consumption, which makes up a significant portion of the lifecycle cost of a datacenter. Google doesn't seem to be one of them, though.
In addition, the growing volume and declining costs of SSDs means that a longstanding performance constraint that people had to architect around has both improved by a couple of orders of magnitude in the space of a few years, and looks poised to continue improving after being nearly flat for a decade or more.
And finally, as noted above, the datacenter network, which was the last bastion of mainframe-style hardware in modern internet-scale datacenters, is becoming commodified and open-sourced.
For a bit of an insider perspective on this stuff, check out James Hamilton's blog. He had a lead-role in Microsoft's efforts to build out large-scale low-cost computing infrastructure, and for the last 4 years has been responsible for Amazon Web Services infrastructure efficiency, reliability and scaling.
posted by Good Brain at 1:18 AM on August 21, 2013
Overall though, most indications are that the same basic philosophy is at work at Google and has taken hold elsewhere (like Facebook), but the manifestations are different.
They aren't buying "enterprise-class" servers and components, they are instead buying what you might call "internet scale" class servers and components. Anything running at internet scale has to have an overall architecture that 1) scales over thousands of machines 2) is resilient in the face of the failure modes that crop up in infrastructures containing thousands of machines. Given that context, avoiding enterprise-class hardware still makes sense.
That doesn't mean that they are doing what they were doing the better part of a decade ago because the world has changed since then. More companies are operating large-scale infrastructures, meanwhile, the traditional desktop and enterprise-server markets have flattened out or even started shrinking. Together, its shifted the dynamics of the industry. Google and the like have more pull, and existing OEM and upstarts are doubtless clamoring for their business. One of those companies, Isilon, now owned by EMC, had a clustered-storage product that Apple reportedly is using a lot of in their new datacenter(s).
Some people now have their eye on components from much more vital areas of the commodity technology markets: Mobile, which has the advantage of being very sensitive to energy consumption, which makes up a significant portion of the lifecycle cost of a datacenter. Google doesn't seem to be one of them, though.
In addition, the growing volume and declining costs of SSDs means that a longstanding performance constraint that people had to architect around has both improved by a couple of orders of magnitude in the space of a few years, and looks poised to continue improving after being nearly flat for a decade or more.
And finally, as noted above, the datacenter network, which was the last bastion of mainframe-style hardware in modern internet-scale datacenters, is becoming commodified and open-sourced.
For a bit of an insider perspective on this stuff, check out James Hamilton's blog. He had a lead-role in Microsoft's efforts to build out large-scale low-cost computing infrastructure, and for the last 4 years has been responsible for Amazon Web Services infrastructure efficiency, reliability and scaling.
posted by Good Brain at 1:18 AM on August 21, 2013
Philosophically, the standard unit that Google now uses for infrastructure design is the datacentre, and the standard deployment unit is about $600m.
The evolution of Buy Cheap to reflect that is in terms of land and power and existing fibre-optic capacity.
posted by holgate at 8:07 AM on August 21, 2013
The evolution of Buy Cheap to reflect that is in terms of land and power and existing fibre-optic capacity.
posted by holgate at 8:07 AM on August 21, 2013
Holgate makes good points and reminds me that one interesting thing that Google, Microsoft, and others have investigated are containerized datacenter infrastructure. The basic deployment unit is a standard-sized shipping container. To deploy them, you plop one down on an appropriate level footing, connect a supply of fresh water, power, and network connectivity. They can be deployed directly in the great outdoors, but I think they typically are placed in some sort of covering structure to avoid heating by the sun.
They can be deployed individually, or in small numbers where a lot of computing power is needed nearby, or assembled in much larger installations like a traditional internet-scale datacenter. I know that Microsoft had one or two in a parking lot in Redmond a few years back where they were onboarding and processing all the ariel photo imagery they used for the 3D Bing maps.
Given the emphasis on siting DCs at the nexus of cheap land, cheap power, and existing fiber, along with a shift to doing more air cooling the containerized datacenter module might be a bit of an evolutionary cul-de-sac, since they seem to be, in part, optimized for density. I think though that the idea of creating relatively large packages that can be assembled and serviced at a central location while being deployed en mass is likely to have continued in some form.
posted by Good Brain at 1:21 PM on August 26, 2013
They can be deployed individually, or in small numbers where a lot of computing power is needed nearby, or assembled in much larger installations like a traditional internet-scale datacenter. I know that Microsoft had one or two in a parking lot in Redmond a few years back where they were onboarding and processing all the ariel photo imagery they used for the 3D Bing maps.
Given the emphasis on siting DCs at the nexus of cheap land, cheap power, and existing fiber, along with a shift to doing more air cooling the containerized datacenter module might be a bit of an evolutionary cul-de-sac, since they seem to be, in part, optimized for density. I think though that the idea of creating relatively large packages that can be assembled and serviced at a central location while being deployed en mass is likely to have continued in some form.
posted by Good Brain at 1:21 PM on August 26, 2013
« Older Help Avoiding Sunburn in Hawaii | Help me find a multi-line (4-8 lines preferred)... Newer »
This thread is closed to new comments.
posted by rmd1023 at 7:35 PM on August 20, 2013