The Big blocks of technology & internet services
December 19, 2012 2:14 PM Subscribe
Twitter, Ticketmaster, Google, Facebook how do their architecture and infrastructure work. Looking for a book/site which explains how these tech services may be assembled. Obviously not looking for the secret sauce or anything. Just wanting to understand the bigger picture of how large scale technology work and come together.
Not looking to understand the code aspect or nitty gritty details of how each component works. Just broad understanding is what I am looking for. I am a UI engineer looking to get more background on middleware and backend systems.
Please recommend any sources/sites/blogs/books to help me gain decent insight.
Response by poster: You guys rock! Thanks. Exactly what i was looking for.
posted by gadget_gal at 2:41 PM on December 19, 2012
posted by gadget_gal at 2:41 PM on December 19, 2012
There's a really interesting looking class on Coursera
posted by halseyaa at 2:57 PM on December 19, 2012
posted by halseyaa at 2:57 PM on December 19, 2012
They're all different. Google publishes papers on occasion, as axiom points out. There's an open source reimplementation called Hadoop. Facebook's tech is comparatively standard open source LAMP technology, but they're investing major dollars into making that work for them at scale; sharding MySQL is a major pain. Netflix has an engineering blog, but here's also a presentation at OSCON on what tech they deploy in EC2.
Conferences are a popular place to pick up things. The aforementioned OSCON, Velocity, and LISA are all good sources and produce varying levels of materials available online. I have my students read parts of The Art of System and Network Administration, who's major benefit is basically covering all the basic equipment.
posted by pwnguin at 9:13 PM on December 19, 2012
Conferences are a popular place to pick up things. The aforementioned OSCON, Velocity, and LISA are all good sources and produce varying levels of materials available online. I have my students read parts of The Art of System and Network Administration, who's major benefit is basically covering all the basic equipment.
posted by pwnguin at 9:13 PM on December 19, 2012
Here's a paper on the Google File System that Hadoop is apparently based upon.
posted by Diag at 2:23 AM on December 20, 2012
posted by Diag at 2:23 AM on December 20, 2012
I'll just add that the way Facebook and Google do it is NOT the way most large IT infrastructure operations run. In more 'traditional' companies, things are nowhere near as ... evenly distributed on commodity hardware as they are at these new internet companies. Most companies will have servers or clusters with a specific function (such as, a 3 node cluster that runs the Oracle database that is the backend of their general ledger application), connected to a centralised SAN and disk array(s).
(Disclosure: I'm an Infrastructure Storage Architect. If all companies ran their core systems like Google and Facebook, I'd be out of a job)
posted by Diag at 2:40 AM on December 20, 2012 [1 favorite]
(Disclosure: I'm an Infrastructure Storage Architect. If all companies ran their core systems like Google and Facebook, I'd be out of a job)
posted by Diag at 2:40 AM on December 20, 2012 [1 favorite]
This thread is closed to new comments.
posted by mkb at 2:20 PM on December 19, 2012 [2 favorites]