Hosting web service on S3
September 24, 2009 12:49 PM Subscribe
Is it possible to host a commenting system on Amazon's S3 servers? I know most people use it for file storage and access, but would hosting a web service on their servers be possible?
Response by poster: Actually, it seems like Amazon SimpleDB would be even better. Thanks for the tip, GuyZero. I was unaware of Amazon's other offerings.
posted by bjork24 at 1:01 PM on September 24, 2009
posted by bjork24 at 1:01 PM on September 24, 2009
EC2 would work, but I would be interested in whether it's easier and cheaper than using a regular VPS hosting company like Rimuhosting ($35/mo includes a decent machine, modest storage, and a good amount of bandwidth).
posted by zippy at 1:28 PM on September 24, 2009
posted by zippy at 1:28 PM on September 24, 2009
SimpleDB is just a DB though. You can only access it programatically. If you want to create dynamic web pages you need EC2. If you already have a web server and just want to store data then SimpleDB is fine, although it's not a SQL database, it's really just a massive hashtable, so you can't just plug it in to most off-the-shelf commenting systems.
posted by GuyZero at 1:45 PM on September 24, 2009
posted by GuyZero at 1:45 PM on September 24, 2009
Response by poster: VPS might be a solution for a smaller site, but this is a massive property that averages nearly 90k new comments per month. I'm not sure a VPS could handle that kind of traffic.
posted by bjork24 at 1:49 PM on September 24, 2009
posted by bjork24 at 1:49 PM on September 24, 2009
You seem extremely confused. S3 is simple file storage; SimpleDB is a simple database. Neither one can "host" a commenting system. EC2 is Amazon's VPS-like solution, upon which you could run whatever commenting system you like. Unless they all come in at once, 90K messages per month is hardly "massive" and could easily be handled on a low-end VPS or small EC2 instance.
posted by nicwolff at 2:14 PM on September 24, 2009
posted by nicwolff at 2:14 PM on September 24, 2009
Yes, 90k comments a month is pretty "meh". That's ~100MB a year assuming 100 byte comments. 1K comments would be about a gig a year. Yawn.
posted by GuyZero at 2:49 PM on September 24, 2009
posted by GuyZero at 2:49 PM on September 24, 2009
Response by poster: But considering the site averages over 150m page views a month, the database would be polled too frequently for a small VPS to keep up.
posted by bjork24 at 3:07 PM on September 24, 2009
posted by bjork24 at 3:07 PM on September 24, 2009
But considering the site averages over 150m page views a month, the database would be polled too frequently for a small VPS to keep up.
Are those database-intensive page views? How intensive? A medium (say 512MB RAM instance) VPS should be able to tackle that. EC2 is really expensive compared to most VPS and dedicated service providers.
posted by signalnine at 3:21 PM on September 24, 2009
Are those database-intensive page views? How intensive? A medium (say 512MB RAM instance) VPS should be able to tackle that. EC2 is really expensive compared to most VPS and dedicated service providers.
posted by signalnine at 3:21 PM on September 24, 2009
Response by poster: Without breaking the NDA, I can say that the site is blog-like, where the comments only show on the permalink page. This is primarily just talk at the moment, but they're seriously considering creating an off-solution for commenting. I was under the impression that a cheap VPS would buckle under that sort of traffic.
posted by bjork24 at 3:36 PM on September 24, 2009
posted by bjork24 at 3:36 PM on September 24, 2009
You can buy/rent/lease a co-located server of whatever size and capacity you want. I'm no cap planning expert but you have a specific idea of disk, RAM & bandwidth you can get quotes from a lot of places. It's not like the only kind of hosted servers you can get are small ones.
EC2's main advantage is that you only pay for what you use whereas with a server rental/lease/VPS you get what you get and you may be over or under provisioned.
posted by GuyZero at 4:01 PM on September 24, 2009
EC2's main advantage is that you only pay for what you use whereas with a server rental/lease/VPS you get what you get and you may be over or under provisioned.
posted by GuyZero at 4:01 PM on September 24, 2009
90k comments per month is only two comments a minute on average. Let's say you get 10 times average at a time, that's still twenty a minute. That's nothing. Anything fast enough to run the software at all is fast enough to handle that kind of traffic. Most people greatly overestimate the kind of processing power they need.
90k comments sounds inconsistent with the 150M pageview figure you quoted, though. I'm wondering how they're serving that part of it?
posted by signalnine at 4:05 PM on September 24, 2009
90k comments sounds inconsistent with the 150M pageview figure you quoted, though. I'm wondering how they're serving that part of it?
posted by signalnine at 4:05 PM on September 24, 2009
Normally most of the page views wouldn't hit the database since those elements should be cached either by the Web server or the application. Only requests that update content should hit the database and invalidate the cache. Again, you should have a professional Web architect investigating this for you.
Amazon EC2 now offers automatic load balancing and scaling, so you can create various server disk images and then let Amazon boot as many of each as the load requires. It isn't simple to set up, but it is very flexible and just about infinitely scalable.
posted by nicwolff at 5:49 PM on September 24, 2009
Amazon EC2 now offers automatic load balancing and scaling, so you can create various server disk images and then let Amazon boot as many of each as the load requires. It isn't simple to set up, but it is very flexible and just about infinitely scalable.
posted by nicwolff at 5:49 PM on September 24, 2009
You should be able to stick that on a VPS with no problem. I recommend Linode. I'm just a happy customer, no referral link here.
No, you cannot host an entire site on S3. No, you cannot host an entire site on SimpleDB.
posted by Brian Puccio at 6:26 PM on September 24, 2009
No, you cannot host an entire site on S3. No, you cannot host an entire site on SimpleDB.
posted by Brian Puccio at 6:26 PM on September 24, 2009
EC2's main advantage is that you only pay for what you use whereas with a server rental/lease/VPS you get what you get and you may be over or under provisioned.
I'm not sure this is an entirely accurate description of how EC2 pricing works. You pay for how long each instance is "on", not for how many resources it uses. Since rebooting an EC2 instance means you effectively lose your machine (this is why you have EBS and things like S3, the former for your db, the latter for stuff like avatar uploads and the like), your machine will be ON all the time. So while it's true you "pay for only what you use", that happens in increments of instances that are typically 4 or 5 times more powerful than the smallest VPS you can get. If you're only using half-an-instance, tough luck. Anyways, this works out to about $70 a month for the single, smallest instance (or even as low as $40-ish, if you pay slightly more upfront to get a lower per-minute cost), a machine that will kick all kinds of ass over anything you'll get from the VPS providers (well.... see below).
The main advantage of EC2 is you can spin up a new box in like, under 10 minutes if necessary. The other advantage is you, in theory, have faster connectivity to the rest of amazon's web services than you would if you were coming from outside of amazon's network.
Still, I think EC2 is probably not suitable for anyone that has to ask if they should use it (because they should either know, or have employees that they can ask). EC2 is cheap as far as processing power goes, but often extremely expensive in developer/ops time. You will need someone who knows what they are doing to get started, and that person is not going to be cheap. If you think you will have scaling needs, you should probably look at a managed service running on top of EC2 (which is going to cost a poopload more than a regular ol' VPS, and waaaaay more than EC2, but is probably cheaper than hiring a dedicated ops team). Even then I'd expect you'd need developers to make scaling a possibility.
For your particular situation, I'd actually look at hosted commenting solutions unless there's some sort of customization needs that preclude them. At that point I'd look at rolling-something-yourself, and figuring out your needs by looking at historical usage. If you don't need to scale, you're on a tight budget, or you lack in-house technical know-how, EC2 is probably a poor choice.
posted by fishfucker at 7:57 AM on September 25, 2009
I'm not sure this is an entirely accurate description of how EC2 pricing works. You pay for how long each instance is "on", not for how many resources it uses. Since rebooting an EC2 instance means you effectively lose your machine (this is why you have EBS and things like S3, the former for your db, the latter for stuff like avatar uploads and the like), your machine will be ON all the time. So while it's true you "pay for only what you use", that happens in increments of instances that are typically 4 or 5 times more powerful than the smallest VPS you can get. If you're only using half-an-instance, tough luck. Anyways, this works out to about $70 a month for the single, smallest instance (or even as low as $40-ish, if you pay slightly more upfront to get a lower per-minute cost), a machine that will kick all kinds of ass over anything you'll get from the VPS providers (well.... see below).
The main advantage of EC2 is you can spin up a new box in like, under 10 minutes if necessary. The other advantage is you, in theory, have faster connectivity to the rest of amazon's web services than you would if you were coming from outside of amazon's network.
Still, I think EC2 is probably not suitable for anyone that has to ask if they should use it (because they should either know, or have employees that they can ask). EC2 is cheap as far as processing power goes, but often extremely expensive in developer/ops time. You will need someone who knows what they are doing to get started, and that person is not going to be cheap. If you think you will have scaling needs, you should probably look at a managed service running on top of EC2 (which is going to cost a poopload more than a regular ol' VPS, and waaaaay more than EC2, but is probably cheaper than hiring a dedicated ops team). Even then I'd expect you'd need developers to make scaling a possibility.
For your particular situation, I'd actually look at hosted commenting solutions unless there's some sort of customization needs that preclude them. At that point I'd look at rolling-something-yourself, and figuring out your needs by looking at historical usage. If you don't need to scale, you're on a tight budget, or you lack in-house technical know-how, EC2 is probably a poor choice.
posted by fishfucker at 7:57 AM on September 25, 2009
Also, keep in mind externally hosted commenting solutions are going to come at some sort of integration cost, some of which may end up being user-facing (ie, loading via Ajax or some other sort of solution, for those pages that need to combine blog posts and comments). You may be best off sticking the entire app on something with a little more power if you're running into performance problems. There's a host of small-fix things to try before you do this, however. You could get better answers from someone who could spend a couple hours analyzing your bottlenecks.
posted by fishfucker at 7:59 AM on September 25, 2009
posted by fishfucker at 7:59 AM on September 25, 2009
This thread is closed to new comments.
posted by GuyZero at 12:49 PM on September 24, 2009