Help keep this community alive
February 11, 2008 6:09 PM Subscribe
How can we reign in the costs of running this forum?
I've been asked to submit a proposal for reigning in costs and improving functionality on a highly trafficked/used web forum. vBulletin is currently being used. Membership numbers into the 65k area. Obviously the number of regular users is much smaller. But those that use the site are heavy users. They use smilies, galleries, graphic signatures, avatars, etc. Lots of bandwidth.
One of the things they are open to is moving to another software solution - IF the migration wouldn't lose all the data. Possible?
The site is set up across three servers at FastServers and they are being charged about $1700/mo. I *know* there has to be a way to improve this without having to tell the membership that they can't use their beloved smilies any longer.
Seriously - they love those freakin' smilies.
I'm happy to share the site via email or MeFi mail if there is someone who is willing to provide info.
BONUS: Currently they are not monetizing the site very well. I'm going to suggest adsense with some deep integration. Other suggestion welcome.
I've been asked to submit a proposal for reigning in costs and improving functionality on a highly trafficked/used web forum. vBulletin is currently being used. Membership numbers into the 65k area. Obviously the number of regular users is much smaller. But those that use the site are heavy users. They use smilies, galleries, graphic signatures, avatars, etc. Lots of bandwidth.
One of the things they are open to is moving to another software solution - IF the migration wouldn't lose all the data. Possible?
The site is set up across three servers at FastServers and they are being charged about $1700/mo. I *know* there has to be a way to improve this without having to tell the membership that they can't use their beloved smilies any longer.
Seriously - they love those freakin' smilies.
I'm happy to share the site via email or MeFi mail if there is someone who is willing to provide info.
BONUS: Currently they are not monetizing the site very well. I'm going to suggest adsense with some deep integration. Other suggestion welcome.
One of the things they are open to is moving to another software solution
If this is simply an issue of total bandwidth and costs, I don't think moving to another solution will solve your problems. Typically the reason for upgrading software is to get something more robust. If your forum software isn't struggling with the active member load, sticking with vBulletin should be fine.
posted by junesix at 6:27 PM on February 11, 2008
If this is simply an issue of total bandwidth and costs, I don't think moving to another solution will solve your problems. Typically the reason for upgrading software is to get something more robust. If your forum software isn't struggling with the active member load, sticking with vBulletin should be fine.
posted by junesix at 6:27 PM on February 11, 2008
Yeah, a breakdown of the costs would help to figure out where the fixable problems are, but AdSense + subscriptions to remove ads are one way of pulling in money, as are board sponsorships like approaching Newegg to sponsor the graphics card board, a gaming retailer to sponsor the Solitaire board, and so on, depending on your subject matter. Also consider not hosting the images in postings, suggesting the use of a place like PhotoBucket to defray bandwidth loads.
posted by rhizome at 6:36 PM on February 11, 2008
posted by rhizome at 6:36 PM on February 11, 2008
1. Assuming you're using a standard forum smiley set, I find it hard to believe that those are using up as much bandwidth as you imply. So you need to verify that smileys are cacheable.
2. Find out what is really eating your bandwidth. Any log analysis product will do this (you don't mention what platform you are on, but awstats might work for you) but I don't believe the hosted solutions (urchin, mint, ...) work this way.
3. You can stop hosting sigs yourself, or move them to much cheaper hosting (I do some free image hosting for a similarly-sized, but very graphically intensive forum on dreamhost's cheapest account.) Honestly, there's so much cheap/free image hosting out there, there's no real need to do this yourself with premium bandwidth/servers.
4. Do you compress your html?
5. Three servers? One web server, one db server, one static content server? As mentioned, you can probably shift much of the static content to cheaper/free hosting. Now can you combine the web and db servers? You might need to ditch a forum mod or two (some are apparently horribly written and really load the db.)
6. As you don't seem to be in this for the money, one thing to consider for monetisation -- only show ads to non-logged-in users, or people coming from search engines.
posted by rjt at 6:41 PM on February 11, 2008
2. Find out what is really eating your bandwidth. Any log analysis product will do this (you don't mention what platform you are on, but awstats might work for you) but I don't believe the hosted solutions (urchin, mint, ...) work this way.
3. You can stop hosting sigs yourself, or move them to much cheaper hosting (I do some free image hosting for a similarly-sized, but very graphically intensive forum on dreamhost's cheapest account.) Honestly, there's so much cheap/free image hosting out there, there's no real need to do this yourself with premium bandwidth/servers.
4. Do you compress your html?
5. Three servers? One web server, one db server, one static content server? As mentioned, you can probably shift much of the static content to cheaper/free hosting. Now can you combine the web and db servers? You might need to ditch a forum mod or two (some are apparently horribly written and really load the db.)
6. As you don't seem to be in this for the money, one thing to consider for monetisation -- only show ads to non-logged-in users, or people coming from search engines.
posted by rjt at 6:41 PM on February 11, 2008
Is this on the most current version of vB? I recently upgraded a client's vB installation to 3.6.8, and it was remarkably faster than his old 2.xx version. The more recenty versions also have subscription-based capabilities, so (as suggested above) you could combine subscriptions with removing ads or granting other capabilities.
Also, is it really necessary to host avatars on your servers? Seems to me vB allows you to have avatars hosted elsewhere, if the user puts the image's URL in the appropriate place in his/her profile.
posted by Robert Angelo at 6:54 PM on February 11, 2008
Also, is it really necessary to host avatars on your servers? Seems to me vB allows you to have avatars hosted elsewhere, if the user puts the image's URL in the appropriate place in his/her profile.
posted by Robert Angelo at 6:54 PM on February 11, 2008
Seems to me vB allows you to have avatars hosted elsewhere, if the user puts the image's URL in the appropriate place in his/her profile.
Do this with caution. I've seen too many forum pages bogged down mid-load by waiting for imageshack or some other free imagehost to respond.
posted by chrisamiller at 7:03 PM on February 11, 2008
Do this with caution. I've seen too many forum pages bogged down mid-load by waiting for imageshack or some other free imagehost to respond.
posted by chrisamiller at 7:03 PM on February 11, 2008
What sort of defaults do you have set?
What sort of maximum limits do you have?
If you're letting people display 200 posts per page and they're consistently slurping that down although they're only reading the last 10, you need to rejigger the settings.
If you're showing 50 posts a page by default, that's too much.
Also, it's hard to tell without the details of your hosting contract, but $1700 for 3 servers seems to be possibly $500 or more too high.
I also wouldn't poo-poo possibly visiting performance issues -- maybe if you aggressively cache you can get away with only using two servers -- get someone who knows something about server tuning to take a look.
Effectively, though, it sounds like you've got an issue with hosting media -- have you looked into people who provide solutions for these sorts of things? Although a lot of content delivery is geared towards speed rather than savings, I'm sure there MUST be some solutions out there that sacrifice in the other direction.
More specific information, if you can reveal it (maybe even the URL), might help people make some more suggestions.
oh, and GZIP compression should be on. That should be just a flipped switch for you and you'll immediately see results. Also very heavy cache directives for non-logged in users.
posted by fishfucker at 7:11 PM on February 11, 2008
What sort of maximum limits do you have?
If you're letting people display 200 posts per page and they're consistently slurping that down although they're only reading the last 10, you need to rejigger the settings.
If you're showing 50 posts a page by default, that's too much.
Also, it's hard to tell without the details of your hosting contract, but $1700 for 3 servers seems to be possibly $500 or more too high.
I also wouldn't poo-poo possibly visiting performance issues -- maybe if you aggressively cache you can get away with only using two servers -- get someone who knows something about server tuning to take a look.
Effectively, though, it sounds like you've got an issue with hosting media -- have you looked into people who provide solutions for these sorts of things? Although a lot of content delivery is geared towards speed rather than savings, I'm sure there MUST be some solutions out there that sacrifice in the other direction.
More specific information, if you can reveal it (maybe even the URL), might help people make some more suggestions.
oh, and GZIP compression should be on. That should be just a flipped switch for you and you'll immediately see results. Also very heavy cache directives for non-logged in users.
posted by fishfucker at 7:11 PM on February 11, 2008
Reining in, as horses. Not reigning, as kings.
posted by zadcat at 7:17 PM on February 11, 2008 [1 favorite]
posted by zadcat at 7:17 PM on February 11, 2008 [1 favorite]
One more thing, from our old 2.xx version of vB: Hiring a MySQL professional to review and revise the database configuration was more than worth the money.
posted by Robert Angelo at 7:42 PM on February 11, 2008
posted by Robert Angelo at 7:42 PM on February 11, 2008
I 2nd rjt's suggestion to make absolutely sure the images are being HTTP cached as much as they possibly can so they get saved in the visitor's web browser or in a proxy cache instead of getting downloaded every time.
And a separate static content server on a cheaper host if it really is the images that are consuming the bandwidth.
posted by XMLicious at 7:57 PM on February 11, 2008
And a separate static content server on a cheaper host if it really is the images that are consuming the bandwidth.
posted by XMLicious at 7:57 PM on February 11, 2008
Response by poster: Thanks to all.
Couple of notes/answers:
1. smilies and galleries are non-negotiable. There are a couple of custom smilies, but I believe the majority if them are standard.
2. I don't have access (as yet) to much of the information/analysis I would like to - which makes everything harder, of course.
3. The community is a non-technical group. So non-technical that a good chunk of members have a hard time figuring out how to post to begin with. Off site hosting would likely be a support nightmare.
4. It's true, not in this for the money and will likely offer the ability to turn off ads for those that are logged in.
5. Agreed about the contract - I have to believe there's better deals out there. Moving the setup isn't something high on the list of things any of us want to do.
Will check into several of the options above. Please keep thoughts coming.
posted by FlamingBore at 7:59 PM on February 11, 2008
Couple of notes/answers:
1. smilies and galleries are non-negotiable. There are a couple of custom smilies, but I believe the majority if them are standard.
2. I don't have access (as yet) to much of the information/analysis I would like to - which makes everything harder, of course.
3. The community is a non-technical group. So non-technical that a good chunk of members have a hard time figuring out how to post to begin with. Off site hosting would likely be a support nightmare.
4. It's true, not in this for the money and will likely offer the ability to turn off ads for those that are logged in.
5. Agreed about the contract - I have to believe there's better deals out there. Moving the setup isn't something high on the list of things any of us want to do.
Will check into several of the options above. Please keep thoughts coming.
posted by FlamingBore at 7:59 PM on February 11, 2008
Yeah... rjt's got the right idea -- get a grip on what you're using in terms of resources. It might not be as much as you think, but it's impossible to make informed decisions without at least an overall look at the data. Are you seeing performance problems? What kinds?
Also... what's going on with those servers? What are the specs? What does each one of 'em do?
$1,700 could be a ripoff or it could be a steal, depending on the quality of the hardware, level of support, etc... if you're looking to defer costs, sometimes hosting companies are cool to do a "Hosted by..." ad deal for a price break.
As far as moving between systems, I've had to do it once or twice -- I used to run forums for a living during the height of the dotcom era -- and really, any competent programmer should be able to move the data around enough that it'll work. It might not be perfect, but forums are pretty much all the same in the end... at least as far as threaded messages attributed to particular users go.
You mention galleries? I'd worry about the loss of any of the extra modifications you've added. If you've got all kinds of stuff tied into the forums, then it's probably going to be a bigger headache then you can imagine, if only it's going to piss off the users who've finally gotten used to the quirks your particular setup.
posted by ph00dz at 9:23 PM on February 11, 2008
Also... what's going on with those servers? What are the specs? What does each one of 'em do?
$1,700 could be a ripoff or it could be a steal, depending on the quality of the hardware, level of support, etc... if you're looking to defer costs, sometimes hosting companies are cool to do a "Hosted by..." ad deal for a price break.
As far as moving between systems, I've had to do it once or twice -- I used to run forums for a living during the height of the dotcom era -- and really, any competent programmer should be able to move the data around enough that it'll work. It might not be perfect, but forums are pretty much all the same in the end... at least as far as threaded messages attributed to particular users go.
You mention galleries? I'd worry about the loss of any of the extra modifications you've added. If you've got all kinds of stuff tied into the forums, then it's probably going to be a bigger headache then you can imagine, if only it's going to piss off the users who've finally gotten used to the quirks your particular setup.
posted by ph00dz at 9:23 PM on February 11, 2008
They use smilies, galleries, graphic signatures, avatars, etc. Lots of bandwidth.
Dont host their images. Make them get their own hosting and you'll have pretty small bandwidth bills.
posted by damn dirty ape at 9:47 PM on February 11, 2008
Dont host their images. Make them get their own hosting and you'll have pretty small bandwidth bills.
posted by damn dirty ape at 9:47 PM on February 11, 2008
If totally cutting out the hosting of user images, perhaps set limits on what you'll host for them, and charge for increased storage. You should be able to limit the dimensions of uploaded images as well, I've found that a surprising number of people will upload 5+ megapixel photos only to display them at 640x480.
Or you could just go wild and charge $5 per smiley.
posted by jjb at 11:11 PM on February 11, 2008
Or you could just go wild and charge $5 per smiley.
posted by jjb at 11:11 PM on February 11, 2008
Wow, I fail. That should say "If totally cutting out the hosting of user images is not an option..."
posted by jjb at 11:12 PM on February 11, 2008
posted by jjb at 11:12 PM on February 11, 2008
You should be able to get some ideas using the YSlow Firefox extension. As suggested above, pushing as much of the design into external CSS, putting all the JavaScript into external files, then compressing them would make a big difference if its not already happening. I have some bookmarks on this kind of stuff at delicious, but I should warn you my del.icio.us could use some optimization itself (it will hang your browser for 15-30 seconds).
posted by yerfatma at 6:40 AM on February 12, 2008 [1 favorite]
posted by yerfatma at 6:40 AM on February 12, 2008 [1 favorite]
Yeah, you should minimize the JS and compress/reduce the CSS if possible. That's a great suggestion.
However, I wouldn't worry too much about 'bad' YSlow results -- read this article by Jeff Atwood about why Yslow results may not be a concern for your website. In summary: it's a good guidline, but scoring poorly doesn't necessarily mean you've got something that needs to be fixed. Atwood also has a post about reducing your website's bandwidth which might be useful to you as well.
posted by fishfucker at 9:23 AM on February 12, 2008
However, I wouldn't worry too much about 'bad' YSlow results -- read this article by Jeff Atwood about why Yslow results may not be a concern for your website. In summary: it's a good guidline, but scoring poorly doesn't necessarily mean you've got something that needs to be fixed. Atwood also has a post about reducing your website's bandwidth which might be useful to you as well.
posted by fishfucker at 9:23 AM on February 12, 2008
If it's busy and it uses MySQL... alter the table types from MyISAM to InnoDB.
MyISAM is indescribably evil. Actually, just look very hard at the database generally or find someone who can do it for you.
posted by genghis at 5:20 PM on February 12, 2008
MyISAM is indescribably evil. Actually, just look very hard at the database generally or find someone who can do it for you.
posted by genghis at 5:20 PM on February 12, 2008
I wouldn't worry too much about 'bad' YSlow results
Crap, that's a really good point and I forgot to mention that. I meant that it might show you some bottlenecks you didn't know you had (looking at the page's pieces' load times in Firebug is also helpful). YSlow's suggestions about edge networks and all that are for people working at Yahoo, not normal sites. Atwood's article is spot-on.
posted by yerfatma at 5:57 AM on February 16, 2008
Crap, that's a really good point and I forgot to mention that. I meant that it might show you some bottlenecks you didn't know you had (looking at the page's pieces' load times in Firebug is also helpful). YSlow's suggestions about edge networks and all that are for people working at Yahoo, not normal sites. Atwood's article is spot-on.
posted by yerfatma at 5:57 AM on February 16, 2008
This thread is closed to new comments.
The more drastic measure would be to impose some limits on the graphics. Announce that you'll be removing all smilies and avatars over a certain filesize and give users a chance to replace them with more optimized images or change altogether. Take a look at your galleries section and see if it's worth keeping vs. your users' wishes and income from the gallery pages.
On the monetization side, you may want to consider selling banner space or sticky threads, ie. allow an advertiser to buy a permanent spot as the first topic in a forum category.
Lastly review your contract with FastServers and do some shopping around.
posted by junesix at 6:23 PM on February 11, 2008