How long is "beta" supposed to last?

December 28, 2004 2:58 PM

How long is "beta" supposed to last?
posted by brent to computers & internet (14 comments total)

It's a vague question, and any answers will be just as vague. I'm assuming that you're talking about beta software. If so, then the answer is "as long as the software house says". The term "beta" can be used correctly to signify that a piece of software is being tested by a number of people in the real world. The beta for this will usually finish after a set period. This period is larger for more complicated software. Think anything from a month to a year.

In recent times, software houses use the phrase "beta" to simply mean "software that is not finished yet". This software will be pushed out to beat the competition to market, or to keep users happy and usually has nothing to do with testing. These beta versions can be in circulation for months and months.

There is also a category of software which is never finished, and is always on "beta". This can be used to cover up the fact that the software isn't very good and probably doesn't work properly.

So in answer to your question... It depends.
posted by seanyboy at 3:08 PM on December 28, 2004

til it's done, I'd say.
posted by John Kenneth Fisher at 4:13 PM on December 28, 2004

Yeah, it's gotten very vague over the years. I mean, my understanding of the term is that it describes software that is feature complete but still being tested to remove bugs. But the more general definition that seanyboy describes is becoming more common.
posted by selfnoise at 4:16 PM on December 28, 2004

Yeah...the term "beta" has been stretched beyond all recognition (especially by Microsoft).

It used to mean a very specific period in the testing phase. "Alpha" usually meant a test phase under very controlled conditions, either entirely internal, or with a very small group of external testers. "Beta" generally meant having a feature-complete app released to a wider group of external testers, but still an environment where (a) you knew everyone who had it, and (b) they had an obligation to report and help squash bugs.

_Some_ companies still handle "beta-testing" that way, but many others (again, especially Microsoft) now just use it to mean "unsupported". The "beta" label is supposed to add enough cachet that you don't care about the fact that you're being thrown to the wolves, support-wise.
posted by LairBob at 4:19 PM on December 28, 2004

It lasts until gamma.
posted by sic at 4:48 PM on December 28, 2004

A product isn't declared "beta" until it's feature-complete. Prior to this point, it's "alpha." Beta lasts until all of the big, nasty, easily reached bugs are squashed and the product team feels that it can safely be released.
posted by glyphlet at 4:57 PM on December 28, 2004

Yeah...the term "beta" has been stretched beyond all recognition (especially by Microsoft).

Actually, I think Microsoft is less 'guilty' of this than other companies, especially Google which still has News in beta. MS does long betas for their operating systems and Office because they need to real world test the largest combination of hardware and other factors you can imagine. Google News is still in beta because they haven't figured out how to monetize it without raising legal issues but Google Froogle has been in beta as long or longer and that one I'm not really sure why. I think these two instances are the one most responsible for perceptions like Brent's. FYI, I used to be a product manager and ran betas for three different companies, none of which had cycles that lasted more than three months but all of which could have used a bit more baking.
posted by billsaysthis at 8:01 PM on December 28, 2004

what has ms done that doesn't reflect the usual definition of beta? as billsaythis says, it's google that springs to mind as the big abuser (and open source stuff that simply never gets finished, but that's more an annoyance than an abuse of the term).
posted by andrew cooke at 6:09 AM on December 29, 2004

Well, I'm not arguing that MS is alone in this, but just look at the new "Desktop Search" tool, at (of all places) http://beta.toolbar.msn.com/

Yes, a lot of other places are doing it, too, but they're blatantly just using some kind of vague "open beta" phase to get a big PR push before they have to actually stand behind the product.

There's little or no "bug tracking" involved--no mention on that home page of this being in some kind of "test phase", no bug-tracking instructions in the installation process (that I could see), and nothing built into the tool itself to help report and track bugs. It's got the standard "I want to participate in the Customer Experience Quality Program" they're putting into everything now, but that's not bug-tracking.

Sure, they'll eventually release it as a shrink-wrapped product, but they're using the "beta" label as a PR ploy, not as some kind of legitimate product development/QA phase.
posted by LairBob at 6:56 AM on December 29, 2004

ah, ok. maybe it's just that i don't use ms's software so much (although i work a lot on win2k i use cygwin mainly), but i'd never seen that before.
posted by andrew cooke at 10:21 AM on December 29, 2004

Joel on Software has some advice for running beta tests.
posted by mmascolino at 12:51 PM on December 29, 2004

LairBob, I think MS does get useful information out of the Customer Experience Quality Program, not to mention that I'm pretty sure they use Windows' phone home utility (the request to send an error report any time an application crashes) extensively. And other collection methods/channels. Google, OTOH, has none of these and frankly I'm not sure how Google News could even be considered a beta except for blocking the potential liability involved.
posted by billsaysthis at 5:51 PM on December 29, 2004

Where I've worked in the past Beta means "feature complete, but not bug free." Beta lasts until you have fixed all A-class (crasher or other high priority failure), all B-class (serious or failure to meet spec) and until all C-class (cosmetic defect) bugs have either been fixed or considered acceptable by management and QA. C bugs may be closed acceptable or deferred to a later release. At this point, the software is RC (release candidate). There is no fixed amount of time, but sometimes management planned for intermediate releases on a 2 week cycle, so beta 1 lasts 2 weeks before beta 2.
posted by plinth at 6:51 PM on December 29, 2004

Umm, hopefully you have a program manager who defined what alpha, beta and RC (or GM) entail. I generally always defined them thusly:

Alpha: not feature complete, fully functional or even tested, but "up and running" enough to bang away and play with it. A good time to begin bug reporting. Alpha ends when development has completed coding the features in the spec into a somewhat stable build, with proof the features are at least testable. A good PM won't allow graduation to beta until this is the case.

Beta: Development of features is no longer a consideration. If something didn't make it into the product at this point, it is cancelled (and removed from all packaging, promotional materials, etc.) and scheduled for the .1 rev. This is the real deal testing period. All features should be tested to the point of bug resolution. If bugs persist in a feature that is critical, it delays release. But it's dangerous to begin removing features, as they could introduce new bugs; this is supposed to be your end product.

Beta ends when development has resolved all high priority bugs. Low priority bugs deserve scrutiny and review for their effect on the release. In a perfect world, you think you're bug-free.

Now you go into a Release Candidate, with the objective being your Gold Master. More bug squashing, lots of config. testing, redundancy, bug resolution verification. Re-verification of old resolved bugs. More config. testing. Ahhh. Looks good. Now burn that GM CD and test it too. Bug count = 0? Good to go.

I have seen alpha last months, beta last months, RC last weeks. But ideally, alpha: months (10+ builds); beta: weeks (6+ builds); RC: days (4- builds).
posted by ValveAnnex at 10:05 PM on December 29, 2004