What should we test for our web software pilot?

June 3, 2008 8:24 AM

Doing a "pilot" for an new internet service product, what should be in our plan to verify it is a viable idea?

So the company I work for is new and will be launching a new internet service product. We are going to do a pilot to verify that the idea makes sense, people will use it, etc before we go searching for round 2 of investor money. We currently have private investors.

So we need to do a valid test to make sure of a number of things such as:

What price will people pay, if they pay at all? What is the best way to test this price point, etc?
Are we getting the demographics we expect?
Can our servers handle the load?

As well as thinking about future business plans:

What happens if we succeed? How do we grow the staff, etc?



So without telling you any more specifics and without listing everything we already have...

Any suggestions?

What things need to be tested/proven out for a new internet service to verify it is a good service and a valid business & technical model?

Suggested websites, etc would be a great help as well.

Assume this is not similar to much else out there and is essentially a new service so we cannot easily compare to rivals because there are not enough that are close enough to our space.

posted by UMDirector to computers & internet (3 comments total)

Whoa. From the wording of this question it appears that you have no technical or business people on staff. Is this really true? If so, that's going to be a pretty big stumbling block even if you manage to get the answers to your questions.

The first thing that jumps out at me is your reluctance to list anything specific other than "How do we know how much money we can make?" and "What happens when (not if) we are popular?" I'm no entrepreneur, but what I do know says that there are a lot of steps in between doing your first user tests and being successful. One step might be to consider that these tests may not give you what you're looking for, since you are not popular and are not making money. This is to ask, how do you know these tests won't be a waste of money?

The second thing, which actually follows from the first, is that your reluctance to list anything specific tells me that you think this idea is super unique, possibly a lightning bolt out of the blue into the world of popular culture and/or business. I can almost guarantee this is not the case, and that there are probably several other companies doing (or trying to do) exactly what you're doing. One step in figuring out the answers to your questions is to figure out who your competition is and to look into how they're doing what they do. If you don't think you have any competition, then the idea is either genius or has been previously rejected by the marketplace. Seriously, spend some time wondering why there is not much similar out there. "People are dumb" is not a good answer to this quandary and will likely not help you in your second round of funding.

I'm not trying to discourage you, but you ask questions about verifying things that cannot be verified (due to our inability to predict the future), and you ask questions about qualities of the business itself that should be answerable by the people who already work there.

So, things to test for:

1) Usability. Do regular people like to use your service? Is it confusing to interact with?
2) Server metrics. You should be using monitoring tools to gauge your capacity and to inform your technical planning.
3) Marketing surveys. "Would you pay for a subscription to this service?" (or similar) is probably as specific as you'll be able to get at this point. Ask other questions related to usability and mock up various versions of the service geared toward prospective revenue models.

Good luck!
posted by rhizome at 10:26 AM on June 3, 2008

I can't give you any specifics, because you didn't really share any with us. But I can tell you that I'm not going to pay for a service that doesn't have a good track record. I paid for this because I've been reading it for a while and knew that it would benefit me. However, I don't pay for other fora because I know I can get the information elsewhere.

As far as testing the price point for your product, would it hurt to ask the people testing it for you?
posted by theichibun at 1:13 PM on June 3, 2008

Your question is too vague to give any but vague answers. But I'll give it a shot:

Option 1: Spend your money on a market research firm. They will get all the information about interest, price points, demographics and general audience response. It will probably be expensive, especially if you have tight target demographics. However, if you're shooting for 2nd round VC, you're going to need some solid numbers to make that money come in, and a market research firm will produce a nice slick report. They can also give recommendations that can help skew demographics or price points. How much and how good the information you get out of them is up to how much you're willing to spend. Don't go cheap.

Option 2: Do a limited (invite only) free beta. Have an engineer spend an afternoon coding up how to handle the invite codes, send out a few press releases, and invites to prominent people, blogs that might cover your service, sites like InviteShare, etc. Bring in a marketing consultant to write some surveys for you. This will answer most of your questions including server load. (Though your engineers really should be able to answer that question within tolerances. If they can't, get new ones. I'm very serious.)

Use option 1 if you won't be able to produce a working beta with your R1VC. Option 2 if it's at all possible to get something working in front of eyes.

"What happens if we succeed?"

Right now worry about getting a product out there. Success is best dealt with as it happens, any energy worrying about it now is wasted.

And read Getting Real. Valuable information for anyone starting internet based service.
posted by Ookseer at 3:31 PM on June 3, 2008