Software development: How to best get useful feedback from users?

October 25, 2007 1:51 PM

Software development: How to best get useful feedback from users?

We are developing a customer service / tech support customer management software. It is web-based and being developed entirely in-house.

We have a call center staff who is not hugely technical oriented. They will be the primary users of this system. They have been using/testing some of the system in early stages of development.

The biggest issues have been how it works for these users. We want the system to work for them and realize that their feedback/input will be hugely valuable to making this system the best it can be.

Is there any standard method of getting feedback from system testers like this? Any list of questions that will help them think about how the systems works and be able to give suggestions that will benefit the development towards improving the various aspects?

I am having a hard time figuring out how to best facilitate the feedback process and make it easier for the users to put their feedback into words that I can take and formulate into a document that will support the developers in understanding the point of view of the users and how the system works for them.

Any suggestions very much appreciated!

posted by doomtop to computers & internet (11 comments total)

list of questions is a good idea.
Other than that how about sending a mail to them saying that on a specific day you will be there to talk to them, and if they have any requests/improvements they can talk to you.

If they are paid hourly, mention that they still get paid for the time they spend with you, otherwise they may not want to spend their time talking to you.
posted by WizKid at 2:06 PM on October 25, 2007

Thanks for the response WizKid!

I won't have any problem getting them to talk to me and there is no concern about being paid to work with me. I am not so much looking for ways to get them to talk to me, but more so how to make the conversation we have more effective.

I agree that a list of questions is a good idea, but what kind of questions get the best responses?
posted by doomtop at 2:12 PM on October 25, 2007

I've wondered how a feedback form on the bottom of every single page would work. It could be labeled something like "Have a question or comment?" and allow users to report problems on any page they're visiting. The form could include the page the user's on, as well as useful information about the user (to help debugging) and send the reports directly to the software developers. This type of system would let your users contact you without needing to open up their email client ("just fill in a form and click send!") and it would also give you information about the site, such as "problem pages" (pages with the most number of complaints), etc.
posted by null terminated at 2:16 PM on October 25, 2007

The best way to get feedback is to watch over their shoulder as they use it. Flat out, you'll get more info that way than any list of questions.
posted by SpecialK at 2:17 PM on October 25, 2007

It may also be useful to know that the staff is a group of about half dozen who work closely together in person. The system is designed to facilitate their work and allow collaboration, while keeping track of the customers and workload.
posted by doomtop at 2:23 PM on October 25, 2007

I personally use null terminated's idea and it works surprisingly well. Don't give them a lot of radio/checkbox options, just a simple link with a feedback textarea and you'll be surprised.

Within two months of a new product release I received almost 3-400 feedback submissions. Some were bogus, but a good part of them fed into the next release.
posted by purephase at 2:30 PM on October 25, 2007

Assign one of the team the role of collecting complaints and requests from the users. Meet with this person from time to time. The users will already have a conversational relationship with the "usability coordinator" and the issues can be brought to your attention anonymously.

A feedback link is always useful, as well.
posted by rhizome at 2:54 PM on October 25, 2007

If you're in the development phase, it helps to have them try to use wireframe prototypes. So make up some html mockups of what they would see and observe how they use it and what they find confusing.
posted by advicepig at 4:02 PM on October 25, 2007

You might look into the XP user stories model of gathering requirements and doing design. It's hard to tell if you're too far along in the process for this, but combining that with a flexible development approach (specifically, one that makes it easy to rewrite elements that don't work well for the users) is a very effective way of writing software fast to meet their requirements.
posted by ch1x0r at 4:57 PM on October 25, 2007

SpecialK's suggestion is great - you need to understand the reality of their job, not just the standard process (and it's great PR for your team). The issues tend to come up in the non-standard situations that may not have been a part of the original business analysis - these are the issues that cause problems later on.

My approach is to set up User Groups early on in the process. If the company culture is that applications are released without much consultation and users just have to use them, they will always find convoluted ways of getting around problems, and probably won't report them, because they don't think they'll ever be resolved. So you end up with an application that you think works fine, but users hate.

In your case, your users are your User Group. Get them all together. Talk them through what they do on a day to day basis. Ask them about situations that don't fit the standard model ("but what if they...?"), and how they handle it. If anything they say sounds inefficient or convoluted, get a discussion going about how it could be better handled. Encourage them to think about their "wish list" - often a lot of suggestions are really simple to implement but can make a huge difference to their day-to-day work (quick wins).

I don't have any standard questions, it's more about getting a discussion going and listening really carefully to what they're saying. And making sure that I understand the process - they're the experts, not me. Users may not be able to formulate feedback in a way that you need - that's why the group discussion format works - you translate the dicussion into actual requirements in a way that they can't always do.

And make sure you deliver the changes they need - once you've done that once, they'll be on your side forever. But still meet with them regularly to identify further improvements.

Good luck!
posted by finding.perdita at 5:08 PM on October 25, 2007

I've always gotten better data, and less noise, by just videotaping people using the application. Especially for the first time. Especially with nobody helping them and no manuals.

You can get by with screen recordings, but actual video with the user in-frame (complete with puzzled expressions) is the best thing.

People, sadly, don't know what they want when you ask them. But their fingers and and their mouse know how they EXPECT the thing to work. Watch.
posted by rokusan at 7:38 PM on October 25, 2007