Help me improve the experience of testing the experience
June 29, 2010 1:50 AM Subscribe
How does your IT department introduce new widescale products to your large company? I'm talking workstation policies, web browsers, collaboration and office applications, intranet. If you're an end user what is your company doing well in this regard? Are you invited to test the product? Do they test the subjective user experience? Do they make it easy for you test and give feedback? What would you change? And if you're in the IT department, what works well? What doesn't? What have you done to improve things.
In a company of over 7500 staff, the functionality, usability and overall experience of these tools can have dramatic effects on the workforce, both good and bad. I want to help improve how we introduce tools to our end-users and would also appreciate any good links, articles, best practice that could help in my research.
In a company of over 7500 staff, the functionality, usability and overall experience of these tools can have dramatic effects on the workforce, both good and bad. I want to help improve how we introduce tools to our end-users and would also appreciate any good links, articles, best practice that could help in my research.
My experience in large companies has been similar to that of rhapsodic.
Generally, the thinking seems to be that they want to roll out a particular change as quickly and efficiently as possible, and, in order to avoid the problem of too many cooks spoiling the broth, roll out the changes en masse, without regard to how the end user is affected.
Frankly, though this is annoying for the end user, I don't really see what other alternative IT departments in large organizations have, especially organizations where the majority of staff are not technically proficient. Of course, given that I like to think of myself as more technically savvy than the median, I find this practice annoying.
This is probably why I work at a much smaller company now.
posted by dfriedman at 6:07 AM on June 29, 2010
Generally, the thinking seems to be that they want to roll out a particular change as quickly and efficiently as possible, and, in order to avoid the problem of too many cooks spoiling the broth, roll out the changes en masse, without regard to how the end user is affected.
Frankly, though this is annoying for the end user, I don't really see what other alternative IT departments in large organizations have, especially organizations where the majority of staff are not technically proficient. Of course, given that I like to think of myself as more technically savvy than the median, I find this practice annoying.
This is probably why I work at a much smaller company now.
posted by dfriedman at 6:07 AM on June 29, 2010
I recently went through a rollout that wasn't handled well. The biggest lesson I learned is that the companies who do this well often use a concerted, considered adoption plan. They typically consist of things such as but not limited to:
1) Make sure there is an upper level manager/exec who acts as the champion/evangelist for the firm's need for this program/application.
2) Get your early adopters/power users (typically folks on the IT staff) to start using it first as a pilot group. They become the subject matter experts (SMEs).
3) Enlarge the pilot group to a few key business stakeholders. Make sure to recruit people who will use it differently (like, say, HR and marketing); you want to be prepared for the different use case scenarios.
4) Refer the larger pilot folks to the SMEs for informal questions & lessons learned.
5) Set up formal training sessions, but remember that training consists of two things: a) HOW to use it, and b) WHY they would want to use it.
6) Distribute cheat sheets on 4a and 4b.
7) Make available online training/tutorials from 4a and 4b for ongoing learning.
8) Get upper level managers to model the use of the program/application for their teams.
posted by ImproviseOrDie at 6:52 AM on June 29, 2010 [2 favorites]
1) Make sure there is an upper level manager/exec who acts as the champion/evangelist for the firm's need for this program/application.
2) Get your early adopters/power users (typically folks on the IT staff) to start using it first as a pilot group. They become the subject matter experts (SMEs).
3) Enlarge the pilot group to a few key business stakeholders. Make sure to recruit people who will use it differently (like, say, HR and marketing); you want to be prepared for the different use case scenarios.
4) Refer the larger pilot folks to the SMEs for informal questions & lessons learned.
5) Set up formal training sessions, but remember that training consists of two things: a) HOW to use it, and b) WHY they would want to use it.
6) Distribute cheat sheets on 4a and 4b.
7) Make available online training/tutorials from 4a and 4b for ongoing learning.
8) Get upper level managers to model the use of the program/application for their teams.
posted by ImproviseOrDie at 6:52 AM on June 29, 2010 [2 favorites]
1) Make training a component of the roll-out plan from day one (full-disclosure: I'm a software trainer). And include time for developing whatever in-house training will have to be developed. That means giving the training developers access to the software early in the process and, if it's in-house developed software, setting a feature freeze date so the existing product can be documented.
2) Stage the roll-out. Don't blast the whole organization with new 'ware overnight. Try it in a few departments first. Record the feedback and learn from it. Find the pain points and address them in the software configuration (if possible) or in the training.
3) Screencasts are your friend. Shoot as many features as possible, not in one long video but in discrete, reusable units. Then tie them together with a table of contents or some other navigation. If you shoot it with enough granularity, you can create customized aggregations of these for different departments/divisions, focusing on what they need. If your screencasting product supports text-to-speech, you can speed up development time and make it all 508-compliant from the get go. You'll likely also need more traditional documentation, but try screencasts. You might be surprised how well they work and how much people like them.
4) Some people need face-to-face training. So you'll have to run some traditional classes and/or open labs as well. But, many are receptive to video and to self-directed learning, so long as you're willing to provide the content and have it available from the get go.
5) Forcing people to go to training can lead to hostility toward the product you're rolling out, and to IT generally. What you want to do is make training, in various modes, available and encourage manager to encourage (but, one hopes, not force) their staff to attend.
ImproviseOrDie's advice is spot on.
posted by wheat at 7:42 AM on June 29, 2010 [1 favorite]
2) Stage the roll-out. Don't blast the whole organization with new 'ware overnight. Try it in a few departments first. Record the feedback and learn from it. Find the pain points and address them in the software configuration (if possible) or in the training.
3) Screencasts are your friend. Shoot as many features as possible, not in one long video but in discrete, reusable units. Then tie them together with a table of contents or some other navigation. If you shoot it with enough granularity, you can create customized aggregations of these for different departments/divisions, focusing on what they need. If your screencasting product supports text-to-speech, you can speed up development time and make it all 508-compliant from the get go. You'll likely also need more traditional documentation, but try screencasts. You might be surprised how well they work and how much people like them.
4) Some people need face-to-face training. So you'll have to run some traditional classes and/or open labs as well. But, many are receptive to video and to self-directed learning, so long as you're willing to provide the content and have it available from the get go.
5) Forcing people to go to training can lead to hostility toward the product you're rolling out, and to IT generally. What you want to do is make training, in various modes, available and encourage manager to encourage (but, one hopes, not force) their staff to attend.
ImproviseOrDie's advice is spot on.
posted by wheat at 7:42 AM on June 29, 2010 [1 favorite]
"we are actually about to roll out Office 2007 with no training because ... well.. i'm not sure why, but someone upstairs decided it was a good idea."
This is a very, very bad idea. We rolled out Office 2007 a long while back. And it was a good roll-out; but it would have been disastrous without training. It's GUI is very different from previous versions of Office. Office 2007 is a fine product, but end-users who grew up on older versions will be very confused by it initially (and for some time afterwards). Power users in particular will be confused by it. You absolutely need to convince whoever it is "upstairs" to rethink this and add a training component to the roll-out.
posted by wheat at 7:48 AM on June 29, 2010
This is a very, very bad idea. We rolled out Office 2007 a long while back. And it was a good roll-out; but it would have been disastrous without training. It's GUI is very different from previous versions of Office. Office 2007 is a fine product, but end-users who grew up on older versions will be very confused by it initially (and for some time afterwards). Power users in particular will be confused by it. You absolutely need to convince whoever it is "upstairs" to rethink this and add a training component to the roll-out.
posted by wheat at 7:48 AM on June 29, 2010
From my experience as an end user, my previous company handled this sort of thing very poorly. They rolled out an Oracle system for our timecards, expenses, annual reviews, and project management. The UI was horrible and we were all made extremely inefficient at these tasks, which used to take perhaps 5 to 10 minutes for timecards and expenses, now took over half an hour. It was also unbearably slow.
So, definitely follow Improviseordie's advice. It's really solid.
Additionally, if there is a choice in products, don't let technical specifications or price be the only decision points. Also include UI elements. The easier a product is to use, the fewer IT requests there will be and the more efficient people will be once they learn it. For example, how many screens and clicks will it take to complete the tasks? Are the the buttons in the right locations. (True story about the Oracle rollout. The trainer kept hitting the cancel button b/c ok and cancel were swapped from their normal locations in Windows. She did that three times in a row and kept losing her work.)
That doesn't mean that a product has to be easy to pick up and learn without training of some kind. Some tasks or processes are just complicated and while they may be easier to do once learned, may not be the easiest to learn w/o assistance.
Also, make your training efficient and useful. If you can, have different levels of training for the different groups of people using it. You may even want to have advanced vs basic levels. That way, people who are more skilled at general computer use can be fed the information more efficiently.
posted by reddot at 7:49 AM on June 29, 2010
So, definitely follow Improviseordie's advice. It's really solid.
Additionally, if there is a choice in products, don't let technical specifications or price be the only decision points. Also include UI elements. The easier a product is to use, the fewer IT requests there will be and the more efficient people will be once they learn it. For example, how many screens and clicks will it take to complete the tasks? Are the the buttons in the right locations. (True story about the Oracle rollout. The trainer kept hitting the cancel button b/c ok and cancel were swapped from their normal locations in Windows. She did that three times in a row and kept losing her work.)
That doesn't mean that a product has to be easy to pick up and learn without training of some kind. Some tasks or processes are just complicated and while they may be easier to do once learned, may not be the easiest to learn w/o assistance.
Also, make your training efficient and useful. If you can, have different levels of training for the different groups of people using it. You may even want to have advanced vs basic levels. That way, people who are more skilled at general computer use can be fed the information more efficiently.
posted by reddot at 7:49 AM on June 29, 2010
I introduce changes to workgroups whose productivity (i.e. bonuses, sales commissions, annual review numbers) rely entirely on how well they integrate new tools into the mix. Slowdowns don't just cost the company money, it is real, countable cash coming straight out of these people's pockets, so it is in everyone's best interest to make rollouts and changeovers as smooth as possible. Dropping something into this ecosystem without floating it in a bag first is going to kill all your fish.
Experienced people who have been in their job for a while are more resistant to change -- they've got methods, they are in a position where they can use their expertise to predict volume and avoid problems, and they've lived through other rollouts that caused conflict. Often, the most experienced people include some who feel threatened by a loss of status when they become just one of the crowd learning a new technology instead of a go-to person who is expert in an orphaned piece of software. They can be a big morale issue if you let them stew and complain, or they can be your trusted peer coach, and a great source of feedback, as they have no reason to blow smoke up anyone's ass.
I like to choose a pilot group of at least six and no more than ten:
- a smart, unafraid, techy one who is moving up,
- a smart, experienced one who is expert on the old software and plans to sit tight in their current job,
- a high-sales person who may or may not have any other skills in any other areas,
- someone "not working to potential" who is skilled but not an overachiever,
- someone who shows up on my error report (makes more than five workflow errors in a week) and is conceivably still someone who ought to be considered for pilot group membership
- and the person who is the answer to "who in this workgroup knows how to do this (job-specific weird task that is important but rare)?"
If a larger group is required, I top off with a volunteer, a random choice, and whoever the boss can spare.
I assemble my group while they can still make suggestions for improvements and changes. I run simulations and mockups past them. I test them for speed and let them know that future performance indicators will be geared to reflect the learning speed of the other (non-pilot) employees, so they're not resistant to setting high standards. I answer any and every question they have, including ones that could have been answered "I don't know, I'll need to ask the developer" or "That's not your job function" or "You're not allowed to know that because you can use it to break things". I pay attention to what mistakes are made, and write up instructions for how to recover from these errors, how to know if you've made one, and develop processes to check workflow before customer-affecting mistakes escape the barn.
What I end up with is a viciously loyal, completely expert, internally motivated, utterly functional team of evangelists and informal trainers who can reassure ANYONE, answer questions and engage with other departments as subject matter experts. They run the sales floor while everyone else is in training, and act as peer coaches when everyone is back.
The byproduct (byproduct! you won't even have to put in any extra effort!) of this teambuilding is excellent, useful training material for the formal trainers, walkthrough documentation, and initial budgetary impact numbers (average time to competency, cost of errors, workload and staffing figures that normally would only be estimates at this stage of a project).
The first couple of times I did this, I had to get managers two levels up to force team leads to release their staff for my project. Now, they'd fight each other with pointy sticks to get their people in.
posted by Sallyfur at 10:16 PM on June 29, 2010
Experienced people who have been in their job for a while are more resistant to change -- they've got methods, they are in a position where they can use their expertise to predict volume and avoid problems, and they've lived through other rollouts that caused conflict. Often, the most experienced people include some who feel threatened by a loss of status when they become just one of the crowd learning a new technology instead of a go-to person who is expert in an orphaned piece of software. They can be a big morale issue if you let them stew and complain, or they can be your trusted peer coach, and a great source of feedback, as they have no reason to blow smoke up anyone's ass.
I like to choose a pilot group of at least six and no more than ten:
- a smart, unafraid, techy one who is moving up,
- a smart, experienced one who is expert on the old software and plans to sit tight in their current job,
- a high-sales person who may or may not have any other skills in any other areas,
- someone "not working to potential" who is skilled but not an overachiever,
- someone who shows up on my error report (makes more than five workflow errors in a week) and is conceivably still someone who ought to be considered for pilot group membership
- and the person who is the answer to "who in this workgroup knows how to do this (job-specific weird task that is important but rare)?"
If a larger group is required, I top off with a volunteer, a random choice, and whoever the boss can spare.
I assemble my group while they can still make suggestions for improvements and changes. I run simulations and mockups past them. I test them for speed and let them know that future performance indicators will be geared to reflect the learning speed of the other (non-pilot) employees, so they're not resistant to setting high standards. I answer any and every question they have, including ones that could have been answered "I don't know, I'll need to ask the developer" or "That's not your job function" or "You're not allowed to know that because you can use it to break things". I pay attention to what mistakes are made, and write up instructions for how to recover from these errors, how to know if you've made one, and develop processes to check workflow before customer-affecting mistakes escape the barn.
What I end up with is a viciously loyal, completely expert, internally motivated, utterly functional team of evangelists and informal trainers who can reassure ANYONE, answer questions and engage with other departments as subject matter experts. They run the sales floor while everyone else is in training, and act as peer coaches when everyone is back.
The byproduct (byproduct! you won't even have to put in any extra effort!) of this teambuilding is excellent, useful training material for the formal trainers, walkthrough documentation, and initial budgetary impact numbers (average time to competency, cost of errors, workload and staffing figures that normally would only be estimates at this stage of a project).
The first couple of times I did this, I had to get managers two levels up to force team leads to release their staff for my project. Now, they'd fight each other with pointy sticks to get their people in.
posted by Sallyfur at 10:16 PM on June 29, 2010
« Older Cheapest way from SE Asia to Canada?! | Grek translating: If 'Good Place/No place' is... Newer »
This thread is closed to new comments.
posted by rhapsodie at 5:37 AM on June 29, 2010