You’re a market researcher. What do you do all day?
August 11, 2012 2:04 PM Subscribe
How do you market researchers do what you do? I’m curious about the profession and I’d like some information on how researchers actually spend time. Special projects at my design service firm often include an element of conjecture about potential audience. When we work on projects that might be legitimately useful or tool-like, I’d like to introduce some rigor into our process and know better when to think about hiring a contractor or outside vendor for research.
Things I’m curious about:
Things I’m curious about:
- Is success most closely tied to familiarity with known sources of data, novel qualitative/quantitive research, or closely guarded connections and secret sauce?
- What do your deliverables look like? If you’re investigating the market for a new product or service, do you have existing sources of industry information that you look to first?
- Do you usually buy information and reports and summarize them?
- Do you work in teams, or alone?
- When you receive a brief from a client, what kinds of detail does it include?
- Do you stick to industries you already know, or do your skills transfer from one to another?
- How do you know when you’re done?
Response by poster: Dave, thank you for your extensive answer. In the focus group sessions, are participants being shown pre-release products or campaigns after having signed an NDA? Does the researcher have a hand in iteratively generating what’s presented, or do they (for example) go back to the client with recommendations for the next round of focus grouping?
posted by migurski at 4:34 PM on August 11, 2012
posted by migurski at 4:34 PM on August 11, 2012
Generally it's a pre-release thing although sometimes it's a 'public opinion check' on something already released - I'm never involved in the candidate signup process, so I don't know if they sign an NDA but I'd assume they do. The researchers then feedback to the client (sometimes the actual client, sometimes a creative or product agency working for them, sometimes both) and the conclusions form one of the inputs into the finalisation of the design or user experience. In my case, things I've directly heard while observing a focus group I then incorporated into the next draft of the work I was doing. Most of the work I do is digital, so I can say I've directly changed how interfaces work and wording/look and feel of advertising because something wasn't working at the focus group stage. I think you can focus group things too much sometimes, but they are a very useful way of checking whether an idea is basically workable.
posted by Happy Dave at 12:45 AM on August 12, 2012
posted by Happy Dave at 12:45 AM on August 12, 2012
Best answer: This is from my dad, a retired market researcher:
I would be happy to answer from the perspective of a recently-retired market research executive with 30 years of experience.
For openers, there are two basic type of market research firms. One type, syndicated research, examines a particular area, often collating publicly available databases, and puts it together into a fat report which may retail for upwards of a thousand dollars a pop, or perhaps a continuing series of reports. IMS Health, for example, is a major provider in the healthcare industry. The other kind (my kind) is custom research, which does original research for a single client to address one or more linked questions, usually concerning a specific product or family of products. There are variants on this; for example, custom research companies may do multi-client surveys, with a common core of questions to which everybody gets the results, and questions specific to certain subscribers who have paid extra to have them included. Custom researchers range from multinational giants like ORC, GfK and Ipsos to one-person shops working off their dining room table. In my experience, there is no necessary correlation
between the quality of the final product and the size of the agency, and I worked for one of the three giants mentioned above.
I’ll be answering your questions from the perspective of a custom researcher.
Things I’m curious about:
• Is success most closely tied to familiarity with known sources of data, novel qualitative/quantitive research, or closely guarded connections and secret sauce?
All of these contribute to success, although I would be very wary of someone’s secret sauce if it cannot be shared with clients; as a client, you need to know how the answers were obtained in order to know how much to trust them. However, the biggest single factor IMHO is the researcher’s understanding of what is actually being sought, and his/her ability to present the results in such a way as to address what the client needs to know, which may or may not be the same thing as what the client wants to know. The researcher’s skill is the ability to take a meta-question like “should we invest a lot of money in this new product/service?” and turn it into a series of questions which can be answered by members of the target market (perhaps first identifying the target market along the way).
What do your deliverables look like? If you’re investigating the market for a new product or service, do you have existing sources of industry information that you look to first?
Only in the sense that I need to get really familiar with the market which I am to research in order to be able to evaluate the answer I will get. If I want to sit down with a room full of doctors to talk to them about a new opioid painkiller, I’d better have a pretty clear understanding of who treats what kind of pain; how many anesthesiologists should I talk with vs. orthopedists vs. palliative care doctors. I also better be really familiar with how opioids work, so that if the doctors start talking about COPD patients I understand that this is not off the subject, since opioids can cause respiratory depression.
Deliverables are typically a personal presentation from a deck of Power Point slides, perhaps with animations or video clips of interviews; they can get very fancy. If a survey was done, the survey database (with respondent identifiers removed) or cross-tabulations of data may also be a deliverable. For the kind of research that I did, spending some time working with client teams as a consultant was not unusual.
• Do you usually buy information and reports and summarize them?
On the custom side, no. On the syndicated side, yes.
• Do you work in teams, or alone?
In medium-sized to large companies, usually in teams. Most typically, there is an officer who designed the study and wrote the proposal and will supervise its execution and be primary liaison with the client, and a mid-level person who will do a lot of the day-to-day study management, and one or more research assistants for some of the necessary grunt work. In some companies, the officer is more of a salesperson and disappears after the study is sold, while in the places where I worked, the officer would be hands-on in tasks like questionnaire design, writing focus group topic guides, doing one-on-ones or focus groups, and writing chunks of the analysis, most importantly the conclusions and recommendations. In such companies, the study team has the advantage of support teams like statisticians, data processors and
data collection managers (with online surveys, these last two functions tend to merge).
The one-person shop either subcontracts some of these functions, or will to stay with less team-intensive methodologies such as focus groups or other type of interviews or observation.
• When you receive a brief from a client, what kinds of detail does it include?
It varies all over the map, from a 20-page request for proposal with lots of background on the product and market to a simple one-line question. One of my more fun studies was for a maker of luxury cars interested in assessing price sensitivity in their market; the entire RFP was: “how high is up?” But I had done a lot of work for this person, and was known to be very familiar with the market and the client’s products.
Do you stick to industries you already know, or do your skills transfer from one to
another?
Research skills are transferable, and those who become successful researchers are typically quick studies. I shifted from luxury cars to pharmaceuticals (although of course, my respondents in both cases were often physicians). One of my bosses maintained that it is easier to teach pharma to a researcher than to teach research to a person with a pharmaceutical background.
• How do you know when you’re done?
Ideally, when you are able to answer the client’s meta-question – “no, don’t invest a dime more in this product, the benefit which it provides is not valued by the market.” But sometimes, you’re done when you run out of budget; this might happen when you get an endless stream of follow-up questions for new cuts on the data, or new issues which need to be addressed but will stretch a half-hour interview to an hour, with consequent additional incentive for the respondent, or an additional presentation that has to be made at European headquarters, etc. Respondents are paid for their time by ethical researchers, and the more time, the more pay. Market researchers themselves, like lawyers, bill by the hour, and while a smart researcher will build up goodwill by doing some work beyond the formal constraints of the study, there comes a point when requests go “out of scope,” and one needs to offer to do a follow-up study.
In the focus group sessions, are participants being shown pre-release products or campaigns after having signed an NDA? Does the researcher have a hand in iteratively generating what's presented, or do they (for example) go back to the client with recommendations for the next round of focus grouping?
Most market research that I was involved in was about pre-launch products. We typically did not require NDAs because we rarely communicated whose product it was; when brand names were used, the product was often far enough along that the impact of a security breach would be minimal. And occasionally the brand name used in the group was either made up or that of
a competitor to our actual client.
A good moderator will reshape a series of focus groups based on what was heard in earlier groups (in fact, failure to do so is a sign of a lazy moderator or a rigid client), but more leisurely analysis might pick out some pattern which was not noticed during the groups themselves and which might be important enough to justify follow-up groups.
* * *
I mentioned meta-questions. These are the big questions which typically trigger market research. The most common meta-questions, in my experience, are:
Is it worthwhile investing money to develop this product or service?
Who is likely to buy this product or service?
What are the best things to say about our product/service to attract the most likely users?
With what other products or services will it compete, and how does the market assess our product/service against what is out there?
How much can we charge for our product/service?
And each of these can be answered with varying levels of rigor for varying amounts of money. It never ceased to amaze me that multi-million dollar decisions were being made by my clients on the basis of six focus groups in three cities, when a well-designed national survey could add precision to the essentially anecdotal data of the focus groups. I’ve done many hundreds of focus groups and probably a few hundred surveys, but each can be misused; and since focus groups are cheaper than surveys, guess which gets misused more.
posted by amro at 8:22 AM on August 12, 2012 [4 favorites]
I would be happy to answer from the perspective of a recently-retired market research executive with 30 years of experience.
For openers, there are two basic type of market research firms. One type, syndicated research, examines a particular area, often collating publicly available databases, and puts it together into a fat report which may retail for upwards of a thousand dollars a pop, or perhaps a continuing series of reports. IMS Health, for example, is a major provider in the healthcare industry. The other kind (my kind) is custom research, which does original research for a single client to address one or more linked questions, usually concerning a specific product or family of products. There are variants on this; for example, custom research companies may do multi-client surveys, with a common core of questions to which everybody gets the results, and questions specific to certain subscribers who have paid extra to have them included. Custom researchers range from multinational giants like ORC, GfK and Ipsos to one-person shops working off their dining room table. In my experience, there is no necessary correlation
between the quality of the final product and the size of the agency, and I worked for one of the three giants mentioned above.
I’ll be answering your questions from the perspective of a custom researcher.
Things I’m curious about:
• Is success most closely tied to familiarity with known sources of data, novel qualitative/quantitive research, or closely guarded connections and secret sauce?
All of these contribute to success, although I would be very wary of someone’s secret sauce if it cannot be shared with clients; as a client, you need to know how the answers were obtained in order to know how much to trust them. However, the biggest single factor IMHO is the researcher’s understanding of what is actually being sought, and his/her ability to present the results in such a way as to address what the client needs to know, which may or may not be the same thing as what the client wants to know. The researcher’s skill is the ability to take a meta-question like “should we invest a lot of money in this new product/service?” and turn it into a series of questions which can be answered by members of the target market (perhaps first identifying the target market along the way).
What do your deliverables look like? If you’re investigating the market for a new product or service, do you have existing sources of industry information that you look to first?
Only in the sense that I need to get really familiar with the market which I am to research in order to be able to evaluate the answer I will get. If I want to sit down with a room full of doctors to talk to them about a new opioid painkiller, I’d better have a pretty clear understanding of who treats what kind of pain; how many anesthesiologists should I talk with vs. orthopedists vs. palliative care doctors. I also better be really familiar with how opioids work, so that if the doctors start talking about COPD patients I understand that this is not off the subject, since opioids can cause respiratory depression.
Deliverables are typically a personal presentation from a deck of Power Point slides, perhaps with animations or video clips of interviews; they can get very fancy. If a survey was done, the survey database (with respondent identifiers removed) or cross-tabulations of data may also be a deliverable. For the kind of research that I did, spending some time working with client teams as a consultant was not unusual.
• Do you usually buy information and reports and summarize them?
On the custom side, no. On the syndicated side, yes.
• Do you work in teams, or alone?
In medium-sized to large companies, usually in teams. Most typically, there is an officer who designed the study and wrote the proposal and will supervise its execution and be primary liaison with the client, and a mid-level person who will do a lot of the day-to-day study management, and one or more research assistants for some of the necessary grunt work. In some companies, the officer is more of a salesperson and disappears after the study is sold, while in the places where I worked, the officer would be hands-on in tasks like questionnaire design, writing focus group topic guides, doing one-on-ones or focus groups, and writing chunks of the analysis, most importantly the conclusions and recommendations. In such companies, the study team has the advantage of support teams like statisticians, data processors and
data collection managers (with online surveys, these last two functions tend to merge).
The one-person shop either subcontracts some of these functions, or will to stay with less team-intensive methodologies such as focus groups or other type of interviews or observation.
• When you receive a brief from a client, what kinds of detail does it include?
It varies all over the map, from a 20-page request for proposal with lots of background on the product and market to a simple one-line question. One of my more fun studies was for a maker of luxury cars interested in assessing price sensitivity in their market; the entire RFP was: “how high is up?” But I had done a lot of work for this person, and was known to be very familiar with the market and the client’s products.
Do you stick to industries you already know, or do your skills transfer from one to
another?
Research skills are transferable, and those who become successful researchers are typically quick studies. I shifted from luxury cars to pharmaceuticals (although of course, my respondents in both cases were often physicians). One of my bosses maintained that it is easier to teach pharma to a researcher than to teach research to a person with a pharmaceutical background.
• How do you know when you’re done?
Ideally, when you are able to answer the client’s meta-question – “no, don’t invest a dime more in this product, the benefit which it provides is not valued by the market.” But sometimes, you’re done when you run out of budget; this might happen when you get an endless stream of follow-up questions for new cuts on the data, or new issues which need to be addressed but will stretch a half-hour interview to an hour, with consequent additional incentive for the respondent, or an additional presentation that has to be made at European headquarters, etc. Respondents are paid for their time by ethical researchers, and the more time, the more pay. Market researchers themselves, like lawyers, bill by the hour, and while a smart researcher will build up goodwill by doing some work beyond the formal constraints of the study, there comes a point when requests go “out of scope,” and one needs to offer to do a follow-up study.
In the focus group sessions, are participants being shown pre-release products or campaigns after having signed an NDA? Does the researcher have a hand in iteratively generating what's presented, or do they (for example) go back to the client with recommendations for the next round of focus grouping?
Most market research that I was involved in was about pre-launch products. We typically did not require NDAs because we rarely communicated whose product it was; when brand names were used, the product was often far enough along that the impact of a security breach would be minimal. And occasionally the brand name used in the group was either made up or that of
a competitor to our actual client.
A good moderator will reshape a series of focus groups based on what was heard in earlier groups (in fact, failure to do so is a sign of a lazy moderator or a rigid client), but more leisurely analysis might pick out some pattern which was not noticed during the groups themselves and which might be important enough to justify follow-up groups.
* * *
I mentioned meta-questions. These are the big questions which typically trigger market research. The most common meta-questions, in my experience, are:
Is it worthwhile investing money to develop this product or service?
Who is likely to buy this product or service?
What are the best things to say about our product/service to attract the most likely users?
With what other products or services will it compete, and how does the market assess our product/service against what is out there?
How much can we charge for our product/service?
And each of these can be answered with varying levels of rigor for varying amounts of money. It never ceased to amaze me that multi-million dollar decisions were being made by my clients on the basis of six focus groups in three cities, when a well-designed national survey could add precision to the essentially anecdotal data of the focus groups. I’ve done many hundreds of focus groups and probably a few hundred surveys, but each can be misused; and since focus groups are cheaper than surveys, guess which gets misused more.
posted by amro at 8:22 AM on August 12, 2012 [4 favorites]
Response by poster: Amro, please pass on my thanks to your father, that’s more relevant detail than I dared hope for.
posted by migurski at 10:05 AM on August 12, 2012
posted by migurski at 10:05 AM on August 12, 2012
Wow, yeah, great answer - please take my answer as a brief skim over a specific kind of market research I've experienced as a third party, compared to a pretty detailed overview of the whole industry from amro's dad!
posted by Happy Dave at 4:30 AM on August 13, 2012
posted by Happy Dave at 4:30 AM on August 13, 2012
« Older How much spending money does a college student... | The comforter set from HIMYM (marshall and Lily's... Newer »
This thread is closed to new comments.
In answer to your specific questions:
- The successful researchers I've met base their success on a consistent way of doing things and a roster of 'name' clients that they can use to sell their services. Basically, you need to be able to show the conclusions you draw for clients are more than just a finger in the air.
- The deliverables are Powerpoint presentations, fully produced reports and the various paraphernalia that goes with running a session (moderator scripts, participant forms etc)
- I've only worked with direct research firms, so in their case it's all original
- Moderators generally work on their own in a session (sometimes with someone observing or taking notes) but there's always a team involved (booking people, facilities people, client management etc)
- No idea about the detail of briefs, although if it's anything like most briefs they are pretty vague. Part of the value a good research firm brings is helping you to figure out precisely what it is you want to find out.
- No idea on transferability, except to say the researchers i've known might be testing a booze campaign one week and working on a government health campaign the next.
- You know when you're done largely by experience. The skills a good researcher brings will mainly be to get a good level of participation from multiple people, make sure the conversation stays focused and relevant without influencing it (too much) and be able to know when a conversation or topic has run its course.
posted by Happy Dave at 3:54 PM on August 11, 2012 [1 favorite]