I love surveys but this is nuts
May 30, 2020 8:29 AM Subscribe
My hospital has sent us a survey about our experiences as healthcare workers during the pandemic. The survey is apparently from WHO and I really wanted to participate and give them my opinions but the questions are phrased in a truly bizarre way. Can you help me figure out what they are asking? I know I can just skip this survey but I'd rather not.
"Below are some common risks for the health and safety of health workers; we are asking you to rate the current level of these risks, now during the COVID-19 pandemic.
Question Title
[number] How would you rate the level of these risks for health workers, now?
Shift work with night shifts:
Risk is negligible
Risk is acceptable for a short time
Risk is not acceptable at all
Don’t know/Unsure
What are they asking? Are they asking if the health risk associated with night shifts is acceptable during the pandemic? It appears that the question conflates the magnitude of risk ("negligible") with the level of acceptance of that risk "acceptable for a short time/not at all". Shift work is part of medicine so I guess it's "acceptable or find a different job" but that does not mean it's "negligible" - there are significant health risks associated with shift work including involuntary weight gain and cancer, so... I'm left scratching my head.
Then there is this nugget:
Sexual harassment:
Risk is negligible
Risk is acceptable for a short time
Risk is not acceptable at all
Don’t know/Unsure
I just - ???
Insufficient access to facilities for personal hygiene, such as, shower and menstrual hygiene
Risk is negligible
Risk is acceptable for a short time
Risk is not acceptable at all
Don’t know/Unsure
I mean... it would be completely unacceptable but does not happen where I work, so... A?
Could you help me figure out the intent of the asker(s)? Bonus points for suggestions of what to write under "comments" at the end of the survey to explain why these questions suck.
"Below are some common risks for the health and safety of health workers; we are asking you to rate the current level of these risks, now during the COVID-19 pandemic.
Question Title
[number] How would you rate the level of these risks for health workers, now?
Shift work with night shifts:
Risk is negligible
Risk is acceptable for a short time
Risk is not acceptable at all
Don’t know/Unsure
What are they asking? Are they asking if the health risk associated with night shifts is acceptable during the pandemic? It appears that the question conflates the magnitude of risk ("negligible") with the level of acceptance of that risk "acceptable for a short time/not at all". Shift work is part of medicine so I guess it's "acceptable or find a different job" but that does not mean it's "negligible" - there are significant health risks associated with shift work including involuntary weight gain and cancer, so... I'm left scratching my head.
Then there is this nugget:
Sexual harassment:
Risk is negligible
Risk is acceptable for a short time
Risk is not acceptable at all
Don’t know/Unsure
I just - ???
Insufficient access to facilities for personal hygiene, such as, shower and menstrual hygiene
Risk is negligible
Risk is acceptable for a short time
Risk is not acceptable at all
Don’t know/Unsure
I mean... it would be completely unacceptable but does not happen where I work, so... A?
Could you help me figure out the intent of the asker(s)? Bonus points for suggestions of what to write under "comments" at the end of the survey to explain why these questions suck.
They're asking if you feel like you are being mistreated, in different areas.
The shift one is asking if you feel like you have been asked to take on unacceptable risk in doing night shift work on a scale of 1-3.
The sexual harassment one should be read as "do you feel like sexual harassment has been a problem in your work on a scale of 1-3."
The insufficient access one is asking if you feel like you have insufficient access.
Etc.
posted by fingersandtoes at 8:37 AM on May 30, 2020 [10 favorites]
The shift one is asking if you feel like you have been asked to take on unacceptable risk in doing night shift work on a scale of 1-3.
The sexual harassment one should be read as "do you feel like sexual harassment has been a problem in your work on a scale of 1-3."
The insufficient access one is asking if you feel like you have insufficient access.
Etc.
posted by fingersandtoes at 8:37 AM on May 30, 2020 [10 favorites]
So this survey comes from WHO who is probobly questioning people in a variety of countries in varying conditions and hospitals with varing cultural norms, resource access and general safety . These are generalized risks of healthcare workers that have been established already. It's asking if you to rate of there is no risk, and increase in risk which is acceptable for now, this risk unacceptable no matter the current circumstances, or don't know. At least that's how I'm reading it?
Its likely asking some generalized demographic data (country of origin, gender, profession etc).
I think it is kind of convoluted. Risk is negliable vs Risk is not acceptable at all aren't... Scaled? I think that's because these are known risks. I wonder if you can request it in a different language and compare.
posted by AlexiaSky at 8:44 AM on May 30, 2020 [2 favorites]
Its likely asking some generalized demographic data (country of origin, gender, profession etc).
I think it is kind of convoluted. Risk is negliable vs Risk is not acceptable at all aren't... Scaled? I think that's because these are known risks. I wonder if you can request it in a different language and compare.
posted by AlexiaSky at 8:44 AM on May 30, 2020 [2 favorites]
I would just answer "don't know/unsure" for all of these and in comments, point out that the answers conflate two axes of risk (magnitude and tolerance) and are therefore unanswerable.
posted by basalganglia at 8:44 AM on May 30, 2020 [3 favorites]
posted by basalganglia at 8:44 AM on May 30, 2020 [3 favorites]
Response by poster: I wonder if you can request it in a different language and compare.
That's a great suggestion. I actually tried it in German and it was about the same level of weird. There is also French, Spanish, Italian and a few others if anyone's interested in checking it out.
posted by M. at 8:53 AM on May 30, 2020 [1 favorite]
That's a great suggestion. I actually tried it in German and it was about the same level of weird. There is also French, Spanish, Italian and a few others if anyone's interested in checking it out.
posted by M. at 8:53 AM on May 30, 2020 [1 favorite]
As you note, there is evidence that night shifts may be linked to higher cancer rates. That is what they may be asking about, though it is weird in a specifically COVID context. But maybe staff shortages means that more people have to work nights, or feel like they are more vulnerable to sexual harassment.
I administer health surveys for a living. Sometimes, the wording is not ideal - translation is an issue, and there is a trade off between clarity and specificity. Also, once you have a set of questions used in one survey, you can't change how they are worded because that's means you can compare the responses as easily to the last time.
As the person who doesn't design the questions, but does program them into the computer and proofread them: don't think too hard. Answer the questions to the best of your ability and as literally as possible.
The researchers are interested in whether these common risks have been increased by the COVID crisis. Maybe the answer key should have been "less, the same, more" but if they have previous surveys about these risks with these answer keys, they may wish to do a comparison between two time points. So, the thing to do is to answer as best you can thinking about right now and your own situation. e.g. Do you think that the risks posed by shift work is at this moment negligible, okay for a short time, or unacceptable? They are also asking about risk, not necessity, so you can answer that you think a given risk is "not acceptable at all", even though the situation is such that people may be facing that risk anyways - like long term care workers not having proper PPE. It happens, but it's not acceptable.
posted by jb at 9:36 AM on May 30, 2020 [3 favorites]
I administer health surveys for a living. Sometimes, the wording is not ideal - translation is an issue, and there is a trade off between clarity and specificity. Also, once you have a set of questions used in one survey, you can't change how they are worded because that's means you can compare the responses as easily to the last time.
As the person who doesn't design the questions, but does program them into the computer and proofread them: don't think too hard. Answer the questions to the best of your ability and as literally as possible.
The researchers are interested in whether these common risks have been increased by the COVID crisis. Maybe the answer key should have been "less, the same, more" but if they have previous surveys about these risks with these answer keys, they may wish to do a comparison between two time points. So, the thing to do is to answer as best you can thinking about right now and your own situation. e.g. Do you think that the risks posed by shift work is at this moment negligible, okay for a short time, or unacceptable? They are also asking about risk, not necessity, so you can answer that you think a given risk is "not acceptable at all", even though the situation is such that people may be facing that risk anyways - like long term care workers not having proper PPE. It happens, but it's not acceptable.
posted by jb at 9:36 AM on May 30, 2020 [3 favorites]
Response by poster: Answer the questions to the best of your ability and as literally as possible.
I literally don't understand them. I am serious.
Could you rephrase one of them as an example?
posted by M. at 9:52 AM on May 30, 2020
I literally don't understand them. I am serious.
Could you rephrase one of them as an example?
posted by M. at 9:52 AM on May 30, 2020
I'll give one a shot:
Insufficient access to facilities for personal hygiene, such as, shower and menstrual hygiene
> Risk is negligible (We can't shower after shifts anymore, and the tampon dispenser is empty, but given what's going on that's not a big deal for at all - we can do all that at home/bring our own supplies)
> Risk is acceptable for a short time (We can't shower after shifts anymore, and the tampon dispenser is empty, given what's going on that's not a big deal RIGHT NOW, but this cannot be our new normal.)
> Risk is not acceptable at all (We can't shower after shifts anymore, and the tampon dispenser is empty, and this is A BIG DEAL and unacceptable even though we're in the midst of a pandemic.)
> Don’t know/Unsure (I don't have an opinion on this.)
posted by kimberussell at 10:18 AM on May 30, 2020 [8 favorites]
Insufficient access to facilities for personal hygiene, such as, shower and menstrual hygiene
> Risk is negligible (We can't shower after shifts anymore, and the tampon dispenser is empty, but given what's going on that's not a big deal for at all - we can do all that at home/bring our own supplies)
> Risk is acceptable for a short time (We can't shower after shifts anymore, and the tampon dispenser is empty, given what's going on that's not a big deal RIGHT NOW, but this cannot be our new normal.)
> Risk is not acceptable at all (We can't shower after shifts anymore, and the tampon dispenser is empty, and this is A BIG DEAL and unacceptable even though we're in the midst of a pandemic.)
> Don’t know/Unsure (I don't have an opinion on this.)
posted by kimberussell at 10:18 AM on May 30, 2020 [8 favorites]
Best answer: Risk is negligible - It's not a problem for me in general and it hasn't been worse by cause of the pandemic.
Risk is acceptable for a short time - It's a bit worse of a problem for me, but is due to the necessity of dealing with the pandemic and I can stand it for a while as long as it goes away eventually.
Risk is not acceptable at all - This shit's all fucked up for me due to pandemic reasons.
Don’t know/Unsure - dunno
Are they asking about you and your environment and history between the before time and now? Or are they asking about your general global feelings about healthcare in general during the pandemic or even just in general.
I agree that they've probably asked these same questions before and will ask these same questions again in the future and will eventually be able to maths out the pandemic blip. Or even compare it to flu season, or other outbreaks, or other sorts of emergency situations that task healthcare outside of the normal day-to-day.
posted by zengargoyle at 10:22 AM on May 30, 2020 [1 favorite]
Risk is acceptable for a short time - It's a bit worse of a problem for me, but is due to the necessity of dealing with the pandemic and I can stand it for a while as long as it goes away eventually.
Risk is not acceptable at all - This shit's all fucked up for me due to pandemic reasons.
Don’t know/Unsure - dunno
Are they asking about you and your environment and history between the before time and now? Or are they asking about your general global feelings about healthcare in general during the pandemic or even just in general.
I agree that they've probably asked these same questions before and will ask these same questions again in the future and will eventually be able to maths out the pandemic blip. Or even compare it to flu season, or other outbreaks, or other sorts of emergency situations that task healthcare outside of the normal day-to-day.
posted by zengargoyle at 10:22 AM on May 30, 2020 [1 favorite]
I think they're listing some things that everyone agrees are bad for workers and asking how likely you feel those things are to happen right now. For instance, I would rephrase the second one as:
How high do you feel the risk of sexual harassment at your job is right now?
-Practically nonexistent
-There's some risk, but I can live with it, at least for now.
-Risk is unacceptably high.
posted by Redstart at 10:24 AM on May 30, 2020 [2 favorites]
How high do you feel the risk of sexual harassment at your job is right now?
-Practically nonexistent
-There's some risk, but I can live with it, at least for now.
-Risk is unacceptably high.
posted by Redstart at 10:24 AM on May 30, 2020 [2 favorites]
This looks to me like a questionaire they have used pre-covid, so they are asking the exact same questions now to do a comparison.
So 6 months ago you might have thought night shifts held a negligible risk, but now you find the night shift is when they run out of PPE and so the risk has become 'not acceptable'.
This would probably be more understandable if they had just phrased it like: "we are asking you to rate the current level of these risks for the work you have done over the last 14 days."
I dont think they are asking: "would this be acceptable in theory" rather "has your actual recent experience been at a level you find acceptable"
posted by Lanark at 11:15 AM on May 30, 2020 [6 favorites]
So 6 months ago you might have thought night shifts held a negligible risk, but now you find the night shift is when they run out of PPE and so the risk has become 'not acceptable'.
This would probably be more understandable if they had just phrased it like: "we are asking you to rate the current level of these risks for the work you have done over the last 14 days."
I dont think they are asking: "would this be acceptable in theory" rather "has your actual recent experience been at a level you find acceptable"
posted by Lanark at 11:15 AM on May 30, 2020 [6 favorites]
Will your returned work survey be anonymous, or linked to you by your hospital/healthcare system?
posted by Iris Gambol at 11:39 AM on May 30, 2020
posted by Iris Gambol at 11:39 AM on May 30, 2020
Response by poster: Will your returned work survey be anonymous, or linked to you by your hospital/healthcare system?
We are filling it out directly on surveymonkey.
I'm not sure whether anyone would bother to look up the results even if they could.
posted by M. at 12:06 PM on May 30, 2020
We are filling it out directly on surveymonkey.
I'm not sure whether anyone would bother to look up the results even if they could.
posted by M. at 12:06 PM on May 30, 2020
I think they are coaching questions about your own subjective experience (subjective either psychologically or because your specific personal workplace) in objective-sounding language ('is negligible', 'is acceptable'), and that is confusing.
Like, it sounds as though they're asking if people in general (or maybe just experts?) would agree that, for example, your current risk of being sexually harassed is negligible/ acceptable for a short period of time etc, but the only thing that actually makes sense is the question of whether you personally feel unbothered by sexual harassment, or bothered, but willing to roll with it for now (maybe because it's bothersome but not too bad) etc.
That's what is a bit weird to me, at least. It sounds a bit like they are asking you to opine about what the consensus would be on your situation, rather then reflect on your own feelings and experience. All the while, the latter is clearly more likely to be what interests them, since if they wanted to measure 'objective' expertise or get a feel for general opinion, they wouldn't ask you but go straight to the respective demographics.
posted by doggod at 1:25 PM on May 30, 2020
Like, it sounds as though they're asking if people in general (or maybe just experts?) would agree that, for example, your current risk of being sexually harassed is negligible/ acceptable for a short period of time etc, but the only thing that actually makes sense is the question of whether you personally feel unbothered by sexual harassment, or bothered, but willing to roll with it for now (maybe because it's bothersome but not too bad) etc.
That's what is a bit weird to me, at least. It sounds a bit like they are asking you to opine about what the consensus would be on your situation, rather then reflect on your own feelings and experience. All the while, the latter is clearly more likely to be what interests them, since if they wanted to measure 'objective' expertise or get a feel for general opinion, they wouldn't ask you but go straight to the respective demographics.
posted by doggod at 1:25 PM on May 30, 2020
since if they wanted to measure 'objective' expertise or get a feel for general opinion, they wouldn't ask you but go straight to the respective demographics.Assumption that this survey is targeted at this specific individual which would cause a sampling bias from only asking the preconceived groups.
My (oh really) is WHO using surveymonkey instead of doing it themselves. WHO outsourcing surveys??? Really?
posted by zengargoyle at 2:03 PM on May 30, 2020
Surveymonkey do seem to have a rather tarnished reputation. Giving them your email address might not be a good idea.
posted by Lanark at 2:58 PM on May 30, 2020
posted by Lanark at 2:58 PM on May 30, 2020
Lots of legitimate research uses Survey Monkey - they have a paid service. My work uses Qualtrics, which I like better, but then I've never seen the backend of Survey Monkey. The survey isn't being programmed or administered by Survey Monkey, they are just the site host and software providers.
As for making their own: WHO are scientists, not software engineers - I know someone who used to be on an important committee for them, and she's brilliant when it comes to theorising about disability, but don't ask her to program a survey, even WYSIWYG. The scientists set the questions, and it will be their research assistants (aka people like me) who administer the survey using a commercially available survey software and site - because I'm no programmer either. I just use the software to set up the skip logic and variables and response options. All this is also done on a shoestring budget, so why would we clunk together a bad homebrew software when we can buy a good product that works?
as for translating the questions: thank you to those up thread, you've done a better job than I have. They aren't well written questions - I can imagine they were written by a health scientist who has never been trained in survey design. My boss, who teaches survey design, has noted there are many big data scientists who are trained to work with secondary data previously collected in surveys, but don''t have much experience in writing surveys (or taking them) - and sometimes they don't understand that it is a specialized skill just like their statistical methods. And then they start writing surveys for big international studies in multiple languages and don't pilot test them ... and you have situations like this.
posted by jb at 4:48 PM on May 30, 2020 [4 favorites]
As for making their own: WHO are scientists, not software engineers - I know someone who used to be on an important committee for them, and she's brilliant when it comes to theorising about disability, but don't ask her to program a survey, even WYSIWYG. The scientists set the questions, and it will be their research assistants (aka people like me) who administer the survey using a commercially available survey software and site - because I'm no programmer either. I just use the software to set up the skip logic and variables and response options. All this is also done on a shoestring budget, so why would we clunk together a bad homebrew software when we can buy a good product that works?
as for translating the questions: thank you to those up thread, you've done a better job than I have. They aren't well written questions - I can imagine they were written by a health scientist who has never been trained in survey design. My boss, who teaches survey design, has noted there are many big data scientists who are trained to work with secondary data previously collected in surveys, but don''t have much experience in writing surveys (or taking them) - and sometimes they don't understand that it is a specialized skill just like their statistical methods. And then they start writing surveys for big international studies in multiple languages and don't pilot test them ... and you have situations like this.
posted by jb at 4:48 PM on May 30, 2020 [4 favorites]
It appears that the question conflates the magnitude of risk ("negligible") with the level of acceptance of that risk "acceptable for a short time/not at all".
The negligible part at least I feel like I've got a handle on. Risk is usually thought of along two axes: likelihood and impact. But often when you want to talk about the most "important" risks, you want to pull out from the nuance of a two-dimensional scatterplot, so you prioritize a list, saying stuff that's both high likelihood and high impact is the most important to mitigate or avoid; stuff that's either low likelihood but high impact, or low impact but high likelihood, is moderately important to think about; stuff that's both low likelihood and low impact is least important to worry about. Here they are basically mapping "most important" to "unacceptable," "moderate" to "acceptable for a short time"*, and "negligible" to " least important" (it would have been loads more consistent if they'd just said "acceptable," or maybe "acceptable for any length of time.")
*Like I said, I get the "negligible" part. The "acceptable for a short time" stuff is weirder; my best guess is that they are trying to indicate that by dint of it being a short time, the likelihood is reduced (while the impact would remain unchanged), so that might change where you put it in the priority list.
posted by solotoro at 8:34 AM on May 31, 2020
The negligible part at least I feel like I've got a handle on. Risk is usually thought of along two axes: likelihood and impact. But often when you want to talk about the most "important" risks, you want to pull out from the nuance of a two-dimensional scatterplot, so you prioritize a list, saying stuff that's both high likelihood and high impact is the most important to mitigate or avoid; stuff that's either low likelihood but high impact, or low impact but high likelihood, is moderately important to think about; stuff that's both low likelihood and low impact is least important to worry about. Here they are basically mapping "most important" to "unacceptable," "moderate" to "acceptable for a short time"*, and "negligible" to " least important" (it would have been loads more consistent if they'd just said "acceptable," or maybe "acceptable for any length of time.")
*Like I said, I get the "negligible" part. The "acceptable for a short time" stuff is weirder; my best guess is that they are trying to indicate that by dint of it being a short time, the likelihood is reduced (while the impact would remain unchanged), so that might change where you put it in the priority list.
posted by solotoro at 8:34 AM on May 31, 2020
Response by poster: I filled out the survey using zengargoyles' interpretation.
Turns out there was no option to comment.
Thank you for your answers and suggestions!
posted by M. at 9:58 AM on May 31, 2020
Turns out there was no option to comment.
Thank you for your answers and suggestions!
posted by M. at 9:58 AM on May 31, 2020
Ah, cool jb. From university IT work I totally get the idea of scientists / researchers that couldn't punch their way out of a paper bag when it comes to tech stuff that's not their thing. I assumed that they would have an IT branch that does that bit for them all faculty / staff like.
posted by zengargoyle at 10:44 AM on May 31, 2020
posted by zengargoyle at 10:44 AM on May 31, 2020
This thread is closed to new comments.
posted by AugustWest at 8:37 AM on May 30, 2020 [7 favorites]