Employee listening is only as good as the questions you ask
Research lives and dies by the questions within your survey. All your subsequent analysis, insight, and action flow from them, and getting this questionnaire design wrong will stop you being able to genuinely improve employee experience within your organisation.
A common sign that something isn’t right? When the results seem far too positive and don’t line up with the feedback you get when speaking to employees and leaders on the ground. For example, you find that 95% of employees say they understand your organisation’s strategy, but no one can tell you what this strategy actually is.
Often, the cause of this is ‘social desirability bias’ – an unconscious self-defence response that encourages people to answer questions in a way that makes them look good. The classic example is the difficulty in collecting healthcare data, such as units of alcohol per week or calories consumed. Respondents almost universally report a more positive reflection on themselves than reality.
Subtle adjustments to question wording can go a long way to mitigating this. Changing our strategy question from ‘I understand my organisation’s strategy’ to ‘Where I work, people around me understand the organisation’s strategy’ gives a much more honest result. By removing ‘self’ from the question, respondents are no longer answering based on what they think they should know, or what the ‘right’ answer is. Understanding this kind of effect on survey responses is crucial, especially when it comes to topics with a ‘right’ answer like safety and risk.
Questioning your questions
This is just one of several vital checks for questions in your survey. You should also ask yourself if your questions are:
- Actionable: ask ‘so what?’ for every question – what will you be able to do with the results?
- Comparable: do you need to compare your score to other organisations? If so, you’ll need to stick to a question format other organisations ask.
- Objective: is your question leading or priming respondents?
- Clear: are you using any jargon respondents might not understand?
- Precise: could any of the words in your question be interpreted in different ways? Do any of the words have an alternate meaning in your employees' line of work?
- Singular: does your question have more than one ask? Watch out for double-barrelled queries like ‘what did you think of our product and the customer service you received?’ - your questions shouldn’t have an ‘and’.
Are you asking the right questions to set you up for survey success?
Questionnaire design is the most crucial stage of your research programme and sets out a path you can’t deviate from later.
If it’s something you want to explore and improve, do sign up for a 60-minute Survey Surgery session and discuss the possibilities with one of our experts.
Miles Crosby
