Why pilot testing should be the holy grail of research
The cornerstone of reliable, actionable and analyze-able data lies in many factors. If we could single out just one that we consider super critical it would have to be training and tool testing.
Training data collection teams goes hand in glove with pilot testing data collection tools before embarking on any data collection for research. Here are our top 4 reasons why this should be a fundamental part of your research process.
- Respondents do not understand what is being asked of them: While questions formulated by a think tank of research practioners and scrutinized by intense client interest may pass muster, it could be that the research process suffers from failure to launch at the point where it matters most: with the subjects of interest in the study i.e. the respondents themselves. Untried surveys and discussion guides could result in challneging questions that force respondents to think too hard , get frustrated, apathetic, and indifferent in their responses thus resulting in poor data quality.
- Inherent biases in the phrasing of questions: Another pitfall that researchers may be blind to is the inherent bias in how questions are framed or presented to respondents. Pilot testing a survey or discussion guide can open one’s eyes to such biases and help in the resturcturing of stronger questions that yield better data outcomes for research
- Long, repetitive survey length: In the desire to maximise data outputs our enthusism will often get the better of us as researchers. We lose sight of the fact that respondents are first and foremost…… human beings. Boring, lengthy and repetitive questionnaires will yield token answers at best, and false, incomplete interviews at worst. Pre-testing how a questionnaire reads or how long it takes to complete is not only an important way of ensuring data quality but also an empathetic consideration for your research subjects.
- Data anlaysis calls into question research design rather than data analysis: Flying blind through data collection without any idea of what to expect in your data is often a recipe for analysis disaster. The real outcomes of research become inherent when analysis happens and by this time it is too late to course correct and too expensive in terms of time and resource to walk back. By pre-testing tools and teams, one is able to project what sort of data they are likely to accquire, what challenges are likely to arise and how to preempt potential crisis in research execution well before the dam bursts its banks.