Data seems pretty cut and dried, but don’t be fooled. There are plenty of ways to fold in bias. Here are some concrete steps to help you do your best to counteract the most common pitfalls.
Start with a clean tool/ protocol to collect data.
1. Keep data clean
There are plenty of ways to keep you tidy. First, have everyone use the same protocol. Ideally, keep your data collector pool down to a minimum. More people means potentially multiple interpretations. Train everyone the same. Take out the protocol and make sure everyone understands it. And, make sure everyone uses the same data collection tool. I used to work in a team of three data collectors. We had to agree to everything, and often huddled up to make sure we were on the same page. Be vigilant
2. Observe First, Interpret Later
Years ago, when I worked on hiring teachers for the public schools, I had to take a course on legal job interviews. The fear that the trainer burned into my soul always returns to me when I do interviews. Only write what people say–word for word. Do not interpret. This goes against your human nature. And, if you have a hard time writing, ask respondents if you can record them. Also, feel free to ask the people who provided the data whether your interpretations seem to be representative of their beliefs. Once all the data comes in, then you have the joy of interpretation. That said, once you get familiar with interpretation-free listening, you will also find joy in data collection.
3. Check out the competition.
After your initial interpretations, look to others to see how they are tackling this issue. What are their findings? What other issues might be occurring in the literature. This is sometimes called triangulation. If you can find other sources of data that support your interpretations, then you can have more confidence that what you’ve found is legitimate.
4. Check for alternative explanations.
False conclusions are absolutely the most likely place that bias comes into understanding data. Jumping to conclusions can feel normal, like finishing someone’s sentence. But, just was you can’t fill in the blanks for your respondents, don’t fill in the blanks for yourself too quickly. Consider whether there are other reasons why you obtained your data. If you can rule out or account for alternative explanations, your interpretations will be stronger
5. Review findings with peers
Don’t be an island. Unless confidentiality prevents you, let others look at your data. You will only become better at your work with critical assessments. Additionally, when you allow peers to review your work, you might find commonalities. You might even be able to augment your argument.