"Check Your Bias"

I talk about bias a lot because bias is kind of important. And as if it weren't already difficult enough to constantly be on the lookout for the ways that biases can sneak and slither their way into a dataset or analysis or presentation, there's another source of bias that is much more pernicious and insidious. Thankfully it's surprisingly easy to find its source: just look in the mirror.

It's so easy. You go into an experiment or observation or study or analysis expecting a certain result. You can't help it, even when you're trying to be impartial; it's human nature. And as soon as the data start to lean in a particular direction, or the analysis starts to confirm your suspicions, it's oh so tempting to call it a success, write it up, and move on.

But this is what gets you into trouble. What if you're looking at a false positive? What if there aren't enough data to justify your statistics? What if you made a mistake somewhere and overlooked it because you got the answer you thought you wanted?

Replication is one of the keys to the scientific paradigm, but we can't be lazy about it and just assume that someone else will do the boring repetitions for us. Nobody will be as close to the original data and setup as we are - it's up to us to perform the first rounds of repeating and cross-checking results ourselves.

It's easy to lie with data and give everything a veneer of respectability. And the person we most often lie to is ourselves. So one key to cracking internal bias is simple and straightforward: get more data, and do the whole thing again.