From The Desk of...The Chief Scientist

"Getting Your Hands Dirty with Data"

Written by Paul Sutter on Monday, 03 September 2018. Posted in From The Desk of...The Chief Scientist

We always question everything, right? Right. But what if we're faced with a chart, or a graph, or a table of raw numbers? It's easy to fetishize the data, and assume that as long as we have data our arguments must be sound. But as I've talked about before, the data can lie - and more twistedly, the presentation of the data can lie even more. So here are some handy-dandy questions to ask yourself when confronted with data:

- Is the presentation of the data hiding something? Was anything excluded or minimized? Was anything glossed over? Was one part of the graph highlighted or emphasized to draw attention away from something else?

- How were the uncertainties calculated? You can't just pick error bars out of a hat - there's a methodology behind it, and that technique must be explained.

- What are the limitations of the data? What were they not able to cover? Does this introduce a bias?

- What choices were made to lead to the result? Are they explained? Are those choices reasonable or unreasonable?

- What choices were made in the presentation of the data? Percentages or absolutes? Linear or logarithmic? Broken axes? Is there a curve "fit" to the points? Are the data grouped?

- What are some other interpretations of the data?

This is, of course, far from a complete list, and each of these questions could be a note of its own. But it's a good start...

"Inference is Destiny"

Written by Paul Sutter on Monday, 27 August 2018. Posted in From The Desk of...The Chief Scientist

If there's one question that is my most favorite to answer, it's the very simple "How do we know?" Explaining cool facts about the universe is always fun, but getting a chance to share how science is done, beyond the overly-simplified grade school "memorize the scientific method for the quiz on Friday" level, is where some really deep science communication happens.

And when we know something based on direct observation or experiment, folks are usually easily satisfied. "We know this because we made a hypothesis and tested for it" wraps up everything in a nice little easily-appreciated package. Unfortunately, most of the time it's not that simple.

Take the case of the interior of a black hole (a question I get a lot). We will never see inside a black hole, and if you were to visit one you would a) die horribly, and b) never be able to communicate your gruesome results to the outside world. So how do we know what's inside?

The answer is inference, or using reasoning and evidence to follow logical paths to make ever-more complicated statements. We can infer about the interiors of black holes because we understand the mathematics and the observations of what we can see - we don't have to actually go there to have a handle on the situation.

But what if we're wrong? Well, that's a possibility...but one that is omnipresent in science. Science is all about inference - about making provisional statements based on the accumulated evidence. The statements can always be overturned with future evidence or rational arguments. Even in the case of direct observation or experiment - which on the surface appear oh-so-comforting - inference is still the tool that we use to build our entire scientific worldview.

And while that subtlety is really hard to communicate, we can't avoid it.

"Challenge Accepted"

Written by Paul Sutter on Sunday, 19 August 2018. Posted in From The Desk of...The Chief Scientist

There's a strange habit that develops in scientific life. Well, to be fair, there are a lot of strange habits that develop, but I'll focus on just one today. It's a habit of questioning everything, and I mean everything.

It's a part of the training to become a scientist, but it's not one you learn through any class. Instead, over the course of years you begin to recognize that what at first feels like harsh, personal critique is actually a vital part of the scientific process itself.

See an obvious mistake? Call it out. Don't understand something? Ask the question. Spot a hidden assumption? Bring it to the surface. Hear a conclusion asserted without evidence? Challenge it. Find a sacred cow? Turn it into hamburgers.

The practice of questioning everything is necessary, because the game of making science actually work isn't something done personally - you need a community. Sure, you might try your best to limit bias or carefully control your experiment or make the best assumptions in the math or apply the proper statistics, but you absolutely rely on your colleagues to check your work. Science is a process that thrives on that constant ever-vigilant critical examination from your peers, from your advisor to your research group to your journal referee to your presentation audience.

The rules of this game are simple: challenge everything.

<<  1 2 [34 5 6 7  >>