Visualizing Levels of Confidence in Science Communication

Published in 2017
74

These days, science is increasingly seen as the source of truth. If someone plays a science card, then that’s often the end of the discussion.

But science is complex, and what good scientists understand is that the body of scientific knowledge is closer to an evolving organism than a fossil set in stone.

So the way we discuss and share science needs to reflect variations in certainty, not absolute certainty.

For individuals, that’s easier said than done. People are proud of what they know, and knowledge tends to have a strong connection with the mind’s ego. It’s just human.

And incentives aren’t always good either. In academics, journals and institutions incentivize researchers to defend rather than refute their own work, going against the intellectual honesty required for good science. Null hypotheses, as powerful of a method as they are, can’t outweigh the need for someone to feed their family at the end of the day, and can act as a subliminal force despite good intentions. These incentives apply to companies selling science-driven products, too.

We’ll save the academic analysis for another day, and focus on the companies selling science. Below, I’ve reviewed some popular products and services, and have seen that companies are being incredibly thoughtful about their responsibilities, in a variety of interesting ways.

So, how are levels of confidence being communicated to consumers today?

Care/of

Care/of is a company that mails you custom packages of supplements. In world of supplements, there’s an incredible amount of variation in what can be trusted and what cannot.

So the first time I saw the way Care/of went out of their way to communicate scientific certainty, I about jumped out of my seat. Take a look.

To add bonus, their icons are animated. This is candy to me.

23andMe

Many people look forward to the promises of genetic testing, but it’s still a field that has a long ways to go. So, the trick is to help people understand what might be true, not what certainly is true. And this applies to the entire product offering.

So 23andMe takes two approaches: First, to educate people up front on what they should know. Second, to give them control over the confidence levels themselves when viewing the data.

From a sample on their website, not my own results.

uBiome

The importance of the microbiome is one the most important discoveries of recent years — we’re still wrapping our head around it, and the implications will be enormous. They mainly take an educational and social statistic approach to certainty.

You’re looking at the bacteria in my stomach.

Ancestry.com

Ancestry offers a kit very similar to 23andMe. They use Q&As to fill people in on how to read the results and interpret them themselves.

For the geneology portion of their product, they offer hints to guide people towards making their own conclusions.

Elysium Health

This is some next level stuff: They’re developing supplements to supply and repair a key biological chemical called NAD+, which is essential for providing energy to cells. But how do you get people to trust something so complex, and that they’ve never even heard of?

Elysium has taken to Medium to start their own journal and blog, and is doing an incredible job of it. But furthermore, they’ve conducted their own primary research and share secondary research in an intuitive way.

Apple Watch

Apple Watch uses fun rings to pull people away from the raw data, and distills it down into three metrics. (Below is my first week of owning an Apple watch, and clearly the game mechanics were effective motivators.)

Strava

Similarly, Strava distills heart rates into ranges, then spits out a metric to sum them all up. The confidence in interpreting the result is all packed into this “smart” variable, cleverly called the Suffer Score.

IBM Watson

Finally, I was fortunate enough recently to get to see IBM’s Watson in action. Its role in the medical world is to increase the speed and certainty of diagnoses for doctors.

Below is a slight recreation of what I saw. The size of the purple circles indicates diagnosis certainty, and as you drag the bottom symptoms up to the top, that certainty changes.

Summary of Methods

We can see the wide variety of methods used for this. Let’s sum them up:

  • Control over certainty (Exploratory)
  • Evolutionary certainty based on changing inputs
  • Educational material
  • Word choice and language
  • Glanceable symbols (hints, specific badges)
  • Heuristic zones (diffuse the precision)
  • Computed metrics and scores
  • Clearly share primary research
  • Clearly share secondary research
  • Reference to a mean, median, or mode

Final Thoughts

I doubt that there’s one good way to communicate certainty in all scenarios. But what this analysis could lead to is a set of best practices, and hopefully over time the way we talk about scientific results in the public sphere can become a bit less black and white.