The Well-being Measurement Problem

One of the lesser-appreciated consequences of the 2020 Covid-19 pandemic will be the lasting effects of deferred healthcare. It will manifest on two fronts: First, in terms of preventive care, a huge number of routine screenings have been cancelled or delayed this year: mammograms, colonoscopies, routine checkups and exercise that keeps people healthy.

One of the greatest dangers of the pandemic has been the possibility that other medical

Al Fasola

problems would go untreated because of the resources dedicated to controlling COVID-19. Coupled with the fact that many people are scared to go to the doctor’s office because of possible exposure, it will be some time before we understand what this will mean for the nation’s long-term health outcomes.

The second front where challenges await is mental health. The COVID-19 pandemic has been a profound trauma for the entire world. Beginning with lockdowns, fear and disruption in the early days and mercilessly building with job losses, remote schooling, social isolation and political upheaval, the pandemic will leave lasting scars on people’s psyches, not unlike the PTSD soldiers experience after leaving the battlefield.

In order to understand what this will mean for the nation, the CDC has conducted an ongoing, comprehensive survey of U.S. households — their material and personal circumstances —in an effort to better understand the impacts of these challenges on daily life. The survey contains a wide range of questions, ranging from ones like, “How confident are you that your household will be able to afford the kind of food you need in the next four weeks,” to “Over the last seven days, how often have you had problems feeling nervous, anxious or on-edge?”

These are important questions, and the CDC is right to collect and analyze this kind of data from the American public. However, consider this: The CDC survey takes 20 minutes to complete. Participants had to self-select to participate in this government survey, and only the ones with the patience to complete the 20-minute survey had their answers counted. If you take a moment to think about all of the online survey requests you’ve ever gotten versus how many you considered versus how many you completed, you’ll realize that the group that finished the CDC survey might not well represent the average American household.

The limitations of surveys have been well-documented over the years. They can be quite expensive — and part of that is because they’re hard to design well. When surveying consumers, there are a lot of variables to take into consideration.

Furthermore, the nature of how people take surveys means that the responses researchers receive are not always reliable or honest. Because respondents have time to mull their answers while selecting their responses, they often answer in ways that they think they should answer or in ways that they aspire to think rather than the ways they actually do. Honest replies and intuitions are lost. As a result, researchers are working with data based on a highly polished view of consumer reactions, rather than one based on the instincts and motivations that people employ when making decisions.

All of these factors make surveys difficult to conduct properly, costly, and only somewhat reliable. In order to overcome the statistical shortcomings of survey methodology, the surveys themselves would have to be fielded at a scale that’s not practical. The world needs better ways of measuring things than self-reported surveys. With any luck, technology and ingenuity will provide a workable solution.

Al Fasola is co-founder and CEO of ADoH Scientific,LLC