David Leonhardt, NY Times, August 26, 2014 http://www.nytimes.com/2014/08/26/upshot/a-new-reason-to-question-the-official-unemployment-rate.html

….

A new academic paper suggests that the unemployment rate appears to have become less accurate over the last two decades, in part because of this rise in nonresponse. In particular, there seems to have been an increase in the number of people who once would have qualified as officially unemployed and today are considered out of the labor force, neither working nor looking for work.

The trend obviously matters for its own sake: It suggests that the official unemployment rate – 6.2 percent in July – understates the extent of economic pain in the country today. ….

Yet the research also relates to a larger phenomenon. The declining response rate to surveys of almost all kinds is among the biggest problems in the social sciences. It’s complicating our ability to understand how people live and what they believe. ….

In 1997, the response rate to a typical telephone poll was a healthy 36 percent, according to Pew. By 2012, it had fallen to 9 percent. ….

The response rate of the Labor Department’s monthly jobs survey is far higher (about 89 percent) than that of a political poll, but it has also fallen (from 96 percent in the 1980s). Not surprisingly, the people who do not respond have different experiences in the job market than those who do.

The trouble with the unemployment rate revolves around a technical concept known as “rotation-group bias,” explain the paper’s three authors, Mr. Krueger, Alexandre Mas and Xiaotong Niu. The government surveys people for four consecutive months, gives them eight months off and then surveys them for four more months. This pattern allows the Labor Department to track people’s experiences for more than a year in a way that is less burdensome than 16 months of monthly surveys would be.

Over time, the kinds of answers that people give — or the kinds of people who respond — change. In later months as part of the survey panel, people who aren’t working are less likely to report being available to work and having looked for a job in the previous four weeks, which is the definition for unemployment. The differences are big, too.

The unemployment rate in the first half of 2014 among people in the first month of being interviewed was 7.5 percent. Among people in the final month of being interviewed, it was only 6.1 percent. Because the Labor Department weights later panelists – for whom there is historical data – more heavily, the official unemployment rate during this period was 6.5 percent.

That number seems too low. The authors note that the higher jobless rate among early-month panelists correlates more strongly with some other economic indicators than the rate among later-month panelists.

…. The problem has existed, and been growing, for decades. A redesign of the survey in 1994, to move it from paper-based to computer-based, seems to be one cause. ….

For decades, survey research has revolved around the telephone, and it’s worked very well. But Americans’ relationship with their phones has radically changed. It’s no surprise that survey research will have to as well.

….

See also from 2006 “Study Finds that Labor Department Overstates Share of Working Americans By 1.4 Percentage Points”