A Tale of Two Surveys

You'll Take the High Road, and I'll Take the Low Road

The labor market remains a surprising source of strength in the economy even as it has normalized somewhat from its extremely tight state following the pandemic. However, the resilience of hiring varies depending on the measure of employment. According to the establishment survey, which produces the widely-watched nonfarm payrolls number and is officially known as the Current Employment Statistics (CES) program, U.S. employment continues to grow at a robust pace, up 1.9% year-over-year in March. However, employment growth according to the household survey—officially known as the Current Population Survey (CPS)—has come close to stalling, with the 0.4% one-year gain in March the smallest outside the pandemic since October 2013 (Figure 1).

The trends in employment between the two surveys often diverge over the short term, and the difference in the year-over-year growth rates (1.3pp on a three-month average basis) is not totally unprecedented (Figure 2). Nevertheless, the gap has been nearly four times larger over the past year than its average over the past cycle. At a time when the FOMC has noted the risks to both sides of its mandate are becoming more balanced, catching turning points in the labor market is especially important. What might be driving the conflicting narratives in household and payroll employment, and what does this mean for the strength of hiring ahead?

Let's Agree to Disagree

At the core of the gap between the household and the payroll measures of employment is that the two surveys capture the labor market picture from different angles. The household survey is primarily a demographic survey, meant to offer insight into the labor status of different segments of the population. The survey sample includes 60K households each month who answer questions about members’ labor force activities and demographic traits. The major outputs from the household survey include the unemployment rate and the labor force participation rate.

The establishment survey, however, is intended to measure employment and earnings from an industry perspective, and it provides the marquee “nonfarm payroll” net job changes over the month. As its name implies, the sample is made up of businesses and government agencies that provide detail on their employees and the wages and salaries paid to them. The sample size is substantially larger than the household survey, covering about 630K worksites and approximately 30% of nonfarm employment. Data from both surveys are included in the Bureau of Labor Statistics' (BLS) closely watched monthly Employment Situation Report to offer an encompassing view of labor market conditions.

Because the surveys differ in their samples and objectives, they also track distinct definitions of employment. "Employment” in the household survey is a count of employed persons, while the establishment survey estimates the number of nonfarm jobs in the U.S. economy. The household survey's measure of employment is also wider in scope, as it includes individuals working in agriculture, a family business without pay, private households (e.g., housekeepers or nannies) or those who are self-employed or taking a temporary leave of absence (e.g., workers on strike). Meantime, the establishment survey only counts jobs that show up on a company’s payroll during the reference period (the pay period including the 12th day of the month). This excludes agricultural workers, the unincorporated self-employed and workers on unpaid leave. Finally, people working multiple jobs are counted as employed once in the household survey, whereas they are counted in the establishment survey for each position held.

Employment Trends Diverge Even After Adjusting for Conceptual Differences

The difference in how the CPS and CES account for multiple jobholders is at first glance a leading contender for the divergence between household and payroll employment trends. If Americans are increasingly turning to moonlighting to shore up their spending power, nonfarm payroll jobs would rise, but household employment would not. Indeed, multiple jobholders' share of household employment has grown starkly since the pandemic, with about 500K multiple jobholders added over the past year (Figure 3). Yet even after adjusting the household measure to account for multiple job holding, a wide gap remains.

The BLS produces an “adjusted” household measure, which accounts for definitional differences between CPS and CES employment.¹ The adjusted household series strips out agricultural workers, the self-employed, unpaid family workers, private household workers and workers on unpaid leave to match the scope of the establishment survey. It then adds in the number of multiple jobholders to match the “job count” concept of the payroll survey.² The adjustment is not perfect, but the adjusted employment level tends to track the payroll measure well over time (Figure 4). However, recently, even adjusted household employment has diverged from the establishment measure; in Q1 the U.S. economy added 400K more jobs per month by the payroll survey relative to the adjusted household measure (Figure 5). With conceptual differences not driving the divergence, could methodological and sampling technicalities be driving it?

Birth-Death Employment Factor

The birth-death factor is still providing a historically large lift to monthly payroll gains (Figure 6). The birth-death model is used to scale payrolls up or down in accordance with the net new change in businesses, since newly formed businesses are not immediately included in the establishment survey sample. Over the past year, the birth-death model has boosted nonfarm employment by an average of 114K per month, about 25K more than its pre-COVID lift following a post-pandemic surge in business formation. However, given the CPS is a survey of households and not businesses, it neither requires nor receives such an adjustment. We do not see the birth-death factor as a substantial contributor to the employment trend gap at present. The birth-death model is used specifically to combat sampling error in the payroll survey and should help to reconcile the establishment survey to the household.

Population Controls

A more likely explanation for the slower rate of employment growth in the household survey is that the CPS population controls may be understating the size of the civilian noninstitutional population (CNIP). Both the household and the payroll survey are regularly anchored to the universe that each survey represents. Given the job count concept of the CES, nonfarm payrolls are benchmarked annually using administrative data of employment counts from the State Unemployment Insurance (UI) system. For the household survey, the “population controls”—independent population estimates used to weight the CPS sample to reflect the CNIP—are adjusted annually by the Census Bureau and are incorporated into the household employment measure with the release of January data each year. For decennial census years, these population controls are from official counts and reflect the population base, but between census years, the controls are approximations based on births, deaths and net immigration. If the population controls are underestimating the total population, then the household measure of employment will be too low.

The CPS currently could be undercounting the CNIP and employment if the recent surge in immigration to the United States is not yet well reflected in the population controls. The Census net immigration estimates rely on lagged data and currently sit below other approximations—notably from Congressional Budget Office, which sees a swelling in net immigration post-pandemic (Figure 7). Accordingly, researchers at Brookings recently demonstrated the BLS' published CNIP has trended lower than would be implied using net immigration estimates from other agencies over the past year.³

The establishment survey's count of jobs, however, does not rely on population estimates and is likely to be faster to include an influx of foreign workers. In fact, researchers at the BLS found that an underestimation of the population controls explained one-third of the large employment gap between the CPS and CES (cumulatively 4.5 million) that arose in the late 1990s, the most notable historical period of divergence between the two surveys (Figure 8).⁴ During this period, reports from the Social Security Administration demonstrated a notable increase in the use of invalid Social Security numbers on payrolls, especially in industries that commonly employ undocumented workers.⁵ This evidence suggests that payroll employment estimates are more nimble to reflect foreign-born workers, regardless of their legal status.

While we will not have a full picture on the true population base for the household survey until the 2030 decennial census becomes available, we expect the updated CPS population controls in coming years to more closely reflect the heightened net immigration that other sources currently estimate. Accordingly, we would not be surprised to see the level of the adjusted household measure of employment eventually shift higher to more closely align with the level of payroll employment.

Survey Size and Nonresponse

A simpler reason behind the divergence between the establishment and household measures of employment may be the smaller sample of the household survey. The household measure tends to be more volatile as a result of its smaller sample size, and month-to-month movements in household employment are rarely statistically significant.⁶ Volatility in growth rates is worsened by the revisions practice of the household survey. Whereas the benchmarking of the payroll data revises historical level data, the household survey population controls are not normally applied to historical data, making level comparisons over the year noisy. Meantime, even as benchmark revisions are incorporated into historical establishment data, the CPS has an inconsistent record of presaging a slowdown in nonfarm payrolls; that is, payroll employment does not tend to be substantially revised to match the household course. All said, CPS employment typically ends up oscillating around the more stable payroll trend (Figure 9).

The volatility of the household survey has been compounded post-pandemic by a declining response rate. In 2019, the average response rates were 83% and 74% for the household and payroll surveys, respectively; these have since fallen to 70% and 65% for 2023 and show few signs of stabilizing (Figure 10). However, whereas both surveys have suffered from heightened nonresponse, the impact is felt much more pointedly in the household survey. With a sample size of 60K, only about 42K households are represented each month in the CPS while roughly ten-times that many worksites are incorporated into the monthly payroll release on Jobs Friday. Furthermore, while the initial establishment survey response has fallen precipitously, responses by the “final” third release of a reference month's payroll data are in-line with the pre-pandemic rate.

Conclusion: Don't Mind the Gap

We do not believe the marked downturn in household employment over the past year is a harbinger of an imminent collapse in nonfarm payrolls. A meaningful divergence in employment trends between the CPS and CES is not unprecedented and can be somewhat cyclical. Refer back to Figure 8, where we plotted the ratio of nonfarm payroll employment to nonagricultural wage and salary household employment (the closest series conceptually before the adjusted household series began in 1994). In many instances, payroll employment increases relative to household employment during expansions and falls in contractions. The tight labor market today is a continuation of this trend, and the increasingly different narratives from household and payroll employment may just be a result of an economy in expansion.

The divergence between the two surveys may also be a matter of the truth lying somewhere in between. Data from the Quarterly Census of Employment and Wages (QCEW), which is derived from the same state UI system as payrolls and serves as a basis for the establishment survey's benchmark revisions, show employment rising about half a percent slower in the year through September 2023 than as currently reported by payrolls (Figure 11). This implies the gap between the household and establishment surveys may also narrow through a more moderate—but still healthy—pace of payrolls once the 2024 annual benchmark revisions are released early next year.

We continue to put considerably more weight on the establishment survey's telling of the employment story. Birth-death adjustments may be adding a little extra to nonfarm payrolls at present, consistent with the QCEW data foreshadowing some downward adjustment in the next annual benchmarking, but these factors are unlikely to be significantly overstating payroll gains given the surge in business formations in recent years. We suspect difficulty estimating population controls is likely playing a larger role. Population controls are hard to estimate at the best of times but have been particularly challenging in recent years due to the jump in immigration and challenging circumstances when collecting the 2020 census base-year counts. The inherent volatility of the household survey's relatively small sample, combined with its declining response rates, also keeps us from reading much into the gap between the establishment and household surveys' measures of employment.

Although the gap between the household and establishment survey of employment has been eye-catching, ultimately the household survey is not designed to measure the monthly or even annual change in the level of employment. Instead, it is designed to provide information on labor force activities via rates of employment, unemployment and labor force participation across the overall population as well as sub-segments. Like the payroll survey's measure of job growth, the household survey's unemployment rate points to a still-strong jobs market, even if both surveys' primary outputs indicate conditions have cooled from a year or two ago. We will thus continue to keep our eyes on payrolls to gauge the strength of hiring and not fret over the near-stalling in household employment growth over the past year.

Endnotes

1 – The BLS “adjusted” household survey employment research series is not available prior to January 1994 because monthly multiple jobholding data is not available before then. For a longer-run view of the relationship between household and payroll employment, the BLS suggests using the household survey's nonagricultural wage and salary employment data, which is the closest proxy series. For more information on the adjusted household series, see the appendix of Comparing employment from the BLS household and payroll surveys. (Return)

2 – While the adjusted household series tracks well with payroll employment over time, the adjustment is not perfect. In particular, the “adjusted” household survey employment series does not precisely match the nonfarm jobs count of the CES because of how it accounts for multiple jobholders. The adjustment for multiple jobholders is based on only the respondent's primary job classification. If their secondary job is agricultural in nature or if they are self-employed in their secondary jobs, then this would lead to an overestimation of the multiple jobholder effect in the adjusted household measure. For additional information, see Bowler, Mary and Teresa Morisi (February 2006). “Understanding the employment measures from the CPS and CES survey.” Monthly Labor Review. (Return)

3 – Edelberg, Wendy and Tara Watson (March 2024). “New immigration estimates help make sense of the pace of employment.” The Hamilton Project at Brookings. (Return)

4 – Mary Bowler, Randy E. Ilg, Stephen Miller, Ed Robison, and Anne Polivka (February 2003). “Revisions to the Current Population Survey Effective in January 2003.” Employment and Earnings; Bowler, Mary and Teresa Morisi (February 2006). “Understanding the employment measures from the CPS and CES survey.” Monthly Labor Review. (Return)

5 – Office of the Inspector General (September 1999). “Patterns of Reporting Errors and Irregularities by 100 Employers with the Most Suspended Wage Items.” Audit Report A-03- 98-31009. Social Security Administration; Office of the Inspector General (February 2000) "The Social Security Administration’s Earnings Suspense File Tactical Plan and Efforts to Reduce the File’s Growth and Size." Evaluation Report A–03–97–31003. Social Security Administration. (Return)

6 – A month-to-month change in the household survey of +/- 600K is required for statistical significance at the 90% confidence level compared to +/- 130K in the establishment survey. (Return)