Survey Info

Summary

The Nonprofit Research Activities (NPRA) module of the Annual Business Survey measures research and experimental development (R&D) performance and funding at U.S. 501(c) nonprofit organizations.

Areas of Interest

Survey Administration

This survey was conducted by the Census Bureau in partnership with the National Center for Science and Engineering Statistics within the National Science Foundation.

Survey Details

Status Active
Frequency Annual
Reference Period FY 2020
Next Release Date October 2025

Methodology

Survey Description

Survey Overview (FY 2020 Survey Cycle)

Purpose

The Nonprofit Research Activities (NPRA) module of the Annual Business Survey collects information on research and experimental development (R&D) performed or funded by nonprofit organizations in the United States. 

Data collection authority

Title 13, United States Code, Sections 8(b), 131, and 182; Title 42, United States Code, Section 1861-76 (National Science Foundation Act of 1950, as amended); and Section 505 within the America COMPETES Reauthorization Act of 2010, authorize this collection. Sections 224 and 225 of Title 13 require mandatory response. Office of Management and Budget No. 0607-1004.

Major changes to recent survey cycle

Not applicable.

Key Survey Information

Frequency

Annual.

Initial survey year

2021. (The questions in the FY 2020 module were first developed and used on the FY 2016 Nonprofit Research Activities Survey, which was the first national survey of R&D activities in the U.S. nonprofit population since 1997. Due to the differences between the previous surveys and the new module, 2021 is considered the initial survey year for the annual series.)

Reference period

Fiscal year 2020.

Response unit

Organizations.

Sample or census

Sample.

Population size

48,500 organizations.

Sample size

8,000 organizations.

Key variables

Key variables of interest are listed below.

  • R&D expenditures by source of funds (federal government, state and local government, business, foundations, universities, other nonprofits, internal funds, individual donors, and other) 
  • R&D expenditures by type of R&D (basic research, applied research, and experimental development)
  • R&D expenditures by field of R&D (agricultural sciences and natural resources and conservation; biological, biomedical, and health sciences; engineering; geosciences, atmospheric sciences, and ocean sciences; mathematics, statistics, and computer and information sciences; physical sciences; psychology and social sciences; humanities; and other fields)
  • Headcounts and full-time equivalent counts of R&D personnel (researchers, technicians, and other support personnel)
  • Headcounts for contract employees and volunteers working on research activities
  • R&D funding provided to others, by type of funding (grants and contracts vs. subawards and subcontracts)

Survey Design

Target population

Included are all businesses with at least one in-scope location filing Internal Revenue Service (IRS) Form 990 as a tax-exempt organization and with FY 2019 annual payroll of $500,000 or more.

Sampling frame

The sampling frame was constructed from the final 2019 Business Register and the Exempt Organizations Business Master File Extract (EO BMF). The Business Register is the Census Bureau’s comprehensive database of U.S. businesses. Organizations were excluded from the frame if they were considered outside the scope of the survey (e.g., churches, government organizations, educational institutions, or organizations located outside the United States). A financial threshold was also imposed to increase the efficiency of reaching organizations that perform research.

Sample design

The nonprofit R&D frame is stratified by state and primary type of organization (hospitals, other health care, science/technology, and all others). Within these strata some nonprofits were selected with certainty based on the following criteria:

  • firms with known R&D activity (1,000 organizations)
  • firms larger than stratum-specific payroll cutoffs (1,200 organizations)

The nonprofit R&D sample consisted of 8,000 organizations; 2,200 were selected with certainty.

The remaining 5,800 noncertainty cases were selected using the systematic stratified random sample selection. The maximum sample weight was 8.3.

Data Collection and Processing

Data collection

The survey was mailed to 8,000 nonprofit organizations in July 2021. Organizations were sent a letter informing them of their requirement to report under Title 13, United States Code, Sections 224 and 225. The letter also provided instructions on how to access the survey and submit online. There were three mail follow-ups conducted to increase response. The third mail follow-up included a paper questionnaire for select nonrespondents.

Additionally, the Census Bureau conducted email follow-ups to respondents who logged into the electronic system but did not submit the questionnaire. The collection period closed in January 2022.

Data processing

Prior to tabulating the data, response data were reviewed and edited with both automated and manual procedures to correct reporting errors. R&D data were tabulated for records reporting $50,000 or more in R&D expenditures. Survey analysts reviewed the R&D reported by the survey respondents. Research was done by evaluating the reported R&D to expenses ratio and the organization website information.

Additional data errors were detected and corrected using an automated data edit system designed to review the data for reasonableness and consistency. The editing process interactively performed corrections by using standard procedures to fix detectable errors. Quality control techniques were used to verify that operating procedures were carried out as specified.

Estimation techniques

Where possible, missing data were imputed using previous survey data or publicly available documents such as annual reports and financial statements. Weights were used to account for unequal probabilities of selection, nonresponse, and to calibrate sample estimates of expenses to match total expenses on the frame. Measures of sampling variability were estimated using the delete-a-group jackknife variance estimator.

Survey Quality Measures

Sampling error

Estimates of sampling errors associated with the detailed statistical tables will be available by request.

Coverage error

Coverage error occurs when the frame fails to completely enumerate the population of interest. There can be both undercoverage error, where units are not included in the frame, and overcoverage error, where units included in the frame are out of scope for the population of interest. The ABS uses the prior-year Business Register to construct the frame so any changes in businesses that would change the inclusion or exclusion of the business to the survey scope could be sources of coverage error. Prior to tabulation, the survey unit’s information is updated with the most recent available Business Register data to mitigate this source of error.

Nonresponse error

Unit nonresponse is handled by adjusting weighted reported and imputed data by multiplying each organization's sampling weight by a nonresponse adjustment factor. Detailed descriptions of the adjustments for nonresponse are available in the Technical Notes accompanying the data tables.

Measurement error

The most common source of measurement error was reporting in different units (for example, reporting in whole dollars rather than in thousands of dollars). This was corrected during data processing. Another source of error involved incorrect inclusion of organizations already represented in different R&D data collections. The R&D of these respondents was set to 0 where it was determined their R&D was already represented in other survey responses. These cases included nonprofit organizations managing federal laboratories and some university-affiliated hospitals.

Data Availability and Comparability

Data availability
Data comparability

The questions in the FY 2020 module were first developed and used on the FY 2016 Nonprofit Research Activities Survey, which was the first national survey of R&D activities in the U.S. nonprofit population since 1997. Due to the differences between the previous surveys and the new module, data are not comparable for trend analysis.

Data Products

Publications.

NPRA data will be published in NCSES InfoBriefs and data tables available at https://www.nsf.gov/statistics/srvynpra/.

Electronic Access

The NPRA module contains confidential data that are protected under Title 13 and Title 26 of the United States Code. Two types of data are currently available: public-use tabular statistics and restricted microdata. Public-use tabular statistics can be obtained on the NCSES Web site (https://ncses.nsf.gov/) and by contacting NCSES. Restricted microdata will be available at any of the 15 secure Research Data Centers administered by the Center for Economic Studies (CES) at the Census Bureau. Researchers interested in accessing microdata can apply for a restricted-use license by submitting a proposal to the CES, which evaluates proposals based on their benefit to the Census Bureau, scientific merit, feasibility, and risk of disclosure. To learn more about the Research Data Centers and how to apply, please visit the CES page on research with restricted-use data. For additional information about the application process, including how to initiate a project, please contact the administrator at the primary site where the research will be conducted. Per the Federal Cybersecurity Enhancement Act of 2015, the data are protected from cybersecurity risks through screening of the systems that transmit the data.

 

Technical Notes

Survey Overview (FY 2020 Survey Cycle)

Purpose. The NPRA module of the ABS collects information on R&D performed or funded by 501(c) nonprofit organizations in the United States. The nonprofit sector is one of four sectors (business, government, higher education, and other private nonprofit) that fund or perform R&D.

NCSES combines nonprofit sector data with data from the other sectors to estimate total national R&D expenditures. Results of the research activities data collected from nonprofit organizations will be used to report updated, valid, and reliable estimates of U.S. nonprofit R&D in National Patterns of R&D Resources and the Bureau of Economic Analysis system of national accounts.

The data collected will also be incorporated into the National Science Board’s biennial report, Science and Engineering Indicators. The R&D data from the nonprofit module will be reported in the Organization for Economic Cooperation and Development (OECD) periodic publications and used for international comparisons of R&D efforts. NCSES also anticipates professional associations will use data from the nonprofit R&D module. Likely users in this category include, but are not limited to, the Science Philanthropy Alliance, the Association of Independent Research Institutes, and the Health Research Alliance.

Data collection authority. Title 13, United States Code, Sections 8(b), 131, and 182; Title 42, United States Code, Section 1861-76 (NSF Act of 1950, as amended); and Section 505 within the America COMPETES Reauthorization Act of 2010, authorize this collection. Sections 224 and 225 of Title 13 require mandatory response. Office of Management and Budget No. 0607-1004.

Survey sponsor. NCSES within NSF.

Survey collection and tabulation agent. The survey is conducted annually by the Census Bureau in accordance with an interagency agreement with NCSES.

Key Survey Information

Frequency. Annual.

Initial survey year. 2021. (The questions in the FY 2020 module were first developed and used on the FY 2016 Nonprofit Research Activities Survey, which was the first national survey of R&D activities in the U.S. nonprofit population since 1997. Due to the differences between the previous surveys and the new module, 2021 is considered the initial survey year for the annual series.)

Reference period. Fiscal year 2020. The fiscal year referred to throughout this report was the nonprofit organizations’ fiscal year; for the majority (62%) of organizations reporting R&D performance, this fiscal year ended in either September or December of 2020.

Response unit. Organizations.

Sample or census. Sample.

Population size. 48,500 organizations.

Sample size. 8,000 organizations.

Survey Design

Target population. Included are all businesses with at least one in-scope location filing Internal Revenue Service (IRS) Form 990 as a tax-exempt organization and with FY 2019 annual payroll of $500,000 or more.

Sampling frame. The sampling frame was constructed from the final 2019 Business Register and the Exempt Organizations Business Master File Extract (EO BMF). The Business Register is the Census Bureau’s comprehensive database of U.S. businesses. Business Register data are compiled from a combination of business tax returns, data collected from the economic census, and data from other Census Bureau surveys. The Business Register includes sole proprietorships, partnerships, and corporations reporting business activity to the IRS. The EO BMF is a publicly available list from the IRS of all organizations that are exempt from filing federal income taxes.

The Business Register contains establishments that are out of scope for the nonprofit R&D module. These establishments are removed from the sampling universe. They include:

  • Establishments engaged in the following North American Industry Classification System (NAICS) industries:
    • 110000, 111, or 112 - Agriculture production
    • 482 - Railroads
    • 491 - U.S. Postal Service
    • 521 - Monetary authorities–central bank
    • 525 - Funds, trusts, and other financial vehicles
    • 8131, 8134 - Religious and civic organizations
    • 814 - Private households
    • 92 - Public administration
  • Establishments located in American Samoa, Northern Mariana Islands, Guam, Puerto Rico, or the Virgin Islands
  • Establishments belonging to foreign entities
  • Government establishments
  • Establishments with zero payroll and zero employment
  • For-profit establishments or nonprofit establishments within multi-establishment firms that have a majority of firm payroll in for-profit establishments

Information on industry classification, receipts, payroll, and employment was extracted from the Business Register during the frame construction. Nonprofit status for each establishment was determined by matching the Business Register to the IRS nonprofit list.

The ABS sample is selected at the firm level, so firms with multiple establishments were combined into one firm-level unit for sampling. Firm payroll, receipts, and employment are set to the respective sums across all establishments within the firm. Firm industry is set to the 6-digit NAICS code with the highest aggregate payroll within the highest 4-digit NAICS within the highest 3-digit NAICS within the highest 2-digit NAICS sector. Firm industry is set to sector 55 (management of companies and enterprises) only if there are no establishments within the firm that belong to other sectors. Firm National Taxonomy of Exempt Entities (NTEE) code is set to the NTEE code with the highest aggregate payroll within the NTEE group (first digit of NTEE code) with the highest aggregate payroll.

After firm-level units are created and firm-level codes are assigned, firms are removed from the sampling frame if any of the following are true:

  • Primary activity is education services (NAICS sector 61 or NTEE code starting with B).
  • The firm is also in the Business Enterprise Research and Development Survey (BERD) sampling frame.
  • Firm payroll is less than $500,000.

All records in the nonprofit sample universe are assigned a primary nonprofit activity stratum as follows:

  • If the organization’s NAICS code begins with “622” or the first 2 characters of its NTEE is “E2” then its stratum is set to “A” (Hospitals).
  • Else if the NAICS code begins with “5417” then its stratum is set to “C” (Science and technology).
  • Else if the NAICS code begins with “62” or the first character of NTEE equals “E” then its stratum is set to “B” (Other healthcare).
  • Remaining unclassified organizations have their stratum set to “D” (Other).

Sample design.

The nonprofit R&D frame is stratified by state and primary type of organization (hospitals, other health care, science/technology, and all others). Within these strata some nonprofits were selected with certainty based on the following criteria:

  • firms with known R&D activity (1,000 organizations)
  • firms larger than stratum-specific payroll cutoffs (1,200 organizations)

The nonprofit R&D sample consisted of 8,000 organizations; 2,200 were selected with certainty.

The remaining 5,800 noncertainty cases were selected using the systematic stratified random sample selection. The maximum sample weight was 8.3.

Data Collection and Processing Methods

Data collection. The survey was mailed to 8,000 nonprofit organizations in July 2021. Organizations were sent a letter informing them of their requirement to report under Title 13, United States Code, Sections 224 and 225. The letter also provided instructions on how to access the survey and submit online. There were three mail follow-ups conducted to increase response. The third mail follow-up included a paper questionnaire for select nonrespondents.

Additionally, the Census Bureau conducted email follow-ups to respondents who logged into the electronic system but did not submit the questionnaire. The collection period closed in January 2022.

Mode. The data were collected using both a paper form and the electronic instrument.

Response rates.

Check-in rate. The check-in rate is defined as the unweighted number of surveys that were submitted online by in-scope organizations, divided by the unweighted total number of all in-scope organizations in the sample. Response to individual questions did not factor into this metric. At the close of the collection period in January 2022, there were 6,800 responses submitted. Of those, 99% reported online and 1% reported using the paper form.

Unit response rate (URR). Unit response is defined as an organization providing total expenses or employment and answering the R&D performance and R&D funded questions. URR is the ratio between the number of unit respondents in a sample (numerator) and total sample size (denominator), expressed as a percentage.

For the nonprofit R&D module, the URR was 83%.

Item response rates. The distribution of values reported by sample organizations in the nonprofit module is highly skewed. Thus, rather than report unweighted item response rates, total quantity response rates are calculated, which are based on weighted data.

Total quantity response rate (TQRR). For a given published estimate other than count or ratio estimates, TQRR is the percentage of the weighted estimate based on data that were reported by units in the sample or on data that were obtained from other sources and were determined to be equivalent in quality to reported data and weighted only by sampling but not nonresponse weights. The TQRR for total expenditures for R&D performed by nonprofit organizations in the United States in 2020 was 77%.

Total quantity nonresponse rate (TQNR). For a given published estimate, TQNR, defined as 100% minus TQRR, is calculated for each tabulation cell from the nonprofits, except for cells that contain count or ratio estimates. TQNR measures the combined effect of the procedures used to handle unit and item nonresponse on the weighted nonprofits estimates. Detailed imputation rates are available upon request.

Data editing. Prior to tabulating the data, response data were reviewed and edited with both automated and manual procedures to correct reporting errors. R&D data were tabulated for records reporting $50,000 or more in R&D expenditures.

Survey analysts reviewed the R&D reported by the respondents. Research was done by evaluating the reported R&D to expenses ratio and the organization website information.

Additional data errors were detected and corrected using an automated data edit system designed to review the data for reasonableness and consistency. The editing process interactively performed corrections by using standard procedures to fix detectable errors. Quality control techniques were used to verify that operating procedures were carried out as specified.

Imputation.

Item nonresponse. If detailed R&D data was not reported by a nonprofit and could not be inferred by survey analysts, it was imputed in the same ratio as reported by other nonprofits in the same sample stratum. These imputations are reflected in the reported imputation rates.

Unit nonresponse. Estimates produced from the ABS include adjustments to account for organizations that did not respond to the survey (unit nonresponse). If available, data from public tax filings, annual reports, or audits were used to impute expenses and R&D for select nonprofits known to have performed large amounts of R&D based on public information or prior surveys. Otherwise, unit nonresponse is handled by adjusting weighted reported data as follows. Each organization’s sampling weight is multiplied by a nonresponse adjustment factor. To calculate the adjustment factors, each organization in the sample that is eligible for tabulation is assigned to the adjustment cells. The adjustment cells for nonprofits are based on certainty or noncertainty sampling strata and NAICS sector. For NAICS sector, there are three categories: healthcare, science and technology, and all other. For a given adjustment cell, the nonresponse adjustment factor is the ratio of the sum of the sampling weights for all organizations in the cell to the sum of the sampling weights for all organizations in the cell with reported data. For the nonresponse adjustment, an organization is considered a respondent if it satisfies the definition of response as available in the URR section above.

Weighting. The survey data are weighted for sampling and unit nonresponse.

Industry classification. Nonprofits are classified into one of four types of organization at the time of sampling: hospitals, other healthcare, science and technology, and all other organizations. Organizations tabulated based on the classification at the time of sampling with hospitals and other healthcare were collapsed into one tabulation group called healthcare. Classification is based on both the 2017 NAICS (https://www.census.gov/naics/) and the NTEE. NTEE code is not available for all organizations.

Organizations with more than one domestic establishment are assigned a single industry classification using a hierarchal system based on the largest payroll. For NAICS, the hierarchy is largest payroll sector, largest payroll 3-digit NAICS (within the largest sector), largest payroll 4-digit NAICS (within the largest 3-digit), and largest payroll 6-digit NAICS (within the largest 4-digit). For NTEE the hierarchy is largest first letter of the NTEE code then largest full 3 characters of the NTEE code.

Organizations are first classified as hospitals if the first two characters of their NTEE code are “E2” or the first three characters of their NAICS code are “622.” Remaining organizations are classified as science and technology if the first four characters of their NAICS code are “5417.” Remaining organizations are classified as other healthcare if the first character of their NTEE code is “E” or the first two digits of their NAICS code are “62.” All remaining organizations are classified as other.

Variance estimation.

This survey uses delete-a-group jackknife variance estimator. Note that certainty cases do not contribute to the sampling variance. The delete-a-group jackknife variance estimator requires that every sampling stratum contains at least two sampled firms. Sampling strata that do not meet this requirement are collapsed as needed to create a new set of variance estimation strata that satisfies this requirement.

Detailed relative standard errors are available on request.

Survey Quality Measures

The estimates produced from the NPRA module are subject to both sampling and nonsampling errors.

Sampling error. The sampling error is described above in the variance estimation section.

Coverage error. Coverage error occurs when the frame fails to completely enumerate the population of interest. There can be both undercoverage error, where units are not included in the frame, and overcoverage error, where units included in the frame are out of scope for the population of interest. The ABS uses the prior-year Business Register to construct the frame so any changes in businesses that would change the inclusion or exclusion of the business to the survey scope could be sources of coverage error. Prior to tabulation, the survey unit’s information is updated with the most recent available Business Register data to mitigate this source of error.

Nonresponse error. Nonresponse error refers to the differences in key estimates between units (i.e., organizations) in the sampling frame that were sampled for data collection and those that responded. For unit nonresponse, multiple follow-ups were conducted with nonresponding organizations, and multiple contact and data collection modes were used (i.e., phone, mail, and e-mail) to mitigate nonresponse error. The final survey weights incorporated nonresponse adjustments to reduce the risk of nonresponse bias in the final estimates. 

Nonresponse bias for survey estimates cannot be directly measured. However, the impact of nonresponse can be incorporated into the variability of survey estimates assuming the data is missing at random. Missing at random for this survey means that, conditional on the nonresponse adjustments, the propensity for an organization to respond is not related to R&D performance.

For item nonresponse, organizations were encouraged to report estimates of expenditures when actual dollar amounts could not be provided. This approach reduces item nonresponse error risk but may introduce measurement error. Imputation was conducted to help mitigate item nonresponse error.

Measurement error. The most common source of measurement error was reporting in different units (e.g., reporting in whole dollars rather than in thousands of dollars). This was corrected during data processing. Another source of error involved incorrect inclusion of organizations already represented in different R&D data collections. The R&D of these respondents was set to 0 if it was determined their R&D was already represented in other R&D survey responses. These cases included nonprofit organizations managing federal laboratories and some university-affiliated hospitals.

Data Comparability

The questions in the FY 2020 module were first developed and used on the FY 2016 Nonprofit Research Activities Survey—which was the first national survey of R&D activities in the U.S. nonprofit population since 1997. Due to the differences between the previous surveys and the new module, data are not comparable for trend analysis.

Definitions

  • Employment. Paid employment consists of full- and part-time employees, including salaried officers and executives of corporations who were on the payroll in the pay period including 12 March 2021. Included are employees on sick leave, holidays, and vacations; not included are proprietors and partners of unincorporated businesses.
  • Research and experimental development (R&D). R&D is planned, creative work aimed at discovering new knowledge or devising new applications of available knowledge. This includes (1) activities aimed at acquiring new knowledge or understanding without specific immediate commercial applications or uses (basic research); (2) activities aimed at solving a specific problem or meeting a specific commercial objective (applied research); and (3) systematic use of research and practical experience and resulting in additional knowledge, which is directed to producing new or improved goods, services, or processes (experimental development). R&D includes both direct costs, such as salaries of researchers and administrative and overhead costs clearly associated with the organization’s R&D. However, R&D does not include expenditures for routine product testing, quality control, and technical services unless they are an integral part of an R&D project. R&D also does not include market research; efficiency surveys or management studies; literary, artistic, or historical projects, such as films, music, or books and other publications; and prospecting or exploration for natural resources.
    • Basic research. Activities aimed at acquiring new knowledge or understanding without specific immediate commercial applications or uses.
    • Applied research. Activities aimed at solving a specific problem or meeting a specific commercial objective.
    • Experimental development. Systematic work, drawing on research and practical experience and resulting in additional knowledge, which is directed to producing new products or processes or to improving existing products or processes.

Data

Product ID  NSF 23-317
  |  
Published  December 2022
 

General Notes

These tables present the results of the Nonprofit Research Activities (NPRA) FY 2020 module of the 2021 Annual Business Survey (ABS), conducted by the Census Bureau for the National Center for Science and Engineering Statistics (NCSES) within the National Science Foundation (NSF). It is the first collection of nonprofit research and experimental development (R&D) data by the Annual Business Survey and the beginning of an annual data series for this population. The questions in the FY 2020 module were first developed and used in the FY 2016 Nonprofit Research Activities Survey, which was the first national survey of R&D activities in the U.S. nonprofit population since 1997. This survey is the primary source of information about 501(c) nonprofit organizations’ R&D performance and funding in the United States.

The Census Bureau has reviewed the data product for unauthorized disclosure of confidential information and has approved the disclosure avoidance practices applied. For the FY 2020 tabular statistics, the approval ID is CBDRB-FY22-333, on 25 July 2022.

 

Acknowledgments and Suggested Citation

Acknowledgments

Ronda Britt of the National Center for Science and Engineering Statistics (NCSES) developed and coordinated this report under the guidance of Gary Anderson, Acting NCSES Program Director, and under the leadership of Emilda B. Rivers, NCSES Director; Vipin Arora, NCSES Deputy Director; and John Finamore, NCSES Chief Statistician. Jock Black reviewed the report. In partnership with NCSES, the Census Bureau conducted the survey and prepared the tables.

Suggested Citation

National Center for Science and Engineering Statistics (NCSES). 2022. Nonprofit Research Activities: FY 2020. NSF 23-317. Alexandria, VA: National Science Foundation. Available at https://ncses.nsf.gov/pubs/nsf23317.

Analysis

Survey Contact

For additional information about this survey or the methodology, contact

Ronda Britt
Survey Manager
Phone
(703) 292-7765
E-mail
rbritt@nsf.gov
Address
National Center for Science and Engineering Statistics, 2415 Eisenhower Avenue, Suite W14200, Alexandria, VA 22314