Purpose. The Nonprofit Research Activities (NPRA) module of the Annual Business Survey collects information on research and experimental development (R&D) performed or funded by private nonprofit organizations as defined in the Frascati Manual. Private nonprofit R&D organizations are tax-exempt nonprofit organizations whose R&D is not controlled by institutes of higher education, governments, or businesses.

NCSES combines nonprofit sector data with data from the other sectors to estimate total national R&D expenditures. Results of the research activities data collected from nonprofit organizations will be used to report updated, valid, and reliable estimates of U.S. nonprofit R&D in National Patterns of R&D Resources and the Bureau of Economic Analysis system of national accounts.

The data collected will also be incorporated into the National Science Board’s biennial report, Science and Engineering Indicators. The R&D data from the nonprofit module will be reported in the Organization for Economic Cooperation and Development (OECD) periodic publications and used for international comparisons of R&D efforts. NCSES also anticipates professional associations will use data from the nonprofit R&D module. Likely users in this category include, but are not limited to, the Science Philanthropy Alliance, the Association of Independent Research Institutes, and the Health Research Alliance.

Data collection authority. Title 13, United States Code, Sections 8(b), 131, and 182; Title 42, United States Code, Section 1861-76 (NSF Act of 1950, as amended); and Section 505 within the America COMPETES Reauthorization Act of 2010, authorize this collection. Sections 224 and 225 of Title 13 require mandatory response. Office of Management and Budget No. 0607-1004. The disclosure review number is NCSES-DRN24-060.

Survey sponsor. NCSES within NSF.

Survey collection and tabulation agent. The survey is conducted annually by the Census Bureau in accordance with an interagency agreement with NCSES.

Frequency. Annual.

Initial survey year. FY 2020. (The questions in the FY 2020 module were first developed and used on the FY 2016 Nonprofit Research Activities Survey, which was the first national survey of R&D activities in the U.S. nonprofit population since 1997. Due to the differences between the surveys prior to FY 2020 and the current module, FY 2020 is considered the initial survey year for the annual series.)

Reference period. FY 2022. The fiscal year referred to throughout this report was the nonprofit organizations’ fiscal year; for the majority of organizations reporting R&D performance, this fiscal year ended in either September or December 2022.

Response unit. Organizations.

Sample or census. Sample.

Population size. 42,000 organizations.

Sample size. 8,000 organizations.

Target population. Included are all nonfarm organizations with at least one in-scope location filing Internal Revenue Service (IRS) Form 990 as a tax-exempt organization and with an annual payroll of $500,000 or more.

Sampling frame. The sampling frame was constructed from the final 2021 Business Register and the Exempt Organizations Business Master File Extract (EO BMF). The Business Register is the Census Bureau’s comprehensive database of U.S. businesses. Business Register data are compiled from business tax returns, data collected from the economic census, and other Census Bureau surveys. The Business Register includes sole proprietorships, partnerships, and corporations reporting business activity to the IRS. The EO BMF is a publicly available list from the IRS of all organizations that are exempt from filing federal income taxes. Both files collect data at the establishment level; organizations may have multiple establishments.

The Business Register contains establishments that are out of scope for the nonprofit R&D module. These establishments are removed from the sampling universe. They include:

• Establishments engaged in the following North American Industry Classification System (NAICS) industries:
• 110000, 111, or 112 - Agriculture production
• 482 - Railroads
• 491 - U.S. Postal Service
• 521 - Monetary authorities – central bank
• 525 - Funds, trusts, and other financial vehicles
• 8131, 8134 - Religious and civic organizations
• 814 - Private households
• 92 - Public administration
• Establishments located in American Samoa, Northern Mariana Islands, Guam, Puerto Rico, or the Virgin Islands
• Establishments belonging to foreign entities
• Government establishments
• Establishments with zero payroll and zero employment
• For-profit establishments or nonprofit establishments within multi-establishment organizations that have a majority of organization payroll in for-profit establishments

Information on industry classification, receipts, payroll, and employment was extracted from the Business Register during the sampling frame construction. Nonprofit status for each establishment was determined by matching the Business Register to the IRS nonprofit list.

The sampling frame is stratified by primary type of organization, primary nonprofit activity, and state. Details on how organization-level sampling units are assigned to these strata follow.

The sample is selected at the organization level, so organizations with multiple establishments were combined into one organization-level unit for sampling. Organization payroll, receipts, and employment are set to the respective sums across all establishments within the organization.

Organizations with multiple establishments are assigned a single industry classification using a hierarchal system based on the largest payroll. For NAICS, the hierarchy is largest payroll sector, largest payroll 3-digit NAICS (within the largest sector), largest payroll 4-digit NAICS (within the largest 3-digit), and largest payroll 6-digit NAICS (within the largest 4-digit). For organization National Taxonomy of Exempt Entities (NTEE) code, the hierarchy is the first letter of the NTEE code with the largest payroll then the largest payroll of the full 3 characters of the NTEE code.

After organization-level units are created and organization-level codes are assigned, organizations are removed from the sampling frame if any of the following are true:

• Primary activity is identified as any of the following:
• Education services (NAICS sector 61 or NTEE code starting with B).
• Housing and shelter (NTEE begins with L except those with the last 2 digits of “05”)
• Recreation and sports (NTEE begins with N except those with last 2 digits of “05”)
• Youth development (NTEE begins with O except those with last 2 digits of “05”)
• Religion-related (NTEE begins with X except those with last 2 digits of “05”)
• Mutual benefit (NTEE begins with Y except those with last 2 digits of “05”)
• The organization is also in the Business Enterprise Research and Development (BERD) Survey sampling frame.
• Organization payroll is less than $500,000 annually.

Education services organizations are out of scope because the R&D for those organizations is measured by the Higher Education Research and Development (HERD) Survey. Organizations also in the BERD Survey sampling frame are removed because their R&D is measured by the BERD Survey. The remaining criteria identify other organizations that are unlikely to perform R&D activities, and these organizations are removed.

All records in the nonprofit sample universe are assigned a primary nonprofit activity stratum as follows:

• If the organization’s NAICS code begins with “622” or the first 2 characters of its NTEE is “E2” then its stratum is set to “A” (Hospitals).
• Else if the NAICS code begins with “5417” then its stratum is set to “C” (Science and technology).
• Else if the NAICS code begins with “62” or the first character of NTEE is “E” then its stratum is set to “B” (Other healthcare).
• Remaining unclassified organizations have their stratum set to “D” (Other).

Sample design. The nonprofit R&D frame is stratified by state and primary type of organization (hospitals, other healthcare, science and technology, and all others) (table A-1). Within these strata some nonprofits were selected with certainty based on the following criteria.

• Organizations with known R&D activity (950 organizations)
• Organizations larger than stratum-specific payroll cutoffs (1,200 organizations)

The nonprofit R&D sample consisted of 8,000 organizations, and 2,100 were selected with certainty.

The remaining 5,900 noncertainty cases were selected using systematic stratified random sample selection, with a maximum sample weight of 7.1.

Data collection. The survey was mailed to 8,000 nonprofit organizations in July 2023. The organizations were sent a letter informing them of their requirement to report under Title 13, United States Code, Sections 224 and 225. The letter also provided instructions on accessing and submitting the survey online. There were three mail follow-ups and four e-mail follow-ups conducted to increase response. The collection period closed on 29 December 2023.

Mode. The data were collected using an electronic instrument.

Response rates.

Check-in rate. The check-in rate is defined as the unweighted number of surveys that were submitted online by in-scope organizations, divided by the unweighted total number of all in-scope organizations in the sample (sample size). Response to individual questions did not factor into this metric. At the close of the collection period, the check-in rate was 79% (table A-2).

Unit response rate (URR). For this survey, the unit response is defined as an organization providing total expenses or employment and answering the R&D performed and R&D funded questions. URR is the ratio between the number of unit respondents in a sample (numerator) and the total sample size (denominator), expressed as a percentage. The URR was 77% (table A-3).

Item response rates. The distribution of values reported by sample organizations in the nonprofit module is highly skewed. Thus, rather than report unweighted item response rates, total quantity response rates are calculated, which are based on weighted data.

Total quantity response rate (TQRR). For a given published estimate other than count or ratio estimates, TQRR is the percentage of the weighted estimate based on reported data (including data obtained from other sources determined to be equivalent in quality to reported data) and weighted by sampling but not nonresponse weights. The TQRR for total expenditures for R&D performed by nonprofit organizations in the United States in 2022 was 76%.

Total quantity nonresponse rate (TQNR). For a given published estimate, TQNR, defined as 100% minus TQRR, is calculated for each tabulation cell from the nonprofits, except for cells that contain count or ratio estimates. TQNR measures the combined effect of the procedures used to handle unit and item nonresponse on the weighted nonprofit estimates. Detailed imputation rates are available upon request.

Data editing. Prior to tabulating the data, response data were reviewed and edited with both automated and manual procedures to correct detectable reporting errors. R&D data were tabulated for records reporting $50,000 or more in R&D expenditures. Survey analysts reviewed the R&D data reported by the respondents. The data were evaluated by calculating the reported R&D expenses to expense ratios and reviewing the organization’s website information.

Additional reporting errors were detected and corrected using an automated data edit system designed to review the data for reasonableness and consistency. The editing process interactively corrected detected errors using standard procedures. Quality control techniques were used to verify that procedures were carried out as specified.

During the editing of FY 2022 data, errors in reporting and eligibility determinations for organizations in FY 2020 and FY 2021 were discovered and revisions were made to correct these. For accurate historical data, use only the most recently released data tables.

Imputation.

Item nonresponse. If detailed R&D data was not reported by a nonprofit and could not be inferred by survey analysts, it was imputed in the same ratio as reported by other nonprofits in the same sample stratum. These imputations are reflected in the reported imputation rates.

Unit nonresponse. Estimates produced from the survey data include adjustments to account for organizations that did not respond to the survey (unit nonresponse). If available, data from public tax filings, annual reports, or audits were used to impute expenses and R&D for select nonprofits known to have performed large amounts of R&D based on public information or prior surveys. Otherwise, unit nonresponse is handled by adjusting weighted reported data as follows. Each organization’s sampling weight is multiplied by a nonresponse adjustment factor. To calculate the adjustment factors, each organization in the sample that is eligible for tabulation is assigned to the adjustment cells. The adjustment cells for nonprofits are based on certainty or noncertainty sampling strata and NAICS sector. For NAICS sector, there are three categories: healthcare, science and technology, and all other organizations. For a given adjustment cell, the nonresponse adjustment factor is the ratio of the sum of the sampling weights for all organizations included in the cell to the sum of the sampling weights for all organizations included in the cell with reported data. For the nonresponse adjustment, an organization is considered a respondent if it satisfies the definition of response as detailed in the URR section above.

Weighting. The survey data are weighted for sampling and unit nonresponse.

Industry classification. Nonprofit organizations are classified into one of four types of organization at the time of sampling: hospitals, other healthcare, science and technology, and all other organizations. Organizations tabulated based on the classification at the time of sampling with hospitals and other healthcare were collapsed into one tabulation group called healthcare. Classification is based on both the 2017 NAICS and the NTEE. An NTEE code is not available for all organizations.

Organizations are first classified as hospitals if the first two characters of their NTEE code are “E2” or the first three characters of their NAICS code are “622.” Remaining organizations are classified as science and technology if the first four characters of their NAICS code are “5417.” Remaining organizations are classified as other healthcare if the first character of their NTEE code is “E,” or the first 2 digits of their NAICS code are “62.” All remaining organizations are classified as other.

Variance estimation. This survey uses delete-a-group jackknife variance estimator. Note that certainty cases do not contribute to the sampling variance. The delete-a-group jackknife variance estimator requires that every sampling stratum contains at least two sampled organizations. Sampling strata that do not meet this requirement are collapsed as needed to create a new set of variance estimation strata that satisfies this requirement.

The estimates produced from the NPRA module are subject to both sampling and nonsampling errors.

Sampling error. Sampling error is the difference between estimates obtained from the sample and results theoretically obtainable from a comparable complete enumeration of the sampling frame. This error results because only a subset of the sampling frame is measured in a sample survey. For published estimates from NPRA, standard errors are produced for estimated percentages, while relative standard errors (RSEs) are produced for all other estimates. Tables of the estimated measures of sampling variability corresponding to each data table are available upon request.

Coverage error. Coverage error occurs when the sampling frame fails to completely enumerate the population of interest. There can be both undercoverage error, where organizations are not included in the frame, and overcoverage error, where organizations included in the frame are out of scope for the population of interest. The NPRA module uses the prior year’s Business Register to construct the frame so any changes in organizations that would change the inclusion or exclusion of the organization to the survey scope could be sources of coverage error. Prior to tabulation, the survey unit’s information is updated with the most recent available Business Register data to mitigate this source of error.

Nonresponse error. Nonresponse error refers to the differences in key estimates between organizations in the sampling frame that were sampled for data collection and those that responded. For unit nonresponse, multiple follow-ups were conducted with nonresponding organizations to mitigate nonresponse error. The final survey weights incorporated nonresponse adjustments to reduce the risk of nonresponse bias in the final estimates. 

Nonresponse bias for survey estimates cannot be directly measured. However, the impact of nonresponse was incorporated into the variability of survey estimates using the assumption that the data is missing at random. Missing at random for this survey means that, conditional on the nonresponse adjustments, the propensity for an organization to respond is not related to R&D performance.

To minimize item nonresponse, organizations were encouraged to report estimates of expenditures when actual dollar amounts could not be provided. This approach reduces item nonresponse error risk but may introduce measurement error. Imputation was conducted to help mitigate item nonresponse error.

Measurement error. The most common source of measurement error was reporting in different units (e.g., reporting in whole dollars rather than in thousands of dollars). This was corrected during data processing. Another source of error involved incorrect inclusion of organizations already represented in different R&D data collections. The R&D of these respondents was set to 0 if it was determined their R&D was already represented in other surveys. These cases included nonprofit organizations managing federal laboratories and some university-affiliated hospitals.

The questions in the FY 2020 module were first developed and used on the FY 2016 Nonprofit Research Activities Survey—which was the first national survey of R&D activities in the U.S. nonprofit population since 1997. Due to the differences in the surveys prior to FY 2020, those data are not comparable for trend analysis.

Contract employees. Contract employees are individuals contracted to work on projects otherwise fully performed by organization staff.

Depreciation and amortization. Depreciation on tangible research assets, such as buildings or equipment, as well as amortization of intangible assets, such as patents.

Employment. Paid employment consists of full- and part-time employees, including salaried officers and executives of corporations who were on the payroll in the pay period including 12 March 2022. Included are employees on sick leave, holidays, and vacations; not included are proprietors and partners of unincorporated businesses.

Other support personnel. Staff who provide direct support services for the research project.

Research and experimental development (R&D). R&D is planned, creative work aimed at discovering new knowledge or devising new applications of available knowledge. This includes (1) activities aimed at acquiring new knowledge or understanding without specific immediate commercial applications or uses (basic research); (2) activities aimed at solving a specific problem or meeting a specific commercial objective (applied research); and (3) systematic use of research and practical experience and resulting in additional knowledge, which is directed to producing new or improved goods, services, or processes (experimental development). R&D includes both direct costs, such as salaries of researchers and administrative and overhead costs clearly associated with the organization’s R&D. However, R&D does not include expenditures for routine product testing, quality control, and technical services unless they are an integral part of an R&D project. R&D also does not include market research; efficiency surveys or management studies; literary, artistic, or historical projects, such as films, music, or books and other publications; and prospecting or exploration for natural resources.

• Basic research. Activities aimed at acquiring new knowledge or understanding without specific immediate commercial applications or uses.
• Applied research. Activities aimed at solving a specific problem or meeting a specific commercial objective.
• Experimental development. Systematic work, drawing on research and practical experience and resulting in additional knowledge, which is directed to producing new products or processes or to improving existing products or processes.

Researchers. Professionals engaged in the conception or creation of new knowledge.

Research technicians. Staff who work under the supervision of researchers to conduct research activities.

Research volunteers. Unpaid workers contributing appreciable and essential activity to research performed by the organization. Their research skills should be comparable to the organization’s employees. Individuals funded by other sources to conduct research at the organization are not considered volunteers.

Salaries, wages, and fringe benefits. Costs for all compensation and benefits of employees who are included in the total R&D expenditures reported.