Purpose. The FFRDC Research and Development Survey is conducted by the National Center for Science and Engineering Statistics (NCSES) within the National Science Foundation (NSF). It is the primary source of information on separately accounted for R&D expenditures at FFRDCs in the United States.

Data collection authority. The information is solicited under the authority of the National Science Foundation Act of 1950, as amended, and the American COMPETES Reauthorization Act of 2010. The Office of Management and Budget control number for the FY 2019 FFRDC R&D Survey is 3145-0100, with an expiration date of 31 August 2022.

Survey contractor. ICF.

Survey sponsor. NCSES.

Frequency. Annual.

Initial survey year. 2001.

Reference period. FY 2019.

Response unit. Establishment.

Sample or census. Census.

Population size. 42.

Sample size. The survey is a census of all known eligible FFRDCs.

Target population. All FFRDCs.

Sampling frame. The total survey universe is identified through the NSF Master Government List of FFRDCs. NSF is responsible for maintaining this list and queries all federal agencies annually to determine any changes to, additions to, or deletions from the list.

Sample design. The FFRDC R&D Survey is a census of all eligible organizations.

Data collection. The FY 2019 survey announcements were sent by e-mail to all FFRDCs in November 2019. Respondents could choose to complete a questionnaire downloaded from the Web or use a Web-based data collection system to respond to the survey. Every effort was made to maintain close contact with respondents to preserve the consistency and continuity of the resulting data. Survey data reports were available on the survey website for each institution; these reports showed comparisons between the current year and the 2 prior years of data and noted any substantive disparities. Questionnaires were carefully examined for completeness upon receipt. Respondents were sent personalized e-mail messages asking them to provide any necessary revisions before the final processing and tabulation of data. These e-mail messages included a link to the FFRDC R&D Survey Web-based collection system, allowing respondents to view and correct their data online.

Respondents were asked to explain significant differences between current year reporting and established patterns of reporting verified for prior years. They were encouraged to correct prior year data, if necessary. When respondents updated or amended figures from past years, NCSES made corresponding changes to trend data in the 2019 data tables and to the underlying microdata. For accurate historical data, use only the most recently released data tables.

Mode. Respondents could respond to the survey by completing an Adobe PDF or Microsoft Excel questionnaire downloaded from the Web or by using a Web-based data collection system. All FFRDCs submitted data using the Web-based survey.

Response rates. All 42 FFRDCs included on the NSF Master Government List of FFRDCs during the FY 2019 survey cycle completed the key survey questions.

Data editing. The FFRDC R&D Survey was subject to very little editing; respondents were contacted and asked to resolve possible self-reporting issues themselves. Questionnaires were carefully examined by survey staff upon receipt. These reviews focused on unexplained missing data and explanations provided for changes in reporting patterns. If additional explanations or data revisions were needed, respondents were sent personalized e-mail messages asking them to provide any necessary revisions before the final processing and tabulation of data.

Imputation. No data were imputed for FY 2019.

Weighting. FFRDC R&D Survey data were not weighted.

Variance estimation. No variance estimation techniques were used.

Sampling error. Because the FY 2019 survey was a survey distributed to all organizations in the universe, there was no sampling error.

Coverage error. Given the availability of a comprehensive FFRDC list, there is no known coverage error for this survey. FFRDCs are identified through the NSF Master Government List of FFRDCs. NSF is responsible for maintaining the master list and queries all federal agencies annually to determine changes to, additions to, or deletions from the list.

Nonresponse error. Most FFRDCs have incorporated the data needed to complete most of the survey questions into their record-keeping systems. Twelve FFRDCs chose not to complete Question 5 of the survey, which asks for expenditures by type of cost. Eleven of those FFRDCs are managed by private companies for whom salary information is considered proprietary. One FFRDC, which is managed by a university, could not separate its expenditures by type from those of the managing institution. Other FFRDCs did not answer all sections of Question 5: five FFRDCs could not provide information on software expenditures, and four could not provide data on equipment expenditures. One FFRDC did not report its operating budget (Question 6). One FFRDC could not provide expenditures funded by nonprofit organizations separately from those funded by businesses (Question 1); the combined amount was reported as business-funded.

Measurement error. NCSES discovered during the FY 2011 survey cycle that seven FFRDCs were including capital project expenditures in the R&D totals reported on the survey. Corrections made for the FY 2011 survey cycle lowered total expenditures by $468 million. However, previous years still include an unknown amount of capital expenditures in the total. The amount is estimated to be less than $500 million per year.

Prior to the FY 2011 survey, the five FFRDCs administered by the MITRE Corporation had reported only internally funded R&D expenditures. After discussions with NCSES, these five FFRDCs agreed to report all FY 2011 operating expenditures for R&D and to revise their data for FYs 2008–10.

NCSES discovered during the FY 2013 survey cycle that Los Alamos National Laboratory (LANL) was reporting some expenditures that were not for R&D, as defined by this survey. Corrections made for the FY 2013 survey cycle lowered the laboratory’s total expenditures by $349 million. LANL also incorrectly was reporting that all expenditures were for basic research. In corrections made for FY 2013, LANL reported that $1,554 million (91%) of its total research expenditures was for applied research. LANL data from previous years still include an unknown amount of expenditures that were not for R&D and categorize all expenditures as basic research.

Prior to FY 2014, the Aerospace FFRDC reported only expenditures on internal R&D projects. After discussions with NCSES, the Aerospace Corporation agreed to report all R&D expenditures for FY 2014 and provide revised data to include all R&D expenditures for FYs 2010–13. R&D expenditures increased by more than $800 million each year.

During the FY 2014 survey, NCSES discovered that the National Optical Astronomy Observatory had been including data for the National Solar Observatory since FY 2010. The Association of Universities for Research in Astronomy, the administrator of both FFRDCs, provided revised data for both FFRDCs for FYs 2010–13.

During the FY 2016 survey, NCSES discovered that the Judiciary Engineering and Modernization Center was incorrectly classified as an industry-administered FFRDC in FYs 2011–15. This FFRDC is administered by the MITRE Corporation, a nonprofit organization, and should have been classified as a nonprofit-administered FFRDC. The classification was corrected for FY 2016, and the Judiciary Engineering and Modernization Center’s FYs 2011–15 data were reclassified as coming from a nonprofit-administered FFRDC.

Annual data are available for FYs 2001–19. When the review for consistency between each year’s data and submissions in prior years reveals discrepancies, it is sometimes necessary to modify prior year data. For accurate historical data, use only the most recently released data tables. Individuals wishing to analyze trends other than those in the most recent NCSES publication are encouraged to contact the Project Officer for more information about the comparability of data over time.

Changes in survey coverage and population. Most years, there are some changes to the FFRDC population that may affect trend analyses. FFRDCs have been created, decertified, renamed, or restructured, as described below:

• On 20 December 2006, the National Biodefense Analysis and Countermeasure Center was created.
• Prior to FY 2009, the Center for Enterprise Modernization was listed as the Internal Revenue Service FFRDC.
• On 5 March 2009, the Homeland Security Studies and Analysis Institute and the Homeland Security Systems Engineering and Development Institute were created. These new FFRDCs replaced the Homeland Security Institute.
• On 1 October 2009, the National Solar Observatory split from the National Optical Astronomy Observatory, with both retaining their FFRDC status.
• On 2 September 2010, the Judiciary Engineering and Modernization Center was created.
• Prior to FY 2011, the National Security Engineering Center was listed as C3I FFRDC.
• On 1 October 2011, the National Astronomy and Ionosphere Center was decertified as an FFRDC.
• Prior to FY 2012, the Frederick National Laboratory for Cancer Research was listed as the National Cancer Institute at Frederick.
• On 27 September 2012, the Centers for Medicare and Medicaid Services FFRDC was created. On 15 August 2013, its name was changed to the CMS Alliance to Modernize Healthcare.
• Prior to FY 2013, the Systems and Analyses Center was listed as the Studies and Analyses Center.
• On 19 September 2014, the National Cybersecurity Center of Excellence was created.
• On 15 September 2016, the Homeland Security Operational Analysis Center was created.
• The Homeland Security Studies and Analysis Institute was phased out on 31 October 2016.
• In June 2020, the National Optical Astronomy Observatory changed its name to NSF’s National Optical-Infrared Astronomy Research Laboratory or NSF’s NOIRLab.

Changes in questionnaire. FFRDCs are asked to provide R&D expenditures by source of funding and type of R&D. In FY 2010, NCSES revised the survey to include three new questions requesting expenditures funded by the American Recovery and Reinvestment Act of 2009 (ARRA), expenditures by type of cost, and total operating budget. In FY 2015, NCSES revised the survey to exclude the question requesting expenditures funded by ARRA. In FY 2016, NCSES added a question (Question 2) asking for R&D expenditures funded by seven specific federal agencies. In FY 2019, NCSES added a question (Question 3) asking which federal agencies funded the expenditures reported under Other federal agencies in Question 2.

Changes in reporting procedures or classification. The FFRDC R&D Survey has been conducted annually for university-administered FFRDCs since FY 1953 and for all FFRDCs since FY 2001.

• Expenditures by federal agency. In Question 2, FFRDCs were asked for the amount of R&D expenditures by federal funding agency. FFRDCs were asked to report expenditures funded by seven specific agencies (Department of Defense; Department of Energy; Department of Health and Human Services, including the National Institutes of Health; Department of Homeland Security; Department of Transportation; National Aeronautics and Space Administration; and National Science Foundation). Any expenditures funded by other federal agencies were reported under Other federal agencies. In Question 3, FFRDCs were asked to list the specific agencies and corresponding expenditures included in Other federal agencies.
• Expenditures by source. In Question 1, FFRDCs were asked to report their total R&D expenditures by funding source, as defined below:
• U.S. federal government. Any agency of the U.S. government. Federal funds that were passed through to the reporting institution from another institution were included.
• State and local government. Any state, county, municipality, or other local government entity in the United States, including state health agencies.
• Business. Domestic or foreign for-profit organizations. Funds from a company’s nonprofit foundation were not reported here; they were reported under Nonprofit organizations.
• Nonprofit organizations. Domestic or foreign nonprofit foundations and organizations.
• All other sources. Sources not reported in other categories, such as funds from foreign governments.
• Expenditures by type of cost. In Question 5, FFRDCs were asked for expenditures by type of cost, as defined below:
• Salaries, wages, and fringe benefits. Included compensation for all R&D personnel, whether full time or part time, temporary or permanent, including salaries, wages, and fringe benefits paid from institution funds and from external support.
• Software purchases. Included payments for all software, both purchases of software packages and license fees for systems.
• Equipment. Included payments for movable equipment, including ancillary costs such as delivery and setup.
• Subcontracts. Payments to subcontractors or subrecipients for services on R&D projects.
• Other direct costs. Other costs that did not fit into one of the above categories, including (but not limited to) travel, computer usage fees, and supplies.
• Indirect costs. Included all indirect costs (overhead) associated with R&D projects.
• Expenditures by type of R&D. In Question 4, FFRDCs were asked for the amount of federal and nonfederal R&D expenditures by type of R&D, as defined below:
• Basic research. Experimental or theoretical work undertaken primarily to acquire new knowledge of the underlying foundations of phenomena and observable facts, without any particular application or use in view.
• Applied research. Original investigation undertaken to acquire new knowledge. It is directed primarily toward a specific, practical aim or objective.
• Experimental development. Systematic work, drawing on knowledge gained from research and practical experience and producing additional knowledge, which is directed to producing new products or processes or to improving existing products or processes.
• Fiscal year. FFRDCs were asked to report data for their fiscal year (or financial year).
• R&D. The FFRDC R&D Survey requested data from FFRDCs on their R&D, defined as systematic study directed toward fuller knowledge or understanding of the subject studied. R&D included basic research, applied research, and experimental development (see Expenditures by type of R&D above for additional information). R&D did not include outreach or nonresearch training programs. Respondents were also asked to exclude capital projects (i.e., construction or renovation of research facilities) from reported expenditures.
• R&D expenditures. FFRDCs were asked to report all current operating expenditures for activities specifically organized to produce R&D outcomes, including those funded by external sponsors or separately budgeted and accounted for by the organization using internal funds. Expenditures included indirect costs, equipment, software, clinical trials, and subcontract expenditures.
• Total operating budget. Total executed operating budget for the FFRDC excluding capital construction costs.