Purpose. The Federal Facilities Research and Development (FFRD) Survey collects information on research and experimental development (R&D) expenditures, funding, and personnel for all federally owned and operated facilities in the United States. This new survey is the unique source of federal performer-reported data on R&D expenditures and R&D personnel. In conjunction with performer-reported data for the remaining sectors of the economy, FFRD Survey data can be used to estimate total national R&D performance and other critical statistics that are increasingly important in the analysis of global R&D competitiveness.

Data collection authority. The information is solicited under the authority of the National Science Foundation Act of 1950, as amended, and the America COMPETES Reauthorization Act of 2010. The disclosure review number is NCSES-DRN24-064.

Survey contractor. ICF.

Survey sponsor. The FFRD Survey is sponsored by the National Center for Science and Engineering Statistics (NCSES) within the U.S. National Science Foundation.

Frequency. Annual.

Initial survey year. A pilot survey that collected FY 2021 data was conducted from September 2022 through December 2022, and then a full implementation of the survey to collect FY 2022 data was conducted from September 2023 through March 2024.

Reference period. FY 2022.

Response unit. Establishment.

Sample or census. Census.

Population size. The population consists of 470 research-performing federal facilities within 39 federal agencies.

Sample size. Not applicable; the survey is a census of all known eligible federal facilities that conduct R&D.

Target population. The population for the FY 2022 FFRD Survey consisted of federally owned and operated facilities in the United States that performed R&D in FY 2022, excluding facilities of the Central Intelligence Agency (due to the classified nature of their work); federally funded research and development centers (surveyed separately in the Federally Funded Research and Development Centers R&D Survey); and University Affiliated Research Centers or facilities associated with universities, both of which may be surveyed as part of the Higher Education Research and Development Survey.

Sampling frame. The facilities for the FY 2022 FFRD Survey were identified from the list of federal agencies with R&D obligations based on the Survey of Federal Funds for R&D and the Federal Laboratory Consortium list of laboratories. A facility is defined as a unit within an agency that is responsible for performing R&D, generally with its own distinct budget and leadership. Since each agency has a different organizational structure, this unit of measurement may be a division, branch, center, lab, or other entity and may span multiple locations. Because of this, the total number of reporting units for the FY 2022 FFRD Survey is 320, which represents the 470 research-performing federal facilities.

Sample design. Not applicable.

Data collection. Data collection began with an e-mail to each agency to verify the reporting units and to identify the name, phone number, and e-mail address of each facility-level respondent. Table A-1 displays a list of facilities within each agency and notes which are reporting units for the FY 2022 FFRD Survey. When the reporting units are totaled, there are 320 total units across 39 agencies. There were 12 agency-level responses and 288 responses at a level lower than the agency-level for a total of 299 responses. The response total is one less than the number of units because for the Center for Veterinary Biologics is included with the National Veterinary Services Laboratories for this survey cycle only.

Respondents could choose to submit a questionnaire downloaded from the Web or sent via e-mail (i.e., PDF or Excel format) or use a Web-based data collection system to respond to the survey. Questionnaires were carefully examined for completeness upon receipt. Respondents were sent personalized e-mail messages asking them to provide any necessary revisions before the final processing and tabulation of data. These e-mail messages included a link to the FFRD Survey Web-based collection system, allowing respondents to view and correct their data online. Respondents were also offered the opportunity to explain their data or provide corrections via e-mail, through PDF, or through Excel and have their data updated by survey staff.

Data collection began on 2023 September 12, and the requested due date for data submissions was 2023 December 31. Data collection was extended until most surveyed agencies provided at least partial data with the last completed survey response verified in March 2024.

Mode. Respondents could choose to submit a questionnaire via the Web-based data collection system or through fillable PDFs or Excel files that were then imported to the Web-based data collection system. All edit and trend checks were accomplished through the Web-based system. Fifty-six facilities submitted data using the fillable PDF, 194 facilities submitted using an Excel version of the survey, and 49 submitted using the Web-based data collection system.

Response rates. By the survey’s closing date in March 2024, forms had been received from 299 facilities out of an eligible population of 319, a response rate of 93.7%. Table A-2 displays a detailed breakdown of response rates by the Department of Defense (DOD) and non-DOD agencies, and table A-3 displays a breakdown of response rates for each survey question. Questions 4, 7, 8, 9, and 10 had much lower response rates for non-DOD agencies than for DOD agencies. Question 4 was a yes or no question on whether the facility’s FY 2022 R&D was funded through public-private partnerships. Questions 7 and 8 asked facilities to report how much funding was provided by type of agreement and type of organization. Questions 9 and 10 asked facilities to report headcounts for R&D personnel by function and job category (Question 9) and federal full-time equivalents (FTEs) by function (Question 10). Due to low item response rates for the questions on funding to others and R&D personnel counts, limited data are available. Data on R&D funding to others are not included in the data tables for this cycle. R&D personnel counts (table 6 and table 7) are only displayed for the facilities that provided these data; no imputed or aggregate estimates are provided for FY 2022.

Data editing. Data inconsistencies or blank fields in the FFRD Survey were flagged automatically by the Web-based data collection system so that respondents could not submit their final data until all required fields were completed without errors. Respondents were contacted and asked to resolve possible self-reporting issues themselves. Questionnaires were carefully examined by survey staff upon receipt. Reviews focused on unexplained missing data, expenditures that significantly differed from the Federal Funds for R&D Survey intramural obligations, expenditures for performing research and funding research that matched, and other data anomalies. If additional explanations or data revisions were needed, respondents were sent personalized e-mail messages asking them to provide any necessary revisions before the final processing and tabulation of data. For any follow-up questions that went unanswered, NCSES was consulted before the data were either accepted without changes or adjusted based on information from other questions or previous contacts with the respondent.

Some agencies provided data outside of survey forms and provided partial data that the survey team added to the database with their approval:

• National Aeronautics and Space Administration (NASA) reported R&D in the survey instrument but then followed up with a report with guidance to update their responses. NASA also provided a report with total civilian FTEs by space center.
• The Federal Highway Administration provided total R&D in the survey but followed up with reports of R&D intramural obligations, which provided the distribution of R&D type and by federal and nonfederal sources.
• The Department of Veterans Affairs provided responses for direct R&D expenditures within the survey form but provided external information for the associated indirect costs (Veterans Equitable Resource Allocation) by facility, which were added proportionally to the type of R&D for each facility.

Responses to Questions 7 and 8 (R&D funding organizations and types and sources) were used to perform a data review of responses to questions in Section 1 (R&D expenditures), and some data edits were made as a result of responses to Questions 7 and 8.

Imputation. Instances of missing data occurred when a facility did not respond to the survey (i.e., unit nonresponse) or when a facility responded but did not answer certain survey questions (i.e., item nonresponse). After removing ineligible facilities, 319 total records remained, with 299 responses and 20 unit nonresponses. Among responding facilities, total R&D expenditures was completely observed (from Questions 1, 2, or 5 totals). The pattern of missing data showed that 37% of responding facilities had complete data. Another 26% were missing only FTE items. Sixteen percent of responding facilities had data for FTE, but were missing personnel headcount data, and 15% were missing both headcount and FTE data. Multivariate imputation by chained equations (MICE) was used to simultaneously impute missing values for all non-personnel (headcount and FTE) variables. The imputation model for total R&D expenditures was based on a linear regression model with the total R&D obligations from the Federal Funds for R&D Survey as a predictor for each facility. The model included separate intercepts and slopes for DOD and non-DOD facilities. When facility-level obligations were not available from the Federal Funds for R&D Survey, facilities responding to the FFRD Survey were aggregated to the level of reporting for the Federal Funds for R&D Survey, referred to below as reporting level found in the Federal Funds for R&D Survey. The imputed amounts were then assigned equally to the FFRD Survey facilities within the same reporting level found in the Federal Funds for R&D Survey. These imputed records display as equal amounts in the tables. This occurs for two Army, two Navy, and six Smithsonian records.

Total R&D expenditures was then allocated to the subcategories as follows:

• Type of R&D was imputed for nonresponding facilities based on (1) the percent distribution of R&D expenditures by R&D type based on responding facilities aggregated within the same reporting level found in the Federal Funds for R&D Survey, (2) the percent distribution of R&D type expenditures based on responding facilities aggregated to the agency level, (3) the percent distribution of obligations by R&D type as reported in the Federal Funds for R&D Survey at the reporting level found in the Federal Funds for R&D Survey, and (4) the percent distribution of obligations by R&D type as reported in the Federal Funds for R&D Survey aggregated to the agency level. For DOD reporting units, when some facilities within the agency responded (partial agency response), total research, development, testing, and evaluation (RDT&E) and operational system development for nonresponding DOD facilities were imputed based on a ratio adjustment using the ratio of RDT&E to R&D at the agency level and applying it to the imputed R&D total for the facility. When no facilities in the agency responded, the ratio adjustment was based on the Federal Funds for R&D Survey ratio at the agency level. All imputed R&D amounts were assigned as federal R&D within each type. 
• Source of R&D funding was imputed as federal, based on 98% of the reported total expenditures coming from federal sources (99% of basic research, 97% of applied research, 99% of experimental development, and 93% of operational system development). The federal source of funding was imputed to the agency where the facility resides, based on 99% of the federal sources originating in the parent agency of the facility.
• Field of R&D was imputed for nonresponding facilities based on (1) the percent distribution of R&D field expenditures based on responding facilities aggregated within the same reporting level found in the Federal Funds for R&D Survey, (2) the percent distribution of R&D field expenditures based on responding facilities aggregated to the agency level, (3) the percent distribution of obligations by R&D field reported in the Federal Funds for R&D Survey at the reporting level found in the Federal Funds for R&D Survey, and (4) the percent distribution of obligations by R&D field reported in the Federal Funds for R&D Survey aggregated to the agency level.

Weighting. Survey data were not weighted.

Sampling error. Because the FY 2022 survey was distributed to all eligible agencies performing R&D, there was no sampling error.

Coverage error. Under the total survey error framework, coverage error describes the difference between reporting units and units in the target population that the frame was developed to reach. The subset of agencies is based on those reporting non-zero intramural R&D obligations on the Federal Funds for R&D Survey. There is a small risk of coverage bias if an agency had federal R&D expenditures but did not have reported obligations.

Nonresponse error. Nonresponse error refers to the differences in key estimates between units (i.e., facilities) in the frame that were selected for data collection and those that responded. Of the 319 eligible reporting units, 299 responded for a response rate of 93.7%. For unit nonresponse, multiple follow-ups were conducted with nonresponding facilities, and multiple contact and data collection modes were used (i.e., phone and e-mail) to mitigate nonresponse error. The imputations include predictions of unit nonrespondents to reduce the risk of nonresponse bias in the final estimates.

The item response rate ranged from 98.2% to 100% for the DOD agencies and from 50.0% to 100% for the non-DOD agencies. For item nonresponse, facilities were encouraged to report estimates of expenditures when actual dollar amounts could not be provided. This approach reduces item nonresponse error risk but may introduce measurement error. Imputation was conducted to help mitigate item nonresponse error.

Measurement error. The FFRD Survey contained various ways for respondents to explain their answers. This included comment boxes for every question with instructions to provide additional information about how they calculated their response and any assumptions they made. Respondents were also prompted to enter comments if they indicated any data was unavailable. Additionally, every question requiring detailed reporting of expenditures or personnel also included a checkbox to allow the respondent to indicate if they were providing estimates. Finally, there were open-ended questions at the end of the survey where respondents could provide additional feedback on their experience completing the survey, specifically, if they found any questions difficult to answer and why. The largest risk of measurement error is likely respondents’ interpretation of the definition of R&D activities and variations in record-keeping procedures used by respondents to answer the survey questions. In addition, other known measurement problems are known to exist in the data, as shown below.

• R&D expenditures. The survey included a question asking respondents to confirm if they were reporting expenditures or obligations. Not all facilities were able to report expenditure data and instead reported obligations. Twelve facilities confirmed they reported obligations as they were unable to provide expenditure data, and six facilities never confirmed if they provided expenditures or obligations.
• Type of R&D. Some facilities said that these amounts were difficult to report because either they could not determine how best to allocate expenditures among the three categories or because they did not track information in that way.
• Federal and military salaries. Due to conversations with a few facilities during data collection, the study team discovered early in data collection that R&D employee salaries, especially military personnel salaries, were not always considered when calculating R&D expenditures, due to being in different budget categories. Therefore, the study team attempted to confirm with all respondents following submission if both federal civilian and military salaries were included in total R&D expenditures. Eight respondents said this information was not included. DOD facilities often noted that although civilian salaries were automatically included in R&D budgets, military salaries were in a different account and needed to be manually included, which some were not able to do during this first year of data collection.

Facility. A unit within an agency that is responsible for performing R&D, generally with its own distinct budget and leadership. This may be a division, branch, center, lab, or other entity. The staff who work within the facility, and the facility itself, may be located in more than one physical location.

Fields of R&D. A list of the 56 fields of R&D reported on can be found on the survey questionnaire. In the data tables, the fields are grouped into 10 major areas: agricultural sciences and natural resources and conservation; biological, biomedical, and health sciences; computer and information sciences; geosciences, atmospheric sciences and ocean sciences; mathematics and statistics; physical sciences; psychology; social sciences; engineering; and other fields.

Fiscal year. The federal government’s financial year: FY 2022 began 2021 October 1 and ended 2022 September 30.

Full-time equivalents (FTEs). Calculated as the total working effort spent on research during a specific period divided by the total effort representing a full-time schedule within the same period. FTE R&D personnel are federal employees and military personnel only.

Public-private partnerships. Those in which the government and private companies share R&D costs.

Research and development (R&D). R&D is creative and systematic work undertaken in order to increase the stock of knowledge—including knowledge of humankind, culture, and society—and to devise new applications of available knowledge. R&D has five major features:

• Novel. Advances current knowledge or creates new knowledge
• Creative. Focuses on original concepts and hypotheses
• Uncertain. Outcomes are not completely determined at the outset of a project
• Systematic. Projects are planned and budgeted
• Transferable or reproducible. Methodology and results are transferable to or reproducible in other situations and locations

R&D covers three activities: basic research, applied research, and experimental development.

Research, development, testing, and evaluation (RDT&E). Exclusive to DOD, this consists of all the activities described above for R&D conduct plus operational system development (Budget Activity 7)—preproduction development of nonexperimental work on a product or system before it goes into full production, including activities such as tooling and development of production facilities.

R&D expenditures. Money the facility spent in FY 2022 for R&D projects, also referred to as outlays. Expenditures include labor costs for R&D projects; noncapital purchases of materials, supplies, equipment, and services to support R&D performance; and general administration costs in support of R&D activities.

Sources of R&D funding/type of organization:

• U.S. federal government. Any agency of the U.S. government. Federal funds that were passed through from another institution were included.
• State and local government. Any state, county, municipality, or other local government entity in the United States, not including funds from state and local universities and colleges or agricultural experiment stations.
• Business. Domestic or foreign for-profit businesses or industrial firms.
• Nonprofit organizations. Domestic or foreign nonprofit foundations and organizations, except universities and colleges.
• Higher education. Domestic higher education institutions, military service academies, and consortia.
• All other organizations. Sources not reported in other categories, such as funds from foreign governments.

Type of R&D:

• Basic research. Experimental or theoretical work undertaken primarily to acquire new knowledge of the underlying foundations of phenomena and observable facts, without any particular application or use in view.
• Applied research. Original investigation undertaken in order to acquire new knowledge. It is directed primarily toward a specific, practical aim or objective.
• Experimental development. Systematic work, drawing on knowledge gained from research and practical experience and producing additional knowledge, which is directed to producing new products or processes or to improving existing products or processes. Like research, experimental development will result in gaining additional knowledge.

Experimental development includes the following:

• The production of materials, devices, and systems or methods, including the design, construction, and testing of experimental prototypes.
• Technology demonstrations, in cases where a system or component is being demonstrated at scale for the first time, and additional refinements to the design (feedback R&D) following the demonstration. However, not all activities that are identified as “technology demonstrations” are R&D.

Experimental development does not include the following:

• User demonstrations where the cost and benefits of a system are being validated for a specific use case. This includes low-rate initial production activities.
• Preproduction development, which is defined as nonexperimental work on a product or system before it goes into full production, including activities such as tooling and development of production facilities. Activities and programs of this type should generally be reported as investments in other major equipment.

R&D personnel. All employees who work on R&D or provide direct support to R&D, such as researchers, R&D managers, technicians, support staff, and others assigned to R&D groups or projects. Personnel may include federal employees, military personnel (civilian and enlisted), contractors, consultants, or volunteers.

• R&D personnel include all R&D personnel, whether full time or part time, temporary or permanent. Employees may perform scientific and technical work for an R&D project (e.g., designing experiments, building prototypes), plan and manage R&D projects, or provide direct support for administration of the financial and personnel aspects of R&D.
• R&D personnel does not include employees who provide only indirect support to R&D, such as services provided by personnel in central finance, computing, printing, maintenance, security, or similar departments in your agency that provide services to R&D and non-R&D projects.

Types of R&D personnel:

• Researchers. Professionals engaged in the conception or creation of new knowledge, products, processes, methods, and systems and also in the management of the projects concerned. Includes R&D managers in this category.
• R&D technicians. Persons whose main tasks require technical knowledge and experience in one or more fields of science or engineering, but who contribute to R&D by performing technical tasks such as computer programming, data analysis, ensuring accurate testing, operating lab equipment, and preparing and processing samples under the supervision of researchers.
• R&D support staff. Not directly involved with the conduct of a research project but support the researchers and technicians. These employees might include clerical staff, financial and personnel administrators, report writers, patent agents, safety trainers, equipment specialists, and other related employees.