Statistical Definition of Development Clarified: Effect on Reported Federal R&D Totals

NSF 21-326

 | 

April 16, 2021

In recent years, several efforts have taken place to better harmonize the statistical definition of federal spending on research and development. There are two main sources of official information on federal government R&D funding, namely, the “Research and Development” chapter in the Analytical Perspectives report accompanying the president’s annual proposal for the Budget of the United States Government (often identified as “the President’s Budget”), issued by the Office of Management and Budget (OMB), and the Survey of Federal Funds for Research and Development from the National Center for Science and Engineering Statistics (NCSES). These data sources are definitionally consistent with each other, and each is definitionally consistent with international guidelines for reporting R&D funding and performance, per the Organisation for Economic Co-operation and Development (OECD) Frascati Manual. While the information in the President’s Budget is presented as fiscal year budget authority, the NCSES statistics are presented as fiscal year obligations. However, in both data sets, a substantial decline in R&D is evident, beginning with FY 2017 budget authority per OMB and with FY 2016 obligations statistics per NCSES. Overall, this decline is not the result of a change in R&D policy per se but, rather, a refinement in the definition of R&D. This InfoBrief discusses this definitional change and provides context on the effect of this change on federal R&D statistics.

Refinement in the Definition of Development

Each year OMB issues an update to Circular A-11 Preparation, Submission, and Execution of the Budget, providing detailed guidance to federal agencies on the budget formulation process. Following the release of the seventh edition of the OECD Frascati Manual in October 2015, which clarified the definition of “development” for the purposes of statistical data collection, OMB updated the definition of R&D for federal agencies in the July 2016 release of Circular A-11 for the FY 2018 budget formulation process. As part of this update, OMB added additional guidance for agencies to differentiate and exclude “preproduction development” from experimental development, citing the pre-production category of the Department of Defense (DOD) Research Development Test and Evaluation (RDT&E)—specifically, Budget Activity 7 for operational system development (OSD):

Pre-production development, which is defined as non-experimental work on a product or system before it goes into full production, including activities such as tooling and development of production facilities. For example, exclude activities and programs that are categorized as “Operational Systems Development” in DOD’s budget activity structure. Activities and programs of this type should generally be reported as investments in other major equipment.

Pre-production development, or OSD for DOD, activities that were previously reported in R&D as development activities were now excluded under the revised guidance for experimental development. Although the Frascati Manual always referred to the “D” in R&D as experimental development, the issue of pre-production development was made more explicit with the 2015 edition of the manual, effectively moving the topic from what was an annex to the 2002 edition directly into the definition and exclusions presented in the 2015 manual.

Effect on Federal R&D Budget Authority Data

The effect of this refinement in the definition of experimental development can be seen in the R&D budget authority data initially released with the FY 2018 President’s Budget in the Analytical Perspectives report’s chapter on R&D. These exclusions were updated and clarified with the release of the FY 2019 President’s Budget as actual budget authority, particularly the difference in total R&D budget authority between the previous and revised definitions (table 1). OMB specifically noted that

unlike previous years, totals for Experimental Development spending in FY 2017–2019 do not include the DOD Budget Activity 07 (Operational System Development) due to changes in the definition of development. These funds are requested in the FY 2019 Budget request and support the development efforts to upgrade systems that have been fielded or have received approval for full rate production and anticipate production funding in the current or subsequent fiscal year.

OMB further differentiates this change from the previous definition by providing the total R&D budget authority under both definitions. If the pre-production development activities were to be included, the FY 2017 R&D budget authority would have been $155.0 billion instead of the $125.3 billion in actual budget authority (table 1).

Federal R&D spending by mandatory and discretionary budget authority, by agency: FYs 2017–19

(Millions of current dollars and percent change)

CR = continuing resolution.

a The totals for experimental development spending in FYs 2017–19 do not include Department of Defense Budget Activity 7 (operational system development) due to changes in the definition of development. These funds are requested in the FY 2019 Budget request and support the development efforts to upgrade systems that have been fielded or have received approval for full-rate production and anticipate production funding in the current or subsequent fiscal year.
b The total uses the new experimental development definition across the three fiscal years.

Note(s):

This table shows funding levels for departments or independent agencies with more than $200 million in R&D activities in 2019.

Source(s):

Budget of the United States Government, Fiscal Year 2019, Analytical Perspectives, Chapter 18: Research and Development, https://www.govinfo.gov/app/collection/budget/2019/BUDGET-2019-PER.

The R&D budget authority time series illustrates the effect of the OSD data on total federal R&D and R&D plant and the defense R&D and R&D plant. Both federal-wide R&D budget authority and the DOD R&D budget authority show the impact of the OSD amounts over the past 20 years (figure 1).

Keyboard instructions

Total federal and defense R&D and R&D plant budget authority with revised total federal and defense R&D and R&D plant, FYs 2000–20

(Millions of current dollars)
Fiscal year Total federal R&D and R&D plant Total defense R&D and R&D plant Revised total federal R&D and R&D plant Revised total defense R&D and R&D plant
2000 83,581 43,161 72,681 32,261
2001 91,505 46,202 78,538 33,235
2002 103,057 53,731 88,665 39,339
2003 118,065 63,708 98,767 44,410
2004 126,270 70,344 106,246 50,320
2005 131,259 74,641 111,332 54,714
2006 136,019 78,737 114,442 57,160
2007 141,890 82,658 117,691 58,459
2008 144,391 85,129 116,710 57,448
2009 164,292 85,642 136,599 57,949
2010 148,962 86,789 119,382 57,209
2011 144,379 83,226 113,386 52,233
2012 143,737 79,875 116,796 52,934
2013 132,477 70,781 109,608 47,912
2014 136,159 70,992 112,502 47,335
2015 138,544 72,950 115,220 49,626
2016 150,392 80,202 126,093 55,903
2017 na na 127,306 57,530
2018 na na 144,459 68,916
2019 na na 149,971 72,027
2020 na na 164,453 80,923
Note(s):

Because of rounding, detail may not add to total. Revised totals reflect the Office of Management and Budget revision to the definition of R&D for federal agencies in the July 2016 release of Circular A-11.

Source(s):

National Center for Science and Engineering Statistics, Federal R&D Funding, by Budget Function.

Although OMB Circular A-11 specifically cites the exclusion of DOD Budget Activity 7 (OSD) funds from R&D, DOD has also excluded Budget Activity 6 (RDT&E management support) funds from its reports to OMB since FY 2018. However, the effect of the Budget Activity 6 funds on the total R&D were $6.8 billion in the FY 2020 proposed budget authority compared to the Budget Activity 7 funds of $38.7 billion.

Effect on Federal R&D Obligations Statistics

For an agency to incur an obligation for R&D, that agency should have been granted R&D budget authority. Therefore, the NCSES Survey of Federal Funds for Research and Development is tied to the universe of agencies presented in the President’s Budget. OMB’s Circular A-11 highlights this connection not just in terms of the agencies themselves but also with regard to each agency’s R&D portfolio. Thus, following the release of the July 2016 update to Circular A-11, NCSES revised the Survey of Federal Funds for Research and Development FYs 2016–17 (volume 66) reporting instructions to exclude OSD from experimental development. This resulted in a noticeable change in the obligations for R&D reported by the NCSES survey between FY 2015 and FY 2016 (table 2).

Federal obligations for R&D, by type, and for operational system development: FYs 2015–18

(Millions of current dollars and percent change)

na = not applicable.

Note(s):

Because of rounding, detail may not add to total. Percentages are computed using actual dollars reported.

Source(s):

National Center for Science and Engineering Statistics, Survey of Federal Funds for Research and Development.

Pre-Production Development Data

Although the removal of pre-production development most clearly impacted DOD’s development data, NCSES created an additional series to present both R&D and the DOD RDT&E, which includes the DOD OSD data and provides some comparability with the previous development time series. The R&D data reported by the National Aeronautics and Space Administration (NASA) were impacted as well. NASA noted that approximately $2 billion in its development obligations was no longer included in its R&D reporting effective with FY 2016 survey results. The Department of Energy (DOE) was the only other agency that would have had some possible OSD-type pre-production development obligations. However, DOE staff noted they had never included these data in their reports to NCSES. No other agencies had any R&D activities resembling the DOD OSD category.

Defense S&T, R&D, and RDT&E

Defense-related R&D activities are typically ordered into three categories, namely, science and technology (S&T) programs, R&D, and RDT&E (figure 2). To differentiate between the part of the federal R&D budget that supports science and key enabling technologies (including technologies for military and nondefense applications) and the part that primarily supports testing and evaluation (mostly of defense-related weapons systems), DOD since 1994 has reported its development obligations into two separate development categories, namely, advanced technology development (Budget Activity 3) and major systems development (Budget Activities 4–7). Thus, DOD defined its S&T budget as one that comprises basic research (Budget Activity 1), applied research (Budget Activity 2), and advanced technology development (Budget Activity 3). DOD has suggested most other federal agencies’ entire R&D programs are equivalent in nature to DOD’s S&T program. Excluding the OSD data from the R&D calculus had no impact on S&T totals, and Budget Activity 7 is not part of the DOD S&T programs. However, the exclusion of OSD did result in changes to DOD’s major system development category—and, thus, to what is reported in the various DOD R&D and RDT&E totals.

Department of Defense research, development, test, and evaluation categorization in National Center for Science and Engineering Statistics data

(Categories)

DOD = Department of Defense. NCSES = National Center for Science and Engineering Statistics. RDT&E = research, development, test, and evaluation. S&T = science and technology.

Source(s):

National Center for Science and Engineering Statistics.

The consequence of the 2016 Circular A-11 revisions is that R&D effectively is defined as a subset of RDT&E. NCSES still collects two categories for DOD’s development; beginning with volume 66 of the Survey of Federal Funds for Research and Development, however, the explicit exclusion of pre-production development resulted in changes to major systems development, only covering Budget Activities 4–6 (advanced component development and prototypes, system development and demonstration, and RDT&E management support, respectively). DOD’s Budget Activity 7 (OSD) is now excluded from DOD’s R&D totals. Because NCSES still continues to collect the DOD Budget Activity 7 obligations separately, though, its total RDT&E obligations are available as part of the survey’s data set. Of the $24.6 billion in OSD obligations in FY 2016, 72% ($17.6 billion) were reported by the Department of the Air Force. OSD obligations increased to nearly $30.8 billion in FY 2018, and the Air Force still accounted for 68% of the total, but the Department of the Navy’s share of the OSD has increased from 5.3% in FY 2016 to 8.3% in FY 2018 (table 3). Further, of the $30.8 billion in FY 2018 total OSD, $22.9 billion (75%) was obligated to businesses, $7.0 billion (23%) to DOD intramural performers, and $826.0 million (2%) to all other performers, including federally funded research and development centers, higher education institutions, nonprofit institutions, and foreign performers.

Although OMB reported totals for R&D exclude DOD Budget Activity 6 amounts, NCSES continues to report Budget Activity 6 as part of experimental development because it includes the administrative and management support costs for DOD’s S&T, R&D, and RDT&E. The OECD standard for the collection of R&D maintains that administrative and support costs to manage R&D programs are part of the cost to perform R&D and should be included. In addition, DOD’s funding for the Small Business Innovation Research and Small Business Technology Transfer programs originate from category Budget Activity 6 and these programs are within the scope of R&D.

Department of Defense obligations for research, development, test, and evaluation, by agency: 2015–18

(Millions of current dollars)

NA = not available.

RDT&E = research, development, test, and evaluation.

a Department of Defense (DOD) development obligations have been reported in two categories, advanced technology and major systems, since volume 44 (FYs 1994–96). As of volume 66 (FYs 2016–17), the definition of major systems development was changed to represent DOD Budget Activities 4–6 instead of Budget Activities 4–7.
b Funding for DOD's Budget Activity 7 (operational system development) was first reported as a separate category for volume 66.          

Note(s):

Because of rounding, detail may not add to total.          

Source(s):

National Center for Science and Engineering Statistics, Survey of Federal Funds for Research and Development.

Data Sources and Limitations

NCSES provides comprehensive definitions of research and development from U.S. and international sources. For additional information regarding the definition of R&D, see Definitions of Research and Development: An Annotated Compilation of Official Sources.

The Analytical Perspectives of the President’s Budget contain analyses highlighting specific subject areas or other important aspects of the budget with additional context. Data shown in the Analytical Perspectives report’s chapter on R&D are collected directly by OMB under a specific Budget Data Request for R&D during the budget formulation process. The federal budget does not include a separately identified R&D account. Furthermore, most appropriations for R&D are not directly labeled as such (except in certain program areas, such as defense, energy, health, and environment). Thus, most funds for R&D are not line items in agency budget submissions but are included instead as part of general program funding.

To provide information on federal R&D funding, OMB requires all agencies with R&D funding levels greater than $10 million annually to submit data on their R&D programs as part of their annual budget submissions. Such agencies are requested to provide data on their funding levels for basic research, applied research, experimental development, R&D facilities, and capital equipment for R&D, in accordance with OMB’s Circular A-11, Section 84: “Character Classification (Schedule C).”

Notes

1The most recent international standard definitions and guidelines for the collection of R&D statistics can be found in Organisation for Economic Co-operation and Development (OECD). 2015. Frascati Manual 2015: Guidelines for Collecting and Reporting Data on Research and Experimental Development. The Measurement of Scientific, Technological and Innovation Activities. Paris: OECD Publishing. R&D comprises basic research, applied research, and experimental development. These definitions reflect the international consensus of statistical organizations that collect R&D data in OECD member countries. These categories and their key features have not fundamentally changed over the years, but the associated descriptive texts have on occasion been updated and refined to better clarify the intended content of each of these types of R&D. See https://www.oecd.org/publications/frascati-manual-2015-9789264239012-en.htm.

2Budget authority means the authority provided by law to incur financial obligations that will results in outlays (see OMB Circular A-11, Section 20.4). Thus, budget authority sets a limit on new obligations an agency may incur each fiscal year. See Office of Management and Budget (OMB), Executive Office of the President. 2016. Circular A-11: Preparation, Submission, and Execution of the Budget, Section 20-4, page 20-11. Washington, DC. https://obamawhitehouse.archives.gov/sites/default/files/omb/assets/a11_current_year/a11_2016.pdf.

3Obligations represent the amount for orders placed, contracts awarded, services received, and similar transactions during a given period, regardless of when the funds were appropriated or when future payment of money is required. Obligations include funds from direct appropriations, trust funds, special accounts, fees and charges, and other federal sources for the year of the obligation. Obligations include the full cost of R&D, both specific project costs and overhead costs. Interagency transfers for R&D are reported by the transferring agency as R&D or R&D plant, not by the agency receiving the funds.

4The 2015 Frascati Manual defines experimental development as “systematic work, drawing on knowledge gained from research and practical experience and producing additional knowledge, which is directed to producing new products or processes or to improving existing products or processes.” Frascati Manual 2015: Guidelines for Collecting and Reporting Data on Research and Experimental Development. §1.35, page 29.

5The 2016 edition of OMB Circular A-11 is used to inform agencies on the budget formulation process for the FY 2018 proposal for the President’s Budget submission to the Congress, which was issued in 2017. OMB and the federal agencies begin work on the FY 2018 budget before FY 2016 has closed and before the full appropriations processes have been completed for FY 2017. Therefore, the FY 2017 budget authority in the 2018 proposed budget are still on the previous basis, and any revisions to the FY 2017 to account for definitional changes in experimental development do not present themselves until the issuance of the FY 2019 President’s Budget proposal.

6The DOD RDT&E budget is comprised of seven Budget Activity categories. For additional information, see Department of Defense, Under Secretary of Defense (Comptroller) (DOD/OUSD[C]). 2017. Research, Development, Test, and Evaluation Appropriations. In Department of Defense Financial Management Regulation, Volume 2B: “Budget Formulation and Presentation (Chapters 419), pages 5-1–5-24. DOD 7000.14-R. Washington, DC: DOD/OUSD(C). https://comptroller.defense.gov/Portals/45/documents/fmr/Volume_02b.pdf.

7OMB Circular A-11 (2016), Section 84.2(c), pages 84-2–84-3. https://obamawhitehouse.archives.gov/omb/circulars_a11_current_year_a11_toc.

8See Frascati Manual 2015: Guidelines for Collecting and Reporting Data on Research and Experimental Development (2015), §2.35 and §2.36. Specifically, Frascati Manual 2015 notes the importance of differentiating “experimental development” from more expansive product development (which includes commercialization) and from pre-production development, a term often used in large-scale government defense or aerospace projects, which includes non-experimental work on products or systems such as final design engineering, tooling and industrial engineering, and user demonstrations—and even low-rate initial production activities (§7.47).

9OMB. 2018. Analytical Perspectives, Budget of the United States Government, Fiscal Year 2019. Chapter 18, Research and Development, pages 233–41; Table 18-2, pages 238–40. https://www.govinfo.gov/content/pkg/BUDGET-2019-PER/pdf/BUDGET-2019-PER-7-5.pdf.

10Ibid.

11See Congressional Research Service (CRS). 2020. Department of Defense Research, Development, Test, and Evaluation (RDT&E): Appropriations Structure. CRS Report R44711. Washington, DC: CRS. https://fas.org/sgp/crs/natsec/R44711.pdf.

12National Center for Science and Engineering Statistics. 2019. Federal R&D Funding, by Budget Function: Fiscal Years 2018–20. NSF 20-305. Table 6. Alexandria, VA: National Science Foundation. https://ncses.nsf.gov/pubs/nsf20305/.

13See OMB Circular A-11, Schedule C, Section 84.3(g), page 84-7. https://obamawhitehouse.archives.gov/omb/circulars_a11_current_year_a11_toc.

14See National Center for Science and Engineering Statistics. 2018. Federal Funds for Research and Development, Fiscal Years 2016–17. Technical Notes. Alexandria, VA: National Science Foundation. https://ncsesdata.nsf.gov/fedfunds/2016/fedfunds_2016_tech_notes.pdf.

15See National Center for Science and Engineering Statistics. Federal Funds for Research and Development. Fiscal Years 2016–17, 2017–18, 2018–19. Tables 9 and 10. Alexandria, VA: National Science Foundation. https://nsf.gov/statistics/srvyfedfunds/.

16Ibid.

17As per the Survey of Federal Funds for Research and Development, most of DOD’s OSD funds were obligated to industry and intramural performers (see the Department of Defense obligations, by agency and performer, National Center for Science and Engineering Statistics, Federal Funds for Research and Development, 2018. Detailed Statistical Table 9. https://nsf.gov/statistics/srvyfedfunds/. Further, based on NCSES and U.S. Census Bureau interviews with defense contractors, as part of the Business Research and Development and Innovation Survey there is considerable evidence that private sector contractors (those receiving the DOD funding) do not consider a substantial share of OSD-funded activities to be R&D.

18See Frascati Manual: Guidelines for Collecting and Reporting Data on Research and Experimental Development (2015), §4.29 and §4.30.

19National Center for Science and Engineering Statistics. 2018. Definitions of Research and Development: An Annotated Compilation of Official Sources. Alexandria, VA: National Science Foundation. Available at https://www.nsf.gov/statistics/randdef/.

Suggested Citation

Pece C, Jankowski J; National Center for Science and Engineering Statistics (NCSES). 2021. Statistical Definition of Development Clarified: Effect on Reported Federal R&D Totals. NSF 21-326. Alexandria, VA: National Science Foundation. Available at https://ncses.nsf.gov/pubs/nsf21326/.

Contact Us

Report Author

Christopher Pece
Senior Analyst
Research and Development Statistics Program, NCSES
Tel: (703) 292-7788
E-mail: cpece@nsf.gov

John Jankowski
Program Director
Research and Development Statistics Program, NCSES
Tel: (703) 292-7781
E-mail: jjankows@nsf.gov

NCSES

National Center for Science and Engineering Statistics
Directorate for Social, Behavioral and Economic Sciences
National Science Foundation
2415 Eisenhower Avenue, Suite W14200
Alexandria, VA 22314
Tel: (703) 292-8780
FIRS: (800) 877-8339
TDD: (800) 281-8749
E-mail: ncsesweb@nsf.gov