Separator.

Glossary
magnifying glass on a desk of scattered letters.

Data Tools and Methodology

Data series

The collection of data characteristics of the target population repeated over time.

Dimension

Reference information about a data series, responding unit, or single observation. Dimensions are characteristics used to group data into distinct groups such as race, gender, or fields of study.

Measure

A summary (mean, mode, total, index, etc.) of variable values for the responding units, such as the total number of doctorate recipients, median salary, and average annual R&D expenditures. In a dataset, the instance of a measure is often referred to as an observation.

Statistical significance

Attained when statistical procedure applied to a set of observations yields a p value that exceeds the agreed level of probability which determines the null hypothesis will be rejected.

Statistical survey

A data collection whose purposes include the description, estimation, or analysis of the characteristics of groups, organizations, segments, activities, or geographic areas. This type of survey may be a census or may collect information from a sample of the target population. Examples of population characteristics that may be explored in NCSES surveys are people, universities, or government entities.

Topic

The subject or area of analytical interest for users exploring data or conducting research. Example topics may include fields of study, research and development activity, or college or university enrollment.

Unweighted count

The actual number of responding units found in a particular dataset, table, or measure before any necessary weighting has been applied to the table.

Variable

A characteristic of an object of study in a dataset that may take on more than one of a set of values to which a numerical or categorical measure can be assigned. Examples of common variables include occupation, sex, age, and industry.

Weighted count

A weighted count represents the proportion of the target population that the responding unit in a survey represents. Users should use the weighted count when creating measures about the larger population from a sample survey.

Standard Application Process (SAP)
 
Access modality
The location, either physical or virtual, where confidential data can be accessed and used. The SAP Data Catalog lists the access modality (or modalities) for each data asset. In the SAP application, applicants are asked to select where they would like to access the requested data. Applicants should consult the SAP Data Catalog to confirm whether the data requested are available in the applicants’ desired location. Access modality is also called “Work Location” in the SAP application.

Advisory Committee on Data for Evidence Building (ACDEB) 
This committee was established by the Foundations for Evidence-Based Policymaking Act of 2018 to review, analyze, and make recommendations to the Office of Management and Budget on how to promote the use of federal data for evidence building. The ACDEB was composed of partners across federal, state, and local governments as well as nongovernmental experts in privacy, technology, and research. It produced a Year 1 Report (2021) and a Year 2 Report (2022) before sunsetting in 2022.

Agency benefit
Some agencies require applicants to specify how their project will benefit the agency, since the agency owns the data being requested. For example, applicants requesting access to Bureau of Economic Analysis (BEA) data are required to show how their project will improve BEA data, surveys, or data collection programs. The SAP application contains a specific field for addressing “Agency Benefits.” The Census Bureau has unique requirements for demonstrating agency benefit, and hence, a separate “Census Benefits” field is in the application. If requesting Census Bureau data, please review the benefit criteria provided in the application before completing this field.

Agency reviewer
A statistical agency or unit staff member who reviews applications using the Common Review Criteria and who issues a determination on the application. The SAP policy provides detailed information on these criteria.

Agent
An individual with approval to use confidential data to perform exclusively statistical activities for an approved project, subject to appropriate control, supervision, and agreement to comply with all relevant legal provisions.

Allowed use
Use that is consistent with any restrictions on the use of a specific data set. These restrictions may include the privacy notices or informed consent statements used when the data were collected or acquired, an agreement or contract, a statutory limitation, or a combination of two or more restrictions.

Amendments 
Application amendments are changes or updates to an approved application. These might include changes to research staff, project methodology, and project timeline.

Appeal 
Individuals may appeal an adverse determination on an application if the specific grounds for the appeal are within the control of the statistical agency or unit that made the determination. They may also submit an appeal if they believe that the agency has not complied with the provisions in the SAP policy, M-23-04.

Applicant credentialing
The process by which an applicant is reviewed to determine whether data access can be granted. Credentialing may involve a background investigation, signing a licensing agreement or a data use agreement, or training. See Authorization levels for additional information.

Application window
By default, the application review period begins when an agency receives an application through the SAP portal. Some statistical agencies and units, however, review applications only during a set window of time every year. The Internal Revenue Service, for example, uses an application window. Applicants should refer to the agency Web pages for specific instructions and information. Any applications received prior to the agency review window are held in the SAP portal. Once the window opens, the review period begins. If an agency or unit uses an application window, they list it clearly in the SAP Data Catalog metadata.

Assessment criteria
See Common Review Criteria.

Authorization levels
The SAP policy (M-23-04) establishes common criteria for applicant review. Authorization levels are a framework that allows agencies to place applicants in a trusted category. M-23-04 establishes four authorization levels, each with varying requirements for credentialing and data access.

Chief Statistician of the United States
A position located in the Office of Management and Budget, the Chief Statistician is charged with leading and coordinating the activities of the federal statistical system and the Interagency Council on Statistical Policy.

Clearance
In the context of the SAP, the term clearance refers to National Agency Check with Inquiries clearance or Moderate Background Investigation clearance.

Commingled data
Data that have been enhanced or combined with data from another agency, organizational unit, or entity outside the executive branch. Commingled data use and onward sharing may be governed by a memorandum of understanding or other contractual arrangement between the statistical agency or unit and the other data owners for the purpose of outlining the permissions protocols to which the commingled data are subject.

Commission on Evidence-Based Policymaking (CEP)
A temporary congressional commission active from March 2016 to the end of 2017 that was tasked with exploring how to increase the availability and use of evidence in the federal government while protecting privacy and confidentiality. The CEP published its findings and recommendations in its 2017 final report, which was a factor in the creation of the SAP policy.

Common Review Criteria 
Applications received through the SAP are assessed using the Common Review Criteria. The Common Review Criteria are outlined in the SAP policy (M-23-04, section 4.1). They include (1) statistical purpose, (2) allowed use, (3) statistical disclosure limitation, (4) demonstrated need, (5) feasibility, and (6) maintaining public trust. To view the full criteria descriptions, refer to M-23-04.

Confidential data 
Information that is collected under a confidentiality pledge or other obligation necessitating the protection of those data from public disclosure. Use of these data is governed by data security protocols and access restrictions. A confidential data asset in the SAP Data Catalog may be accessed only by an agent designated by a statistical agency or unit for an exclusively statistical activity. Such data include surveys of businesses and households, linked employer-employee data, and administrative records from federal state, local, or tribal agencies.

Confidential Information Protection and Statistical Efficiency Act (CIPSEA)
CIPSEA establishes uniform confidentiality protections for information collected for statistical purposes by recognized statistical agencies and units and allows some data sharing between agencies. The 2018 Foundations for Evidence Based Policymaking Act reauthorized CIPSEA and expanded access to CIPSEA-protected data for statistical uses.

Data access
In the context of the SAP policy, the term data access refers to the ability of approved and authorized researchers to securely access and use confidential data assets.

Data asset 
A collection of data elements or data sets that may be grouped together.

Determination
For the purposes of an SAP review, the term determination means the decision by the agency or unit reviewers to grant or deny a request for access to data assets. Agencies may approve, reject, or request changes to the application.

Evidence 
Information produced as a result of statistical activities conducted for a statistical purpose.

Evidence building
The use of data to produce evidence, or statistics. The term evidence building refers to the production of statistics for the purpose of providing evidence that can be used to make informed decisions about public programs and policy.

Federal Statistical Research Data Centers (FSRDCs)
FSRDCs provide secure environments for qualified researchers to use data that has restricted access while protecting respondents’ confidentiality. FSRDCs are partnerships between federal statistical agencies and leading research institutions. To learn more, please visit the Census Bureau website.

Federal statistical system
The federal statistical system collects and transforms data into useful, objective information and makes it readily and equitably available to stakeholders, while protecting the responses of individual data providers. The federal statistical system includes the following:

  • The Office of the Chief Statistician of the United States
  • The Interagency Council on Statistical Policy
  • 16 recognized principal statistical agencies and units
  • Approximately 100 other statistical programs within other federal agencies

Visit www.statspolicy.gov to learn more about the federal statistical system.

Foundations for Evidence-Based Policymaking Act of 2018
This law, passed in 2018, was established to advance evidence-building in the federal government by improving access to data and expanding evaluation capacity. The SAP was established within this legislation (44 U.S.C. § 3583).

Interagency Council on Statistical Policy (ICSP)
Entity that supports implementation of the statistical system’s vision to operate as a seamless system to support evidence-based decision-making. ICSP advises the Office of Management and Budget (OMB) on SAP development and policy. It is chaired by OMB’s chief statistician and comprises the heads of all 16 principal statistical agencies and units and a rotating member from one of the statistical programs within other federal agencies.

Linked data
Data assets that result from joining two or more data assets across common elements. The SAP Data Catalog includes metadata describing linkage capabilities and linkage variables for some data assets. Applicants with questions about potential data linkages are encouraged to contact the data-owning agency or agencies before submitting their applications.

Metadata
In the context of the SAP, metadata is information that describes (1) the basic characteristics and content of confidential data assets and (2) any requirements necessary to obtain access to a confidential data asset. For example, metadata in the SAP includes the title of the confidential data set, information about how the data were collected, and information about where the data may be accessed. Metadata may also include a link to an agency website that provides complete documentation for each data asset. See SAP Data Catalog.

Multi-agency application (or multi-agency request)
An application requesting access to data sets from more than one agency. This includes requests for data sets that are co-owned by two or more agencies.

Nonconfidential data
Information that is not collected under a confidentiality pledge or other obligation necessitating the protection of data from public disclosure.

Nonstatistical purpose
The use of data for activities without a statistical purpose, including activities with an administrative, regulatory, law enforcement, adjudicatory, or other purpose that affects the rights, privileges, or benefits of a particular identifiable respondent.

National Secure Data Service (NSDS) Demonstration Project
Section 10375 of the 2022 CHIPS and Science Act authorized an NSDS Demonstration (NSDS-D) project to pilot potential services for an NSDS. An NSDS would provide a platform of shared services to streamline and innovate data sharing, data linkage, and privacy protections to support expanded data use. This effort facilitates statistical activities in support of increased evidence building for the American public. The goal of the NSDS-D project is to inform efforts for developing a shared services model that would streamline and innovate data sharing and linking to enable decision-making at all levels of government and in all sectors.
The idea of an NSDS was first introduced by the bipartisan Commission on Evidence-Based Policymaking in 2016, and it was later envisioned through specific recommendations provided by the Advisory Committee on Data for Evidence Building in 2022.

Office of Management and Budget (OMB)
OMB, an office within the Executive Office of the President of the United States, produces the presidential budget, and it also coordinates interagency policy initiatives. OMB is home to the Chief Statistician of the United States.

OMB M-23-04
Establishment of Standard Application Process Requirements on Recognized Statistical Agencies and Units: This Office of Management and Budget memorandum, released in December 2022, establishes the policy governing the SAP. This memorandum outlines the standardized review time frames, sets forth the Common Review Criteria, and establishes other aspects of the SAP.

Personally Identifiable Information (PII)
Information that can be used to distinguish or trace an individual’s identity, either alone or when combined with other personal or identifying information that is linked or linkable to a specific individual. See M-07-16 for more information.

Principal statistical agency
One of 16 federal agencies or units that have statistical work as their principal mission. These include the following:

  • Bureau of Economic Analysis (Department of Commerce)
  • Bureau of Justice Statistics (Department of Justice)
  • Bureau of Labor Statistics (Department of Labor)
  • Bureau of Transportation Statistics (Department of Transportation)
  • Census Bureau (Department of Commerce)
  • Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration (Department of Health and Human Services)
  • Economic Research Service (Department of Agriculture)
  • Energy Information Administration (Department of Energy)
  • Microeconomic Surveys Unit (Board of Governors of the Federal Reserve System)
  • National Agricultural Statistics Service (Department of Agriculture)
  • National Animal Health Monitoring System, Animal and Plant Health Inspection Service (Department of Agriculture)
  • National Center for Education Statistics (Department of Education)
  • National Center for Health Statistics (Department of Health and Human Services)
  • National Center for Science and Engineering Statistics (National Science Foundation)
  • Office of Research, Evaluation, and Statistics (Social Security Administration)
  • Statistics of Income Division (Department of the Treasury)

Programmatic benefit
See Agency benefit.

Protected data
See Confidential data.

Public data asset
A data asset, or part of an asset, maintained by the federal government that has been, or may be, released to the public. Some public use versions of data are initially collected under a confidentiality protection, but disclosure limitation methods have been applied, such as removing Personally Identifiable Information and aggregation of information, to prevent the risk of disclosing the identities of individuals.

Public reporting
The Confidential Information Protection and Statistical Efficiency Act requires that the SAP provide the public with information on each application received, the status of each application, the determination made for each application, and any other information, as appropriate, to ensure the transparency of the process. Appropriate transparency for this process requires providing the public with an understanding of who is accessing confidential data as a result of applications processed through the SAP, what data are being accessed, why the data are being accessed, when the data are being accessed, and how the data are being accessed.

Research Data Gov
The SAP portal is located at www.ResearchDataGov.org.

Restricted use data (or restricted access data)
See Confidential data.

Review time frames
The SAP policy (M-23-04) requires that agencies review applications in a timely manner and establishes standardized review time frames. For applications involving review by a single agency, the agency has 12 weeks to provide a final determination (approve or reject). For applications involving review by more than one agency (this includes co-owned data), the reviewing agencies have 24 weeks to provide a final determination. These time frames allow agencies to work with applicants to revise their applications, if needed. The review time frames do not apply to applications that request access to confidential data assets that require approval from a third party outside the federal statistical system (e.g., state or local governments).

SAP
The SAP is a common application and process for requesting access to confidential data from federal statistical agencies and units. The 2018 Foundations for Evidence-Based Policymaking Act (44 U.S.C. § 3583) required this standard process to be developed. Through the SAP, agencies, the Congressional Budget Office, governments (state, local, tribal, and territorial), researchers, and other individuals, as appropriate, may apply to a statistical agency or unit for access to confidential data assets (accessed or acquired by the agency or unit under the Confidential Information Protection and Statistical Efficiency Act) for purposes of developing evidence. The SAP is fully defined in the OMB M-23-04.

SAP application
This application is a standard form submitted through www.ResearchDataGov.org that is used to apply for access to confidential data. It includes questions on researchers’ information (such as contact information and affiliation) and a description of the proposed research project. Applications for data from some agencies may also require additional documents. To preview the SAP application questions, select this link (1.1 MB) .

SAP Data Catalog
A listing of confidential data assets owned by the nation’s federal statistical agencies and available for discovery on the SAP portal. For each data asset, the SAP Data Catalog provides metadata that describe the content and characteristics of that data asset. Applicants can search the Data Catalog by agency, topic, or keyword to identify data assets that are relevant to their project. For more information on metadata in the SAP Data Catalog, see Metadata. The SAP Data Catalog is available at www.ResearchDataGov.org.

SAP Governance Board
The SAP Governance Board, a subcommittee chartered by the Interagency Council on Statistical Policy, represents the agencies and units using the SAP and assists OMB in the oversight and coordination of the SAP. The SAP Governance Board approves key decisions of the SAP Program Management Office (PMO) and coordinates with the PMO to implement, operate, and maintain the SAP as established in the SAP policy. View the SAP Governance Board Charter to learn more.

SAP policy
See OMB M-23-04.

SAP portal
The SAP portal is the software implementation of the SAP and includes a Web-based interface that facilitates data asset discovery, the application process, and the application review process. The SAP portal is available at www.ResearchDataGov.org.

SAP Program Management Office (PMO)
The PMO's role is to implement, operate, and maintain the SAP, under the guidance of the Governance Board. The National Center for Science and Engineering Statistics is designated by the Office of Management and Budget as the SAP PMO.

Single-agency application (or single-agency request) 
A single-agency request is an application requesting access to a data asset or assets held by only one statistical agency or unit. The SAP policy (M-23-04) establishes a review time frame of no more than 12 weeks for single-agency applications.

Special Sworn Status (SSS)
Title 13 U.S.C. § 23(c) authorizes the Census Bureau to provide access privileges, Special Sworn Status, to individuals who have gone through an application and vetting process to assist the Bureau in performing work authorized by Title 13 of the United States Code. Applicants for data are required to obtain Special Sworn Status prior to being granted data access at the Census Bureau or a Federal Statistical Research Data Center.

Standardized review criteria
See Common Review Criteria.

Statistical activity
The collection, compilation, processing, or analysis of data for the purpose of describing or making estimates concerning the economy, society, or the natural environment as a whole or relevant groups or components therein.

Statistical agency or unit
An agency or organizational unit of the executive branch whose activities predominantly consist of the collection, compilation, processing, or analysis of information for statistical purposes. See Principal statistical agency or unit for a list of the 16 principal federal agencies and units that have statistical work as their principal mission.

Statistical purpose
The description, estimation, or analysis of the characteristics of groups, without identifying individuals or organizations that make up such groups.

Unauthorized disclosure
Occurs when data or information acquired by an agency under a pledge of confidentiality for exclusively statistical purposes is disclosed for any use other than an exclusively statistical purpose, without respondent consent, without being authorized by the head of the agency which gathered the data, or when prohibited by any other law.