National Survey of College Graduates: 2017

Purpose. The National Survey of College Graduates (NSCG) provides data on the characteristics of the nation’s college graduates, with a focus on those in the science and engineering (S&E) workforce. It samples individuals who are living in the United States during the survey reference week, have earned at least a bachelor’s degree, and are younger than 76. By surveying college graduates in all academic disciplines, the NSCG provides data useful in understanding the relationship between college education and career opportunities, as well as the relationship between degree field and occupation.

The NSCG is designed to provide demographic, education, and career history information about college graduates and to complement another S&E-focused survey conducted by the National Center for Science and Engineering Statistics (NCSES): the Survey of Doctorate Recipients (SDR, https://www.nsf.gov/statistics/srvydoctoratework/). The NSCG and the SDR share a common reference date, and they use similar questionnaires and data processing guidelines.

These technical notes provide an overview of the 2017 NSCG. Complete details are provided in the 2017 NSCG Methodology Report, available upon request from the NSCG Survey Manager.

Data collection authority. The information collected in the NSCG is solicited under the authority of the National Science Foundation Act of 1950, as amended, and the America COMPETES Reauthorization Act of 2010. The Census Bureau collects the NSCG data, on behalf of NCSES, under the authority of Title 13, Section 8 of the United States Code. The Office of Management and Budget control number is 3145-0141.

Survey contractor. Census Bureau

Survey sponsor. NCSES.

Major changes to the recent survey cycle. The 2017 NSCG questionnaire added two new questions on U.S. military veteran status.

Frequency. Biennial.

Initial survey year. 1993.

Reference period. The week of 1 February 2017.

Response unit. Individual.

Sample or census. Sample.

Population size. Approximately 61.2 million individuals.

Sample size. Approximately 124,000 individuals.

Target population. The NSCG target population includes individuals who meet the following criteria:

• Earned a bachelor’s degree^{​“Bachelor’s degrees” include equivalent undergraduate academic degrees awarded by colleges and universities in countries that may name their degrees differently.} or higher prior to 1 January 2016
• Are not institutionalized and reside in the United States or Puerto Rico as of 1 February 2017
• Are younger than 76 years as of 1 February 2017

Sampling frame. Using a rotating panel design, the 2017 NSCG includes new sample cases from the 2015 American Community Survey (ACS) and returning sample cases from the 2015 NSCG.

The NSCG sampling frame for new sample cases included the following eligibility requirements:

• Were residing in the United States or Puerto Rico as of the ACS interview date
• Were noninstitutionalized as of the ACS interview date
• Had earned at least a bachelor’s degree as of the ACS interview date
• Would be under the age of 76 as of 1 February 2017
• Did not have an inaccurate name or incomplete address on the ACS data file

Returning sample cases from the 2015 NSCG originated from three different frames (the 2009 ACS, 2011 ACS, and 2013 ACS) and had the following eligibility requirements:

• Were a complete interview or temporarily ineligible during their initial NSCG survey cycle
• Would be under the age of 76 as of 1 February 2017
• During the 2015 NSCG survey cycle, did not refuse to participate and request to be excluded from future NSCG cycles

Sample design. The NSCG sample design is cross-sectional with a rotating panel element. As a cross-sectional study, the NSCG provides estimates of the size and characteristics of the college graduate population for a point in time. As part of the rotating panel design, every new panel receives a baseline survey interview and three biennial follow-up interviews before rotating out of the survey.

The NSCG uses a stratified sampling design to select its sample from the eligible sampling frame. In the new sample, cases in the S&E strata were selected using systematic sampling, and cases in the non-S&E strata were selected with probability proportional to size (PPS) sampling.^{​With PPS sampling, the probability of selection was proportional to the ACS final person-level weight, adjusted to account for imputed educational attainment, incomplete addresses, or invalid names.} Among the returning sample, all eligible cases were selected. The sampling strata were defined by the cross classification of the following four variables:

• Young graduate oversample group eligibility indicator (2 levels)
• Demographic group (9 levels)
• Highest degree type (3 levels)
• Detailed occupation group (25 levels, with over half in S&E)

As was the case in the 2015 NSCG, the 2017 NSCG includes an oversample of young graduates to improve the precision of estimates for this important population. The 2017 NSCG includes approximately 124,000 sample cases drawn from the following:

• Returning sample from the 2010 NSCG who were originally selected from the 2009 ACS
• Returning sample from the 2013 NSCG who were originally selected from the 2011 ACS
• Returning sample from the 2015 NSCG who were originally selected from the 2013 ACS
• New sample selected from the 2015 ACS

Approximately 76,000 cases were selected from the returning sample members for one of the three biennial follow-up interviews that are part of the rotating panel design. For the baseline survey interview, about 48,000 new sample cases were selected from the 2015 ACS.

Data Collection and Processing Methods

Data collection. The data collection period lasted approximately 6 months (17 April 2017 to 29 October 2017). The NSCG used a trimodal data collection approach: self-administered online survey (Web), self-administered paper questionnaire (via mail), and computer-assisted telephone interview (CATI). Individuals in the sample generally were started in the Web mode, depending on their available contact information and past preference. After an initial survey invitation, the data collection protocol included sequential contacts by postal mail, e-mail, and telephone that ran throughout the data collection period. At any time during data collection, sample members could choose to complete the survey using any of the three modes. Nonrespondents to the initial survey invitation received follow-up contacts via alternate modes.

Quality assurance procedures were in place at each data collection step (e.g., address updating, printing, package assembly and mailing, questionnaire receipt, data entry, CATI, coding, and post-data collection processing).

Mode. About 79% of the participants completed the survey by Web, 12% by mail, and 9% by CATI.

Response rates. Response rates were calculated on complete responses, that is, from instruments with responses to all critical items. Critical items are those containing information needed to report labor force participation (including employment status, job title, and job description), college education (including degree type, degree date, and field of study), and location of residency on the reference date. The overall unweighted response rate was 70%; the weighted response rate was 71%. Of the 123,861 persons in the 2017 NSCG sample, 83,672 completed the survey.

Data editing. Response data had initial editing rules applied relative to the specific mode of capture to check internal consistency and valid range of response. The Web survey captured most of the survey responses and had internal editing controls where appropriate. A computer-assisted data entry (CADE) system was used to process the mailed paper forms. Responses from the three separate modes were merged for subsequent coding, editing, and cleaning necessary to create an analytical database.

Following established NCSES guidelines for coding NSCG survey data, including verbatim responses, staff were trained in conducting a standardized review and coding of occupation and education information, certifications, “other/specify” verbatim responses, state and country geographical information, and postsecondary institution information. For standardized coding of occupation (including auto-coding), the respondent's reported job title, duties and responsibilities, and other work-related information from the questionnaire were reviewed by specially trained coders who corrected respondents’ self-reporting errors to obtain the best occupation codes. (See technical table A-2 for a list and classification of coded occupations reported in the NSCG.) For standardized coding of field of study associated with any reported degree (including auto-coding), the respondent’s reported department, degree level, and field of study information from the questionnaire were reviewed by specially trained coders who corrected respondents’ self-reporting errors to obtain the best field of study codes. (See technical table A-1 for a list and classification of coded fields of study reported in the NSCG.)

Imputation. Logical imputation was primarily accomplished as part of editing. In the editing phase, the answer to a question with missing data was sometimes determined by the answer to another question. In some circumstances, editing procedures found inconsistent data that were blanked out and therefore subject to statistical imputation.

The item nonresponse rates reflect data missing after logical imputation or editing but before statistical imputation. For key employment items—such as employment status, sector of employment, and primary work activity—the item nonresponse rates ranged from 0.00% to 1.07%. Nonresponse to questions deemed sensitive was higher: nonresponse to salary and earned income was 4.54% and 6.30%, respectively, for the new sample members and 5.33% and 7.43%, respectively, for the returning members. Personal demographic data of the new sample members had variable item nonresponse rates, with sex at 0.00%, birth year at 0.02%, marital status at 0.24%, citizenship at 0.13%, ethnicity at 0.92%, and race at 2.33%. The nonresponse rates for returning sample members were 1.52% for marital status and 1.41% for citizenship.

Item nonresponse was typically addressed using statistical imputation methods. Most NSCG variables were subjected to hot-deck imputation, with each variable having its own class and sort variables chosen by regression modeling to identify nearest neighbors for imputed information. For some variables, there was no set of class and sort variables that was reliably related to or suitable for predicting the missing value, such as day of birth. In these instances, random imputation was used, so that the distribution of imputed values was similar to the distribution of reported values without using class or sort variables.

Imputation was not performed on critical items or on verbatim-based variables. In addition, for some missing demographic information, the NSCG imported the corresponding data from the ACS, which had performed its own imputation.

Weighting. Because the NSCG is based on a complex sampling design and subject to nonresponse bias, sampling weights were created for each respondent to support unbiased population estimates. The final analysis weights account for several factors, including the following:

• Adjustments to account for undercoverage of recent immigrants and undercoverage of recent degree-earners
• Adjustment for incorrect names or incomplete address information on the sampling frame
• Differential sampling rates
• Adjustments to account for non-locatability and unit nonresponse
• Adjustments to align the sample distribution with population controls
• Trimming of extreme weights
• Overlap procedures to convert weights that reflect the population of each individual frame (2009 ACS, 2011 ACS, 2013 ACS, and 2015 ACS) into a final sample weight that reflects the 2017 NSCG target population.

The final sample weights enable data users to derive survey-based estimates of the NSCG target population. The variable name on the NSCG public use data files for the NSCG final sample weight is WTSURVY.

Variance estimation. The successive difference replication method (SDRM) was used to develop replicate weights for variance estimation. The theoretical basis for the SDRM is described in Wolter (1984) and in Fay and Train (1995). As with any replication method, successive difference replication involves constructing numerous subsamples (replicates) from the full sample and computing the statistic of interest for each replicate. The mean square error of the replicate estimates around their corresponding full sample estimate provides an estimate of the sampling variance of the statistic of interest. The 2017 NSCG produced 320 sets of replicate weights.

Disclosure protection. To protect against the disclosure of confidential information provided by NSCG respondents, the estimates presented in NSCG data tables are rounded to the nearest 1,000.

Data table cell values based on counts of respondents that fall below a predetermined threshold are deemed to be sensitive to potential disclosure, and the letter “D” indicates this type of suppression in a table cell.

Sampling error. NSCG estimates are subject to sampling errors. Estimates of sampling errors associated with this survey were calculated using replicate weights and are available upon request from the NSCG Survey Manager. Data table estimates with coefficients of variation (that is, the estimate divided by the standard error) that exceed a predetermined threshold are deemed unreliable and are suppressed. The letter “S” indicates this type of suppression in a table cell.

Coverage error. Coverage error occurs in sample estimates when the sampling frame does not accurately represent the target population and is a type of nonsampling error. Any missed housing units or missed individuals within sample households in the ACS would create undercoverage in the NSCG. Additional undercoverage errors may exist because of self-reporting errors in the NSCG sampling frame that led to incorrect classification of individuals as not having a bachelor's degree or higher when in fact they held such a degree.

Nonresponse error. The weighted response rate for the 2017 NSCG was 71%; the unweighted response rate was 70%. Analyses of NSCG nonresponse trends were used to develop nonresponse weighting adjustments to minimize the potential for nonresponse bias in the NSCG estimates. A hot deck imputation method was used to compensate for item nonresponse.

Measurement error. The NSCG is subject to reporting errors from differences in interpretation of questions and by modality (Web, mail, CATI). To reduce measurement errors, the NSCG questionnaire items were pretested in focus groups and cognitive interviews.

Data comparability. Year-to-year comparisons of the nation’s college-educated population can be made among the 1993, 2003, 2010, 2013, 2015, and 2017 survey cycles because many of the core questions remained the same. Because the 1995, 1997, 1999, 2006, and 2008 surveys do not provide full coverage of the nation’s college-educated population, comparisons between these cycles and other cycles should be limited to those individuals educated in S&E fields or employed in S&E occupations. Also, because of the use of different reference months in some survey cycles, seasonal differences may occur when making comparisons across years. Thus, use caution when interpreting cross-cycle comparisons.

There is overlap in the cases included in the 2010 NSCG through the 2017 NSCG (see figure 1). The overlap among cases allows for longitudinal analysis of a subset of the NSCG sample using restricted use data files within NCSES’ Secure Data Access Facility (SDAF). Cases can be linked across survey years using a unique identification variable and single-frame weights are available for each survey year, allowing for the evaluation of estimates from each frame independently. If you are interested in applying for a license to access restricted use NSCG data via the SDAF, please visit the NCSES Restricted-Use Licensing page. Moreover, the Census Bureau offers NSCG restricted use data files that include a few additional data elements. These files can be accessed via the Federal Statistical Research Data Centers (census.gov).

Changes in survey sampling

• Starting with the 2010 survey cycle, the NSCG has used a rotating panel sample design in which a new sample is selected every survey cycle from the most recent ACS and the cases are followed over four survey cycles. After the fourth cycle, the cases rotate out of the NSCG and are replaced by a newly selected panel of cases from the most recent ACS.
• Beginning in the 2013 cycle, NCSES discontinued the National Survey of Recent College Graduates (NSRCG) and expanded the sample of young college graduates in the NSCG. The 2010 NSRCG sample cases were carried forward through the 2015 NSCG survey cycle as the NSCG transitioned to a design that included a young college graduate oversample.
• Figure 1 summarizes the NSCG rotating panel design and the sample sizes for the 2010 through 2017 NSCG survey cycles. Each new NSCG survey cycle adds new cases from the most recent ACS survey year to offset the rotating out of the oldest NSCG panel.

Changes in questionnaire

• 2017. The 2017 NSCG questionnaire added two new questions about U.S. military veteran status that are asked on the ACS.
• 2015. The 2015 NSCG questionnaire added a section on professional certifications and licenses.
• 2013. The 2013 NSCG questionnaire added questions about attendance at community colleges, amounts borrowed to finance undergraduate and graduate degrees, and sources of financial support for undergraduate and graduate degrees. The 2013 questionnaire also differed from the 2010 questionnaire by splitting the first response category for the indicator of sample member location on the survey reference date into two categories. “United States, Puerto Rico, or another U.S. territory” became “United States or Puerto Rico” and “Another U.S. territory.”
• 2010. The 2010 NSCG questionnaire added items on components of job satisfaction, importance of job benefits, year of retirement, whether employer is a new business, and degree of difficulty concentrating, remembering, or making decisions.

Changes in reporting procedures

• In past years, NSCG data were combined with data from the SDR and the NSRCG to form the Scientists and Engineers Statistical Data System (SESTAT). The last series of tables produced from SESTAT used 2013 NSCG data. Since then, NSCG data have been used as a sole source in numerous tables for NCSES’s two congressionally mandated reports (Science and Engineering Indicators and Women, Minorities, and Persons with Disabilities in Science and Engineering).

Changes in microdata

• NSCG public use data files can no longer be linked across cycles. The NSCG public use data files now include a case identifier (OBSNUM), which is a random number for each case in each cycle.

Field of degree. NSCG respondents are asked to report each degree they have earned at the bachelor’s level or higher, along with the major field of study for each degree. The 2017 NSCG used a taxonomy of 142 “detailed” fields of study from which respondents could select the field that best represented their major. These “detailed” fields of study were aggregated into 31 “minor” fields, 7 “major” fields, and 3 “broad” fields (S&E, S&E-related, and non-S&E). S&E fields of degree are those included in the following five major categories: (1) computer and mathematical sciences; (2) biological, agricultural, and environmental life sciences; (3) physical sciences; (4) social sciences; and (5) engineering. S&E-related fields of degree are those included in the following four categories: (1) health, (2) science and mathematics teacher education, (3) technology and technical fields, and (4) other S&E-related fields. (See technical table A-1 for a list and classification of fields of degree reported in the NSCG.)

Full-time and part-time employment. Full-time (working 35 hours or more per week) and part-time (working less than 35 hours per week) employment status is for the principal job only and not for all jobs held in the labor force. For example, an individual who works part time in his or her principal job but full time in the labor force would be tabulated as part-time.

Highest degree level. NSCG respondents report the degrees they have earned at the bachelor’s level (e.g., BS, BA, AB), master’s level (e.g., MS, MA, MBA), and doctorate level (e.g., PhD, DSc, EdD), as well as other professional degrees (e.g., JD, LLB, MD, DDS, DVM). Because the NSCG is focused on the S&E workforce, the sampling strategy does not include a special effort to select individuals with professional degrees. As such, there is not always sufficient data for the professional degrees to be reported separately in the tables.

Occupation data. The occupational classification of the respondent was based on his or her principal job (including job title) held during the reference week—or on his or her last job held, if not employed in the reference week (survey questions A5 and A6 as well as A16 and A17). Also used in the occupational classification was a respondent-selected job code (survey questions A7 and A18). S&E occupations are those included in the following five major categories: (1) computer and mathematical scientists; (2) biological, agricultural, and other life scientists; (3) physical scientists; (4) social scientists; and (5) engineers. S&E-related occupations require science and technology expertise but are not part of the five major categories of S&E occupations. S&E-related occupations are those included in the following five categories: (1) health-related occupations, (2) S&E managers, (3) S&E precollege teachers, (4) S&E technicians and technologists, and (5) other S&E-related occupations. (See technical table A-2 for a list and classification of occupations reported in the NSCG.)

Race and ethnicity. Ethnicity is defined as Hispanic or Latino or not Hispanic or Latino. Values for those selecting a single race include American Indian or Alaska Native, Asian, Black or African American, Native Hawaiian or Other Pacific Islander, and White. Those persons who report more than one race and who are not of Hispanic or Latino ethnicity also have a separate value.

Research and development (R&D). R&D includes the work activities of (1) basic research, (2) applied research, (3) development, and (4) the design of equipment, processes, structures, and models.

Salary. Median annual salaries are reported for the principal job, rounded to the nearest $1,000, and computed only for individuals employed full time. For individuals employed by educational institutions, no accommodation was made to convert academic year salaries to calendar year salaries.

Science and engineering (S&E) workforce. Individuals working in an S&E or S&E-related occupation or who earned an S&E or S&E-related degree and are working in a non-S&E occupation.

Scientists and engineers. The NSCG data tables use a narrow definition of scientists and engineers based solely on occupation: individuals employed in an S&E occupation. In other NCSES publications, the definition of scientists and engineers may be broadened to include individuals employed in S&E-related occupations, as well as individuals who may not be working in an S&E or S&E-related occupation but who earned an S&E or S&E-related degree.

Sector of employment. Employment sector is a derived variable based on responses to questionnaire items A13, A14, and A15. In the data tables, the category 4-year educational institution includes 4-year colleges or universities, medical schools (including university-affiliated hospitals or medical centers), and university-affiliated research institutes. Two-year and precollege institutions include community colleges, technical institutes, and other educational institutions (which respondents reported verbatim in the survey questionnaire). For-profit business or industry includes respondents who were self-employed in an incorporated business. Self-employed includes respondents who were self-employed or were a business owner in a non-incorporated business.

Fay RE, Train GF. 1995. Aspects of Survey and Model-Based Postcensal Estimation of Income and Poverty Characteristics for States and Counties. American Statistical Association Proceedings of the Section on Government Statistics, 154–59.

Wolter K. 1984. An Investigation of Some Estimators of Variance for Systematic Sampling. Journal of the American Statistical Association 79(388):781–90.