Examining Workforce Investment Act Programmatic Outcomes

Occasional Paper No. 1

Evaluation of Federal Training & Education Programs

October 2004

Workforce Investment Act participation appears to be very effective in improving the economic lives of participants. For example, 65 percent of Dislocated Workers and 61 percent of Adults experienced at least one positive outcome from participation; many of those experience more than one positive outcome.

The U.S. Department of Labor, Employment and Training Administration (ETA) currently uses employment, job retention, and earnings increase performance measures for programs funded by ETA (U.S. Department of Labor, ETA, 2003). Section 171 of the 1998 Workforce Investment Act (WIA) requires the Secretary of Labor to prepare a five-year research plan for pilot, demonstration, research, evaluation, multi-state, and multi-service initiatives every two years. ETA’s research plan provides for the evaluation of existing programs but it also covers applied research, including labor market research. Section 171 requires the continuing evaluation of programs and activities using appropriate methodologies and research designs, especially experimental research designs (U.S. Government Accounting Office, 2004). Unfortunately, experimental research is expensive, time consuming, and often raises legal and ethical issues. This paper addresses the current performance measures used by ETA and provides alternative measures that could improve program outcomes.

A basic limitation with the current ETA performance measurement strategy is the failure to measure outcomes in the context of the surrounding labor market. Without the labor market context, neither the state nor ETA can determine whether changes in wages and retention are effects of WIA participation or simply a reflection of the local economy.

ETA’s current evaluation strategy also uses a limited number of outcome measures, preventing a thorough understanding of program outcomes. Research emphasis has been placed on finding new ways to evaluate the current outcomes without determining whether or not those outcomes represent program success. While performance measures can drive program development and management, it is not clear that federally required measures capture relevant information pertinent to employers, workers, or program managers.

To answer a few questions about ETA-prescribed WIA participant performance measures in Wyoming, this paper shows WIA training participants in the context of the Wyoming labor market. This context provides a framework for evaluating labor market outcomes. Furthermore, new outcome measures are investigated as potential enhancements to the strategy already in place.

Literature Review

Several studies report wage and employment increases for disadvantaged adults immediately after completion of an employment and training program. Bloom, et al. (1997) showed significant monthly earnings increases for those who participated in a training program when compared to a control group. Positive effects were greater for those who received on-the-job training, job search assistance, or technical/occupational training.

Other findings show disadvantaged adult women participants are unemployed for shorter periods of time following training than are other participants ( Eberwein, Ham, & LaLonde, 1997). LaLonde (1995) and Couch (1992) found disadvantaged adult women benefited from training services more so than youth or adult men. Finn and Willoughby ( 1996) demonstrated that being unemployed for fifteen weeks or more prior to Job Training Partnership Act (JTPA) enrollment negatively influenced the odds of being employed upon completion of a training program. Receiving training from a current employer rather than a training provider unfamiliar to the trainee positively influenced employment. Similar findings were reported by Hollenbeck and Anderson ( 1993) who found that job-specific training programs improved males’ employment rate and increased females’ wages.

Findings from research on the Wyoming WIA participants were not similar to studies which showed positive economic outcomes for WIA adults. Harris ( 2002), using a quasi-experimental design and administrative records, found no difference between WIA participants and the control group in the year following training.

Few studies have been conducted on the economic outcomes of Dislocated Workers who complete a WIA (or JTPA) training program. Some evidence suggests the effects are insignificant on both wages and employment rates, even when other group differences were controlled for (Leigh, 1997; Kodrzycki, 1996). Alternatively, Jacobson, LaLonde, and Sullivan (1999) found that Dislocated Workers increased their annual earnings by $20-24 per completed credit hour in community college classes (in U.S. Department of Labor, 2001). Greater earnings were seen for courses completed in health related fields, technical and vocational fields, or math and science fields. Other studies report a positive and significant impact on earnings after training ( Benus & Byrnes, 1993; Jacobson, LaLonde, & Sullivan, 1994), but the findings were not generalizable.

Examining only short-term outcome measures, notably wage progression, fails to show the entire picture of WIA outcomes. “The short-run performance measures that are used in its [value added net of social cost] place are either uncorrelated with or negatively correlated with net value added, especially in the long run” (Heckerman, Heinrich, & Smith, 1997). For example, research conducted on Wyoming community college graduates showed wages did not rise above a comparison group until 9 to 18 months after program completion (Gallagher, 2001). Without longitudinal analysis as a measure, community college education would appear ineffective in increasing graduates’ wages. In fact, it simply takes time for the graduates to stabilize in the labor market.

As evident from the data, disadvantaged adults, especially women, tend to fare reasonably well economically compared to control groups in the first year after training. Unfortunately, the long-term consequences are much more dismal.

The United States General Accounting Office (1996) released a report to congressional requesters on the efficacy of JTPA. The report describes a study performed in response to congressional concern about the long-term impact of job training programs. The findings demonstrated that the positive effects of JTPA during the 30 months following training are not permanent. Adult men earned significantly higher wages than the control group one, two, and three years after training, but the effect dissipated by the fourth year. Adult women earned significantly higher wages than the control group every year after training until the fifth year. Both male and female youths never achieved significantly higher wages than the control group in the five years subsequent to training. If the program has no long-term impact on participants and questionable effect in the short term, is the program attaining its goals?

Both WIA and the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996 promote rapid entry into employment. Under WIA, only individuals unable to obtain employment through less intrusive core or intensive services are eligible for job training, meaning that all efforts are made to find an individual employment before the individual qualifies for training. Additionally, WIA participants are required to be economically disadvantaged (U.S. Department of Labor, 2001). With the passage of PRWORA, time-limited benefits and significant work requirements replaced cash benefit entitlements. Instead of receiving government assistance in the form of cash, individuals are required to work. As a result of these policy changes, individuals with few skills or work experience are being placed into entry-level, low-paying jobs because the emphasis is on employment, not employment that pays a living wage. In addition, the monthly income of many families falls after finding work because they do not make high enough wages to replace the lost government subsidies (Friedlander & Hamilton, 1996). Despite the strong economy and the substantial decline in welfare rolls during the latter half of the 1990s, the extent to which low-skill workers and former welfare recipients are able to retain their jobs and advance in them is unknown (U.S. Department of Labor, 2001).

Past evaluations of employment and training programs have demonstrated that many individuals who successfully obtain employment often face additional barriers to retain employment. Recently, more focus has been placed on training that assists individuals in remaining employed (U.S. Department of Labor, 2001). Finding a job is of limited value if the person is unable to keep it. Research on the effectiveness of existing retention strategies is limited and needs to be expanded. This is particularly important due to a strong relationship between employment tenure and earnings, regardless of the type of employment (Glover, 2003).

The logical next question is why continue to fund a program that fails to achieve federal mandated goals that were developed in the absence of empirical statistical knowledge? The simplest answer is that the outcome measures are not capable of reflecting the advantages gained by program participants. Therefore, the research should focus on identifying how WIA participation really affects program participants and how those effects can be quantified rather than how we think WIA should affect participants. When that is determined, the empirical data can drive ways to enhance the program to maximize the advantages.

A starting place for examining program effects is industry of employment. Industry is strongly related to income. Those who work in Mining or Transportation, Communications, & Public Utilities (TCPU) typically earn significantly more than those who work in Retail Trade. In addition, retention is usually higher in those industries than in the lower-paying industries (Glover, 2003; Mishel, Bernstein, & Schmitt, 1999). Bartik (1997) showed welfare recipients were more successful for longer periods of time when they were placed in health care; educational services; Finance, Insurance & Real Estate (FIRE); and durable goods manufacturing. Unfortunately, the highest demand for WIA trained workers in Wyoming and the U.S. is in Retail Trade.

Another area to investigate is turnover. Many individuals who register for WIA services have difficulty maintaining employment, especially employment with a single employer. Turnover is associated with lower wages ( Jones, 2001). If WIA program completers learn job skills that enhance their ability to maintain employment with one employer, their wages will likely increase as their tenure increases. While lowered turnover rates may not result in immediate wage increases, they generally reflect improved employability of the individual.

Another possibility for outcome measurement is the number of Unemployment Insurance (UI) claims filed. Currently, the outcome measures note whether or not a participant was employed in a quarter but there is no indication of how long they remained employed or how attached they were to their employer.

This study sought to evaluate the federally required outcomes of WIA participants in Wyoming (earnings after training) for eight quarters after program completion. A quasi-experimental design was employed using administrative records in the absence of random selection. It was expected that WIA would have a positive effect on wages in that wages for the WIA group would increase after training. Theoretically relevant variables were statistically controlled so that a more accurate test could be conducted between the WIA participant group and the Comparison group. Statistical controls provide for the mathematical removal or control of competing additional theoretical explanations of outcomes. Such controls are introduced after the fact as opposed to experimental designs that insure comparison and experimental group equivalence through pre-treatment random assignment.

In addition, we suggested the addition of new outcome measures: turnover rate, industry of employment, employment rate, and UI claims filed. We also created a combined success variable based on the other suggested outcome measures.

Methods

A broader evaluation of current performance measures was conducted by putting Wyoming WIA training participants in the context of the Wyoming labor market. New outcome measures are investigated as potential enhancements to the strategy already in place.

The study group included Wyoming WIA participants for the Program Year 2001 and 224,217 individuals selected from the Wyoming UI Wage Records file. There were 568 individuals who participated and exited WIA training during the program year, including 217 Adults, 110 Dislocated Workers, and 241 in Youth programs. The Youth groups were removed from the analyses because their outcome measures differ significantly from Adults and Dislocated Workers. The WIA participant file was combined with Wage Records in order to obtain wage and employment information. Turnover, employment industry, UI claims rate, and success were investigated.

The average participant age was 36.1 for Adults and 39.4 for Dislocated Workers (see Table 1). The mean age of individuals in the Comparison group was slightly higher at 40.2. Gender was unknown for 5 (1.5%) of the participants, 185 (56.6%) were men, and 137 (41.9%) were women.

The Comparison group was defined as individuals who had wages in Wyoming during Program Year 2001, were residents of the state (Jones, 2002), and were not participants of WIA in Program Year 2001. The restrictions narrowed the group to approximately 70 percent of all who worked in Wyoming during that time period. All eligible individuals in Wage Records were randomly assigned a termination quarter from the third quarter of 2000 (2000Q3) to 2001Q2. Total wages were then computed for the six quarters prior and eight quarters after the termination quarter. Table 2 shows the means and standard deviations of wages by participation status both four and six quarters prior to the termination quarter.

Demographics for the Comparison group were taken from the Wyoming Department of Transportation (DOT) Driver’s License file. Wage Records were merged with the DOT file using social security number (SSN) as the matching variable. There were 106,779 (47.6%) women and 117,438 (52.4%) men.

Turnover was defined as the number of exits in employment divided by the number of jobs worked during the defined time period (Glover, 2002). If an individual exited all jobs worked, the turnover rate would be 1.0. Table 3 shows the means and standard deviations of the turnover rates by participation status before and after program exit.

We defined employment industry using the North American Industry Classification System (NAICS, 2004) code of the primary employer in each quarter. Most previous studies used the Standard Industrial Classification (SIC) system to identify industries, however, since then all states have adopted NAICS. Primary employer is defined as the employer that paid the individual the largest proportion of wages in a quarter, regardless of the number of employers. For purposes of our study, living-wage industries are defined as those which pay, on average, wages higher than 130 percent of the federal poverty guideline for a family of four (Harris, 2003). The UI claims rate reflects the number of participants who filed a UI claim at any time during a quarter, not those who actually received UI benefits.

The success measure was a two-category measure that was assigned a value of one to reflect a success. Individuals received a success rating if they met any of the following criteria:

Results

Earnings

Research on WIA participant outcomes, with the exception of Harris (2002), demonstrated positive earnings change after training. For this reason it was expected that earnings of WIA participants would increase after the termination quarter. The data partially supported the hypothesis. As seen in Figure 1 and Table 4, earnings of WIA adults and the Comparison group increased during the study period. Dislocated Workers had no significant change.

When regression analysis was applied to test for group differences in performance, statistically there was no significant difference in wages based on WIA participation. Thus, WIA participants do not experience additional wage progression beyond what the Comparison group also experienced. A possible explanation is that Wyoming’s relatively strong economy during this time period could have been driving the increase in wages. The controlled variables all significantly influenced post-training wages eight quarters after program termination (see Table 5). The corrected model accounted for approximately one-third of the variance associated with wages after training (R2 = .31).

Turnover

It was hypothesized that the turnover rate would decline for all groups except the Comparison group after program termination. The data supported the hypothesis (see Table 6). The rates for both WIA Adults and Dislocated Workers significantly differed between the two time periods, while the rate for the Comparison group did not (see Figure 2).

When the regression model was applied (see Table 7), neither Adults nor Dislocated Workers had a significantly different turnover rate than the Comparison group when all other variables were controlled for. The other variables all significantly predicted turnover eight quarters after program termination.

Industry

Figure 3 shows the percentage of Adults employed in living-wage industries before and after program exit. Six quarters before program termination, 43.8 percent were not employed in a Wyoming UI covered job, 39.6 percent were employed in low-paying industries, and 16.6 percent were employed in living-wage industries. Eight quarters after exit, 37.3 percent were not employed by a UI covered job, 29.5 percent were in low-paying industries, and the remaining 33.2 percent were in living-wage industries. Table 8 shows the detailed change by industry.

Figure 4 shows the same breakdown for Dislocated Workers. Before exit, 21.8 percent were not found in UI covered jobs, 41.8 percent were in low-paying industries, and 36.4 percent were in living-wage industries. After program termination the percentages were similar (37.3% were in low-paying industries, 34.5% were in living-wage industries, and 28.2% were not found).

UI Claims Rate

As seen in Figure 5 and Table 9, 3.2 percent of Adult WIA participants and 3.6 percent of Dislocated Workers filed a UI claim six quarters before program exit. This compares to only 0.5 percent of the Comparison group. One quarter before exit, 8.8 percent of Adult participants and 15.5 percent of Dislocated Workers filed claims. At four and eight quarters after exit, the Adult percentage had fallen to 3.7 percent and 3.2 percent respectively. The Dislocated Workers percentage was steady at 2.7 percent for both four and eight quarters after exit.

Employment Rate

Figure 6 and Table 10 show the number and percentage of individuals who were employed in Wyoming before and after WIA participation. Six quarters before exit, 20 percent fewer Adult participants were employed than Comparison group members. By eight quarters after exit, only 8 percent fewer Adult participants were employed.

In contrast, the percentage of employed Dislocated Workers six quarters before program exit was higher than the percentage of employed Comparison group members. At eight quarters after exit, Dislocated Workers were still employed more often than Comparison group members, but the difference was only 1.2 percent.

Success

Table 11 shows the number and percentage of WIA participants who experienced a positive outcome after program completion. Over 60 percent either enrolled in a community college, increased their earnings more than the average for Wyoming workers, decreased their turnover rate, or obtained employment in an industry that pays a living-wage. Many of the participants had more than one positive outcome.

Discussion / Suggested Measures for ETA Outcomes

Study findings were mixed. Other studies which showed short-term earnings gains were replicated because wages of both WIA program segments increased after termination from the program. However, their wages did not increase more than the statistical control group.

While WIA participants initially showed a rather large increase in earnings after program termination (see Figure 1); those high earnings started to decline after the second quarter. This could occur if participants stayed with their On-the-Job Training employer for two quarters after program termination and then needed to look for new employment, or if participants faced retention barriers, such as childcare issues. Contrary to expectations, changes in turnover rates cannot be predicted by WIA participation either, even though turnover does significantly decrease after program exit for both participant groups used in the study (see Figure 2).

The industry measure revealed interesting results. The industrial distribution of Adult participants changed considerably after training (see Figure 3). The percentage of individuals employed in low-paying industries decreased. Alternatively, employment in living-wage industries increased. The implications of this are very positive. WIA participation for Adults appears to result in a shift from no or low-paying employment to employment in higher-paying industries.

Dislocated Workers did not fare as well as Adult participants in the industry measure. More Dislocated Workers were employed six quarters before program termination than eight quarters post-program (see Figure 4). Employment in living-wage industries declined. However, employment in low-paying industries also declined. It is possible that Dislocated Workers are leaving the state for employment elsewhere, becoming employed in non-UI covered jobs, or becoming self-employed. Enrollment in a community college or university without concurrently working a UI covered job would also remove them from Wage Records. Outside influences likely affect Dislocated Workers’ outcomes. For instance, Dislocated Workers tend to be firmly established in a job before they become eligible for the WIA program. After program completion they are encouraged to find employment in an area in which they have no experience. They face competition from applicants with more relevant experience for jobs that often pay less than what they originally earned. Frustration with these situations could lead to discouragement and an eventual withdrawal from the labor market. This hypothesis could be tested using longitudinal and inter-state data.

The proportion of Adults who filed for UI benefits remained fairly constant before and after program exit (see Figure 5). However, the rate for Dislocated Workers dropped rather sharply meaning they filed for UI benefits less often post-program than before training. This adds further support to the idea that those workers are either leaving the state for employment opportunities elsewhere, or are becoming discouraged and withdrawing from the labor market.

Before program termination, the WIA participation groups were less likely to be employed than the Comparison group (see Figure 6). After program termination, there is no noticeable difference between the groups. The WIA program appears to be allowing participants to become indistinguishable from the average Wyoming worker, and that is an important accomplishment. As previously discussed, after program completion the employment rate for Adults increased. This was not the case for either Dislocated Workers or the Comparison group, both of which showed decreases in the percentage of individuals employed in Wyoming.

Findings in the success measure are most interesting. WIA participation appears to be very effective in improving the economic lives of participants. For example, 65 percent of Dislocated Workers experienced at least one positive outcome from participation; many of those experienced more than one positive outcome. The same is true for Adults. Except for Adults who enrolled in a Wyoming community college, who tended to withdraw from the labor market during their education, very few participants had only one positive outcome. The majority of Adults with a successful outcome increased their wages, decreased their turnover rate, and gained employment in a living-wage industry. In essence, they epitomize the unwritten goals of the WIA program.

Limitations

Wyoming has the smallest population in the nation. Because of the small size, findings are not generalizable to other states. In addition, it is impossible to perform detailed analyses, such as the effects of different kinds of training, because the sample size is too small. A remedy is to combine data for several states, primarily bordering states, so that a regional picture of WIA training could be built. That kind of analysis would be fairly easy to complete for Alaska, Nebraska, New Mexico, South Dakota, and Wyoming because the available wage records and demographic data are comparable for those states.

Another limitation is the lack of qualitative program knowledge of WIA. Because the analysts are not case managers, there is no firm understanding of common participant issues that affect the outcomes. For instance, if transportation availability is a common obstacle, the number of training opportunities is limited and outcomes could be affected. In order to give broader coverage to those types of issues, a stronger relationship needs to be developed between research and program staff.

Finally, there are limitations inherent in a quasi-experimental design. It is impossible to completely match the control group to the WIA participant group on only the simple demographic and economic variables available in administrative databases. There are likely many other important variables that were not controlled for because of limitations in the administrative databases. Also, there are some issues that can only be addressed in longitudinal analysis.

Summary

While the capacity to conduct true experimental design and replication is limited, this study offers a powerful alternative analysis. By comparing labor market experience between WIA participants and a comparable segment of Wyoming’s workforce, it was determined that WIA participants basically became indistinguishable from the rest of Wyoming’s workforce, an arguably favorable outcome. However, the Wyoming study shows that no program effect exists for wages or turnover, at least in the 24 months following program completion. In the absence of a strong economy, participants may have failed to become indistinguishable. Training programs likely have difficulty showing success in economically challenged environments. Additionally, the federally mandated outcome measures fail to reflect all of the successes of program completers. ETA should consider changing the focus of WIA program evaluation to measures that capture relevant information pertinent to employers, workers, and program managers.

References

Bartik, T. (1997). Jobs for welfare recipients. Kalamazoo, MI: Upjohn Institute for Employment Research.

Benus, J. M. & Byrnes, R. M. (1993). The St. Louis metropolitan re-employment project: An impact evaluation. Washington, D.C.: U.S. Department of Labor.

Bloom, H.S., Orr, L. L., Bell, S. H., Cave, G., Doolittle, F., & Lin, W. (1994). The national JTPA study: Overview of impacts, benefits, and costs of Title II-A. Bethesda, MD: Abt Associates, Inc.

Bloom, H.S., Orr, L. L., Bell, S. H., Cave, G., Doolittle, F., Lin, W., & Bos, J. M. (1997). The benefits and costs of JTPA Title II-A programs. The Journal of Human Resources, 32(3), 549-576.

Bureau of Labor Statistics. (2004). North American Industry Classification System (NAICS) at BLS. Retrieved April 14, 2004, from: http://www.bls.gov/bls/naics.htm

Campbell, D. P., & Boruch, R. F. (1975). Making the case for randomized assignment to treatments by considering the alternatives: Six ways in which quasi-experimental evaluations in compensatory education tend to underestimate effects. In Carl A. Bennett & Arthur A. Lumsdaine (Eds.), Evaluation and Experiment (pp. 195-296). New York, NY: Academic Press.

Couch, K. A. (1992). New evidence on the long-term effects of employment programs. Journal of Labor Economics, 10(4), 380-388.

Eberwein, C., Ham, J. C., & LaLonde, R. (1997). The impact of being offered and receiving classroom training on the employment histories of disadvantaged women: Evidence from experimental data. Review of Economics Studies, 64, 655-682.

Finn, M. A. & Willoughby, K. G. (1996). Employment outcomes of ex-offender Job Training Partnership Act (JTPA) trainees. Evaluation Review, 20(1), 67-83.

Friedlander, D., Greenberg, D. H., & Robins, P. K. (1997). Evaluating government training programs for the economically disadvantaged. Journal of Economic Literature, 35, 1809-1855.

Friedlander, D., & Hamilton, G. (1996). The impact of a continuous participation obligation in a welfare employment program. Journal of Human Resources, 31(4), 734-756.

Gallagher, T. (2001, July). When does training pay off? Challenging the assumptions of the Workforce Investment Act. Wyoming Labor Force Trends, 1, 9.

Girt, R. (2003). [State workforce development, vocational and educational programs exceed federal benchmarks]. News Release.

Glover, T. (2002). Market dynamics from administrative records; Seven state project report for December 2002. Casper, WY: Wyoming Department of Employment.

Glover, T. (2003). [Retention and wages]. Unpublished raw data.

Harris, M. A. (2002). Examining the impact of training programs on wages through the use of quasi-experimental design. Unpublished.

Harris, M.A. (2003, May). Identifying potential living wage employment growth opportunities in Wyoming. Wyoming Labor Force Trends. Retrieved on April 6, 2004 from http://doe.state.wy.us/LMI/0503/a1.htm

Heckerman, J., Heinrich, C., & Smith, J. (1997, May). Assessing the performance of performance standards in public bureaucracies. The American Economic Review, 389-394.

Hollenbeck, K. & Anderson, W. (1993).
A net impact analysis of adults, job-specific training programs funded by the Ohio Department of Education (Tech. Rep. No. 93-003). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.

Jacobson, L. S., LaLonde, R., & Sullivan, D. (1994). The returns from classroom training for displaced workers. Chicago, IL: Federal Reserve Bank of Chicago.

Jacobson, L. S., LaLonde, R., & Sullivan, D. (1999). Measures of program performance and the training choices of displaced workers. Unpublished manuscript.

James Bell Associates, Inc. (1997). Involving employers in training: Best practices. Washington, D.C.: U.S. Department of Labor.

Jones, S. D. (2001). Demographic analysis of employee turnover in the transportation industry: Fourth quarter, 1999. Wyoming Labor Force Trends. Retrieved April 6, 2004 from http://doe.state.wy.us/LMI/0801/a2.htm

Jones, S. D. (2002, November). Defining residency for the Wyoming workforce. Wyoming Labor Force Trends. Retrieved September 21, 2004, from http://doe.state.wy.us/lmi/1102/toc.htm

Kodrzycki, Y. K. (1996, July/August). Laid-off workers in a time of structural change. New England Economic Review, 3-26.

LaLonde, R. J. (1995). The promise of public sector sponsored training programs. Journal of Economic Perspectives, 9(2), 149-168.

Leigh, D. E. (1997). Does training work for displaced workers? Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.

Mishel, L., Bernstein, J., & Schmitt, J. (1999). The state of working America, 1998-1999. Washington, D.C.: Economic Policy Institute.

Murray, S. (Ed.) (2002). Market dynamics from administrative records: Seven state project report for December 2002. Retrieved April 6, 2004, from http://doe.state.wy.us/LMI/w_r_research/MarketDynamics1202.pdf

Rosenbaum, P. R., & Rubin, D. B. (1985). Constructing a control group using multivariate matched sampling methods that incorporate propensity score. The American Statistician, 39 (1), 33-38.

United States Department of Labor. (2001). 2001 Strategic Plan. Retrieved March 15, 2004 from http://doe.state.wy.us/LMI/0701/0701.pdf

United States Department of Labor, Employment and Training Administration. (2003, December). Training and Employment Guidance Letter No. 15-03.

United States General Accounting Office. (1991). Designing evaluations. (GAO/PEMD-10.1.4). Washington, DC: U.S General Accounting Office.

United States General Accounting Office. (1996). Job Training Partnership Act: Long-term earnings and employment outcomes. (GAO/HEHS-96-40). Washington, DC: U.S. General Accounting Office.

United States General Accounting Office. (2004). Workforce training. (GAO-04-282). Washington, DC: U.S. General Accounting Office.

These pages designed by Susan J. Murray.
Last modified on by Valerie A. Davis.