U. S. Department of Education Office of Planning, Evaluation and Policy Development
|
|  Yüklə 1.39 Mb.
|
B. Context for Interpreting Program Effects Figure I.1 outlines a conceptual framework that indicates how Upward Bound projects structure recruitment and services based upon participant characteristics and policy requirements, which contribute to effects on intermediate and long-term student outcomes. The population under study (furthest-left box in Figure I.1) are high school students who are low-income (family income under 150 percent of the poverty line as defined by the Census Bureau) or potential first-generation college students (neither parent with a bachelor’s degree) and therefore eligible for Upward Bound. Within this population, students were randomly assigned to a treatment group or a control group; all students were allowed to utilize other supplemental services available in their schools and communities.
Upward Bound is designed to help disadvantaged students complete high school and to enter and succeed in postsecondary education. Outcomes related to both of these goals are presented in Figure I.1. Previous reports have examined student outcomes through high school and focused on measures associated with progress on the path toward college completion. Both Upward Bound and intermediate high school outcomes affect the long-term outcomes related to college, such as high school completion, postsecondary application, enrollment, persistence, and completion; postsecondary enrollment and completion are the focus of this report. To interpret the estimated effects of Upward Bound presented in this report, it is important to understand that these effects (1) are indicative of the “value-added” of Upward Bound relative to other programs in which students participate, (2) are based on students who chose to apply to Upward Bound, (3) are based on students who participated in Upward Bound in the mid-1990s, and (4) are based on students who chose to participate in Upward Bound for various lengths of time.
1. Value-added of Upward BoundThis report provides estimates of the value-added of regular Upward Bound above and beyond other available precollege programs and services. Because eligible applicants were randomly assigned to the treatment and control groups, and because—with very few exceptions—only treatment group members were offered the opportunity to participate in regular Upward Bound, the differences between the two groups provide valid estimates of the value of that opportunity relative to the opportunities for participation in other programs. Many Upward Bound projects operate in service-rich environments, and the kinds of students who are eligible for Upward Bound may also participate in other precollege programs (see Appendix H for other precollege supplemental service programs attended by sample members during the same time period, as reported in our surveys). Many of the treatment and control group members participated in precollege services other than regular Upward Bound. In fact, it is critical to the scientific validity of the study that treatment and control group members had the same opportunity to pursue other services as the typical eligible applicant to regular Upward Bound. A Like students in the control group, some treatment group members received supplemental services beyond those offered by Upward Bound. According to the treatment group members themselves, 25 percent attended instructional and tutoring sessions outside of Upward Bound; 20 percent participated in a program with a math or science emphasis; 11 percent participated in Upward Bound Math-Science; and 7 percent participated in Talent Search. While about half of treatment group members would have received precollege services if they had not been given the opportunity to participate in regular Upward Bound (based on the rate for control group members), most would not have participated in programs that are as intensive as regular Upward Bound. For example, while Upward Bound spends more than $4,700 per participant annually, Talent Search and GEAR-UP, two other large Department of Education precollege programs aimed at low-income and disadvantaged youths, spend about $400 and $300 per participant annually. With much higher spending, Upward Bound is able to offer many academic and nonacademic activities during both the academic year and the summer. Specifically, we found that on average, Upward Bound participants attended about 265 academic sessions over their entire Upward Bound career (see Myers et. al. 2004, Table II.2); 174 of the sessions occurred during the summer program and 91 sessions occurred during the academic year. Sessions in English, math, and science courses constituted the bulk of participants’ Upward Bound academic course work. In addition to the academic course work completed through the Upward Bound program, participants engaged in a variety of nonacademic activities. The most common activities attended, as reported by Upward Bound projects, focused on counseling, followed by skills development and college preparation courses. On average, participants attended 212 activity sessions while in Upward Bound, with nonacademic activities split nearly equally between the summer and the academic year. Not only are the treatment group members more likely to receive services, it appears that these services are generally much more intensive (with the exception of control group members who participated in programs such as Upward Bound Math-Science). 2. Upward Bound ApplicantsA comparison of overall postsecondary enrollment rates of Upward Bound applicants with national enrollment rates indicates that Upward Bound attracts students who are much more likely to enroll in postsecondary education than are similarly disadvantaged students; in fact, Upward Bound applicants are just as likely to enroll in postsecondary education as the average eighth-grader in the United States, regardless of socioeconomic status. Data from the National Education Longitudinal Study’s 1988 sample (NELS:88) reveal that of all students in eighth grade in 1988, 76 percent had attended at least one postsecondary institution by 2000, that is, by about eight years after scheduled graduation from high school (Ingels et al. 2002). Of students in the lowest quartile of socioeconomic status, 52 percent had enrolled in some postsecondary education; of students whose parents did not attend college, 56 percent had enrolled in some postsecondary education. These national rates of postsecondary enrollment for disadvantaged students are approximately 25 percentage points lower than the 81 percent of control group members in the Upward Bound evaluation who attended a postsecondary institution within approximately seven to nine years after scheduled high school graduation (based on fifth follow-up survey responses). 3. Effects for Students Who Participated During the Mid-1990sRecognizing that the treatment and control group members applied to participate in regular Upward Bound during the mid-1990s, this report characterizes the effects of Upward Bound as it operated at that time. To the extent that the types of services provided by regular Upward Bound and other precollege programs and the types of students served by these programs have not changed much since the mid-1990s, this report may provide a reasonable assessment of the effects of Upward Bound as it operates today. However, there have been changes in the universe of Upward Bound grantees during the past 15 years, both in terms of new grantees and grantees that no longer operate Upward Bound programs. There have also been changes in student targeting. These program changes suggest that findings based on students who participated in the mid-1990s may not be directly applicable to current participants. Considering positive subgroup impacts in previous reports, the Department of Education set aside $18 million for a Participant Expansion Initiative as a way to encourage Upward Bound projects to serve more higher-risk students. The 2003 initiative allowed grantees to apply for additional funds if they served at least one target school in which at least 50 percent of the students were eligible for a free lunch and wanted to increase the number of participants from such schools who had the “greatest need” for project services. Three indicators of “greatest need” could be used: a participant’s (1) not meeting state standards for eighth-grade proficiency in reading or language arts, (2) not meeting state standards for eighth-grade proficiency in math, or (3) having a GPA of 2.5 or lower in the most recent academic year. Projects could receive $100,000 for serving 20 additional qualified students, $75,000 for 15 students, and $50,000 for 10 students. Priority was given to the approximately 180 projects that had received funds under a prior expansion initiative that began in 2000, but many more projects received the expansion funds as a result of this initiative—219 in 2003, 259 in 2004, and 256 in 2005. In another effort to improve overall program effectiveness, the Office of Postsecondary Education issued rules in 2006 that could substantially modify the composition of new students admitted to Upward Bound. In making the grant awards, priority would be given to applicants who agreed to select all new participants from otherwise eligible students who are in or about to enter ninth grade, and reserve at least 30 percent of these first-time participant slots for students at “high risk of academic failure.” Students would qualify as high risk if they (1) do not meet the proficient level on state assessments for eighth-grade reading or language arts, (2) do not meet the proficient level on state assessments for eighth-grade math, or (3) have a GPA of 2.5 or lower in the most recent academic year. Although the evaluation findings indicated that Upward Bound may have the greatest impact on students with lower educational expectations, it was decided that expectations would be difficult to assess accurately for determining eligibility among entering ninth-grade students and that low grades and performance on state assessments may serve as a proxy for lower educational expectations. Requiring students to begin during or before ninth grade is intended to make it possible for all newly admitted students to participate for four full years. Previous findings from a nonexperimental analysis conducted as part of this evaluation suggested that the longer students stay in the program, the larger are some impacts. Along with the revised targeting initiatives, a new evaluation to examine the effects of these initiatives was started in 2007, though the evaluation was terminated in 2008.
4. Variation in Exposure to Upward BoundIn considering the effects of Upward Bound, it is important to recognize that students’ Upward Bound experiences vary in the length of participation and, as a result, in the amount of services received. The experience of treatment group members can be summarized from previous evaluation reports (see Myers et. al. 2004); no additional data have been collected on the Upward Bound experiences of treatment group members since the previous report because that report was based on data collected after sample members would have finished high school. Approximately 84 percent of treatment group members (unweighted) received some Upward Bound services and can be classified as “participants” (Table I.3). The participation rate was similar across most subgroups, with two exceptions: the subgroups defined by Upward Bound eligibility (low-income and/or first-generation) and the subgroups defined by ninth-grade grade point average. The lack of a statistical difference between the subgroups defined by likelihood of being selected to participate in Upward Bound suggests that expanding the applicant pool for the evaluation did not cause the no-show rate to be artificially higher. Conditional on any participation, the median length of time participants remained in Upward Bound was 20 months, with 64 percent participating for at least one year, 35 percent participating for at least two years, and 14 percent participating for at least three years. About 39 percent completed the program, which we defined as still being in the program in the spring of their senior year in high school. The services participants received in Upward Bound included academic and nonacademic sessions during both the school year and the summer. Although Upward Bound participants typically attended a substantial number of academic sessions, there was variability in the number of sessions attended, just as there was in the duration of participation. For example, while one-quarter of participants attended 104 or fewer Upward 
Bound sessions, another quarter attended 405 or more sessions. Like the academic sessions, we found variability in the number of nonacademic activities attended by Upward Bound participants: about 25 percent of the participants attended 74 or fewer sessions and about 25 percent attended 293 or more sessions. As might be expected, the amount of services received by students is directly related to the duration of participation. The average student who completed the program participated for about 50 percent longer than the average participant and received about 50 percent more services. Chapter V of this report provides estimates of the effects of longer participation and program completion. |
fter applying to Upward Bound, nearly half of the control group members reported participating in some kind of supplemental services in high school (Table I.1). The most common type of supplemental service received by control group members was instructional and tutoring sessions (33 percent), followed closely by programs with a math or science emphasis (31 percent). We also found that 11 percent of control group members reported participating in an Upward Bound Math-Science program and 9 percent reported participating in Talent Search. Control group members were more likely to participate in supplemental services during the academic year than the summer (46 percent versus 20 percent).
Due to participation in Upward Bound, treatment group members received substantially more precollege services than did control group members. More than four out of five treatment group members received services