Over 5200 students (K-3) from sixty four district school boards across the province of Ontario participated in the 2014 Summer Learning Program (SLP). Through a mix of high quality instructional programming and recreational activities, the SLP continued to help elementary students reduce summer learning loss and improve their literacy and numeracy skills over the summer. For 2015, the program will expand and continue to be led by the Council of Ontario Directors of Education (CODE). The Student Achievement Division will once again collect and analyze program data to measure ongoing impact. The longitudinal research study conducted by our university partners with six district school boards is ongoing as well.
Program types offered in the summer of 2014 included those with programming to support focused learning in the following categories:
–Literacy and Numeracy Blend
–First Nations, Métis, and Inuit
Students participating in the programs completed pre- and post-assessments using the STAR computer-adaptive literacy, numeracy and early learning assessment tools.
What is the impact on learning?
An analysis of pre- and post-program student assessment data collected during the summer of 2014 indicates not only a reduction in learning loss for students who participated in the Summer Learning Program, but evidence of gains made as well. A view of all participating students showed a statistically significant average improvement between the start and end of the program, with average gains in reading and mathematics of approximately three weeks.
More good news can be found by digging deeper into specific program types. For example, pre- and post-test scores in reading show that students enrolled in literacy program classes made average gains in reading of approximately five weeks in instructional time. Students enrolled in literacy/numeracy blended program classes made similar gains in reading as well as average gains in mathematics of approximately seven weeks in instructional time.
Overall, students identified as English Language Learners (ELL) made lower gains than other students in most programs with the exception of those who participated in Literacy and Numeracy Blend programs. There were no significant differences in achievement changes between girls and boys, students self-identified as First Nations, Métis & Inuit learners and other students, or students with and without an IEP.
What is the impact on early learning?
Students in kindergarten were assessed using the Early Learning STAR Assessment. Based on this assessment, students were categorized into four groups of readers: 1) Early Emergent Reader; 2) Late Emergent Reader; 3) Transitional Reader; and 4) Probable Reader.
While most students did not move between levels (210 students, 57.2%), a statistically significantly greater number moved up (97 students, 26.5%) compared to moving down (60 students, 16.4%). Participating students did show a statistically significant average gain between pre- and post- assessments in the early numeracy sub-domain within the Early Learning Assessment.
Does the timing of test dates impact achievement?
The majority of boards participating in the 2014 Summer Learning Program held program assessment dates on the first and last day of programming. This pre- and post-test timing protocol was shared by 86.5% of participating boards. Alternatively, 5.8% of participating boards pre- and post-tested students at the end of June and early in September, while another 7.7% of participating boards had non-standard pre- and post-testing dates. No significant differences were found in the analysis of achievement gains between these three different types of testing protocols.
Methodology used in the Analysis
• Data from participating students (Grades 1-4) with both pre- and post-assessment results were used in analysis
• Normal curve equivalent (NCE) scores were used in analysis. These scores are designed to be used for most statistical analyses.
• Pre/post changes and differences in average achievement were assessed using t-tests and ANOVAs:
– Dependent samples t-tests to examine achievement change within groups
– Independent samples t-tests to compare achievement changes between two groups (e.g. males/females)
– Multinomial difference tests examined statistical significance of category changes for participating kindergarten students who had both pre- and post-assessment results.