Successmaker Math And ReadingSm math pilot report PEARSON SUCCESSMAKER MATH PILOT STUDY12109 No part of this report may be reproduced or used for any purpose without permission from Gatti Evaluation Inc. Guido G.Gatti Gatti Evaluation Inc. 162 Fairfax Rd. Pittsburgh, PA 15221 (412) 3719832 Manager Kate Giordano Gatti Evaluation Inc. Funded By For Information Please Contact: Marcy Baughman of Academic Research (724) 8631621 OF CONTENTS EXECUTIVE SUMMARY 13 I. INTRODUCTION 46 a.Instructional technology ; 45 b.Study Goals and Research Questions &#;...56 II. METHODOLOGY > a.Site Recruitment &#; 78 b.Study Sample &#; 811 c.Site Descriptions &#;.. 1115 d. Study Design &#; 1516 e.Program Training &#; 1617 f. Teaching Logs and Classroom Observations ..&#; 1719 III.RESULTS > a.Statistical Analysis of Outcome Measures &#; 1923 b. Baseline Achievement and Attitudinal Measures &#; 2324 c. SuccessMaker Math Achievement Gains &#> d.SuccessMaker Gains by Subpopulations &#; 2838 e. SuccessMaker Gains by Implementation Style &#; 3839 f.Group Comparisons of Achievement Gains &#; 3943 g. Group Comparisons by Subpopulations &#;.. 4448 h.Teacher and Student SuccessMaker Opinions &#; 4866 i.SuccessMaker math Program Usage &#. DISCUSSION 6970 A.1 Teacher Implementation Rating Results _________________________________ 72 A.2 GMADE Standard Score to Percentile Rank Correspondence ________________ 73 A.3 SuccessMaker Math vs. Comparison Group Model Adjusted Grade Differences by Subpopulation Graphs _____________________ 7490 SuccessMaker Math Pilot Gatti Evaluation Inc. 12109  1  EXECUTIVE SUMMARY Pearson partnered with Gatti Evaluation to conduct rigorous research to support the assertion that the SuccessMaker Math computer based learning program effectively increases student mathematics achievement and attitudes. The program was evaluated in fortythree diverse elementary grade classrooms from NJ) during the 200809 school year. Students in SuccessMaker schools made regular use of the program while students in comparison schools received supplemental instruction from noncomputerized district adopted supplemental mathematics programs. The study schools come from public schools show considerable variation in ethnicity, students eligible for reduced priced lunch, as well as a wide range of ability with respect to mathematics and reading achievement. The evaluation team sought out diversity in the study sample to ensure the program would be used by ity that is today’s elementary classrooms. The final study sample consisted of 408 3 grade (i.e., SuccessMaker = 230, comparison = 178) grade (i.e., SuccessMaker = 225, comparison = 159) students. A challenging assessment battery was group administered to studeschool year. The assessment Group Mathematics Assessment and (GMADE), and the mathematics attitude survey developed by the principal investigator where ststions regarding general math attitude, confidence, motivation, and self perceived aptitude. Of the 879 students tested at baseline, 90% remained in the final study sample (i.e., SuccessMaker = 94%, comparison = 85%). Quantitative data collected from site observations indicated that the studyteachers executed their mathematics instruction at similarly high levels with the vast majority to scores were statistically study groups, with the comparison group higher for the 3sample and the SuccessMaker group higher for the 5 grade sample. In both cases the differences are 0.21 standard deviations, a small to moderate sized baseline difference. RQ3.How did teachers and students react to the SuccessMaker Math program? Focus group sessions were conducted at each school during site visits between mid November and early February and again between April and early June. These evaluators with the following insights into teacher and student experiences with the program. Teachers and students have become very comfortable very quickly with the SuccessMaker program and feel the program was a good educational investment.When inresponse to the program was overwhelmingly positive with 81% of the 475 recorded comments being positive in nature. Teachers appreciated the reporting system in informing classroom for remediation, monitoring studerelaying student progress information to curriculum specialists and parents. A majority of teachers felt the initial placement and adaptive motion of students through the program was effective, the learning activities were well differentiated and aligned to their current curricula and SuccessMaker Math Pilot Gatti Evaluation Inc. 12109state educational objectives, and that the program challenged both their lower and higher rare minor technical issues (ex., logging in, activities loading), most likely a result of Teachers firmly believe that their students like using the program and feel that the program makes the learning process more fun. When formally interviewed, teachers were overwhelmingly positive about their students’ interactions with the program. Of the 108 recorded comments, 81% were positive in nature. When surveyed, 90% of 3 liked the program. Studentcharacters, animation, and found the learning activities engaging, especially the speed games. Some students did find the characters immature and the animation sometimes excessive and nts appreciate the capacity of the program to allow them to rtual learning environment. RQ4 How was the SuccessMaker Math program utilized? Proper implementation was supported by the teacher logs as 3 grade SuccessMaker teachers reported Successmaker math efficacy report final Principal InvestigatorGuido G.Gatti Gatti Evaluation Inc. 162 Fairfax Rd. Pittsburgh, PA 15221 412 > CoPrincipal Investigator Katya Petrochenkov Gatti Evaluation Inc. Primary Stakeholder Funded By Pearson For Information Please Contact: Marcy Baughman Director of Academic Research 724 8631621 Math RCT Gatti Evaluation Inc. 91510 TABLE OF CONTENTS EXECUTIVE SUMMARY 14 I. INTRODUCTION 56 a.Instructional Technology Literature... 5 b.Study Goals and Research Questions ...6 II.METHODOLOGY .Student Outcome Measures7 b.Teacher Measures...9 c. Site Recruitment and Selection. 11 d.Math Instruction .... 13 e.SucessMaker Implementation....15 e. Settings.18 f.Participants....31 g.Data Analysis Procedures.. 32 III.RESULTS > a.Baseline Group Equivalence.34 b. Group Comparisons of Achievement Gains. 38 c.Group Comparisons by Subpopulation..43 d.Student Academic Attitudes 49 f.Teacher and Student SuccessMaker Opinions..52 IV. DISCUSSION 5859 A.1 Comparative Study Group Results by Program Usage ______________________ 3943 Table 1 Gatti Evaluation SuccessMaker Math Study Site State Assessment Information 12 Table 2 SuccessMaker Math RCT Training Dates.16 Table 3 Gatti Evaluation SuccessMaker Math RCT Sample Demographic Information...29 Table 4 Third Grade Baseline GMADE Scores Study Group Comparisons..34 Table 5 Fifth Grade Baseline GMADE Scores Study Group Comparisons...35 Table 6 Seventh Grade Baseline GAMDE Scores Study Group Comparisons..35 Table 7 Baseline Math Academic Survey Score Comparisons.. 35 Figure 1 GMADE Total.36 Figure 2 GMADE Concepts and Communication.36 Figure 3 GMADE Operations and Computation... 37 SuccessMaker Math RCT Gatti Evaluation Inc. 91510 Figure 4 GMADE Process and Applications.. 37 Figure 5 Math Academic Attitude Survey.50 Figure 6 Do You Like SuccessMaker Math?52 Figure 7 Do You Like It When the Characters Sing and Dance?....53 SuccessMaker Math RCT Gatti Evaluation Inc. 91510 1  EXECUTIVE SUMMARY Pearson partnered with Gatti Evaluation to conduct rigorous research to support the assertion that the SuccessMaker Math computer based learning program effectively increases student mathematics achievement and attitudes. The program was evaluated in sixtythree diverse elementary and middle grade classrooms from ten schools in seven different states i.e., AZ, AR, CA, IN, KS, NY, PA during the 200910 school year. Students in classrooms randomly assigned to use SuccessMaker made regular use of the program while students in comparison classrooms received supplemental instruction from noncomputerized supplemental mathematics programs. Four widelyused classroom mathematics programs were utilized by the sites at 3rdand 5th grade, and three different programs were utilized at 7th grade. The study schools come from public school districts located in large cities or suburbs of large cities. The study schools show considerable variation in ethnicity, students eligible for reduced priced lunch, as well as a wide range of ability with respect to mathematics and reading achievement. The evaluation team sought out diversity in the study sample to ensure the program would be used by learners of all abilities and backgrounds, thus reflecting the reality that is todays elementary classrooms. Five schools began the study in the first month, three began in the third month, one in the fourth and the last in the fifth month of the school year. The final study sample was large, consisting of 505 3rd grade i.e., SuccessMaker = 282, comparison = 223, 408 5th grade i.e., SuccessMaker = 224, comparison = 184 and 273 7th grade i.e., SuccessMaker = 136, comparison = 137 students. A challenging assessment battery was group administered to students at baseline and again at the end of the school year. The assessment battery consisted of the Group Mathematics Assessment and Diagnostic Evaluation GMADE, and the mathematics attitude survey developed by the principal investigator where students respond to selfreport questions regarding general math attitude, confidence, motivation, and selfperceived aptitude. Comparisons on assessment outcomes were made between study groups using model adjusted endofyear raw score group mean differences. Adjusted group mean differences are calculated holding the effects of confounding variables constant for both groups. The equating of confounding variables and the maintaining of consistent implementation ensures the outcomes may more confidently be attributed to the study conditions randomly assigned to these groups. Results were broken out and analyzed separately for each GMADE subtest i.e., Concepts and Communication, Operations and Computation, and Process and Applications. Results were also broken out and analyzed for separate levels of five key student populations i.e., English proficiency, ethnicity, gender, meal status, math ability. Further, the performance for the comparison group was compared to four blocks of program usage i.e., block 1 = 1 to 9 hours, block 2 = 10 to 19 hours, block 3 = 20 to 29 hours, block 4 = 30 or more hours. RQ: How did teachers and students react to the SuccessMaker Math program? Focus groups were conducted at each school during site visits between April and early June. These sessions provided the Tags: successmaker math and reading,math and reading lesson plans,free math and reading games,math and reading worksheets,math and reading test,math acuity mcgraw hill,acuity math and reading,acuity math assessment,math acuity website 
New Topics
