Learning Cultures School-Wide Pilot in ELA Classrooms: P.S. 126/Manhattan Academy of Technology, Pre-K-8


by cynthia mcccallister

I began working at the Jacob Riis School in Fall 2007, when the principal, disillusioned with the TC Writing program in place at the school, recruited me to implement Genre Practice, a new and innovative writing approach that I invented. The model was so successful in harnessing engagement and motivation in students that the principal asked me to implement my version of a reading program the following year.

This is how, at the Jacob Riis School, a Pre-K through 8th grade public school in Lower Manhattan, the Learning Cultures model came to be was piloted from Fall 2007 to December, 2010, and where it demonstrated consistent growth in state ELA scores every year of implementation. The Jacob Riis School comprises an elementary and middle school. School census data from both the K-5 and middle schools appear in the Figures below.

 

Elementary School

Middle School

Free Meal

87.63%

48.99%

Reduced Meal

6.18%

8.41%

Full Meal

6.18%

42.61%

Asian

57.6%

36.4%

Black

10.5%

13.1%

Hispanic

29%

22%

White

2.6%

29%

Figure 1: K-5 School Census Data, Poverty & Race (2010

Implementation Phases

From Fall 2007 to December 2010, the program was implemented in phases as described below:

  • Phase I: Fall 2007-Spring 2008: Writing Work Time (free choice writing topics), Writing Conferences and Writing Share.
  • Phase II: Fall 2007-Spring 2009: Incorporation of Reading Work Time (daily reading responsibilities of familiar rereading and free choice reading or centers), Conferences, Cooperative Unison Reading, and Content Share.
  • Phase III: Fall 2009-Winter 2010: Program improvement strategies through targeted teacher evaluation and professional development.

 Comparative longitudinal evidence of program effectiveness

In a comparative time-series analysis of the Jacob Riis school and 10 other elementary and middle schools where Learning Cultures was not implemented, achievement on state ELA and math scores were compared from 2006-07 to 2011-12 (coding inconsistencies in 2011-12 are unresolved). The five elementary schools were selected for demographic similarity to the Jacob Riis student population. Five middle schools from district were selected, two of which were similar demographically, and the others reputable as high quality schools (See Figures 2-5).

Middle School Math Analysis: In every year of the implementation, math scores improved at significant rates. Not only is every value significant, but there is also a consistently high effect size in each year, and the effects remain constant across years. The Jacob Riis gains are strong in relation to other schools. For example, the 11.92 increase in Jacob Riis is slightly less than the 12.01 increase in School 6, but School 6 doesn’t demonstrate as high a level of consistency across the study. When Jacob Riis Middle School gains are compared to School 3, which also shows significance across years, the magnitude of effect sizes are smaller in the School 3 than they are in Jacob Riis data.

Elementary School Math Analysis: Elementary math achievement at Jacob Riis was statistically significant the first two years of the study, with large effect sizes. The initial gains are maintained across the course of the study. In comparison to School 1, which also demonstrated significance in the first two years, Jacob Riis gains improved over time and were more consistently large. School 2 and 3 both showed significance in isolated years.

Middle School ELA Analysis: Middle school students made significant gains in the second year of the implementation, when high fidelity was achieved at the middle school level. Also, the data shows steep inclines of 23 and 18 points for ELLs in 06/07-07/08 and 07/08-08/09. Schools 2 and 3 both show significant gains for ELLs in first two years of study. But both schools 2 and 3 show significant decreases in scores in 08/09-09/10, whereas Jacob Riis initial gains are maintained. 

Elementary School ELA Analysis: In elementary school, we see a significant increase in ELA scores in the first year of implementation with high effect sizes. These gains are maintained through the study. we don’t see a significant increase in ELL achievement, which is surprising since one would expect an effect size of 21.1 demonstrated the first year would be significant. However, lack of significance might be due to the small number of students in the ELL subgroup.

Figure 6: Middle School ELA without ELLs

Longitudinal evidence of program effectiveness
The figure below displays school-level data that can be used to make judgments of the effectiveness of the K-8 intervention in reading/writing/English language arts. Data were retrieved from the annual school report card (http://schools.nyc.gov/Accountability/tools/report/default.htm). Most of the DOE progress metrics derive from student achievement scores on the state assessments. During the course of the implementation, the state assessment underwent revisions. Moreover, the school report card was piloted in 2007, and reporting categories and scoring formulas underwent modifications throughout the course of the Learning Cultures program implementation. Any data presented below must be interpreted in light of the shifting accountability context at the state and system levels. But even so, the data demonstrate a general steady upward progression in all categories across all years, with the exception of the School Environment category (it is beyond the scope of this article to provide an account of the challenges of securing school-wide faculty buy-in within the context of an ambitious, school-wide reform effort. These challenges might partly explain declining environment ratings).

Category

2006-2007

2007-

2008

2008-

2009

2009-

2010*

2010-

2011

Student Progress

21.4 out of 55

42.7 out of 60

54.9 out of 60

44.9 out of 60

41.1 out of 60

Category, cont.

2006-07

2007-08

2008-09

2009-10

2010-11

Student Performance

18.0 out of 30

17.8 out of 25

22.5 out of 35

7.7 out of 25

13.7 out of 25

School Environment

8.3 out of 15

8.4 out of 15

5.7 out of 15

7.7 out of 15

6.0 out of 15

Closing the Achievement Gap

Not reported

Not

reported

Not reported

Not

reported

7.0 out of 15

Attendance

95%

90%

95.7%

96%

95.5%

ELA: ELLs: Percent at 75th Growth Percentile or Higher in ELA

13.0%

35.7%

18.8%

59.0%

44.3%

ELA: Lowest Third Citywide: Percent at 75th Growth Percentile or Higher

 

 

   ----

36.0% (non Black or Hispanic)

33.3% (non Black or Hispanic)

58.7%

71.1%

ELA: Self-Contained/ICT/SETSS: Percent at 75th Growth Percentile or Higher

17.2%

30.9%

38.1%

39.3%

52.9%

ELA: Black and Hispanic Males in Lowest Third Citywide: Percent at 75th Growth Percentile or Higher

B: 17.9%

H: 32.5%

B: 40.7%

H: 42.9%

B: 43.8%

H: 45.5%

Not reported

65.7%

Figure 2: Jacob Riis Report Card Data, 2008-2011
Each year during the Bloomberg Administration, the New York City Department of Education issued school report cards with overall grades, scores on student performance and progress, and the number of additional credit points awarded to the school for its record of performance in meeting the needs of subgroups of students including English language learners, special education students, and students who performed in the lowest third academically.


*Note: In 2009-2010 New York State adjusted the cut scores on the state ELA exam. That year, scores across schools in NYC trended down dramatically.

Report Card Category

2006-2007

(Prior to Learning Cultures curriculum)

2007-2008

(After implementation of Learning Cultures in Writing)

2008-2009

(After implementation of Learning Cultures Reading and Writing)

2009-2010

(After implementation of Learning Cultures in reading and writing)

Overall Results

C

A

A

A

Student Performance

18 out of 30

17.8 out of 25

22.5 out of 25

14.9 out of 25

Student Progress

21.4 out of 55

42.7 out of 60

54.9 out of 60

44.9 out of 60

Additional Credit

0.8 out of 15

7.5 out of 15

9.0 out of 15

7.5 out of 15

Total Score

48.5 out of 100

76.4 out of 100

91.2 out of 100

75 out of 100

Figure 3: Jacob Riis Report Card Summary

Comparative progress to high-performing schools and notable charter schools

The student progress scores provide a metric to gauge students’ academic growth from one year to the next. When compared to schools citywide that are recognized for high rates of achievement, students at the Jacob Riis School make relatively large gains in academic progress. In 2009-2010, Jacob Riis students made greater rates of progress than students in any of the high-profile Harlem charter schools (KIPP, Harlem Children’s Zone and Harlem Success Academies), most schools in the district with much lower rates of poverty that have been consistently recognized for high rates of student performance (P.S. 3, 6, 41, 234, Salk Middle School), and schools with similar demographics (P.S. 198 & M.S. 131). Students made only .1% lower rates of progress than students in the renowned Manhattan New School (Harwayne, 1999) and seven points lower than students of the Anderson School, a program for “gifted and talented” students within the district (See Figure below). 

 Notable public schools

Anderson school for “gifted and talented”: 52.0/60

Manhattan New School (P.S. 290): 45/60
P.S. 126: 44.9 out of 60
P.S. 189: 41.0/60

P.S. 6: 36.7/60

P.S. 234: 29.9/60

P.S. 41: 29.0/60

P.S. 198: 28.5 out of 60

MS 131: 27.8
P.S. 3: 26.5/60

Salk: 23.4/60

Notable Charter Schools

KIPP Infinity: 39.9/60
KIPP Academy: 39/60
Harlem Success Academy: 34.2/60

KIPP STAR: 26.0/60
Harlem Children’s Zone/Promise Academy: 22.2/60

KIPP AMP: 16.1/60

Harlem Children’s Zone/Promise Academy II: 10.6/60

Figure 4: Comparative student progress gains to notable public and charter schools  

Analysis of K-8 literacy progress-monitoring assessment data from 2009-2010

Data from the K-8 literacy progress monitoring system used in the school allow judgments to be made about the efficacy of the Learning Cultures model. These assessments consist of the Dynamic Indicators of Basic Literacy Skills (DIBELS) in kindergarten and first grade and the Degrees of Reading Power assessment (DRP) in grades 2-8, both of which are administered three times a year in each grade.

Data were probed with three questions: How well are children as a whole faring in response to the Learning Cultures implementation? How well are the lowest performing students faring in response to the intervention? And how do the highest-performing students fare?

To assess the progress of primary students, we compared the number of children in kindergarten and first grade who scored at the “intensive” level on DIBELS (i.e., most academically at risk) mid year in 2009, before we began Unison Reading, to those who scored at the “intensive” level on DIBELS in January 2010, eight months after Unison Reading was mandated in all classrooms. We found that the number of students in both grades in the intensive category dropped by over 75%.


Grade

             Intensive

               Strategic

             Benchmark

 

Beginning of Year

Middle of year

End of year

Beginning of year

Middle of year

End of year

Beginning of year

Middle of year

End of year

K

08-09

0

 

16

6

0

37

12

0

 

10

47

K

09-10

6

3

0

31

25

0

31

43

0

 

1st 08-09

0

17

5

0

24

14

0

31

54

1st 09-10

6

4

3

12

15

15

38

35

37

Figure 5: Dynamic Indicators of Basic Literacy Skills (DIBELS) Comparative Analysis, K/1, 2008-09 to 2009-10

Gains in achievement were further documented in the annual gains scores on the Degrees of Reading Power® assessment, the progress monitoring assessment administered in second through eighth grade in the Fall, Winter and Spring of each year (Maculaitis, 2001; Questar Assessment, 2000). Students in all grades outperformed the national average in annual gains in reading comprehension. Children in grades 3, 6 and 7 doubled the national average rate of achievement and students in 5th grade made gains more than four times greater than the national average. Students in grade 8 made five times more progress than the national average (See Figure 8).

Grade

N of

Students

Average Change in DRP units, September-June

National Average Gain in DRP units September-June *

Jacob Riis DRP rates of change compared to DRP growth in relation to national average

2nd

57

+14.3

10

Approximately 1.5X

3rd

53

+15.9

8

Approximately 2X

4th

54

+9.5

6

Over 1.5X

5th

60

+13.4

3

Over 4X

6th

112

+8.8

4

Over 2X

7th

102

+6.9

3

Over 2X

8th

108

+11.0

2

Over 5X

Figure 6: School-wide Degrees of Reading Power Achievement Data, September, 2009 through June, 2010

Since students are not grouped by reading level or ability in Cooperative Unison Reading, it was important to test the claim that self-selected groups consisting of students of mixed reading abilities support the achievement of all students—high- and low-functioning students alike. In order to gauge the effectiveness of the program on students in both the highest and lowest performing subgroups, students with the highest and lowest raw scores on the DRP were identified in September. Average rates of DRP progress between Fall 2009 and Spring 2010 for theses students were calculated. When compared to the national average DRP gains, students in both the highest and lowest performing 20% made rates of progress that were higher. Notably, the lowest 20% of students in fifth grade made more than five times the national average rate of progress, and the highest 20% of students in fifth grade made more than three times the national average rate of progress. The lowest 20% of students in eighth grade made four times the national rate of progress, and the highest 20% of students in eighth grade made six times the national average rate of progress.


Grade

N of

Students

National Average Gain in DRP units

September-June *

Average Change in DRP units for Lowest 20% Subgroup,

September-June

 

DRP growth in relation to national average for Lowest 20%

Average Change in DRP units for Highest 20% Subgroup,

September-June

 

DRP growth in relation to national average for Highest 29%

2nd

12

10

+16.8

Over 1.5X

+14.4

Approx. 1.5X

3rd

11

8

+16.8

Over 2X

+13.0

Above average

4th

11

6

+11.9

Approx. 2X

+8.6

Approx. 1.5X

5th

12

3

+16.3

Over 3X

+10.2

Over 3X

6th

23

4

+12.7

Over 3X

+9.1

Over 2X

7th

21

3

+7.1

Over 2X

+7.4

Over 2X

8th

22

2

+9.1

Over 4X

+12.6

Over 6X

Figure 7: Student’s Average Gains in DRP Units for Lowest 20% and Highest 20% Subgroups Compared to National Average Gains in DRP Units, September through June, 2010

The program also proved beneficial to English language learners, with higher rates of progress than the national average in all grades. Fifth grade English language learners made more than seven times the national average in annual rates of progress.

Grade

N of

Students

National Average gains in DRP units

for ELLs September-June **

Average Change in DRP units for ELL Subgroup,

September-June

 

DRP growth in relation to national average for ELLs

2nd

23

Not Available

12.1

Data not available

3rd

16

Not Available

15.2

Data not available

4th

8

5

7.5

Above average

5th

11

2

14.6

7X

6th

5

2

2.0

Average

7th

5

2

9.2

Between 4-5X

8th

6

3

9.0

3X

Figure 8: English Language Learners Gains on the Degrees of Reading Power Assessment, September through June

Utilization of exiting space and resources

The logic of the model relies on substantive changes in classroom discourse patterns through social groupings for activities that are not stratified by ability or level. These alterations required, in some cases, the purchase of new desks. But otherwise, alterations to classroom spaces were not necessary in order to implement the intervention.

The Learning Cultures model requires that reading and writing materials are procured by choice, for the most part, by students themselves, from their own environments. Children’s magazines, encyclopedias, texts from internet news and education sites, are all that was required for the curriculum. With the exception of a large magazine order each year of approximately $10,000 (for 700+ students), no other funding was required for curriculum materials.

Reduced cost of primary remediation programs

Early reading intervention programs tend to be quite costly to maintain. This was the case at the Jacob Riis School, where prior to implementation of the Learning Cultures/Unison Reading program, Reading Recovery was used as the early intervention program for academically at-risk first grade students. Reading Recovery is an evidence-based intervention that provides 30 minutes of one-on-one instruction to students on a daily basis for one semester. One full time teacher can serve six students per semester (12 students per year). Prior to implementation of the Learning Cultures curriculum, three full-time Reading Recovery teachers were employed to provide intervention support to 36 first-grade students. In addition to three full-time teacher salaries, the university-based Reading Recovery training program cost an additional $15,000 per year.

After analyzing the progress monitoring data for second grade after a full year of implementation of Learning Cultures, the principal of the school was confident that the program was proving more effective than the guided reading/Reading Recovery program combination that was previously used in primary classrooms. In addition to the fact that substantially fewer kindergarten and first grade students required intensive intervention at the half-year point according to DIBELS, as previously noted, comparisons of the end-of-year DRP scores between 2009-10 for the lowest performing 20% of second grade students revealed that they made nearly two times more progress than the national average. The apparent success of the Learning Cultures model justified the elimination of the Reading Recovery program, allowing two of three Reading Recovery teachers to be re-assigned as general education teachers, saving the school the equivalent of two teacher salaries per year (one of the teachers continued to serve in a resource capacity, providing push-in and pull-out literacy support to high-need elementary students).

Reduction of professional development costs

The Learning Cultures model provides a strategy to support the cultivation of in-house expertise and teaching capacity through a comprehensive and coherent set of practices. The combination of efficient and effective practices and a low-cost and effective PD structure is economical. In 2007-08 the Jacob Riis school employed a total of 5 literacy staff developers who contributed a combined total of 11 days per week at a cost of $360,000 per year. In 2010-11, the total extent of professional development support for the whole school was provided in two days per week, reducing the cost of professional development by more than 80%.




cynthia mcccallister
cynthia mcccallister

Author



Leave a comment

Comments will be approved before showing up.


Also in News

Cooperative Unison Reading: A Socio-Cultural Instructional Method, K-12
Cooperative Unison Reading: A Socio-Cultural Instructional Method, K-12

by Cynthia McCallister

In a new e-book, by Cynthia McCallister, is a brief guide to the Cooperative Unison Reading approach. It provides an innovative angle on reading instruction by looking at the reading process as a cooperative human process and applying theories from science that explain cooperation. 

McCallister also looks at the reading process as a form of action, as opposed to the accumulation of discrete skills that are accumulated through linearly transmitted instruction. 

The book is packed with practical applications of the method to real classroom situations, and comes alive with media assets such as audio and slideshows. 

Even if you don't buy the book, you can access a free chapter that provides a training guide to the Cooperative Unison Reading method, led by an expert teacher in the method, Tara Silva. 

The book is a companion to the online course, Cooperative Unison Reading/Mindful Reading, found on www.LearningCultures.net. 

Visit the book's url and get yourself a copy today! 

Read More

Learning Cultures Is...
Learning Cultures Is...

by Cynthia McCallister

Teacher practitioners of the Learning Cultures approach describe what Learning Cultures means to them in this short video montage.

Read More

'Integrative Math'
'Integrative Math'

by Cynthia McCallister

Summer 2016. A new instructional procedure, developed by Cynthia McCallister, called Integrative Math©, combines elements of Cooperative Unison Reading®: and modes of representation Jerome Bruner presented in his book, Toward A Theory of Instruction (1974). Students use the rules of Unison to read story problems. Then, using manipulatives, crayons, markers, and stories, they integrate enactive, iconic, and symbolic modes of representation, strengthening their mathematical thinking.

 

Read More