The one time a district collected the recommendations and then ignored them

A necessary aside

The one time a district collected the recommendations and then ignored them

Everything so far rests on a comparison that is normally impossible to make. To know whether teacher recommendation is missing capable children, you need to know what happened to the children who were not recommended. Ordinarily you cannot, because not being recommended is the end of the story. The child does not take the course, so there is no result to compare.

One large North Carolina district produced the exception, and the children whose futures it turned on were never told.

What was done. Seven of the district's roughly thirty middle schools agreed to place every sixth grader into one rigorous course. No tracking, no separate advanced section: in the authors' words, “they would have only one mathematics track, that of advanced math”. Meanwhile the fifth grade teachers at the eighty-five elementary schools that fed those seven schools filled in their recommendation forms exactly as they always had, not knowing the forms would be set aside. So the district ended up holding two things at once: a written record of which children the adults would have placed where, and a year of results for those same children after they were all placed in the same room. That is the experiment nobody ever gets to run.

One detail makes the record sharper than it looks. Fifth grade recommendations in this district are made before the state test results come back. The forms therefore asked teachers to predict each child's score. Those predictions were written down, and they could be checked.

First, how often the recommendations were right

Among students who went on to score at the top achievement level, teachers' predictions of their performance were correct 61% of the time. That is the headline number, and on its own it is not damning. Teachers were being asked to forecast a score that did not exist yet, and nobody would expect perfection.

The problem is where the errors went. Overall they balanced: teachers over-estimated about as often as they under-estimated. Broken out, they did not balance.

Students at the top achievement level	Teacher prediction correct
White students (n = 1,427)	65%
Non-Asian minority students (n = 892)	54%
Asian students (n = 125)	70%
All students (n = 2,444)	61%

Stiff, Johnson and Akos, 2011, Table 6.1. Teachers “incorrectly predicted lower scores than non-Asian minority students actually received at statistically significant greater rates than they did for Asian or white students”.

Then, the same children a year apart

This is the finding that is hard to argue with, because the two rows below describe the same students, assessed by the same profession, twelve months apart. The only thing that changed in between is that the adults doing the assessing had watched these children do mathematics.

Top-scoring students recommended for the next advanced course	White	Non-Asian minority	Asian
By 5th grade teachers, into the advanced 6th grade course Before the state test results existed	71%	12%	83%
By 6th grade teachers, into grade 7 prealgebra After a year in the untracked course	68%	61%	70%

Stiff, Johnson and Akos, 2011, Table 6.2. Sixth grade teachers recommended 15% more students for the advanced track overall, a figure that includes the 6% they sent straight to algebra. A quarter of the students the fifth grade teachers had placed in the low track received high-track recommendations a year later, and so did every single student they had placed low who had scored above the mean of the students they placed high.

Twelve per cent to sixty-one per cent. Nothing about these children changed. What changed is that the second set of teachers had evidence and the first set had a form to fill in before the evidence arrived. The fifth grade recommendation was not a judgment about mathematics. It could not have been. It was made before anyone had seen the child do any.

And nobody was held back

The obvious objection to putting every child in the rigorous course is that the strongest students pay for it. They did not. Growth on the state mathematics assessment was compared against matched students in the rest of the district, and the study-group students grew more: in every ethnic group and at every one of the four achievement levels, including the top one, all at p < .001.

What it cost, districtwide

The study also followed the top-scoring fifth graders across the whole district, including the schools that kept tracking, to see who eventually reached algebra in 8th grade.

Top-scoring 5th graders who reached 8th grade algebra	Share
Other students	59%
White students	51%
Hispanic or Latino students	26%
Black students	19%

Stiff, Johnson and Akos, 2011, Figure 6.1; p < .001. Fewer than half of all top-scoring students reached 8th grade algebra. Across the district's middle schools the odds ranged from about 20% to nearly 80%, depending only on which school a child attended.

And the people closest to it believed the opposite. Counsellors across the district believed that top-scoring students were automatically tracked into the advanced course, and, in the authors' words, “even expressed confidence that all Level 4 black students were tracked high… because of the district's initiatives.” The actual figure was 19%. Nobody was lying. The number simply did not exist anywhere a counsellor could see it.

The parent waiver, and why “nothing is stopping anyone” is not an answer

The most common response to all of this is that the door was never locked. A parent who disagreed with a recommendation could sign a waiver and override it. That was true, and the district kept the waivers.

Among students who had scored at the top level for two consecutive years and were nonetheless recommended for the lower track, 98% of whom were formally identified as academically gifted, with mathematics percentile ranks between 93 and 99, here is who exercised the right.

Parents of high-scoring students recommended for the low track	Signed a waiver
Asian families	83%
White families	44%
Non-Asian minority families	20%
Black families	None. Not one waiver.

Stiff, Johnson and Akos, 2011, Table 6.4.

A right that has to be discovered is not the same as a right. The waiver worked exactly as designed for families who knew it existed, knew their child's percentile rank, and were comfortable contradicting a teacher in writing. It did nothing at all for families who were told their child had been recommended for the regular course and reasonably assumed the school knew what it was doing. This is the mechanism behind the advocate: not corruption, not malice, just a system in which the remedy is available only to the people already positioned to use it.

One more consequence nobody was looking for

Because the advanced course had become the marker of a capable student, its absence became the marker of a struggling one. Fifth grade teachers, the study found, “often assigned minority students to the school district's dropout prevention program, having assumed that minority students who were not in the top math track in middle school must be low scoring in math.” The district later discovered it had assigned a disproportionate number of its top-scoring minority students to dropout prevention, and excluded a disproportionate number from rigorous mathematics and science courses.

That is what a placement decision does when nothing checks it. It does not stay a placement decision. It becomes the school's shorthand for the child.

What the district did about it, and why it did not hold

The response was sensible and local. The district's academically gifted, school counselling, and curriculum and instruction departments sat down together and wrote objective criteria: a student qualified on any two of three measures, a supportive teacher recommendation, a high Level 3 or Level 4 test score, or a grade no lower than 3 out of 4 on the standards-based report card backed by two work samples. Note what that is. It is not the abolition of teacher judgment. Teacher recommendation stays, as one of three routes in.

Then came the sentence that explains the next decade. In the authors' words: “since school board policy does not govern such recommendations, the recommendations may be ignored.” Criteria that nobody is required to follow are a suggestion. The chapter records that the counsellors accepted the findings, the mathematics teachers remained reluctant, and the administrators followed the mathematics teachers.

And once, in one school, somebody simply did it

The same two authors published a companion chapter the same year, about the same district, and it contains a single paragraph worth the whole argument. By then the objective criteria above existed. One middle school had overlooked 103 sixth graders who met them.

“One middle school in this large urban school district ‘overlooked’ 103 sixth-grade students who met the placement criteria for the top track in mathematics. After reassignment and placement in the top-track mathematics course, with no additional interventions, students performed consistently with their pro-equity indicators.”

No tutoring, no summer programme, no double period, no catching up. They were moved into the top track and they did the work. Many of them had previously been designated low achieving, and some as behaviourally or emotionally handicapped.

They were moved because one adult insisted.

“these students were nevertheless placed into the top track because the principal of the school insisted that they be given an opportunity to succeed even if the mathematics faculty did not agree.”

And then the principal was reassigned to a high school.

“similar students in the next year's classes did not get the opportunity to engage in mathematical reasoning and sense making because the new principal… would not embrace the pro-equity model used previously if the mathematics faculty did not agree. They did not; the results in figure 5.5 were not enough evidence to change their beliefs and attitudes.”

Read those together. Same school, same written criteria, same kind of children, one year apart. What changed was which adult was in the building. And the evidence that it had worked, a published grade distribution showing those 103 children succeeding, was not enough to keep it going. That is the entire case for a rule rather than a practice, and it is why the words “when practicable” in the statute matter so much. Seven years after these chapters were published, the General Assembly wrote a placement rule into law. It had to.

Why this study answers the objection the others cannot

Studies of what happens when a district adopts a placement rule almost always examine the students right at the cut score, because that is where the statistics are cleanest. Those studies are useful and they are also, structurally, incapable of telling you about the children this piece is about: the ones far above the line who were never recommended at all. This district measured those children, because it collected the recommendations and then declined to act on them.

What it found was not that the bar should be lower. Every student in these tables had already cleared the bar. It found that at an unchanged bar, an unwritten process was admitting 71% of one group and 12% of another; that a year of actually teaching those children closed almost the whole gap; and that when a school did place the overlooked children, with no extra help at all, they were fine, until the person who insisted on it left the building. The research objection, in full →

Sources for this page: Stiff, L. V., Johnson, J. L. and Akos, P., “Examining What We Know for Sure: Tracking in Middle Grades Mathematics”, chapter 6 in Tate, W. F., King, K. D. and Rousseau Anderson, C. (eds), Disrupting Tradition: Research and Practice Pathways in Mathematics Education, Reston, Va.: National Council of Teachers of Mathematics, 2011, 63-75. Tables 6.1, 6.2, 6.3 and 6.4 and Figure 6.1.

Stiff, L. V. and Johnson, J. L., “Mathematical Reasoning and Sense Making Begins with the Opportunity to Learn”, chapter 5 in Strutchens, M. E. and Quander, J. R. (eds), Focus in High School Mathematics: Fostering Reasoning and Sense Making for All Students, Reston, Va.: National Council of Teachers of Mathematics, 2011, from p. 85. The 103 overlooked students and Figure 5.5.

Neither chapter names the district; both describe a large urban school district in North Carolina. The work began as a review by that district's counselling department, which formed the School Counseling/Math Collaborative; Edstar Analytics ran the Data Academies for the school teams. Both chapters are NCTM copyright and are cited and quoted here rather than reproduced.