Part two
The students who did get in were almost never wrong for the class
The prediction was checked against what actually happened. Of the 18,670 students in the top band who were enrolled, 17,914 scored proficient or higher. That is 95.9%. The pattern holds down the scale: students given a 0 to 10% chance succeeded 10.5% of the time, students given 60 to 70% succeeded 70.6% of the time. The model was not guessing. The middle bands rest on very few enrolled students, because almost nobody below the top band was let in, so read those rows as direction rather than precision.
EVAAS prediction against what students actually scored
2008 entering probability of reaching proficiency in 8th grade Algebra I
| Predicted chance of success | Students | Enrolled | Enrolled | Scored proficient | Success rate |
|---|---|---|---|---|---|
| 0 to 10% | 6,364 | 19 | 0.3% | 2 | 10.5% |
| 10 to 20% | 6,146 | 56 | 0.9% | 7 | 12.5% |
| 20 to 30% | 6,092 | 128 | 2.1% | 43 | 33.6% |
| 30 to 40% | 6,041 | 182 | 3.0% | 66 | 36.3% |
| 40 to 50% | 6,354 | 342 | 5.4% | 169 | 49.4% |
| 50 to 60% | 6,278 | 452 | 7.2% | 276 | 61.1% |
| 60 to 70% | 6,881 | 823 | 12.0% | 581 | 70.6% |
| 70 to 100% | 42,144 | 18,670 | 44.3% | 17,914 | 95.9% |
Source: NC DPI EVAAS Longitudinal Database, reproduced in the 2009 SAS analysis commissioned by the NC Association of School Administrators. The student counts sum to 86,300, so this is a statewide cohort. A companion figure in the same material reports the split as 42% enrolled and 58% not, against the 44.3% in this table; the difference does not change the size of the gap.
A misreading worth heading off
The threshold was at least a 70% probability of scoring proficient. Here is what that phrase is routinely taken to mean, and what it says.
“We expect 70% of these students to pass. So three in ten will fail, and putting them in the course sets them up to fail.”
70% is the lowest chance anyone in the group was given. It is a floor, not a forecast. The students above it passed at 95.9%.
The 70% is a floor under each child's chance. It is not a share of the group.
Take every day this year when the forecast gave at least a 70% chance of rain, and put those days on a list. Nobody would look at that list and conclude it must have stayed dry on three days in ten. Most of the days on it were 90% days and 95% days. The 70% days are the bottom of the pile, not the middle of it. It rained on nearly all of them, and that is not the forecast being wrong. That is what the list is.
The 70-to-100 band is that list. It holds every student the model gave a 70% chance and every student it gave a 99% chance, and there are far more of the second kind. So the number to expect was never 70%. It was the average of their individual chances, whatever that came to.
And we know what it came to, because the state enrolled 18,670 of them. 17,914 scored proficient. 95.9%.
Which means the 70% line was never a coin flip with a three-in-ten failure rate attached. It was a deliberately cautious bar, and the students above it passed at a rate any teacher would be pleased with. The argument for using it was never that these students might scrape through. It was that we already knew they would not need to.