In 2014, School District 2 trustees set a goal for Superintendent Terry Bouck: raise scores by 2 percent the next year on standardized tests that the district gives twice a year.
Schools now give the Northwest Evaluation Association Measures of Academic Progress (NWEA MAP) tests three times in grades 3-8 and 10, in the fall, winter and spring. Scores can be tracked on an individual student level and measured by class cohort as they get older and move into new schools.
The district uses the tests, not state-mandated Smarter Balanced tests, as its measuring stick.
"We're not that confident, at this time, with the Smarter Balanced data," Bouck said. "It's really only been there one year."
Only two schools improved 2 percent from 2014-2015 to 2015-2016 in each grade, and only two grades met the mark for districtwide scores. A larger look at the test results shows significant variability among grades and schools from year to year, underscoring the unlikelihood of uniform improvement.
“You want to see improvement in the school overall,” said Trustee Gordon Klasna, a principal at Lockwood’s Eileen Johnson Middle School, which also administers the tests. “To put a set number on it, I hesitate a little bit.”
In data dating to 2012-13, districtwide, grade-level proficiency rates went up at least 1 percent year-to-year eight times in reading and six times in math; they went down at least 1 percent six times in reading and six times in math. School level scores, with a smaller sample, typically show bigger swings.
School principals told trustees that they look more at individual student growth and cohort growth than year-to-year changes at the same grade level at a Performance Monitoring Committee Meeting on Tuesday, one of four such meetings slated for this year.
“The 2 percent goal I don’t worry about as much as the cohort piece,” said Poly Drive Elementary principal Kevin Croff.
Improvements or decreases in scores by cohort are more meaningful because, excluding kids moving in and out of school, they measure the same students. Comparing, for example, third grade in one year to third grade the next year is a different group of students.
“It’s a little easier to set a goal when the kids are staying the same,” said district assessment director Roger Dereszynski.
Principals talked about strategies they used to target improvements, like individual goal setting with students, more early assessment in grades K-2 to identify and address student weaknesses and more teacher collaboration.
NWEA MAP tests are both formative and summative; they're given multiple times to provide feedback for teaching adjustments and they're used as a one-time benchmark.
The scores also have more specific breakdowns that help educators understand where students are succeeding and where they struggle within a subject area. By offering the test multiple times, it give educators a chance to make adjustments targeting students' weakness.
Administrators bluntly said that students don’t start in the same place. When grade-level proficiency by school from 2012-13 to 2015-16 is averaged, scores tend to correlate with the economic status of a school’s population. That poorer students usually have worse academic outcomes isn’t a new finding, nor is it specific to Billings.
“We can’t use that as an excuse,” said Orchard principal Jeremy Carlson.
Carlson’s students had the lowest average scores in the district, and the school has one of the highest poverty levels in the district.
“That’s not good. That’s our reality,” he said.
But they also improved more than most students in the district; looking at the two groups of students that school has three-year reading data for, 20 percent more fifth-graders in 2014-15 scored proficient than when the students were third-graders and last year’s fifth-graders saw the same level of improvement.
For schools where students perform at a high level early, improvement can be more elusive. Arrowhead Elementary, the most economically well-off school in SD2, has the highest multi-year, grade-level test score average.
On last year’s reading test, 92 percent of third-graders, 93 percent of fourth-graders and 95 percent of fifth-graders scored proficient.
“One piece that the teachers and I struggled with … it’s supposed to be two percent (improvement) no matter where you are at,” Arrowhead principal Pam Meier said.
Bouck floated the idea of changing the 2 percent goal at a regular Board of Trustees meeting earlier this month, but there was no change talk Tuesday.
“We’re going to stay the course,” Bouck said, while noting that “one shoe doesn’t fit all.”
“You had to have something because we hadn’t had anything for so long. The 2 percent provides something to shoot for.”
Klasna said he’d be open to tweaking the goal in the future, but understood why it was set.
“Trustees can probably understand those (improvements) more than the breakdown of the RIT scores,” he said, referring to in-depth scores that evaluate specific areas of learning. “If you say, hey, they improve by 1 or 2 percent, the public can understand those more.”