A report examining the submission process for Montana's ACT scores to the feds last year found that all scores were not actually submitted as proficient, as Superintendent of Public Instruction Elsie Arntzen claimed earlier this year, though education officials initially intended to do so.
The report, which Arntzen commissioned, also found issues with the reporting process internally at the Office of Public Instruction, and recommended clearer procedures for handling student test scores.
Previous Superintendent Denise Juneau announced that the state would use the ACT to report federally mandated test scores for high school juniors in late 2015. After raising concerns about Juneau's authority to make that call, Montana's Board of Public Education backed up her decision while being aware that the test did not line up with Montana's education standards.
Arntzen called a press conference in January where she accused Juneau's administration of "falsifying" test scores.
“They did not meet state and federal reporting standards and misrepresented student proficiency,” Arntzen said at the press conference. “It was reported that all Montana (high school juniors) were proficient.”
Juneau chalked up the score reporting to a federal form that forced the state to report scores, no matter whether the information was applicable, and that the reporting was intended to be revised later.
The report, authored by Communication and Management Services, a Helena management and personnel consulting firm that doesn't have specific education expertise, backs up Juneau's claim that OPI intended to change the test scores.
It also found that despite widespread assumptions at OPI, the ACT scores did not require a label from one of four federal score categories, so they were never submitted as proficient. The process appeared to sow significant confusion among agency employees.
OPI officials reviewed a draft copy of the report on March 2, but an employee revised their statement to CMS on March 6, delaying the report's release. That stemmed from access to the reported scores, available since Feb. 20, which showed they weren't submitted as proficient.
The report also delved into why the office never developed cut scores, which would have translated the ACT's 36-point scale into the four federal categories, and never examined the test's alignment with state standards.
Its conclusions called the performance of Judy Snow, OPI's assessment director who retired last fall, into question. CMS did not interview Snow or any former OPI employees. The report does not name employees, but Snow's position is clear based on job titles.
The report says that "there was little or no work done" to address cut scores or standards alignment after Juneau's initial ACT announcement, despite assumptions among OPI employees interviewed for the report that Snow "was taking the necessary steps to implement Superintendent Juneau's directive to use the ACT" and meet federal requirements.
It also notes a lack of communication from Snow to OPI employees and said that despite overlapping employment for her and new assessment director Jessica Eilertson, there was little transfer of departmental knowledge. The report raises questions about accountability within OPI under Jueanu.
It recommends several changes to streamline communication and data handling.
Juneau, in an emailed statement Friday, took issue with the report's lack of contact with her staffers. Several OPI employees left the agency before Arntzen took over, including Chief of Staff Madalyn Quinlan.
"The report proves Elsie is more interested in political games than doing her job," the statement said. "The report takes shots at my former coworkers without consulting any member of my administration to get the facts."
Arntzen said the report's value "cannot be stressed enough."
"Having identified the failures of the past, it is now my job to take leadership and move forward," she said in an emailed statement. "The report indicates several areas of improvement that can be made immediately including proactive management and increased accountability, knowledge transfer and succession planning, and continuous quality improvement. OPI is developing a comprehensive master plan for data accountability."
Arntzen previously said Montana could lose federal funding because the ACT didn't meet federal requirements. However, most other states' testing processes don't meet all federal requirements either.
Twleve states use the ACT or SAT, another college readiness exam, for federal accountability, according to an "Education Week" database. At least seven states “won permission” from the feds to use those tests, but still must go through the peer-review process.
The Montana debate centers on No Child Left Behind, which will be replaced when the Every Student Succeeds Act goes into effect for the 2017-2018 school year. The new law appears to open the door to ACT use, allowing for "nationally recognized high school academic assessments," but it still requires a peer-review process.
Montana’s peer review process has been pushed back a year, according to the federal spokeswoman, because of changes to its assessment program in the 2015-2016 school year — presumably the ACT switch.
In decision letters sent to Wyoming and Wisconsin, who use the ACT, the test was labeled as “partially meeting federal requirements” for NCLB, in part because the test does not align with state standards.
However, the letters make no mention of the potential loss of federal funding, nor do they declare states “non-compliant.” Such letters often point out flaws in submissions for any state and request that states resubmit information with corrections.
A review of the latest round of peer review assessment letters shows that of the 31 states that received decision letters, not one met every requirement under NCLB. And none of the letters threaten the loss of federal funding.
In California, state education officials are in the midst of a showdown with federal officials because its use of a test the feds didn't sign off on. State officials have held their ground despite a response from feds citing "many possible enforcement actions and remedies available to be applied by the department, including the withholding of funds,” according to Edsource, a California education news outlet.
Unlike California's test, 12 states use the ACT or SAT, another college readiness exam, for federal accountability, according to a database of state tests by “Education Week.” At least seven states have “won permission” from the feds to use those tests but still must go through the peer-review process.