Controversial FSA Tests: How Are Results Used?

For almost three weeks now, Grades 4 and 7 students across B.C. have been completing the annual Foundational Skills Assessments measuring their reading, writing, and numeracy comprehension. At the same time, the teachers' union and government have been squaring off over the usefulness of such an exercise.

The BC Teachers' Federation (BCTF) takes umbrage with the time assessment takes away from classroom instruction time. They say there is no need for every student to take a standardized test to prove students are learning the curriculum, when testing a representative sample of students would do.

The B.C. government says the test is key to understanding if the province's children are learning the curriculum, and it helps parents know where their child stands compared to their peers. The results can help pinpoint areas where instruction is strong, and where it can be improved.

"FSA assists parents, teachers and school districts in providing help for the students who need it most and it also allows the ministry and districts to spot trends and correct what needs to be corrected," said Education Minister Don McRae in an op-ed in the Vancouver Sun published in mid-January.

What McRae's op-ed doesn't cover, however, is how the FSAs are used to improve education outcomes for B.C. students. Once the results come back, what happens?

Most use the tests to complete their district achievement contracts each summer, detailing what needs to change to improve learning and how they will do it. The following January a superintendent's report updates the ministry on the progress they've made. But how accountable are the school districts held to reaching these goals and improving on their FSA scores? Not accountable enough according to Victor Froese, a former school trustee and member of the FSA design committee.

"They're supposed to set goals," Froese told The Tyee. "The criticism I have is that they really do not act on those: [achievement contracts are] filed, and that's the end. There's no direct link to anything in the system."

'Nothing has changed': Froese

In 2007 Froese, who in recent years has served as an external reviewer on the technical review team for the reading and writing section of the FSAs, wrote an email to Vancouver Sun reporter Janet Steffenhagen claiming the FSAs, achievement reports, individual school plans, and the ministry's district reviews weren't effectively used to improve student outcomes.

"If districts were strongly encouraged to implement the recommendations in the district review reports alone we could expect fairly radical changes," he told Steffenhagen.

"Do we know if any of these things are being done? I would urge the ministry to practice what it preaches."

He said districts were putting actions in their achievement reports that would have helped increase student learning, but not implementing them. He wrote a similar letter to the Vancouver Sun the year before, which you can read at the bottom of this article, talking about what districts with low FSA reading scores needed to do to improve:

Seven years later, his criticisms still stand. He says if you look at the provincial FSA statistics on the government's website "they have little line graphs at the bottom for the last five years, from 2008 to 2012, and all of them are straight lines, which means that nothing has changed."

The statistics show for all Grade 4 and 7 students who took the tests last year -- and about 17 per cent of them didn't -- 70 per cent met or exceeded expectations in reading, 72 per cent met or exceeded expectations in writing, and 68 per cent met or exceeded expectations in numeracy. Froese says its concerning the average FSA score has remained the same for four years, despite four sets of district achievement contracts being filed.

Froese says not only are FSA results used in achievement contracts to create goals that districts aren't held accountable for reaching, the goals often don't make sense, either.

"Typically what they would say is, 'We'll do two per cent better in this,' and that's the criticism I made is that it actually doesn't tell you anything about what will be or should be changed when you talk about percentages, because those are outcome measures and not the outcome," he said.

"The outcome would be that students could read at a higher level or do more complicated arithmetic."

Districts 'cherry pick' data: Ehrcke

Tara Ehrcke, Greater Victoria Teachers' Association president, isn't shy about saying she doesn't like the FSAs, period. But she doesn't like the way districts are reporting the results of these tests, either.

In a post on her blog last week, Ehrcke accused both the Delta and Greater Victoria school districts of cherry-picking assessment data for achievement contracts and superintendent reports to show only where marks were improving.

"I think the point of that particular article was to point out the hypocrisy in what McRae said with regards to 'We must test every student because that will be used for planning,' and then in the achievement reports, [districts are] really cherry-picking data," she told The Tyee.

"But there's no rational connection between the data and what the plans [for improvement] are."

The Ministry of Education doesn't require districts to do anything with their FSA data, but suggest they make use of them to set educational outcome goals.

"Superintendents of Achievement from the ministry meet with school district leaders to discuss FSA results and ways to make improvements. The ministry uses FSA data as one piece of information to review curriculum and the elementary program, and to assist in setting goals and targets," reads an emailed statement the ministry sent to The Tyee.

The statement adds that "many districts" use the FSA, in addition to other data, as information to set goals in their achievement contracts.

FSA just one indicator

The Tyee spoke with two districts, Arrow Lakes and Coast Mountains, whose 2012 FSA results were below the provincial average. They both confirmed use of the FSAs in their achievement contracts, as well as other district and school tests like District Assessment of Reading and the Whole Class Reading Assessment.

"School districts sign achievement contracts with the ministry, and then we submit a report on that contract. So we use the FSA results to inform us in those areas in regard to how the kids are doing," said Nancy Wells, superintendent of the Coast Mountains district.

"Nothing that you measure in education, if you just use one indicator, is very wise. You've got to use a broad range of indicators."

Terry Taylor, district principal of learning for Arrow Lakes, says the ministry doesn't punish districts with low FSA results, but rather treats the improvement process as a conversation.

"In our district we've always provided data and evidence about what we're working on, and what progress we're making or what progress we're not making, and we're in charge of identifying our own goals and needs," said Taylor.

"What we get from the ministry, particularly the superintendent of achievement that's assigned to our particular district, is we get support. It's not a top-down directive way of working. The ideal is that it's a learning community, and certainly that's my experience."

In the Arrow Lakes district individual school planning councils also take a look at the data to set improvement goals for each school.

"It informs our teaching practice and it informs what support we might be able to give parents and student teachers in moving learning forward. The agenda is moving learning forward, and it's just a part of what helps us know whether what we're doing is making a difference or not," said Taylor.

But Froese says there's very little evidence that assessment data, whether its FSAs or other standardized tests, is used to improve districts' educational outcomes.

"There's no direct link [from assessments] to anything in the system," he said. "The only thing that I can find or see now is they do a kind of early childhood assessment, and that's done by teachers and that's used to supply some funding to districts where there are a lot of needy kids. That's useful."