Marking 20 years
of bold journalism,
reader supported.
News
Education

Do Standardized Tests Measure Up?

Teachers' union says nope; government sticks by 'em. Now, a public forum takes on controversial FSAs.

Katie Hyslop 2 Oct 2013TheTyee.ca

Katie Hyslop reports on education and youth issues with The Tyee Solutions Society. Follow her on Twitter.

image atom
B.C.'s standardized tests have been a sharp point between the teachers' union and government for years. Photo via Shutterstock.

Each year, Grades 4 and 7 students in British Columbia sit down to write a census-style, province-wide test while their high school counterparts prepare for provincial exams in June. Tests for each grade level are standardized, regardless of students' background or learning style -- a fact that continues to cause no end of controversy in education circles. When the conservative Fraser Institute releases its school rankings each year, debate over the effectiveness of standardized testing in the province is renewed.

The Foundation Skills Assessment (FSA), the annual test for Grades 4 and 7 students, is openly disliked by the B.C. Teachers' Federation. Union brass has encouraged parents across the province to pull their kids out of it since 2008, arguing standardized tests heighten student stress and anxiety. The union's campaign has been so successful it's kept up to 60 per cent of kids in some schools out of the test. Consequently, results are perceived to be less representative of province-wide progress.

The B.C. Ministry of Education continues to stand by the standardized assessment that was introduced by the New Democratic Party government in 1999. But in 2011, then-president of the teacher's union Susan Lambert said it was former Education Minister Christy Clark who made the tests about assessing schools.

"(Clark) decided to take what was being designed as a measure of the curriculum and use it to announce to parents that her government was listening to them, and announce to parents that her government was giving them what they wanted," said Lambert.

During the recent provincial election, the NDP promised to replace the FSA with another standardized test. The Liberals promised a provincial advisory group headed by the deans of education at the University of British Columbia and Simon Fraser University, which would examine the current assessments and make recommendations for change. However, no apparent progress has been made on the advisory groups.

The different points of view on standardized tests will be argued at the upcoming BC Assessment Forum, held Oct. 4-5 at the Morris J. Wosk Centre for Dialogue, by the Measurement, Evaluation, and Research Methodology (MERM) program at the University of British Columbia. It will feature presentations by the B.C. Teachers' Federation, Great Schools Project, Vancouver School Board, and academics from UBC, Arizona State University and the University of California Los Angeles.

The forum is likely to reflect the current conversation around standardized testing, which has morphed over the years from one of concern over the validity of test results to a much more polarized debate -- with deep roots in ongoing disputes between the teachers' union and the government.

Teacher-designed testing

Conducted annually over six weeks from January to February, students in Grades 4 and 7 in both independent and public schools take the FSAs. Assessing reading comprehension, writing, and numeracy, the Education Ministry website says the main purpose of the tests is to "help the province, school districts, schools and school planning councils evaluate how well students are achieving basic skills, and make plans to improve student achievement."

Despite objections from the union, which claims all teachers are opposed to the FSAs, B.C. teachers themselves design the tests.

Since 2008, students have answered the multiple-choice portion of the tests online, where possible, and the written-answer portion in paper booklets. The ministry website offers sample multiple-choice questions for both grades, but only to school-district employees with an online log-in password.

Samples for the written-response portion are open to everyone online. The Grade 4 version, for example, asks students to identify similarities between an article about water preservation and a description of an environmental advocacy-based game; draw diagrams arranging 24 school desks in different ways that form a rectangle; and write a short story based on a prompt about a flask holding a "wonder" inside.

The Grade 7 version asks students to analyze and compare a news article and a poem about animals in captivity; to offer advice to future Grade 7s about making the most of their year; and a slightly-anachronistic math problem about purchasing CDs.

Although the tests have been administered for almost 15 years, the baseline year for comparison is 2008. That's the first year tests were administered in February instead of May in order to have results returned by March. Because students are not as far along in their school year as they were when they did the tests in May, results today can't be compared to pre-2008 scores.

Teacher objections to testing

The move to partial online testing in 2008 sparked complaints from teachers and school districts that reported problems stemming from outdated technology, poor instructions and students' inexperience completing tests online.

The B.C. Teachers' Federation has continued to report issues, in particular its objection to the Fraser Institute's use of test results to rank schools.

"The primary purpose of these tests is to provide the Fraser Institute with the results so that they can publish the names of the schools and how they rank relative to each other," reads a March 9, 2007 letter from the union to members of the legislative assembly.

The Tyee contacted the union for an interview, but president Jim Iker, who will present at the BC Assessment Forum, was not available by deadline. However, the teachers' union has consistently maintained students' economic and family background is a bigger predictor of their assessment scores than school or teacher performance.

The government has repeatedly distanced itself from the Fraser Institute's rankings, which uses FSA results to rank elementary schools. Typically the schools that rank highest are independent schools, while the low scorers are found in low-income neighbourhoods.

The union also contends the FSAs disrupt teaching time and require that teachers "teach to the test", instead of using their own individual skills to deliver the provincial curriculum.

It has offered to compromise with the ministry in the past, requesting the FSAs be applied to only a sample of B.C. students, but the ministry has refused to budge on the census implementation of the assessment.

The ministry has acknowledged the FSAs will be adapting to match curriculum changes expected to roll out over the next few years under the B.C. Education Plan. Ministry representatives are unable to present at the forum due to scheduling conflicts.

David Chudnovsky of the Great Schools Project is a presenter. Last winter his organization held its own assessment forum, The Great Schools Teach-In, Skyping-in independent academic Alfie Kohn to provide the keynote. Kohn, an author of several books on education, including The Case Against Standardized Testing: Raising the Scores, Ruining the Schools, argues that standardized-test scores are meaningless and used in the United States solely to push the privatization of public schools.

Instead of census testing, the Great Schools Project argues for a "toolkit" of assessments and evaluations, including sample-based province-wide assessment once every six years, and assessment on the classroom and school level, requiring a broad set of measurements to determine students' "achievement, how students learn, how best to help them, and how to ensure that students also learn to self-assess."

As Chudnovsky told The Tyee in 2011, in response to parents' questions about a school's performance, "We in education often answer, 'The Fraser Institute sucks, and standardized testing doesn't tell you much about how the school's doing.' And that's true, we're right about both of those things -- but we haven't answered [their] question."

The project hopes to show how their toolkit, rather than the FSAs, can answer that question.

'We tend not to trust teachers'

Assessment in U.S. schools is arguably worse than B.C., where poor scores on federally-mandated assessments have led to school closures and the firing of teachers and school board employees.

Joan Herman, co-director emeritus of the National Center for Research on Evaluation, Standards, and Student Testing at UCLA, will present the keynote speech for the BC Assessment Forum. Herman has studied assessment in U.S. schools for over 20 years and said one thing that continually holds assessments back is the country's attitude towards teachers.

"We tend not to trust teachers, which I think is a terrible thing," Herman, a former teacher, said in an interview.

"I think in Canada there is perhaps more confidence and trust in teachers, and that opens a lot more opportunities for doing better and richer assessments that are actually more supportive of student learning than what we're doing in the United States."

Herman's talk will focus on the current evolution in American assessments that will hopefully change testing for the better.

This includes reverting at least part of assessments to performance-based tests, commonly used in the 1990s when the federal government didn't require assessments annually as they do now for students in Grades 3 through 8.

This kind of testing doesn't just ask students to recite facts, but to demonstrate they have the skills and work habits required to solve problems, such as writing an essay outline before they answer an essay question or showing all their calculations on a complex math problem.

"Research showed that when performance assessment was part of the accountability system, teachers spent more time engaging kids on problem solving in mathematics, and in writing and thinking in English Language Arts," Herman said.

But testing also needs to adapt to the times, says Herman, which is why new ones will be entirely computer based and will measure students' "21st-century skills." Some of these skills, which include the ability to analyze, synthesize, critique, prove and compare ideas, are outlined in a list of criteria for high-quality assessment that Herman contributed to as part of a project by the Stanford Centre for Opportunity Policy in Education.

Tools for measuring these skills are still in development, but could include embedding student assessment in curriculum and student projects, allowing teachers to assess students while they complete curriculum objectives.

Herman believes assessment is important for schools and government to see how students are performing, but only in relation to schools with similar student demographics, ruling out Fraser Institute-style rankings.

"On the one hand, I'm very much aware of the limitations of a standardized test. On the other, the absence of any accountability for schools serving low-performing kids is not helping those kids either," she said.

High stakes encourage 'gaming the system'

David Berliner agrees that tests should only be used to compare schools that are demographically similar. Berliner, a professor emeritus of education at Arizona State University who will also present at the forum, agrees with the B.C. teachers' union that a child's economic background and family life have a bigger effect on test results than teacher or school performance.

Using standardized tests to rank schools or teachers is like the "streetlight effect", says Berliner -- it seeks answers in the easiest places.

"We're trying to improve schooling with methods that only account for a small percentage of student achievement," he said.

Attaching high stakes to testing, like ranking schools or penalizing teachers or students based on scores, encourages people to "game the system" -- from students hiring tutors to prepare them for the tests or asking older students what was on the test last year, to teachers siphoning off classroom time to teach to the test.

Berliner, who comes from an assessment background, isn't anti-test. Nor is he against disciplining or firing teachers whose performance is poor and never improved upon. But he is against the assessment-based firings that have made headlines in the United States in recent years, saying teacher performance can't be based on tests alone and firing should be the last resort after consultations with teachers and professional development have failed.

"The American school system has really come under the gun by the business community, and the business community likes metrics, even if they're invalid," he said.

He was quick to add that hasn't happened in Canada yet, though he acknowledged the government is becoming more conservative.

"You've always had a two-tiered system because of your more British model, so you've had some support for Catholic schools and private schools," he said.

The BC Assessment Forum takes place Oct. 4 and 5 at the Morris J. Wosk Centre for Dialogue in Vancouver. Attendance is open to the public but registration is required.  [Tyee]

Read more: Education

  • Share:

Facts matter. Get The Tyee's in-depth journalism delivered to your inbox for free

Tyee Commenting Guidelines

Comments that violate guidelines risk being deleted, and violations may result in a temporary or permanent user ban. Maintain the spirit of good conversation to stay in the discussion.
*Please note The Tyee is not a forum for spreading misinformation about COVID-19, denying its existence or minimizing its risk to public health.

Do:

  • Be thoughtful about how your words may affect the communities you are addressing. Language matters
  • Challenge arguments, not commenters
  • Flag trolls and guideline violations
  • Treat all with respect and curiosity, learn from differences of opinion
  • Verify facts, debunk rumours, point out logical fallacies
  • Add context and background
  • Note typos and reporting blind spots
  • Stay on topic

Do not:

  • Use sexist, classist, racist, homophobic or transphobic language
  • Ridicule, misgender, bully, threaten, name call, troll or wish harm on others
  • Personally attack authors or contributors
  • Spread misinformation or perpetuate conspiracies
  • Libel, defame or publish falsehoods
  • Attempt to guess other commenters’ real-life identities
  • Post links without providing context

LATEST STORIES

The Barometer

Do You Think Naheed Nenshi Will Win the Alberta NDP Leadership Race?

Take this week's poll