As parents and students digested the results of a dizzying week of revelations about scores on state tests, education officials and observers turned toward a question aimed at avoiding future fiascoes: What went wrong?

The tests were supposed to be the culmination of one of State Schools Superintendent Kathy Cox's most ambitious achievements. Shortly after coming to office in 2003, Cox embarked on an effort to overhaul state standards for student learning by reforming the unwieldy state curriculum then in place.

Despite controversies over whether to include the word "evolution" in the new standards and whether the Civil War was given short shrift, the new curriculum finally won board approval.
Then came Monday, when Cox warned school administrators and the public that 70-80 percent of sixth- and seventh-graders were likely to fail the state's social studies test and that 40 percent of eight-graders were expected to fail the math test they needed to pass to advance to high school.

By midweek, Cox was vacating the social studies scores and promising to name a panel of educators to find out what happened. At the same time, she stood by the math results, a move that could send thousands of children into summer school, potentially disrupting local systems' budgets and families' summer plans.

Even for those who were expecting some slip in the numbers as students tried to grasp a more rigorous curriculum and perform on the resulting tests, the scores were surprising.
"I think we know there are going to be bumps along the way," said Tim Callahan, a spokesman for the Professional Association of Georgia Educators, the largest teachers group in the state. "But it seems like these bumps have been far more severe than they need to have been."

Flawed test?

Many suspected that one of the reasons for that, at least with regards to the social studies test, was some sort of glitch with the test itself.

Speculation that the test company might have made an error in scoring the exams was quickly dismissed by Cox. Instead, the theory that seemed to be gaining the most traction late last week was that the test didn't line up with what students were taught.

The best-publicized complaints from students and teachers revolved around "almost inane questions that were on the test that purport to measure knowledge of social studies that sounded more like Trivial Pursuit," as Herb Garrett, executive director of the Georgia School Superintendents Association, put it.

That was the idea Cox endorsed in an opinion piece published in the Atlanta Journal-Constitution.

"As a social studies teacher, it appears to me that in some areas, our social studies curriculum is too broad, while the questions on the test were more specific," Cox said. "This is not fair to the teachers or students."

There was a certain irony in that statement. The main reason given for moving to the new curriculum, called the Georgia Performance Standards, was that the old state checklist was far too broad. The original draft of the new social studies standards tried to fix that by removing some material, only to cause complaints about what was being left out. The draft was changed as a result.

Interest has since pivoted to whether the state knew, or should have known, that students were going to fare poorly on the new test. While noting that he wasn't familiar with the specifics of what happened with the social studies test, former Department of Education official Franklin Shumake said the state doesn't just put an exam before the students.

"We do a lot of trials, usually, on test items before we use them," he said.

For example, a student might get 50 questions on a test -- 40 that count, and 10 that are going through a trial run.

The Department of Education did use some trial questions, officials said, in 2007. And they conceded that students did poorly on them.

"That might have been a red flag for them to go back and slow down a little bit and make sure they had a good match between curriculum and test," Callahan said.

But the agency points out that the students who were stumped by the trial questions hadn't been taught under the new curriculum, which went into effect this past school year.

"The results from those field-test items are only reflective of the strength of those items themselves and provide no indication of how students would perform," department spokesman Dana Tofig said in an e-mail.

Shumake also noted that testing is far more widespread since the passing of No Child Left Behind, a federal act that requires states to keep track of how students are doing. That has put more demand on the companies that design the exams, with all 50 states now requiring their own tests.

"They are challenged, I'm sure, to do what's appropriate in each state," he said.

Enough training, communication?

Other culprits, though, were also being blamed. Teachers and administrators were pointing at state officials, saying the Department of Education didn't do enough to train teachers how to teach students under the new curriculum.

Oglethorpe County Schools Superintendent Jeffery Welch said the new curriculum amounted to a "quantum shift" for teachers as well as for students.

"From my perspective, we have not put in the staff-development money or time to be able to make that shift effectively," he said.

In addition to a lack of training, Callahan said the department didn't keep classroom teachers in the loop when it came to the new curriculum.

"There's a tendency on the part of state departments to think if they communicate with the superintendents or they communicate with the county level officials, that that somehow magically translates to every classroom teacher in the state," he said. "The fact is, it doesn't."

Math scores significant

Going forward, the test that is likely to have the most impact on students and schools is the eighth-grade math exam. Cox has made it very clear that she intends to stand by those results.

"We have full confidence in the math curriculum," Tofig said. "We have full confidence in the math assessment. ... There is no discussion about changing anything related to math."

Students are required to pass the test to advance to high school, meaning many will head to summer school and retake the exam in an effort to move on. The total numbers aren't known; state officials likely won't get their first look at hard figures on how many students passed or failed until late May, and the numbers won't be released until at least early June. School district numbers will follow 10 business days later, and school-level figures will be released 10 business days after that.

"I think there's a confidence on the part of the department that a lot of the kids who retake it will pass it," Callahan said.

If not, students are allowed an appeal to district officials and can be promoted to the next grade anyway, Tofig said.

Meanwhile, some question whether the math scores aren't just as problematic as the social studies marks.

"If I were a classroom teacher and I gave a test to my students, and 40 percent of them failed, I'm not so sure that I wouldn't be taking a look at the test itself," Garrett said.

In any case, schools are bracing for an influx of summer school students that they weren't expecting when budgets were written almost a year ago. And without district-level data, it's still hard to know which areas need to be the focal point at summer school.

"It's not only a budget issue ... but now that we have (the resources), what do we do with them?" Welch asked.

Cox has promised to try to help districts cope with the extra budget hit, and has asked the federal government to give the state some flexibility on the requirements of No Child Left Behind, which takes the math scores into account when determining which schools are meeting standards.

Others are already beginning to look for some upside to the mess.

"I think it's a learning experience for the superintendent and the department, certainly a learning experience in an unpleasant way for a lot of kids and parents and schools," Callahan said. "But I hope that we will learn from this positively and do better moving ahead."