No PISA for American Children?
Reading the PISA 2006 report, I was surprised to find no mentioning of USA reading scores in the Reading section. They did participate, but according to
The PISA has been given every three years since 2000 to fifteen-year-olds in the thirty member countries of the Organisation for Economic Co-operation and Development (OECD) and in twenty-seven partner countries. Students are tested in reading and in mathematical and scientific literacy to assess whether they have “acquired the knowledge and skills essential in everyday life,” according to the OECD. The 2006 results, which concentrate on students’ scientific literacy, also included math and reading, however, a printing error resulted in a decision not to include U.S. students’ reading results for 2006 in the 2007 report.
This brief from the ED gives additional details, although not much:
In the US, because of an error in printing the test booklets, some of the reading items had incorrect instructions and the mean performance in reading cannot be accurately estimated. The impact of the error on estimates of student performance is likely to exceed one standard error of sampling. No results are therefore reported for the US.
The PISA FAQ has this to say:
United States
In PISA 2006, in the United States an error in printing the test booklets, in which the pagination was changed and instructions for some reading items directed students to the wrong page, may have affected student performance. The potential impact of the printing error on student performance was estimated by examining the relative performance of students in the United States on the item set that was common between PISA 2006 and PISA 2003, after controlling for performance on the items that were not likely to be affected by the printing error.The predicted effect of the printing error and the wrong directions on student mean performance on the reading test was up to 6 score points, and thus exceeds one standard error of sampling. Reading performance data for the United States are therefore excluded from this publication and the PISA database.
The predicted effect of the printing error on student mean performance on the mathematics and science tests was one score point. Mathematics and science performance data for the United States, therefore, have been retained.
Another report has more to say:
U.S. READING RESULTS ON INTERNATIONAL COMPARISON THROWN OUT DUE TO PRINTING ERROR: Alliance President Calls for Retest
On November 19, officials from the National Center for Educational Statistics announced that the United States’ reading scores on the 2006 Programme for International Student Assessment (PISA) had been invalidated because of an error in the printing of the test. The other results from PISA, which tests fifteen-year-olds in fifty-seven countries in mathematics, science, and reading literacy, are still scheduled to be released by the Organisation for Economic Co-operation and Development (OECD) on December 4.
The error occurred because of a printing problem by RTI International, the firm with which the U.S. Department of Education contracted to administer the test. Originally, students who opened the test booklets were supposed to find a reading passage on the left page and a series of questions related to it on the right page. However, in printing the test booklets, RTI International noticed that the color from the cover of the test booklet bled through the first sheet. As a result, it decided to begin the test on the first right-hand page. Students opening the booklets were instructed to answer questions related to a story or passage on the "opposite page." Because of the new layout, those directions were incorrect.
Mark S. Schneider, the commissioner of the National Center for Education Statistics, which oversees U.S. participation in the exam, called the printing error a "major loss for the study" and an "embarrassment" to everyone involved.
I am surprised that 4th graders would be thrown off by something like this. Apparently they had similar problems in math and science but the consequences were small enough that they decided to include the results. Were kids who took the reading test somehow more literal in reading the instructions?
April 11th, 2009 at 12:04 pm e
i was an examinar in 2005-2006 in Qatar and this year i want to share them to correct the exam so i want to remind how do we correct and what is the critiria for coding
May 24th, 2009 at 11:45 pm e
It wouldn’t have mattered. The American education system is a total and corrupt failure. Their reading scores would’ve probably fallen behind just as much, if not more, than both the science and mathematics scores.
If you don’t believe me then just visit a public high school. Students are burned out, uncaring, and trouble-makers. How do know? Because I have to attend the damn place every day!
I swear, if education in this country gets any worse (is that even possible?), then we might as well dismantle the entire educational system and be done with it. Otherwise we’ll end up worse than some third world countries.