Defending PISA.

Defending PISA. If Mark Schneider has doubts about the usefulness of the Programmefor International Student Assessment (PISA) ("The InternationalPISA Test," check the facts, Fall 2009), he should consider whetherthe U.S. has used PISA effectively. Among the G8 economies, the U.S.assessed the second smallest number of students for PISA and collectedthe least contextual information, limiting the inferences that can bedrawn for states and the usefulness of PISA for policy. While much ofthe industrialized world has extended PISA toward interactive electronictests, the U.S. stuck to paper-and-pencil versions. While Schneiderrightly notes that only longitudinal studies can establish causality,Australia, Canada, and Denmark are already implementing them, keepingtrack of the students assessed in PISA to find out how their knowledgeand skills shape their subsequent life opportunities. In virtually everyother federal nation, whether it is Canada or Mexico in North America;Belgium, Germany, Italy, Spain, Switzerland, or the U.K. in Europe; orAustralia in the Pacific, individual states have implemented PISAsuccessfully, employ it effectively, and find it useful for policyformation. They recognize that the yardstick for educational success isno longer improvement by state and national standards alone, but by thestandards of the best-performing education systems internationally. Schneider worries that international assessments, like anyevaluations, embody judgments about what should be measured. That is so,and much of their value lies in allowing states to see their ownstandards through the prism of the judgments that the principalindustrialized countries make collectively as to what skills matter forthe success of individuals in a global economy. Do international assessments provide causal evidence on what makesschool systems succeed? No, but they shed light on important features inwhich education systems show similarities and differences, and, bymaking those features visible, can help to ask the right questions. Arethe contextual data currently used for this perfect? Certainly not, andOrganisation for Economic Co-operation and Development (OECD) nationsconstantly review and refine them. The methods used, however, are farmore robust than the ways in which Schneider's organization ispatching together data from U.S. states and international assessments tosuggest to states that they can pass over a process of thoroughinternational benchmarking. ANDREAS SCHLEICHER Directorate for Education Organisation for Economic Co-operation and Development Schneider responds: Mr. Schleicher fails to address my concern that because PISA'spolicy advice is not based on methodologically sound analysis and oftenfits preconceived notions, states will not get reliable guidance.Second, it is for this nation to decide how PISA fits into the U.S.system of testing and data-collection efforts in which we have investedhundreds of millions of dollars, including, for example, the NationalAssessment of Educational Progress (NAEP) and student-based longitudinaldata systems. Mr. Schleicher's comparison of the United States tocountries that have not made similar investments is unhelpful. Finally,states have spent millions of dollars to ensure that their tests matchwhat they expect schools to teach in each subject. How can states alignwith an international test that admits, even celebrates, not testingwhat schools teach?