Institute for Education Policy StudiesInstInI
Graduate School of Education and Human Development

Is It Real for All Kids?
A Framework for Equitable Assessment Policies
for English Language Learners

Mark W. LaCelle-Peterson
State University of New York at Geneseo

Charlene Rivera
The George Washington University

TABLE OF CONTENTS

ACKNOWLEDGEMENTS

We want to thank Mary Ellen Gallegos and Charles Stansfield for their helpful comments on drafts of this article.

CENTER for POLICY STUDIES:
Graduate School of Education and Human Development

Is It Real for All Kids?
A Framework for Equitable Assessment Policies
for English Language Learners ¹

Mark W. LaCelle-Peterson
State University of New York at Geneseo

Charlene Rivera
George Washington University

Many people in education hope that reform will bring positive change for all students in the United States. However, Mark LaCelle-Peterson and Charlene Rivera argue in this article that, unless educational reformers reflect seriously on the implications of assessment reform for specific groups of students, among them students whose first language is not English, little meaningful change will occur. The authors present a demographic profile of English language learners, propose a definition of educational equity and excellence, and outline the range of educational goals the definition implies. They argue that it is erroneous to assume that changes that affect monolingual English students favorably will automatically do the same for English language learners, and offer options and recommendations for more appropriate assessment policy and practice for English language learners.

The rhetoric of the education reform movement promises great things for students in U.S. schools: the rising tide of higher standards coupled with new assessment technologies will, it is hoped, lift all students -- including those who are not now succeeding -- to new levels of educational achievement. The confident assertion that "all students can learn" has become a mantra at educational conferences of all types, and the twin efforts of standards setting and assessment reform are promoted as central means to that end at all policy levels. But will these new efforts deliver on their promises? Will they, as a colleague likes to ask, "be real for all kids?"

If the promises of reform are indeed fulfilled, the United States will achieve new levels of equity and excellence in its educational institutions and in society. However, as the second decade of reform dawns, it appears to us, from our vantage point as observers of the national reform movement and as advocates for English language learners (ELLs), that the fulfillment of promised "success for all students" is still far off. This article is based on the premise that U.S. educators must reflect seriously on the implications of reform, and, in particular, assessment reform, for those students who have not, historically, thrived academically. Unless we do so, current efforts to improve U.S. education will amount to little more than a reformulation, perhaps with new labels, of the status quo. There may be little, if any, meaningful change for those who hope to, and stand to, benefit most from reform in educational thought and practice -- among them English language learners. While the development of common high standards and related innovations in assessment practices constitutes a commendable commitment to the ideals of equity and excellence, wide-scale implementation of changes without consideration of the particular strengths and needs of particular groups of students will only create disappointments.

Because ELLs historically have suffered from disproportionate assignment to lower curriculum tracks on the basis of inappropriate assessment and, as a result, from over-referral to special education (Cummins, 1984; Duran, 1989; Ortiz & Wilkinson, 1990; Wilkinson & Ortiz, 1986), changes in assessment will have much to do with whether or not the promise of reform becomes real for them. Though the motivating impulses and underlying commitments of the current standards-setting and assessment reform efforts are profoundly optimistic and egalitarian, the processes and results have neither considered nor adequately addressed the perspectives and needs of ELLs. The implicit guiding assumption appears to be that whatever curricular revisions and/or assessment innovations contribute to the success of monolingual students will also work for ELLs -- that once ELLs know a little English, the new and improved assessments will fit them too. Research on language learning, however, shows that even though minimal survival skills in English can he achieved fairly quickly, the ability to use English for academic purposes -- that is, school success skills in English -- takes longer to develop (Collier, 1992; Cummins, 1989; Ramirez, Yuen, Ramey, & Billings, 1991). While ELLs can and do learn in accordance with high academic standards, their accomplishments will likely be underestimated if they are assessed in the same way as their monolingual peers. We argue that assessment systems must be designed to 1) help all students -- not just monolingual learners -- meet the educational goals set by and for them, and 2) help educators monitor the effectiveness of educational programs in enabling all students to reach those goals. Assessment -- the gathering and interpreting of information about students' knowledge, achievements, or accomplishments in relation to an educational goal or goals -- must be appropriate for the learners being assessed. Thus, assessment systems must be designed with the whole learning experience -- including both linguistic and academic components -- of each group of students in mind. In a multicultural, multilingual society, assessment policies must seek excellence and equity simultaneously, or they will accomplish neither. This article presents considerations that should guide assessment policy development that seeks to ensure educational equity and excellence for ELLs in the current reform cycle.

Underlying Assumptions

We want, at the outset, to state clearly the assumptions that underlie this article. First, when it comes to languages, we believe that more is better, both for individuals and for the nation. Bilingualism is not an elusive goal, but rather a relatively common human experience and an even more common educational goal. Other national educational systems, the German and Japanese, for example, uniformly expect that students will learn at least one language in addition to their native language, and will study it for a number of years. Rather than viewing the native languages of ELLs as "problems to be overcome" or "excess baggage to be shed," we believe that individuals have the right: 1) to participate fully in the best available academic programs offered through local schools; 2) to receive the best possible educational program to help them acquire proficiency in English; and 3) to participate, if possible, in an academic program that enables them to develop literacy in their native languages.

Our second assumption is that, historically, we as U.S. educators have not applied the best educational thinking to issues facing ELLs. While there have been important breakthroughs and advances in improving the quality of education for ELLs over the past three decades, most, if not all, of these advances have come through litigation and legislation -- not through the disinterested pursuit of educational excellence (Lyons, 1988). Experience and research must form the basis for building educational programs, and, to a larger degree than is generally realized, the experience and research base we need to design and implement appropriate programs for ELLs already exists. Given what we know about language learning and about instruction of ELLS, in most cases the best program options for ELLs are bilingual education programs that provide the opportunity for ELLs to become biliterate.² Currently, however, many instructional programs for ELLs are submersion programs in which ELLs who are just beginning to learn English are instructed entirely in English -- sometimes in special sections taught by teachers who are trained in English-as-a-second-language (ESL) methods, sometimes in classes with monolingual peers taught by non-ESL specialists. Submersion programs have proliferated despite research that shows that students learn more when they are taught in a language they can understand and with materials that are comprehensible to them (Cummins, 1989; Krashen, 1981; McLaughlin, 1992; Ramirez et al., 1991). While second language learners do need exposure to the target language, in this case English, students who are complete novices in the second language can be overwhelmed and exhausted by spending the whole day receiving incomprehensible input; providing content instruction in the students' first language in addition to English language instruction produces both development of greater English proficiency and higher achievement in content areas (Ramirez et al., 1991).³ Students who remain in such bilingual programs for longer, rather than shorter, periods of time achieve more (Collier, 1992; Ramirez, 1992). While the research base may not contain all the answers, educators and researchers with expertise in areas such as second language acquisition and language proficiency assessment are available to contribute to the development of new policies and programs and in the reforms of existing ones. Too often they are not called upon because there is the belief that "one size fits all"; that is, what will work for monolingual students should work with ELLs after they have acquired some proficiency in English. We believe that the failure to recognize and learn from existing expertise regarding second language learning, cultural influences on learning, and program impacts on ELLs results in an unnecessary and unjustifiable "systemic ignorance" about the potential and capacity of ELLS.

Our last assumption is that the current educational reform movement represents a hope-filled new opportunity to focus the best of our collective educational wisdom on the problems facing all U.S. schools, students, and communities. The focus of reform must be on making better educational opportunities and outcomes part of all children's experience, but conscious effort is needed to ensure that these benefits accrue equitably to all groups of students within the system.

Goals

Our goals in this article are to clarify equity issues related to assessment of ELLs and to present a framework that might help structure discussions about how to address those issues. In the first section, we present a demographic profile of ELLs, propose a definition of educational excellence and equity for ELLs, and outline the range of educational goals this definition implies. In the second section, we focus on assessment per se, discussing assessment challenges that face ELLs and presenting a framework to guide efforts to develop comprehensive assessment systems to promote educational excellence and equity for ELLs. In the third section, we discuss the policy options that are available to educators who are leading and implementing assessment reforms and present recommendations for educators and others as they tackle the technical challenges of developing useful, meaningful, educational, and equitable assessment for ELLs.

English Language Learners: Diversity, Equity, and Excellence

Diversity

Who are ELLs? According to the 1990 census, over 6.3 million people between five and seventeen years of age (13.9 percent of all school-aged people in the nation) reported speaking a language other than English in the home (Waggoner, 1992). Statistics from the fifty states, Washington, DC, and Puerto Rico have indicated that approximately 2.26 million K-12 students have "limited English proficiency"; this means that more than one of every twenty students enrolled in public and private schools is learning English as a second or other language (United States Department of Education, 1992).⁴ ELLs are also a growing group; between 1980 and 1990 -- the years when the current school-reform movement took shape -- the number of people in the United States aged five years and older who reported speaking a language other than English at home increased by 38 percent (Waggoner, 1992). Drawing on the data from the U.S. Department of Education, the National Center for Education Statistics, and state sources, Olsen (1993) reports that the number of K-12 ELLs with limited proficiency in English increased by 51.3 percent between 1985 and 1991.

Although ELLs share one educationally relevant variable -- the need to increase their proficiency in English -- they differ in language, cultural background, and family history. The umbrella of "ELL" includes students from Native American communities that have been in what is now the United States from time immemorial; students from other long-established language minority communities, such as Franco-Americans in the Northeast, Latino/as and Chicano/as in the Southwest, and the Amish in the Midwest; and students from migrant and immigrant groups who represent the most recent arrivals in a virtually unbroken series of migrations that have brought linguistic diversity to North America. Research shows that each of these communities may interact with schools differently (Au, 1980; Au & Jordan, 1981; Heath, 1983; Jacob & Jordan, 1987; Ogbu, 1987; Phillips, 1983; Trueba, 1989). In addition to belonging to a wide range of ethnic groups within U.S. society, ELLs come from families of differing socio-economic status and differing educational orientations. The diversity among ELLs needs to be recognized, lest all ELLs be regarded as a monolithic group with a single defining educational characteristic: use of a non-English language. Indeed, while language represents an important, educationally significant variable that is most often conspicuous by its absence from U.S. educational discourse, it is only one of many educationally relevant characteristics of any individual English language learner, whose whole identity, including cultural heritage, ethnic group affiliation, gender, and individual learning differences, must be taken into consideration in educational decisions. For example, a newly arrived eleven-year-old immigrant ELL, whose schooling has been interrupted by war and frequent moves, and who has not developed literacy in his or her native language, faces relatively greater educational challenges than another recently arrived eleven-year-old ELL who has developed literacy skills in his or her native language, succeeded well academically, and already studied some English in school before coming to the United States.

Equity in Educational Programs

Given the large, growing, and diverse groups of ELLs who are entering U.S. schools, virtually every teacher and every school district serves, or will serve, ELLs. Therefore, all assessment programs in all states and localities should be responsible for reporting on the educational progress of ELLs in terms of developing English language proficiency and content knowledge. Since assessment should play an enabling or a supportive role in education and does not exist for its own sake, development of equitable assessment systems and policies to support educational excellence is only possible within the context of an equitable education system that promotes excellence for all students. Four elements must be considered in seeking educational excellence and equity for ELLs: 1) access to the full range of content knowledge that is valued by the school, community, and society; 2) participation in meaningful interaction with challenging subject matter, with classmates, and with teachers; 3) benefit from and success in learning that challenging subject matter (Faltis, 1993); and 4) continued development of their native language abilities to the greatest possible extent.

Access for all students to a commonly valued curriculum is a minimal equity criterion; meeting it would mean a significant increase in excellence and equity for many ELLs. Before students are held accountable for meeting educational standards, schools should be held accountable for giving all students the opportunity to learn the full scope of what is tested or measured (Stevens, 1993). ELLs frequently encounter barriers to this access because schools fail to provide courses in such areas as science and mathematics (Minicucci & Olsen, 1992), or else track ELLs into courses that do not address challenging content (Oakes, 1990), or teach in ways that fail to make the content comprehensible to ELLs who are just beginning to learn English. In a study of ELLs' access to secondary school curricula in California, less than one-fourth of the secondary schools surveyed offered the full range of content courses for ELLs (Minicucci & Olsen, 1992). Among the factors responsible were the lack of trained teachers and assessment practices that failed to identify students' academic needs. On a national scale, U.S. Department of Education reports from recent years indicate that an increasing proportion of identified ELLs with limited proficiency in English are not being served by appropriate programs; in the most recently available report, nearly one in four identified ELLs (23 percent) is not enrolled in a federal or state supported program designed for ELLs (U.S. Department of Education, 1991, 1992). These students have instead generally been placed in monolingual programs and courses designed for monolingual English-speaking students.

The second dimension of educational equity for ELLs, meaningful participation in learning, implies both full participation in well-taught, challenging classes that aim to meet the same educational goals set for all students. The goals for students and the criteria by which their success in meeting those goals will be measured must be made explicit. Too often, educational goals for ELLs are reduced to the acquisition of a minimal level of proficiency in English with the expectation that content can be learned after English proficiency increases. Given that 1) proficiency in using English for academic purposes takes much longer to develop than the ability to use English for social purposes, and 2) academic proficiency increases gradually through using English in academic settings, students cannot afford to wait for academic English proficiency to develop (Collier, 1992; Cummins, 1989; Ramirez et al., 1991). Successful instructional strategies have been developed by teachers working with ELLs. Such strategies are guided by clear, explicit learning goals and provide ELLs who are just starting to develop English proficiency with meaningful interaction in challenging content in the native language and/or in English (Rivera & Zehler, 1990, 1991). ELLs enrolled in programs using such strategies gain content knowledge and proficiency in academic uses of English -- both of which they will need for future educational success. One exemplary program being implemented in Cambridge, Massachusetts, is the Cheche Konen project, which engages ELLs in middle and high school in "doing science" like practicing scientists (Rosebery, Warren, & Conant, 1989, 1992). In a year-long study of students in the Cheche Konen project, researchers found that students with no education in science and little formal schooling developed analytical skills that enabled them to create and use scientific discourse to analyze hypotheses and to "organize and give direction to their reasoning" Rosebery et al., 1992, p. 20).

A third dimension of equity for ELLs is ensuring that students benefit from their educational experiences and succeed in reaching the educational goals they set. Pursuing this dimension demands that schools offer the support structures that students need to succeed. Within the school, this provision requires that the same high expectations be held for all students, including ELLS. This dimension of equity recognizes that, like monolingual English-speaking students, ELLs need both access to higher level courses and the guidance and encouragement to take them. The reasons for lower rates of college entry and progress of ELLs at the university level have been well documented (Astin, 1982). By the time an ELL has acquired sufficient English to compete with his or her monolingual English-speaking peers, she or he has not taken the same number of high-level courses and consequently scores lower on college admission tests (Duran, 1990; Mehan et al., 1992; Pennock-Roman, 1990). The full range of academic requirements necessary to participate in post-secondary educational opportunities must be meaningfully explained early and clarified often for all students; schools vary greatly in the degree to which expectations about college attendance for ELLs are expressed and the degree to which information about college admissions is made available to them (Rodriguez, 1992). Support structures necessary for ELLs's educational benefit must also extend outside the walls of the school; if participation in school activities results in student alienation from their families and language communities, students' educational prospects are seriously compromised (Wong-Fillmore, 1990). Therefore, efforts to incorporate the support of ELL families and community members are essential to motivate ELLs to stay and succeed in school.

Fourth, and finally, continued development of the native language should be fostered, both in the interest of providing equitable education to ELLs and in the interest of the nation in this ever-shrinking, highly interdependent global community. For individuals, the use and development of one's native language along with deepening knowledge of one's own ethnic culture and heritage are invaluable birthrights. We recognize that situations will vary from school to school, depending on the number of linguistic groups present, the desires and available resources within the ELL communities, and the experience and expertise of educational personnel; nevertheless, to the greatest extent possible, ELLs should continue to receive instruction in and through their native language in order to develop full bilingualism (Hakuta, 1986). From the national perspective, the easiest and most effective way to develop the multilingual skills needed to interact effectively in the global marketplace is to preserve our collective linguistic abilities from generation to generation.

Excellence: World-Class Education Goals for ELLs

If our educational system met the equity criteria outlined above, educational goals for ELLs would be explicitly linked to our goals for all students, as embodied in: 1) the National Education Goals; 2) the content standards that have been or are being developed by professional associations (for example, the standards in mathematics developed by the National Council of Teachers of Mathematics); and 3) state and local goals statements and curriculum frameworks. Building on the dimensions of equity for ELLs discussed above -- access, participation, benefit and success, and native language development -- we hold that making the goals of higher academic achievement real for ELLs must begin, but not end, with consideration of the common challenge that characterizes them as learners and distinguishes them from their monolingual English-speaking peers: they are all in the process of learning a second (or third, or even fourth) language, and as a result, they must learn more than their monolingual peers. ELLs need to work toward English proficiency for both social and academic purposes; need to develop knowledge and abilities in the content areas as defined for all students at the same rate as their monolingual peers; and, to the extent possible, continue development of their native language abilities. Figure 1 represents a hypothetical summary of the emerging goals as set by the National Educational Goals and discipline-based standards-setting efforts for monolingual students in the United States. Figure 2 represents parallel goals we must hold for ELLs: all that the monolingual students must accomplish, plus social and academic proficiency in two languages.

Clearly, setting such comprehensive learning goals for ELLs will have implications for how we, as educators, think about the time that must be given for learning, the instructional resources that must be made available, and the intensity of the educational interactions that will be required.

One way for monolingual readers to conceptualize the magnitude of learning that is expected of ELLs is to imagine that, in addition to the other ambitious goals U.S. educators have for native English speakers, we added the goal of "developing complete social and academic proficiency in a second language."⁵ To make learning goals truly equal for all students, we might, for example, require all native English speakers to learn one of the other major North American languages, French or Spanish, or a non-European world language such as Japanese, Chinese, or Hindi, and learn it well enough to complete some or all of their academic studies in high school in that language.⁶ The purpose of this "thought experiment" is not to advocate that such a policy be implemented immediately (though if it were implemented, the cultural and cognitive benefit to all U.S. students, particularly monolingual English speakers, would be great, and considerable progress toward the often-sought educational parity with our European and Asian economic competitors could be approximated). Rather, the intent is to provide a sense of the magnitude of the learning challenges that face ELLs -- challenges at which they routinely succeed, despite limited academic support.

Framework for Equitable and Comprehensive Assessment for ELLs

Educators assess students for many different purposes, but our focus here is on assessment that is undertaken for the purpose of gauging the effectiveness of an educational program or institution. Such assessment is ideally undertaken to provide feedback that will allow instructional leaders to improve instructional programs, as Ralph Tyler (1949) suggested over four decades ago. More recently, assessment has focused on accountability, and assessment programs have been linked to rewards or sanctions for individual teachers, schools, or school systems. We refer to this type of assessment as "accountability assessment" in deference to the current climate (though we hope that eventually, as emphasis shifts, "assessment for program improvement" could become a more appropriate label). The discussion that follows focuses on developing systems of accountability assessment that ensure equity and promote excellence for ELLS.

As part of the current reform movement, assessment policies have increasingly emphasized accountability at state and local levels, and, in pursuing that accountability, a wider range of assessment technologies is being employed. Fundamental dissatisfaction with the "performance" of U.S. schools and students -- linked in the minds of critics to lagging U.S. economic performance -- led to demands for greater accountability from all schools and for the creation of higher educational standards. Likewise, fundamental dissatisfaction with the instructional influences of standardized testing lead those seeking to develop higher standards to recommend the use of performance assessment tasks, long-term projects, and portfolio assessment systems as means of documenting students' progress toward the new, higher standards (Mitchell, 1992; Resnick & Resnick, 1991). The greater flexibility and closer link to instruction that are possible in new assessment technologies suggest that their expanded use may bode well for ELLs because they open the possibility of assessing students' knowledge through a wider range of modalities; however, many ELL educators are cautious, both because development efforts have, to date, paid scant attention to ELLs' assessment needs, and because new assessment technologies have yet to be field tested with ELLS.

Though a variety of assessment "technologies" (Madaus, 1993) are available to educators, accountability assessment generally relies on two types of assessment technology: standardized tests, which have traditionally been used in large-scale assessment programs, and performance assessments, which have traditionally been used to inform instruction but are increasingly being incorporated into large-scale accountability assessment programs. Equity concerns begin with the development of any assessment instrument and extend to the scoring and reporting of results. Any assessment technology used to assess ELLs should conform to the standards for testing linguistic minorities published in the Standards for Educational and Psychological Testing developed by a committee representing the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education (American Psychological Association [APA], 1985). While equity issues related to the administration of standardized tests to ELLs have a longer history of discussion (Duran, 1983, 1989; Geisinger, 1992a, 1992b, 1992c; Haladyna, 1992; Padilla, 1979) than those related to performance assessments, equity concerns over both are similar. However, particular attention must be paid to the scoring process for performance assessments because fair and valid ratings of ELLs' performances depend on the background knowledge of the rater (e.g., knowledge of language acquisition processes).

Figure 3 summarizes the technical questions and other equity considerations that must be addressed in relation to any particular instrument or task used to assess ELLs. To fulfill the educational goals that must be set for ELLs, assessment systems that provide information on each ELL's developing language abilities and academic achievement in content areas are needed: Information on language learning should include measures of the student's proficiency in both academic and social uses of English, and, if it is part of the instructional program, proficiency in his or her native language. Information about the student's academic achievement in content areas should be collected in ways that allow students to show their knowledge, skills, and abilities, through the medium of the language or languages in which the material was taught. Since individuals differ in the process of learning a second language in terms of whether spoken or written expression develops more quickly, latitude should also be provided in terms of the mode of expression used, the length of time allowed for completion of a task, and the degree to which fluency of expression, as distinct from substantive content, is counted in scoring the results.

Such comprehensive, flexible assessment systems do not currently exist in most educational jurisdictions, although efforts by the Council of Chief State School Officers (CCSSO, 1991, 1992) have started discussions aimed at improving, at the state level, assessment and monitoring practices of ELLs who are starting to develop English proficiency (those usually labeled "limited English proficient"). The potential value of such efforts in encouraging common definitions and wise assessment practices is great; however, much work remains to be done in developing optimal assessment systems for ELLS. Assessment development efforts must focus on the creation of systems that provide evidence of what students have accomplished, that facilitate achievement of educational goals, and that meaningfully inform instruction.

Equitable Assessment: What Do Assessments Tell Us When Second Language Is a Variable?

The basic question to be raised in regard to equity is: will new assessment approaches result in greater equity across all groups of students? At the most general level, this question is addressed in considering whether students have been given the opportunity to learn the necessary content covered in performance-based or curriculum-linked assessment measures. For ELLs, however, content coverage is only one dimension of equity to consider; language itself constitutes a separate issue, one that raises important questions of validity and reliability regarding the assessment measures, and poses particular questions of equity. Because some ELLs may have the content knowledge and/or the cognitive ability needed to perform successfully on assessment tasks but are not yet able to demonstrate in English what they know, assessment procedures may be neither equitable nor yield valid results for ELLs (Gandara & Merino, 1993). Assessing diverse groups of students -- some of whom speak English as their first language, others of whom are learning it as a second or other language -- with instruments written in English and normed on monolingual English-speaking students inevitably yields data of unknown validity that cannot be meaningfully aggregated.

In seeking to answer policy questions, such as what elements to include in an assessment system, or which student groups to include in a testing program, policymakers must ensure as a first step that the instruments in question will yield, as far as can he determined, accurate and beneficial information about students. First, policymakers must bear in mind that the validity of any given standardized test is not absolute: a test is valid, meaning that the inferences one makes from the scores are likely to be accurate, only for populations for which it has been validated. Unless a test's manual indicates that particular groups of language minority students were included in the norming groups, the safest assumption would be that they were not. If translations of a test are available, the same validation and norming process should have been repeated for the translation; comparability of different versions of a test cannot be assumed and must be proven. The contents of standardized test items and of materials included in performance assessment tasks must also be scrutinized for evidence of cultural, gender, or other biases. In addition, thought should be given to whether or not test administration procedures unduly penalize students for whom the testing format is unfamiliar or for whom the standard amount of time allotted is inadequate due to unfamiliarity with the test language. In the case of performance assessments, special attention must be given to the adequacy of the scoring criteria for evaluating the work of ELLs, and also to the adequacy of the training given those who need a particular expertise in order to score material produced by ELLs. If reservations exist regarding any of these issues, the most equitable course is not to include ELLs in an assessment event, except on a pilot or test basis. If ELLs are included, data should be disaggregated and scrutinized by ELL, specialists.

As the Standards for Educational and Psychological Testing (APA, 1985) point out, whenever students who are still in the process of learning English are tested in English -- regardless of the content or intent of the test -- their proficiency in English will also be tested. The same is true for any second-language testing situation.

The following example illustrates how, in an educational setting in which the learner is in the process of learning a new language, every test will inevitably measure both what the learner knows about the particular subject matter and the learner's proficiency in the particular language. A monolingual English speaker who had just graduated from high school spent a year as an exchange student in Sweden, where she participated as a student in two standardized achievement tests in science subjects -- physics in the fall, and chemistry in the spring. During her last year in high school she had studied both physics and chemistry and had performed well in them. At the Swedish gymnasium, she enrolled in the second year of the physics and chemistry sequences. During the fall, after about three months of Swedish language study and two months of attending physics classes, she took a standardized, three-hour physics test in Swedish. The results for the visitor were, predictably, miserable. The low score, however, was not because she had forgotten all the physics she knew from the previous year; rather, the test had shown that her proficiency in Swedish for academic purposes was minimal at best. In the spring, after an additional six months of language study and review of chemistry, she participated in a Swedish-language, standardized chemistry exam. This time the results were more or less "average," reflecting a gain in her ability to accomplish academic tasks in Swedish since the fall. The results, however, still understated the student's mastery of chemistry -- the learner's academic language proficiency in Swedish was still not equivalent to her proficiency in English. From the individual's perspective, the tests gave an unfairly low estimation of her achievement; from the assessment system's perspective, the tests resulted in incomparable data because the low scores reported for the visitor did not have the same meaning as equally low scores for native Swedish speakers.

From the individual student's perspective, this example captures the assessment dilemmas that face one small subset of ELLs -- those who come to the United States with strong academic backgrounds from schooling in other countries. It is much less accurate in capturing the assessment dilemmas of another subgroup of ELLs -- those who are learning academic material through native language instruction, but who are forced to participate in English language assessment programs that ask them to give evidence of their content mastery in a language other than the language through which they learned it. And the example barely begins to parallel the assessment dilemmas that face the majority of ELLs -- students who are being taught content in English even before their academic mastery of English is up to the task.

A Comprehensive Assessment System Framework

Given the nature of second-language learning and the ambitious learning goals outlined above, assessment systems for ELLs need to be comprehensive, flexible, and responsive. While the specifics of any assessment system will be determined by state and local educational goals, there are several common criteria for appropriate and comprehensive assessment systems for ELLs: assessment systems should be comprehensive, attempting to assess all that students are learning; flexible, allowing students to show what they know in a variety of appropriate modes; progress-oriented, tracking students' progress from year to year, rather than producing only relative scores at one point in time; and student sensitive, bringing into the process the expertise of educators who know the needs and learning characteristics of particular groups of students -- in this case ELLs.

First, assessment systems should be comprehensive in the sense that they seek to provide an integrated account of all that ELLs are learning, both in language and in academic content areas. While individual tests or performance assessments will inevitably target specific areas, a complete picture of what each individual English language learner is accomplishing in each language and in academic content areas (and other additional areas determined by local or state policies) must emerge if schools are truly to account for ELLs' educational progress. Assessment systems designed with only monolingual, native English-speaking learners in mind would assess only the issues represented in Figure 1; to assess accurately ELLs' learning, all that is represented in Figure 2 must be the target.

Second, assessment systems should be flexible: multiple indicators should be used to assess ELLs' progress in languages and academic areas. Such flexibility will mean tracking language development in all four modes of communication (speaking, listening, reading, and writing), rather than through assessing the written mode alone. It will probably also mean assessing ELLs' achievements in content areas through oral reports and other presentations rather than relying solely on discrete-point assessment technologies. Thinking back to the example of the exchange student cited above, if either the student's grades or the school's ranking depended on the outcomes of the tests described, flexibility in the form of multiple achievement indicators would have benefitted everyone. For example, discussions with the student or observation of lab work in physics or chemistry would have allowed the student to exhibit what she knew through showing and telling rather than through reading and choosing from verbal options. While many educators claim (often rightly) that their students -- ELLs and non-ELLs alike -- are over-tested, it is, in many cases, equally true that students -- particularly ELLs -- are under-assessed in the sense that much of what they know and much of what they can do is not captured in current assessment methods.

Third, assessment systems should be reoriented to focus on students' progress over time toward established goals, rather than on comparisons. Cumulative learning records such as portfolios are particularly well adapted to providing evidence of student progress, though standardized measures can also be used in this way if student gains from year to year are tracked. Common reporting practices, however, do not present individual student gains. Shifting to a focus on student progress has implications for what kind of information about students is gathered in the assessments, as well as for reporting of accountability data for schools. It is important that individual students' progress be profiled for the benefit of the student, the parents, and all teachers who participate in the child's education. It is also important that schools, school districts, and school systems report student gains from year to year (for example, mean student gains between third and fourth grade), rather than tables of averages by grade level over a period of years (for example, comparing this year's fourth graders to last year's fourth graders).

Fourth, and finally, assessment systems should be sensitive to the particular needs of groups of students locally. This sensitivity could be accomplished by teams whose members bring relevant experience and expertise to the task of monitoring assessment systems and outcomes; for schools that enroll ELLs, assessment systems should be designed and monitored by teams that include individuals who are knowledgeable about second language learning and the assessment of second language learners, and who are acquainted with the ELLs in the particular schools.

Assessment Policies: Options and Recommendations

Given the high goals of both education reform and assessment standards, and in light of the educational needs and characteristics of ELLs, what assessment policy options should we pursue? Unfortunately, the two options that have been tried in the past do not offer much help, as both ignore the characteristics and needs of ELLs. Previously, testing of ELLs without consideration of the validity of the instruments for the populations being tested has been the most common option for assessment; excluding ELLs from accountability loops by exempting them from testing for an arbitrary length of time has emerged as a second option. The first option ignores educationally significant differences among learners by including ELLs in assessment programs as if their scores or ratings mean the same thing as those of their monolingual peers. Unless test publishers can provide evidence that ELLs' scores are valid and comparable to scores of monolingual students, such testing and data reporting result in bad experiences for the students and inaccurate information about schools or school systems. This option is neither ethically nor educationally defensible.

The second option, exempting students from testing, is commendable as a first step toward developing appropriate assessment systems, in that it recognizes the unfairness of arbitrary inclusion. Some educators advocate exempting ELLs from assessment programs for a number of years so that students' scores will not be misleading; however, exemption periods are usually so short (one to three years) that even in the best of programs, ELLs may still not have developed sufficient academic English proficiency to be tested accurately. A more serious problem with this option, however, from the perspective of equity, is that it literally creates a kind of systemic "ignorance" about the educational progress of ELLs. While it avoids the problems of aggregating data that have known discontinuities in meaning, this option leaves the school, district, or system utterly unable to account for the learning of these students. In the current reform climate, leaving schools "unaccountable" for their success at educating ELLs has led to calls for inclusion of all students -- even ELLs -- in accountability assessment programs. Yet, in high-stakes accountability contexts, schools that fear that the inclusion of data from ELI,s will lower their average are beginning to argue again, on different grounds, for the exemption or exclusion of ELLs from testing programs.

For the purpose of providing optimal and equitable education for ELLs, the most appropriate discussion regarding assessment policy is not about whether to include, exclude, or exempt ELLs from assessments. Rather, the discussion must center around two questions: how best to assess ELI,S, and how best to incorporate the data into accountability assessments of schools and school systems.

How should we assess ELLs? While no definitive answer can be given to this question, the choice of assessment technology to be used should be guided by the considerations outlined above. Regardless of which means of assessing ELLs is chosen, equity must be safeguarded. The best assessment policies will result not in the hurried establishment of inflexible policies and practices, but in the establishment of processes for experimenting and reviewing assessment strategies in light of the rapidly changing ELL population entering the schools. The best means of safeguarding the interests of ELLs and of assuring that the assessments are valid and useful to educational practitioners and policymakers is to create working groups of ELL, specialists and assessment specialists to develop, monitor, and revise the assessment programs. One characteristic of the assessment practices developed by these bodies, however, should be the ability to track individual students over time, so that their increasing ability in languages and academic achievement can be reported. Given the impossibility of determining to what extent a given assessment tool -- particularly a standardized instrument -- measures language familiarity or content achievement, such tracking is important. In this way, students' progress in both language familiarity and content will be evident, even if the component parts may be difficult to separate.

How should data from ELLs be included in accountability assessments? In collecting assessment data, student background information should be included, together with language background, school background (if student's schooling was interrupted, for example), and the length of time students have been receiving instruction in English as a second language and English-language instruction in content areas. This will make it possible to aggregate and/or disaggregate student data. Aggregated data will provide a snapshot of how the district, school, or class is doing in educating all students, regardless of language background. Disaggregated data will make it possible to compare ELLs' achievement with that of monolingual English students at the district, school, or class level. In reporting data on ELLs, then, students' progress over time should be reported not only in terms of what any grade cohort gains from one year to the next, but also in terms of what students at a given grade level gain from the previous year according to how long they have been studying English and/or studying content in English. Such reporting will allow more accurate judgments about the effectiveness of instructional programs, because baselines will have been established to give meaning to later achievement levels.

Ultimately, the quest for equity in assessment systems comes down to meeting two criteria: information about students must be accurately collected and reported, and the assessment process must benefit the students. The following guidelines for assessment developers and policymakers outline a process that will satisfy these criteria.

1. Involve ELL specialists, including teachers, from the beginning in the processes of developing assessment technologies and assessment policies.
   
2. Develop and field test policies of systematically accommodating differences in English proficiency in the administration of tests; since we recognize that ELLs need appropriate instructional programs, the need for appropriate assessment administration should also he recognized.
   
3. Develop assessment systems with the whole of students' learning goals in mind; for ELLs, this process may mean combining formal and informal assessment strategies in order to gain a comprehensive picture of students' abilities.
   
4. In the assessment process, keep track of students' educational program and years of schooling in English, so that meaningful disaggregations can be made to examine the performance of the schools and programs that are serving them.
   
5. Create assessment collaboratives at the local level, involving assessment experts and ELL educators to develop these kinds of meaningful policies. Collaboration among all levels of individuals with differing expertise, from policymakers to classroom teachers, will be needed, along with administrative structures to foster such interaction.

Conclusion: Does Reform Portend Promise or Peril?

We began this article by questioning whether the assessment changes that are part of the educational reform movement were likely to help or hurt the prospects of educational equity for ELLs. Efforts to reform assessment as part of overall systemic reform do not clearly bode well or ill for ELLs; while there are evident grounds for hope, there are equal grounds for caution. The trend toward utilization of performance assessment tasks is potentially hopeful, in that it represents an openness to expanding the range of ways in which students can show what they know and are able to do. That hope will be realized, however, only if ELL experts -- teachers as well as researchers --are involved at all stages of the assessment development process. Teachers and researchers who understand second-language learning and the educational issues of ELLs must participate fully in task development; must scrutinize task materials (texts, etc.) and processes for appropriateness to ELLs; and must field test the assessment instruments or tasks, evaluation criteria, rater training, and scoring processes with ELLs to ensure their validity and efficacy for this group of students. While improvements in standardized testing in general, and in standardized testing for ELLs in particular, can be expected to add incrementally to the equity of accountability assessment, largely through the incorporation of more performance-linked items, greater educational benefit and, arguably, greater equity can be expected from the expanded use of performance assessments if flexibility is maintained to tailor assessments to the needs and characteristics of students. It might be argued that the optimal assessment system for both instructional and accountability purposes would consist of a comprehensive assessment system at the school level, in which a complete picture of each student's growth and development is collected, largely through performance data. These data would, in turn, be available for matrix sampling for accountability assessment.

While such a system would most likely benefit all students, including ELLS, the ultimate safeguard for equity will be the increase in knowledge of and sensitivity to ELL issues on the part of assessment personnel. The potential peril that will remain if only the technical details of the system are changed and the human factors ignored is illustrated by the comments of an assessment specialist in a district experimenting with performance assessment in the elementary schools. When asked whether the initial results from performance assessments of kindergartners appeared to give useful information, the specialist replied that he felt the data had real validity problems because in some schools, ELLs in the bilingual classes scored higher than their monolingual peers. When asked whether the students' work had been scored again, or objectively compared, he replied that such close examination of the students' work had not been undertaken, yet he was confident that the ELLs simply could not have scored that well. Unless scoring criteria are equitable and rigorously applied, and unless assumption-laden interpretations are systematically kept out of scoring processes, ELLs will continue to be inequitably assessed.

We know of one state commissioner of education who wears a button on his lapel every day: the button reads "All Kids," to emphasize who is to benefit from educational reform. While we are hopeful about the assessment innovations that are rising out of the reform movement, we wish to underscore that making educational reform real for all kids will demand new partnerships, new collaboration, and new, common higher standards for assessment systems. In the past, and all too frequently in the present as well, ELLs and the educators who specialize in working with them have been on the margins of this nation's common schools, and ELLs' distinctive abilities in languages and their potential contributions to the schools have been overlooked. If the schools of the twenty-first century are truly going to be characterized by educational excellence for all students, they must be designed with all learners in mind -- including those who bring linguistic riches with them to school.

NOTES

¹ "English Language Learners" (ELLs) refers to students whose first language is not English, and encompasses both students who are just beginning to learn English (often referred to as "limited English proficient" or "LEP") and those who have already developed considerable proficiency. The term underscores the fact that, in addition to meeting all the academic challenges that face their monolingual peers, these students are mastering another language -- something too few monolingual English speakers are currently asked to do in U.S. schools. The term follows conventional educational usage in that it focuses on what students are accomplishing, rather than on any temporary "limitation" they face prior to having done so, just as we refer to advanced teacher candidates as "student teachers" rather than "limited teaching proficient individuals," and to college students who concentrate their studies in physics as "physics majors" rather than as "students with limited physics proficiency."

Harvard Educational Review Vol. 64 No. I Spring 1994 Copyright (c) by President and Fellows of Harvard College 0017-8055/94/0200-0055 $1.25/0

² These are bilingual education programs that include continued instruction in the student's native language, instruction in the English language, and, optimally, content instruction in both languages, with the majority of content instruction in the native language for the first two or three years while the student is learning English (Hornberger, 1990).

³ Thoughtfully developed strategies for teaching via English are also used, if there is truly no way to provide instruction in a student's first language.

⁴ The exact number of students who are in the process of developing proficiency in English is difficult to know with certainty because definitions and identification processes vary greatly from state to state and within states. We can, however, be reasonably certain that the number is much higher than reports of identified students suggest. It should also be noted that these figures include ELLs whose proficiency in English is still regarded as limited; others, who have developed more proficiency in English, may still perform differentially in their two or more languages, and thus must be considered in assessment decisions.

⁵ In the original version of the National Education Goals, one objective listed under Goal 3 stated that "the percentage of students who are competent in more than one language will substantially increase." Greater support for bilingual education programs involving both ELLs and native English speakers would be one way to meet this objective. Proposed legislation currently before the U.S. Congress would add "foreign languages" to the subjects enumerated in the National Goal that refers to academic content.

⁶ Some languages are mentioned here explicitly for illustrative purposes; the learning of any language would be valuable.

CENTER for POLICY STUDIES:
Graduate School of Education and Human Development

Is It Real for all Kids?
A Framework for Equitable Assessment Policies
for English Language Learners

REFERENCES

American Psychological Association. (1985). Standards for educational and psychological testing. Washington, DC: Author.

Astin, A.W. (1982). Minorities in American higher education. San Francisco: Jossey-Bass.

Au, K (1980). Participation structures in a reading lesson with Hawaiian children. Anthropology and Education Quarterly, 11, 91-115.

Au, K, & Jordan, C. (1981). Teaching reading to Hawaiian children: Finding a culturally appropriate solution. In H. Trueba, P. Guthrie, & K. Au (Eds.), Culture and the bilingual classroom: Studies in classroom ethnography (pp. 139-152). New York: Newbury House.

Collier, V.P. (1992). A synthesis of studies examining long-term language minority student data on academic achievement. Bilingual Education Research Journal, 16(1/2), 187-221.

Council of Chief State School Officers. (1991). Summary of state practices concerning the assessment of and the data collection about limited English proficient (LEP) students. Washington, DC: Author.

Council of Chief State School Officers. (1992). Recommendations for improving the assessment and monitoring of students with limited English proficiency. Washington, DC: Author.

Cummins, J. (1984). Bilingualism and special education: Issues in assessment and pedagogy. Austin, TX: Pro-Ed.

Cummins, J. (1989). Empowering language minority students. Sacramento, CA: California Association for Bilingual Education.

Duran, R.P. (1983). Hispanics' education and background: Predictors of college achievement. New York: College Entrance Examination Board.

Duran, R.P. (1989). Testing of linguistic minorities. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 573-587). National Council on Measurement in Education and The American Council on Education. New York: Macmillan.

Duran, R.P. (1990, April). Teaching the discourse of cooperation. Paper presented at the annual meeting of the American Educational Research Association, Boston.

Faltis, C. (1993). Accountability, assessment and equity. In C. Minicucci & L. Olsen (Eds.), Educating students from immigrant families: Meeting the challenge in secondary schools (Proceedings of a conference, October 22-24, 1992). Santa Cruz: University of California at Santa Cruz, National Center for Research on Cultural Diversity and Second Language Learning.

Gandara, P., & Merino, B. (1993). Measuring the outcomes of LEP programs: Test scores, exit rates, and other mythological data. Educational Evaluation and Policy Analysis, 15, 320-338.

Geisinger, K. F. (Ed.). (1992a). Psychological testing of Hispanics. Washington, DC: American Psychological Association.

Geisinger, K. F. (1992b). Fairness and selected psychometric issues in the psychological testing of Hispanics. In K. F. Geisinger (Ed.), Psychological testing of Hispanics (pp. 17-42). Washington, DC: American Psychological Association.

Geisinger, K. F. (1992c). Testing limited English proficient students for minimum competency and high school graduation. In Proceedings of the Second National Research Symposium on limited English proficient student issues: Focus on evaluation and measurement, Vol. 2 (pp. 33-65). Washington, DC: U.S. Department of Education, Office of Bilingual Education and Minority Language Affairs.

Hakuta, K. (1986). Mirror of language. The small debate on bilingualism. New York: Basic Books.

Haladyna, T. (1992) Test score pollution: Implications for limited English proficient students. In Proceedings of the Second National Research Symposium on limited English proficient student issues: Focus on evaluation and measurement, Vol. 2 (pp. 135-163). Washington, DC: U.S. Department of Education, Office of Bilingual Education and Minority Language Affairs.

Heath, S. B. (1983). Ways with words: Language, life, and work in communities and classrooms. Cambridge, Eng.: Cambridge University Press.

Hornberger, N. (1990). Creating successful learning contexts for biliteracy. Penn Working Papers in Educational Linguistics, 6, 1-21.

Jacob, E., & Jordan, C. (Eds.). (1987). Explaining the school performance of minority students (Special Issue]. Anthropology and Education Quarterly, 18(4).

Krashen, S.D. (1981). Bilingual education and second language acquisition theory. In Schooling and language minority students: a theoretical framework (pp. 51-79). Los Angeles: California State University, Evaluation, Dissemination and Assessment Center.

Lyons, J. J. (1988). Legal responsibilities of education agencies serving national origin language minority students. Washington, DC: Mid Atlantic Equity Center.

Madaus, G. (1993). A national testing system: Manna from above? An historical/technical perspective. Educational Assessment, 1(l), 9-26.

Mehan, H., Datnow, A., Bratton, E., Tellez, C., Friedlander, D., & Ngo, T. (1992). Untracking and college enrollment (Cooperative Agreement No. R117GI0022). San Diego: University of California San Diego, National Center for Research on Cultural Diversity and Second Language Learning.

McLaughlin, B. (1992). Myths and misconceptions about second language learning. What every teacher needs to unlearn. Santa Cruz: University of California, National Center for Research on Cultural Diversity and Second Language Learning.

Minicucci, C., & Olsen, L. (1992, Spring). Programs for secondary limited English proficient students: A California study (Occasional Papers in Bilingual Education, No. 5). Washington, DC: National Clearinghouse for Bilingual Education.

Mitchell, R. (1992). Testing for learning. How new approaches to evaluation can improve American schools. New York: Free Press.

Oakes, J. (1990). Multiplying inequalities: The effects of race social class, and tracking on opportunities to learn mathematics and science. Santa Monica, CA: RAND.

Ogbu, J. (1987). Variability in minority school performance: A problem in search of an explanation. Anthropology and Education Quarterly, 18, 312-334.

Olsen, R.E.W-B. (1993). A survey of LEP and adult ESL student enrollments in US public schools. In Language Minority Student Enrollment Data. Symposium conducted at the TESOL Convention, Atlanta, GA.

Ortiz, A. A., & Wilkinson, C.Y. (1990). Assessment and intervention model for the bilingual exceptional student. Teacher Education and Special Education, Winter, 35-42.

Padilla, A.M. (1979). Critical factors in the testing of Hispanic Americans: A review and some suggestions for the future. In R.W. Tyler & S.H. White (Eds.), Testing, teaching and learning: Report of a conference on research on testing (pp. 219-243). Washington, DC: National Institute of Education.

Pennock-Roman, M. (1990). Test validity and language background. New York: College Entrance Examination Board.

Phillips, S. (1983). The invisible culture. Communication in classroom and community on the Warm Springs Indian Reservation. New York: Longman.

Ramirez, J.D. (1992). Executive summary. Bilingual Education Research Journal, 16(1/2), 1-62.

Ramirez, J., Yuen, S., Ramey, D., & Billings, D. (1991). Final report: Longitudinal study of structured English immersion strategy, early-exit and late-exit bilingual education programs for language-minority children (Vols. I, II) (No. 300-87-0156). San Mateo, CA: Aguirre International.

Resnick, L.B., & Resnick, D.P. (1991). Assessing the thinking curriculum: New tools for educational reform. In B.R. Gifford & M.C. O'Conner (Eds.), Changing assessments: Alternative views of aptitude, achievement and instruction (pp. 37-76) Boston: Kluwer.

Rivera, C., & Zehler, A. (1990). Collaboration in teaching and learning: Findings from the Innovative Approaches Research Project. Washington, DC: Development Associates.

Rivera, C., & Zehler, A. (1991). Assuring the academic success of language minority students: Collaboration in teaching and learning. Journal of Education, 173(2), 52-77.

Rodriguez, C.E. (1992). Student voices: High school students' perspectives on the Latino dropout problem. Report of the Fordham University, College at Lincoln Center Student Research Project. New York: Latino Commission on Educational Reform, Subcommittee on the Causes and Solutions to Latino Dropout.

Rosebery, A., Warren, B., & Conant, F.R. (1989). Cheche Konen formative evaluation report: Teaching teachers to leach science in bilingual classrooms. Cambridge, MA: BBN Systems and Technologies.

Rosebery, A., Warren, B., & Conant F.R. (1992). Appropriating scientific discourse: Findings from language minority classrooms (Research Report No. 3). Santa Cruz: University of California at Santa Cruz, National Center for Research on Cultural Diversity and Second Language Learning.

Stevens, F.I. (1993). Opportunity to learn: Issues of equity for poor and minority students. Washington, DC: National Center for Education Statistics.

Trueba, H.T. (I 989). Raising silent voices: Educating the linguistic minorities for the 21st century. New York: Newbury House.

Tyler, R.W. (1949). Basic principles of curriculum and instruction. Chicago: University of Chicago Press.

U.S. Department of Education. (1991). The condition of bilingual education in the nation: A report to the Congress and the President. Washington, DC: Author.

U.S. Department of Education. (1992). The condition of bilingual education in the nation: A report to the Congress and the President. Washington, DC: Author.

Waggoner, D. (Ed.). (1992, July, September). Numbers and needs: Ethnic and linguistic minorities in the United States. (Available from Numbers and Needs, Box GIH/B, 3900 Watson Place, NW, Washington, DC 20016.)

Wilkinson, C.Y., & Ortiz, A.A. (1986). Characteristics of limited English proficient learning disabled Hispanic students at initial assessment and at reevaluation. Austin, TX: Handicapped Minority Research Institute on Language Proficiency.

Wong-Fillmore, L. (1990). Now or later? Issues related to the early education of minority-group children. In Early childhood and family education: Analysis and recommendations of the Council of Chief Stale School Officers (pp. 122-144). Orlando, FL: Harcourt Brace Jovanovich.