<?xml version="1.0" standalone="yes"?>
<Paper uid="W99-0404">
  <Title>Multimedia Computer Technology and Performance-Based Language Testing: A Demonstration of the Computerized Oral Proficiency Instrument (COPI)</Title>
  <Section position="1" start_page="0" end_page="16" type="abstr">
    <SectionTitle>
Abstract
</SectionTitle>
    <Paragraph position="0"> The field of language testing has long led the way in integrative, performance-based assessment. However, the use of technology in language testing has often meant limiting assessment options. We believe computer-mediated language assessment can enrich opportunities for language learners to demonstrate what they are able to do with their second language. In this paper, we describe the rationale and operation of the Computerized Oral Proficiency Instrument (COPI), a multimedia, computer-administered oral proficiency test. While at present speech performances on the COPI are evaluated by trained raters using a national standard, the COPI affords an excellent opportunity to investigate the use of Natural Language Processing for computer-assisted evaluation.</Paragraph>
    <Paragraph position="1"> Introduction The Computerized Oral Proficiency Instrument (COPI) is a multi-media, computer-administered adaptation of the tape-mediated Simulated Oral Proficiency Interview (SOPI). Both the SOPI and the COPI are oral proficiency tests based on the Speaking Proficiency Guidelines of the American Council on the Teaching of Foreign Languages (ACTFL). Oral proficiency tests like the SOPI and COPI use simulated real life tasks to elicit speech ratable by the ACTFL Guidelines' criteria. The purpose of the COPI is to use the advantages of multi-media computer technology to improve the SOPI by giving examinees more control over various aspects of the testing situation and increasing raters' efficiency in scoring the test.</Paragraph>
    <Paragraph position="2"> In this paper we primarily discuss the Spanish version of the COPI, although an Arabic and a Chinese version are also being prepared. This paper provides the context for the COPI, discusses its rationale, its components and its phases, and introduces the scoring program used by raters who assess an examinee's speech performances using the criteria of the ACTFL Guidelines.</Paragraph>
    <Paragraph position="3">  1. Computer Technology in Performance-Based Assessments of Speaking Ability  Technology has no doubt been a part of language testing since 'before the invention of the pencil. Electronic technology, through the phonograph record, reel-to-reel and later casette tape, and the compact disc, has enhanced the assessment of listening skills for decades.</Paragraph>
    <Paragraph position="4"> Computers allowed for the development of computer-adaptive and computer-administered tests in second languages. Since June of 1998, the Educational Testing Service (ETS) has administered the Test of English as a Foreign Language (TOEFL) by computer in many parts of the world. With almost one million test takers a year, the TOEFL is the world's largest language test. The use of computer technology has allowed ETS to introduce a new variety of selected-response type items not easily presented in paper and pencil format. In addition, the computer-based TOEFL allows examinees the option of word-processing a written essay, as opposed to writing it longhand. Of all sections of the current TOEFL, only the essay can be  regarded as performance-based, since examinees provide a demonstration of their linguistic abilities through producing a text.</Paragraph>
    <Paragraph position="5"> While some have argued that multiple-choice tests of listening comprehension can provide a proxy measure of speaking ability, speaking skills have traditionally been assessed through some type of performance-based assessment, typically a live face-to-face oral interview procedure. The best known formal procedure is the Oral Proficiency Interview (OPI). The OPI is used by various government agencies involved with language training, including the Foreign Service Institute, where it was originally developed in the 1950s to assess the readiness of US personnel for functioning in oversees diplomatic posts. In US academia, the American Council on the Teaching of Foreign Languages (ACTFL) has promulgated the OPI since the early 1980s through professional development workshops and tester training programs (Stansfield, 1996).</Paragraph>
    <Paragraph position="6"> In the mid-1980s, the Center for Applied Linguistics (CAL) began a program of research and development in using technology to elicit speech samples from examinees that can be assessed following the same criteria used in the ACTFL OPI. The impetus for this program was the need to assess speaking skills of students of less-commonly-taught-languages in instructional programs throughout the nation where there was no trained OPI interviewer. Performances elicited by and recorded on tape could then be sent to trained OPI testers for evaluation. The format developed by CAL came to be known as the Simulated Oral Proficiency Interview (SOPI). High correlations (averaging .92) were found between performances on the SOPI and the OPI across a variety of languages (Stansfield and Kenyon, 1992). The testing format was also found to be useful in large-scale testing applications where it was necessary to ensure that all examinees received the same high quality test, and the SOPI format has been used in or adapted for a variety of language testing projects. Other variations of the SOPI appeared, most notably the Video Oral Communication Instrument (VOCI), developed by the Language Acquisition Resource Center at San Diego State University. The VOCI uses a video rather than an audio tape and test booklet to elicit examinee speech performances.</Paragraph>
    <Paragraph position="7"> Based on its work with the SOPI, CAL is currently developing a format for a computer-administered assessment of oral proficiency known as the Computerized Oral Proficiency Instrument (COPI).</Paragraph>
  </Section>
class="xml-element"></Paper>