Computer Mediated Language Assessment 
and 
Evaluation in Natural Language Processing 
Proceedings of a Symposium sponsored by the 
Association for Computational Linguistics 
and 
International Association of Language Learning Technologies 
Edited by: 
Marl Broman Olsen 
22 June 1999 
University of Maryland 
College Park, Maryland USA 
Published by the Association for Computational Linguistics 

Computer Mediated Language Assessment 
and 
Evaluation in Natural Language Processing 
Proceedings of a Symposium sponsored by the 
Association for Computational Linguistics 
and 
International Association of Language Learning Technologies 
Edited by: 
Mari Broman Olsen 
22 June 1999 
University of Maryland 
College Park, Maryland USA 
Published by the Association for Computational Linguistics 
© 1999, Association for Computational Linguistics 
Order additional copies from: 
Association for Computational Linguistics 
75 Paterson Street, Suite 9 
New Brunswick, NJ 08901 USA 
+ 1-732-342-9100 phone 
+ 1-732-342-9339 fax 
acl @aclweb.org 
Computer Mediated Language Assessment 
and 
Evaluation in Natural Language Processing 
A joint ACL-IALL Symposium 
Final Program 
June 22,1999 
830-845 Opening remarks 
Introduce SESSION I: COLLECTING AND EVALUATING ORAL LANGUAGE 
845-920 Eliciting Natural Speech from Non-native Users: Collecting Speech data for LVCSR 
Laura Mayfield Tomokiyo and Susanne Burger - Carnegie Mellon University 
920-955 Speech Comparision in the Rosetta StoneTM 
John Fairfield - James Madison University 
955-1010 BREAK 
1010-1045 Multimedia Computer Technology and Performance-Based Language Testing: 
A Demonstration of the Computerized Oral Proficiency Instrument (COPI) 
Valefie Malabonga and Dorry Kenyon - Center for Applied Linguistics 
1045-1120 Modeling the Language Assessment Process and Result: 
Proposed Architecture for an automatic Oral proficiency Assessment 
Gina-Anne Levow and Mari Broman Olsen - University of Maryland 
1120-1130 Introduce SESSION II: MULTIMODAL AND MULTILINGUAL RESOURCES 
1130-1205 Dual Use of Linguistic Resources: Evaluation of MT Systems and Language Learners 
Lisa Decrozant - University of Maryland and Army Research Laboratory 
Clare R. Voss - Army Research Laboratory 
1205-1240 FAME: a Functional Annotation Meta-scheme for multi-modal and multi-lingual 
parsing Evaluation 
Allessandro Lenci, Simonetta Montemagni, Vito Pirrelli, Claudia Sofia - 
Istituto di Linguistica Computazionale - CNR 
1240-2 LUNCH 
2-215 
215-250 
250-325 
325-340 
340-415 
415-450 
450-515 
Remaining Session I discussion. 
Introduce SESSION III: COLLECTING AND EVALUATING WRITTEN LANGUAGE 
Modding User Language Proficiency in a Writing Tutor for Deaf Learners of English 
Lisa Michaud and Kathleen F. McCoy - University of Delaware 
Exploiting the Student Model to Emphasize Language Teaching Pedagogy in Natural 
Language Processing 
Trude Heift and Paul McFetfidge - Simon Fraser University 
BREAK 
A Web-based System for Automatic Language Skill Assessment: EVALING 
C6drick Fairon - Universit6 Paris, LADL 
Automated Essay Scoring for Nonnative English Speakers 
Jill Burstein - Educational Testing Service 
Martin Chodorow - Hunter College of CUNY 
Closing discussion 
PROGRAM CHAIR: 
Marl Broman Olsen 
University of Maryland Institute for Advanced Computer Studies 
A.V. Williams Building 
College Park, MD 20742 
REVIEW COMMITTEE: 
Charlotte Groff Aldridge (University of Maryland Language Center Director) 
Susan Armstrong (University of Geneva, ISSCO) 
Chris Higgins (University of Maryland Language Center, IALL-99 conference chair) 
Eduard Hovy (USC, Information Sciences Institute) 
Douglas Jones (U.S. Department of Defense) 
Margaret Ann Kassen (Catholic University, Modem Languages) 
Dorry Kenyon (Center for Applied Linguistics) 
Patricia O'Neill Brown (U.S. Department of Commerce) 
Philip Resnik (University of Maryland Linguistics & Computer Studies) 
Roberta Lavine (University of Maryland, Spanish and Portuguese) 
Valerie Malabonga (Center for Applied Linguistics) 
Flo Reeder (MITRE Corporation) 
Carol Van Ess-Dykema (U.S. Dept. of Defense) 
John White (Litton PRC) 
Table of Contents 
Preface .................................................................................................................................................. 1 
COLLECTING AND EVALUATING ORAL LANGUAGE 
Eliciting Natural Speech from Non-native Users: Collecting Speech data for LVCSR 
Laura Mayfield Tomokiyo and Susanne Burger - Carnegie Mellon University .................................. 5 
Speech Comparision in the Rosetta StoneTM 
John Fairfield - James Madison University ........................................................................................ 12 
Multimedia Computer Technology and Performance-Based Language Testing: 
A Demonstration of the Computerized Oral Proficiency Instrument (COPI) 
Valerie Malabonga and Dorry Kenyon - Center for Applied Linguistics .......................................... 16 
Modeling the Language Assessment Process and Result: 
Proposed Architecture for an automatic Oral proficiency Assessment 
Gina-Anne Levow and Mari Broman Olsen - University of Maryland .............................................. 24 
MULTIMODAL AND MULTILINGUAL RESOURCES 
Dual Use of Linguistic Resources: Evaluation of MT Systems and Language Learners 
Lisa Decrozant - University of Maryland and Army Research Laboratory and Clare R. Voss - Army 
Research Laboratory ............................................................................................................................ 32 
FAME: a Functional Annotation Meta-scheme for multi-modal and multi-lingual 
parsing Evaluation 
Allessandro Lenci, Simonetta Montemagni, Vito Pirrelli, Claudia Soria - 
Istituto di Linguistica Computazionale - CNR .................................................................................. 39 
COLLECTING AND EVALUATING WRITTEN LANGUAGE 
Modeling User Language Proficiency in a Writing Tutor for Deaf Learners of English 
Lisa Michaud and Kathleen F. McCoy - University of Delaware ...................................................... 47 
Exploiting the Student Model to Emphasize Language Teaching Pedagogy in Natural 
Language Processing 
Trude Heift and Paul McFetridge - Simon Fraser University ............................................................ 55 
A Web-based System for Automatic Language Skill Assessment: EVALING 
C6drick Fairon - Universit6 Paris, LADL ........................................................................................... 62 
Automated Essay Scoring for Nonnative English Speakers 
Jill Burstein - Educational Testing Service and Martin Chodorow - Hunter College of CUNY ........ 68 

