<?xml version="1.0" standalone="yes"?>
<Paper uid="W99-0410">
  <Title>A Web.Based System for Automatic Language Skill Assessment: EVALING</Title>
  <Section position="1" start_page="0" end_page="62" type="abstr">
    <SectionTitle>
Abstract
</SectionTitle>
    <Paragraph position="0"> EVALING is a Leonardo da Vinci project funded by the European Union, involving four European laboratories 1. The aim of the project is to build an automatic system to evaluate language skills in people's native language. This paper focuses on native French. Other partners are working on their own language and are building specific tests (Italian and German). EVALING is an 'Item Banking '2 system: exercise database allowing dynamic design of questionnaires.</Paragraph>
    <Paragraph position="1"> We present a technique based on the use of NPL tools that assure easy and costless updating of these databases. In addition, we underline the interest of Local Grammars (Finite State Transducers) for scoring exercises on language.</Paragraph>
    <Paragraph position="2"> Introduction EVALING is a Leonardo da Vinci project funded by the European Union, involving four European laboratories. The aim of the project is to build an automatic system to evaluate language skills in people's mother language. Each partner is working on his own language and building specific tests (at the present: i Association pour le traitement automatique des langues (ASSTRIL) for French, Consorzio Lexicon Ricerche from the University of Salerno, for Italian &amp; Piidagogische Hochschule Karlsruhe and Universit~t Mtinehen for German.</Paragraph>
    <Paragraph position="3"> 2 &amp;quot;Item banking covers any procedures that are used to create, pilot, analyse, store, manage and select test items so that multiple test forms can be created from a subset of the total 'bank' of items. &amp;quot;Brown (1997). French, Italian and German). In this paper we will present French.</Paragraph>
    <Paragraph position="4"> The first step consists, for each language, in determining the fields to be tested and the types of exercises which can be computerized to carry out this task. We have observed this task differs from one language to another. For example, spelling exercises are relevant in French, but they are not very interesting for German, since people make few mistakes. For French, we decided to focus on syntax, lexicon, spelling and reading comprehension. Hence, we oriented the reflection on the tests that could be automatized and on those that could not. At this point, automatization in the EVALING sytsem bears on three phases of the evaluation process: - dynamic setting up of tests (exercises are stored in tables of an exercise database). We assume that if a person has to take the test more than once, this person should not get the same set of questions twice, - automatic grading of tests (including storage of marks in a client database), - semi-automatic filling of exercise databases (with the assistance of linguistic tools).</Paragraph>
    <Paragraph position="5"> First, we will discuss some technical aspects: EVALING is a Web-based program interacting with large exercise databases and a client database. We will explain how this ' item Banking' system has been implemented on a  of exercises, but rather to be able to renew them easily. We designed a set of linguistic tools to satisfy this demand. The set of tools is based on the software INTEX, developed at the LADL by Max Silberztein 4. Of course, it is not always possible to automatize the creation of exercises. In certain cases, the work will have to be done manually.</Paragraph>
    <Paragraph position="6"> Second, we will present the 'Administrator side'. We call 'administrator' the person or team who needs to evaluate a large group (students, employees, applicants, etc.). An interface enables the administrator to define the parameters of the test (length, level, etc.) and to perform some statistical analysis on the client database.</Paragraph>
    <Paragraph position="7"> Finally, we will deal with the 'client side'.</Paragraph>
    <Paragraph position="8"> The client registers himself and then has access to the test session through a Web-Client software. All test forms are in HTML form.</Paragraph>
  </Section>
class="xml-element"></Paper>