<?xml version="1.0" standalone="yes"?>
<Paper uid="W97-1503">
  <Title>Participatory Design for Linguistic Engineering: the Case of the GEPPETTO Development Environment</Title>
  <Section position="3" start_page="16" end_page="18" type="intro">
    <SectionTitle>
2 Methodology
</SectionTitle>
    <Paragraph position="0"> A User Centered (UC) approach was adopted for the design of GEPPETTO. Indeed, UC approach takes user needs into account from the very beginning of the design phase till the final evaluation of the system. This way, system design is changed from a mere technical and individual activity into an interdisciplinary and group activity. Importantly, UC approach secures the attainment of such goals as: appropriateness with respect to user needs and desiderata; flexibility with respect to different skills and different user typologies; and overall usability.</Paragraph>
    <Paragraph position="1"> The engagement of users in the system design can occur at different levels: consultative level when users are considered as sources of information or as participants to final evaluations; representative level when they participate in structured design meetings; consensus level when users are part of the design team.</Paragraph>
    <Paragraph position="2"> For GEPPETTO we chose the Participatory Design methodology (henceforth PD, (of the ACM, 1993)) which falls in the third class. In PD, users act as fully empowered participants to the design process, sharing decisions together with system designers. Such a working style promotes mutual learning between users and designers, and facilitates the identification of user needs and of possible misunderstandings. Hence PD is particularly suitable for complex domains where it might be difficult for designers alone to get a knowledge sufficient for proposing meaningful solutions. This is certainly true of LE, because of the complexity of the domain and of the different skills involved in the development of a LEA. Moreover, certain peculiar techniques of PD, such as participatory prototyping, offer a natural way to reduce errors otherwise not detectable until the final system is put to use.</Paragraph>
    <Paragraph position="3"> PD employs a wide range of techniques (Muller et al., 1993) whose applicability depends on such factors as design goals, group size, availability of users for long periods, and the like.</Paragraph>
    <Paragraph position="4"> Concerning GEPPETTO design, PD was implemented by establishing a working group (WG) of five people consisting of system developers (2), users (2), and an interface design expert (1). Different techniques were applied at different stages of the design process: * Envisioning future solutions: in the early phases, informal discussions for clarifying global statements and stimulating the users' creative thinking took place. Outcomes of these meetings concerned the awareness of the different roles, skills and knowledge involved in the development of LEAs, and the identification of a number of basic user typologies. Moreover, it seemed unlikely to the WG that a single per-son could cover the whole process alone. That is, the development of a LEA is a multidisciplinary work which can benefit from some kind of support in its cooperative evolution.</Paragraph>
    <Paragraph position="5"> * Participatory requirement specifications: the discussion focussed on users desiderata and system capabilities and resulted in a list of the required functionalities. Such a list was then divided into subsets, each one corresponding to one of the user typologies. The discussion then centered on how each typology acts in isolation, and how it interacts with the others, during the development of a LEA. Thus, different levels for single and cooperative work were identified. 1 * Collaborative low-fi prototyping: during collaborative prototyping workshops, paper mock-ups (also called low-fi prototypes) were designed and evaluated by the WG. This activity was extremely useful to move from ideas to concrete interface design, to detect and correct misunderstandings, and to elicit original solutions to unforeseen problems. The outcome was the complete definition of the system.</Paragraph>
    <Paragraph position="6"> * Cooperative evaluations: cooperative evaluations of the first implementation supported the final refinements of the implemented environment. At this stage, feedbacks from the users were discussed and taken into account for further improvements.</Paragraph>
    <Paragraph position="7"> * Experimental sessions: even if not required by PD, empirical evaluations with users not involved in PD have been conducted to verify the effectiveness of the design. Method and results are discussed in Section 7.</Paragraph>
    <Paragraph position="8"> In the next section we will focus on the results of the first two steps of the PD methodology, as applied to GEPPETTO design.</Paragraph>
    <Paragraph position="9">  ery systems, modular approach to linguistic development, etc.; * specifications of system facilities: tools for browsing and editing linguistic data, API for integrating external resources, etc.</Paragraph>
    <Section position="1" start_page="17" end_page="18" type="sub_section">
      <SectionTitle>
3.1 Building LE Applications
</SectionTitle>
      <Paragraph position="0"> The working group focused on the abstract definition of the development cycle of LEAs and of the typologies of the involved users. As a matter of fact this is a requirement of an LE approach to NLP systems. null The development cycle of LEAs was defined as in figure 1.</Paragraph>
      <Paragraph position="1"> As a first step, applicative constraints must be considered. In fact, the working context of a LEA determines not only the global behavior of the LEA, but also the way the different modules interact to produce the desired behavior. Another preliminary step is the collection of raw corpora.</Paragraph>
      <Paragraph position="2"> After such a preparatory work, the following development cycle typically takes place: * identification of representative corpora. In this step the aforementioned raw corpora are classified and filtered to find a set of examples that is representative of the characteristics of the whole corpus. The resulting corpus is then split in two parts: one to be used during the system development (training corpus), the other during the testing phase (test corpus); * definition of the architectural requirements.</Paragraph>
      <Paragraph position="3"> Given the applicative constraints and the characteristics of the corpus, the specific requirements of the LEA are defined; * definition, design and implementation of the processors, according to the requirements of the previous point; * development of the linguistic resources, according to the requirements arising from the previous analysis; * testing and refinement of both the processors and the data collection.</Paragraph>
      <Paragraph position="4"> Once all these steps have been gone through, the resulting architecture is delivered (delivery system) and customization can start.</Paragraph>
      <Paragraph position="5"> The working group singled out three different user typologies which can play a role in the tasks above. Each of them corresponds to different backgrounds, knowledge and skills: 2  * Linguistic Engineer (LER): expert on architectures for LE. Background: computer science; knowledge of computational linguistics; * Computational Linguist (CL): expert on linguistic data development. Background: computational linguistics; little knowledge of computer science; * Processor Manager (PM): expert on processors  for language processing. Background: computer science; knowledge of computational linguistics. null Accordingly, the development cycle has been refined as follows: * identification of representative corpora: LER interacts with CL to provide a representative corpus for the application; * definition of architectural requirements: given the corpus and the requirements for processors and linguistic data, LER interacts with PM and CL to define the correct architecture; * definition, design and implementation of the processors: PM chooses (or designs and implements) them; * development of linguistic resources: CL chooses (or designs and implements) them; 2Actually the working group added also an Application Manager, i.e. an expert of the domains and of the users of the LEA. Such a profile is not discussed in this paper.</Paragraph>
      <Paragraph position="6">  * test and refinement: LER checks the correspondence between the current implementation and the architectural requirements; the processors are tested by PM and the data collection by CL.</Paragraph>
      <Paragraph position="7"> In the end, the working group had effectively specified the actions, the tasks, and the skills required to create LEAs. The following step was the identification of the user needs.</Paragraph>
    </Section>
    <Section position="2" start_page="18" end_page="18" type="sub_section">
      <SectionTitle>
3.2 User Needs and Desiderata
</SectionTitle>
      <Paragraph position="0"> The working group discussed some of the desirable features of a LEADS, from the point of view of the users. Results can be summarized as follows:  ternal modules (e.g. Knowledge Bases).</Paragraph>
      <Paragraph position="1"> One of the main outcomes of PD discussions was that the different users would benefit from a single, common tool capable of facilitating and supporting their mutual interactions (even when performing their tasks independently) as well as the integration of resources developed independently. 3 On the other hand, given the different profiles and skills involved, each of the three user typologies needs different facilities and might prefer different interaction modalities. For example CLs tend to favor graphical interfaces that hide as much as possible low-level details (e.g. internal data representation). On the other hand, PMs have to cope with low level details. As it turns out, the ideal environment should both address the differing interaction styles of each user, and, at the same time, provide a</Paragraph>
    </Section>
    <Section position="3" start_page="18" end_page="18" type="sub_section">
      <SectionTitle>
3In this paper we focus on the interactions among
</SectionTitle>
      <Paragraph position="0"> users belonging to the different typologies and on the integration of their work. We will not address the important question of how to support the interactions and integration involving users of the same typology. For instance, we will not discuss here the issue of how the development of large grammars by different CLs can be properly supported by a LEADS.</Paragraph>
      <Paragraph position="1"> uniform environment where their contributions can be easily integrated. These results can be obtained if, at any time, the user can select all and only the functionalities he/she actually needs.</Paragraph>
      <Paragraph position="2"> A similar tension involves also linguistic data and processors. LERs want to see them as units that can be assembled to build the final architecture. PMs are inclined to consider the linguistic data as a unit, but see the processors as complex modules to manipulate. Finally, CLs obviously must be able to single out pieces of linguistic data and organize them in a significant way, while using the processors as black boxes.</Paragraph>
      <Paragraph position="3"> Before discussing how user needs have been implemented in GEPPETTO, we briefly introduce the formalism for linguistic data as it was developed by the CLs of the working group.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>