Visualization of Protocols of the Parsing and Semantic 
Interpretation Steps in a Machine Translation System 
Ulrich Germann 
USC Information Sciences Institute 
Marina del Rey, CA 90292 
email: germannC~isi.edu 
Abstract 
In this paper, we describe a tool for the vi- 
sualization of process protocols produced by 
the parsing and semantic interpretation mod- 
ules in a complex machine translation system. 
These protocols tend to reach considerable sizes, 
and error tracking in them is tedious and time- 
consuming. We show how the data in the pro- 
tocols can be made more easily accessible by 
extracting a procedural trace, by splitting the 
protocols into a collection of cross-linked hyper- 
text files, by indexing the files, and by using 
simple text formatting and sorting of structural 
elements. 
1 Introduction 
The tool described in this paper was developed 
in connection with the Gazelle Machine Transla- 
tion System (Knight et al., 1995), which is cur- 
rently under development at the USC Informa- 
tion Sciences Institute. At the moment, Gazelle 
covers machine translation from Japanese and 
Arabic to English. 
Figure 1 sketches the flow of processing. The 
input text is first segmented and tagged with 
morphological information. It is then parsed 
and interpreted semantically. The result of se- 
mantic interpretation is finally fed into the text 
generation module. 
Almost all modules relevant to the discussion 
here employ bottom up chart parsing mecha- 
nisms. For any given input, they may return 
more than one interpretation, as the sample 
parse sequence for the string saw the ape with 
his binoculars in Figure 2 illustrates. 
The processes of parsing and semantic inter- 
pretation are recorded step by step in process 
protocols. A parse step in our system is equiv- 
alent to the creation of a new parse node. Each 
node receives a category label which determines 
I Text Submission I 
\[ Preprocessing \[ 
I Segmentation and Tagging \[ 
\[ Parsing \[ 
\[ Semantic Interpretation \] 
\[ Text Generation \[ 
\[ Result Presentation \[ 
Figure 1: The machine translation process 
its behavior in the remainder of the parsing pro- 
cess. A new parse node can be created in order 
to 
• integrate a new text element into the parse 
space, 
• combine two or more exisisting nodes into 
one node, or 
• change the category of an existing node by 
making it the daughter of the new node, 
and assigning a different category label to 
the new node. 
If, for any given range of input text, there are 
two interpretations that result in the same cat- 
egory label, they are combined into one node. 
For example, parse step No. 6 in Figure 2 cre- 
ates a node that comprises two interpretations. 
Each node in the parse tree also has a fea- 
ture structure associated with it. This feature 
83 
dominance 
V, 
V' PP 
V NP with his binoculars 
sa~w the ,ape 
V~NP precedence 
the ape with his binoculars 
Parse Sequence (simplified) 
Step h V -- saw 
Step 2: NP ~ the ape 
Step 3: V' -- V NP 
Step 4: PP -- with his binoculars 
StepS: NP-- NPPP 
Step 6: V' -- V NP 
OR: 
V' -- V' PP 
Figure 2: The parse space for the string saw the ape with his binoculars. (simplified) 
structure specifies various properties of the con- 
stituent represented by the respective node. A 
feature structure is a set of attribute-value pairs, 
where a value can either be atomic (something 
that cannot be described in terms of having par- 
ticular features or properties), or an embedded 
feature structure. Typical forms of representa- 
tion are attribute-value matrices (AVMs), and 
graphs with labeled arcs and nodes. Figure 3 
shows a partial description of the word sees (3rd 
person singular form of the verb see with the 
meaning \[see\[) in both formats. 
During semantic interpretation, the feature 
structures introduced by the parser are aug- 
mented by semantic features assigned to the 
nodes by semantic interpretation rules (Knight 
and Hatzivassiloglou, 1995). 
In the protocols, each node receives an indi- 
vidual entry which keeps track of the following 
information: 
• the node number, which uniquely identifies 
the node / parse step; 
• the category label assigned to the node, e.g. 
NP for 'noun phrase'; 
• the nodes dominated by the node, or, if the 
node is a terminal node, the corresponding 
lexical symbol (a word or morpheme) from 
the input text. Daughter nodes of nonter- 
minal nodes are referenced by their node 
SURFACE 
SYN 
SEM 
sees 
pos ,,erq 
PERSON 3rd \] 
NUMBER sg j \[INST IseeJ\] 
/ SURFACE a~_~ N 
"-<:2: "--..-.--..,lsee I *sg 
Figure 3: Partial description of the word sees as 
an attribute-value matrix (top) and as a graph 
with labeled arcs and nodes (bottom) 
numbers. Since ambiguities are packed 
into one node (instead of spelling them out 
by creating multiple nodes), a nonterminal 
node may have several sets of daughters (cf. 
node No. 22 in Figure 4). 
• the range of input text covered by the node; 
84 
category label corresponding lezical symbol ~ the node is a lerminalnode 
node number,.,,,.~ //range 
(~~~\](((SEM ((INSTANCE (*OR* "diameter .... distance across")))) 
I(LEMMA i~.regNi) (SYNSEM ((SYN ((HEAD ((POS-L2 IregNi))))))) 
(EKE '~{~") )____~ \[ .... 
feature strucl'~tre (2 N ..... 3 4 (((SEM ((INSTANCE llI))) (CONTEXT ((RIGHT ((SUFFIX +))))) 
(LEMMA i--.Number\]) (SYNSEM ((SYN ((HEAD ((POS-L2 INumber\[))))))) 
(TREE ..... )))) 
...\] 
(22 NP (~-~'~ ~"~) ~--~) S 9 (*OR* ((SEM ((tiP2 ((INSTANCE (*OR* "globe" 
\ \ t C'"\] A node may have. several sets of daughters. 
The daughters o\] a nonierminaTnode are referenced by lheir node numbers. 
Figure 4: Sample sections from a process protocol. The samples were slighty modified and refor- 
matted in order to achieve a more compact and readable display. 
• the feature structure(s) associated with the 
node. 
Figures 4 and 5 show sample sections from a 
process protocol in raw format and as displayed 
by the visualization tool. 
2 Analysis of the Data 
Apart from the obvious, explicit information ex- 
pressed in the protocol entry of each node -- 
the node number, the immediate daughters of 
the node, and the features associated with it -- 
the process protocol also contains some implicit 
information which can be retrieved by going 
through the protocol as a whole. For exam- 
ple, it is possible to check whether a node is 
used later on in the parsing process and what 
its mother node(s) is. Going down the tree, 
it can be determined which part of the input 
sentence is covered by a particular node. The 
'parse forest' (collection of parse trees) can be 
reconstructed by linking the individual branch- 
ings (minimal trees consisting of a mother node 
and its daughter node(s)) by virtue of the node 
numbers. Last but not least, the original input 
sentence can be recovered by concatenating the 
leaf nodes of the tree. 
Individual parse trees are two-dimensional, 
the two dimensions being precedence (the lin- 
ear order in which the elements occur in the 
sentence) and dominance (th e relation between 
a mother node and its daughter node(s)). Even 
though both relations are preserved in the pro- 
cess protocols, none of them is immediately ob- 
vious from the protocols: Nodes that are neigh- 
bors in terms of linear order, or nodes that are 
connected in terms of dominance do not nec- 
essarily appear next to each other in the parse 
sequence. Consider, for example, the parse se- 
quence in Figure 2. The node created by step 1 
is used again in the steps 3 and 6. Whereas for 
step 3 the sister nodes 1 and 2 happen to be 
adjacent, this is not true for step 6. Here, the 
two nodes that make up the first interpretation 
are separated by the nodes 2, 3 and 4. Equally, 
the dominance relation between parse nodes is 
usually obscured by intermediate parse steps. If 
we also take into account the issue of ambiguity, 
we have to deal with it as a third dimension, as 
illustrated in Figure 2. 
Keeping in mind the purpose of examining 
the data, which is first and above all error trac- 
ing and quality control, we can thus distinguish 
four aspects of the information contained in the 
process protocols. These aspects are: 
the overall structure, i.e., information 
about the constituent structure (parse for- 
est) of the sentence, and about the way it 
is built up from lexical items; 
the details, i.e., information about the 
properties of the constitutents as preserved 
in the feature structures associated with 
85 
21: NP 
• NP (t~) 
• NP (20) 
\[showr~ge\] 
• P~e.nts: NP (22\] $ (~3\]) S (79) 
22: NP : m, eo, ~o. t 
• NP~. 
• NP(m ) 
of, S,, No. 2 
• NP (%5) 
• NP (22) 
oFtilm No. 3 
Notle 22, opXlon 2 ~) 
COMPLEX- + 
SEM 
• INSTPdqCE - ~and~ 
• OP1 
0 LNSTANCE - ('OR* "house (mte'$ own)" "J~skle") 
0 MOD 
0 INSTANCE = (*OR* "asteroid. ~ "planetoid" "small 
planet") 
£3 NIEKT-MOD 
0 INSTANCE - ('OR." "ext+~eding" "this i.s all" 
"ahave" "siru:e" "greater thaJ~" "the end" "as long 
a~" "the ahox~+m, entianed" "mnre titan" "b~ra't" 
"rmt less than" "in excess o1" "arulup" "over') o N~r-MOV - \[1\] 
• OP2 I INSTANCE - (+OR* "globe" ",be earth" "earth" "the 
,~'arm") 
,~mmA . +~..~.~ 
RULE - "NP -) PiP\[no case\] NP = 
SYN • ...... ~-!+ i" l~ ..... ,, "_.,.S~SEM = 
Figure 5: Sample screen. The top frame displays the original input text, which serves as an index 
to the parse sequence in the lower left frame. The category labels and node numbers in the lower 
left frame are hyperlinks. A click on a category label will cause the browser to display the feature 
structure(s) associated with that node in the frame on the right. A click on a node number will 
lead to the respective node within the parse sequence. 
each node; 
• ambiguity, i.e., the number of interpreta- 
tions associated with each node; 
• the relation between the parse sequence 
and the input, i.e., information about the 
connection of each node in the parse se- 
quence to the part of the sentence that is 
covered by this node, and, in reverse, the 
connection of each word or morpheme of 
the input to the node that integrates this 
word or morpheme into the parse sequence. 
Whereas the connection of each node to the 
input can be considered a property of the re- 
spective node, the connection of the input to 
the parse sequence is a matter of indexing that 
will ease access to 'trouble spots' when testing 
changes to the syntactic grammar or the seman- 
tic interpretation rules. 
3 Displaying the Data 
3.1 Display of the Constituent 
Structure 
The obvious solution to the issue of visualizing 
the Overall constitutent structure of a sentence, 
namely displaying it as a conventional parse tree 
representation, turns out to be not very practi- 
cal at closer examination. Since some parsers 
employed in our system may return more than 
just one structure for any given input, and since 
86 
partial parses are very relevant for debugging, 
we are, in fact, not dealing with a single parse 
tree but rather with a 'parse forest' which may 
contain dozens or even hundreds of complete 
and partial trees. A development tool that con- 
fronts the developer with a large number of 
spelled-out ambiguities is not satisfactory, be- 
cause it does not allow efficient access to rele- 
vant areas in the data space. Moreover, even in 
cases where there is only one single parse tree, 
the tree may not fit on the screen, and even if it 
does, there will be little room left to display the 
feature structures associated with each node. 
Therefore, instead of using conventional tree 
representations, the parse sequence is main- 
tained as such and displayed as a cross-linked 
sequence of minimal trees, essentially in the 
same way it is recorded in the process protocol. 
However, it is formatted in a manner that sup- 
ports the correct interpretation of the symbols 
as mother and daughter(s) (cf. the lower left 
frame in Figure 5). In addition, the explicit in- 
formation contained in the individual entries of 
the process protocols is augmented by informa- 
tion that can be inferred from the process proto- 
col as a whole and will not be visible when just 
looking at the protocol. Note, for example, that 
category labels are given not only for the mother 
node but also for the daughter nodes, and that 
the mother-daughter relation contained in the 
individual protocol entry is back-referenced to 
the daughter nodes in the HTML representa- 
tion, so that the subsequent use of the node can 
be traced easily. In contrast, detailed informa- 
tion about the properties of the individual con- 
stituents, which tends to obstruct the view of 
the overall structure, is removed from this trace 
of display and stored in a separate file. Never- 
theless it is readily at hand by means of hyper- 
links, as will be explained immediately below. 
What is not visible in Figure 5 is that all cate- 
gory labels, and the node numbers in subscript 
after the category labels of daughter and par- 
ent nodes, are hyperlinks. A mouse click on a 
category label causes the browser to display the 
feature structure associated with that node in 
the frame on the right. A click on the node 
number leads to the respective node within the 
same (left) frame. This setup provides the user 
with the means to easily navigate the parse tree 
or parse forest, and, at the same time, allows 
him or her to inspect the features associated 
with each node in detail. In both frames, ambi- 
guities (multiple structures associated with one 
node) are preserved by marking the alternatives 
with clear headings. Since parse sequence and 
feature structures are separated and displayed 
in different, adjoining frames in the browser, a 
very compact yet informative form of data vi- 
sualization is achieved. 
3.2 Visualization of the Relation 
between Constitutent Structure 
and Input Text 
Before we turn to the representation of feature 
structures, let us briefly comment on the top 
frame in Figure 5. This frame displays the text 
of the original input sentence and serves two 
purposes. First, a click on the caption \[show 
range\] in either one of the lower frames 1 causes 
the browser to display the range covered by that 
node in red (as opposed to black for the sur- 
rounding context), and, secondly, in the 'click- 
able' mode of the top frame the input sentence 
serves as an index: a click on a word leads to 
the respective node in the parse sequence in the 
lower left frame, so that particular points of in- 
terest can be accessed easily during the gram- 
mar development process. 
3.3 Representation of Feature 
Structures 
Our approach to the display of feature struc- 
tures also diverges from representation formats 
that are well established in the literature. Com- 
monly used representations -- representations 
as graphs with labeled arcs and as attribute- 
value matrices (cf. Figure 3) -- tend to become 
too large once the feature structures reach a cer- 
tain complexity. Moreover, both of them re- 
quire extensive calculations and graphical pro- 
cessing, which have to be paid for in terms of 
processing time. Instead, our tool represents 
feature structures as unordered lists in HTML. 
Each list element represents an attribute-value 
pair. Atomic values are separated from their 
attributes by an equal sign (=); complex values 
(embedded feature structures) are represented 
as embedded lists. 
1The caption is not visible in the lower right frame in 
Figure 5. See Figure 9 for the location of the caption in 
the lower right frame. 
87 
"INST \[prize(giftl v \]bootyl 
v \[trophy, prize\[ 
SEM MOD "INST "best dresser"\] 
T.-LOC. "INST \[time unit\[ \] 
. YEAR \[INST 1199211\] 
Figure 6: AVM representation of the semantic 
content of the phrase the "Best Dresser" Award 
1992. 
• SEM 
o INST -- (,or, Iprize(gift I Ibootyl 
Itrophy, prizeD 
o MOD 
o INST = "best dresser" 
o T.-LOC. 
o INST = Itime unit\[ 
o YEAR 
t, INST = \]19921 
Figure 7: The semantic content of the phrase 
the "Best Dresser" Award 1992, represented as 
an unordered list. 
• SEM 
o INST = (*or, \]prize(gift\[ Ibooty I 
\[trophy, prize I) 
o MOD I INST = "best dresser" 
o T.-LOC. 
o INST = Itime unit\[ 
o YEAR \]INST = \]19921 
Figure 8: The semantic content of the phrase 
the "Best Dresser" Award 1992, represented as 
an unordered list with path abbreviations. 
In this representation format, the feature 
structure represented by the AVM in Figure 6 
would be rendered as shown in Figure 7. 
In order to achieve a more compact represen- 
tation, we use a convention that is commonplace 
in the HPSG literature (Pollard and Sag, 1994, 
and others), namely, we abbreviate single paths 
by the use of a vertical bar (I). Thus the rep- 
resentation in Figure 8 is equivalent to the one 
shown in Figure 7. 
3.4 Sorting and Use of Colors 
In the process protocols, features usually appear 
in random order. In order to ease the locating 
of features and the comparison of feature struc- 
tures, features are generally listed in alphabeti- 
cal order in the display. However, since not all 
information contained in the feature structures 
is equally important, some features receive spe- 
cial treatment. 
The main purpose of the parsing and se- 
mantic interpretation steps within the transla- 
tion process is to build up a more language- 
independent representation of the semantic con- 
tent of the input text. This information is 
stored in the SEM feature of the feature struc- 
ture associated with each node. Within these 
substructures, the INSTANCE feature plays a 
central role: it specifies the concept from the 
Sensus ontology 2 (Knight and Luk, 1994; Hovy 
and Knight, 1993) that the object referred to by 
the expression in the source language is an in- 
stance of. For example the Japanese expression 
1992 year "best dresser" award 
refers to an instance of a prize or award (~) 
which is modified 3 by the string "<.X b V l/,, 
@-- ("best dresser") and temporally located in 
the year 1992: the "Best Dresser" Award 1992. 
Other features on the same 'level' as the IN- 
STANCE feature specify arguments (in the case 
of predicates and relations) and further modifi- 
cations of the object in question. In order to 
accomodate the central role of the INSTANCE 
feature, it is always pushed to the top of the 
list of features under its governing attribute in 
the feature structure, and displayed, together 
with its value, in red instead of black. This 
increases its visibility in the display of the fea- 
ture structures, so that it can be spotted easily. 
The lower right frame of the main window in 
Figure 9 shows the feature structure discussed 
here. 
2If an appropriate concept cannot be found in the 
ontology, an English gloss is used instead. 
ZThis representation does not specify the nature of 
the modification. We currently do not have the means 
to automatically determine the specific character of the 
modification. 
88 
P~.S P output for 
t.~ cJu.~.:~-L~L~-~ I I~ ~~: ~ ' ~ll:'arseNodeNo. 8:N \[sho~,rangel \[exit} 
JUCL~SS: ~\[~:~.~?~i ,!;: :~::i!:.i i:::...~e~e~.N,~+~-l.lm~l~Or~Rle~0~,~J :: ::,: ::, .:l 
p~iz~ Eft I:/l~mall~ rmal~ln~l~4usd~ r - dl~vll~l~mengN- - N- N .sl~ndl~l \[INDEX\] : I: 
..... j (1/I) N->NN j .; POS_L2 : Nv / i 
(IN -> N N) ;; N=no N // N~la N // N=tO H // N N : CAT 
((xo $gnsem) = (×2 sgnsem)) 
( ( ×0 1 ) = ( x2 \[ ) ) 
( (~0 cat) - (x2 cat) ) • ~rBCAT_i . 
(-xoP- (((~ s~rlsem ~'oo~ ~c~'lomin~lzer) =c ~o) 
i " (Ix0 semi = Ix2 semi) : ~EMMA - ~I{..N I (-xor- (((x2 MAP N0) =C (xl SEN))) 
{((xl lemma) =c (-or- I-~a~e-I I*TemDor 
((xO sere temp?val-locatlng) = Cxl semi • INSTANCE * {'OR'~dze<~if~il;'~mgh~'~z.eJ} 
: : ...... " : • MOD I INSTANCE - "~ dot" 
(((xl lemma) =C I-Ouantltg-ll • T~4\]POII.AL-LO,~J~T1NG 
((x0 sel~ me(I) = (xl sere)I) 0 INST,~,IC£ - timetmit~ 
: : ......... O YEAR I LNSTANCE - tL~$1 
.~.~1--~ :~77 : ...... : ~ :.. :~ .:.~':;:~:~5:,;~:";,::~: ~.-.:.:,-,-..--~,-:7-:-::'~: 
Figure 9: Integration of additional information. From left to right: Dictionary output, grammar, 
and main window. 
Also, in order to help the developer distin- 
guish the different frames and windows (see be- 
low) of the visualization tool, we use different 
background colors for different frames. For ex- 
ample, the background of the lower right frame 
of the main window is kept in a light blue, 
whereas other windows have yellow or white 
backgrounds. 
4 Integration of Additional 
Resources 
Whereas the data discussed so far is almost al- 
ways of interest for the developer, other types 
of information are requested less frequently. For 
example, one may want to check a word's en- 
try in the dictionary, or see the grammar rule 
that was used to create a particular node. Ded- 
icating a frame in the main window to the dis- 
play of this kind of data did not seem an ap- 
propriate solution. Display space is far too pre- 
cious to be wasted on a frame that is used only 
occasionally. Instead, this kind of information 
is provided in separate browser windows. The 
grammar rules contain unique IDs that are pre- 
served in the feature structures associated with 
each node. The converter uses these IDs to set 
links to an HTML version of the grammar. The 
links are listed under 'Rules applied' in the lower 
right frame (cf. Figure 9). For terminal nodes, a 
click on the lexical symbol in either of the lower 
frames will access the cgi interface of the dic- 
tionary and deliver the complete lexical infor- 
mation that is available for the respective en- 
try. Figure 9 shows a sample of the developer's 
screen with the main and the two subsidiary 
windows. 
5 Evaluation 
The prime criterion for the evaluation of a visu- 
alization tool like the one described is its effect 
on the efficiency of working with the data. Since 
the tool was mainly developed for in-house use 
by a small team of developers, we did not carry 
out formal experiments in order to assess the 
increase in productivity. Based on my personal 
experience in working with both the simple pro- 
cess protocols and the visualized version as a 
grammar developer for the Japanese modules, 
89 
I estimate the speedup at a factor of two to at 
least ten, depending on the size of the struc- 
ture to be analyzed: the larger the structure, 
the greater the benefit of visualization. 
Also, the tool has proven to be valuable for 
demonstration purposes, as the structuring pro- 
vided by this kind of display helps people who 
have no previous experience with our system 
and maybe are even unfamiliar with natural lan- 
guage processing in general understand the pro- 
cess in more detail. In particular, since many 
people are now familiar with the internet and 
the interfaces provided by web browsers, our im- 
pression is that people who see our system for 
the first time tend to be able to focus on the data 
rather than the interface more quickly. Again, 
this claim has not been tested experimentally. 
6 Technical Information 
The converter was implemented in Perl and cur- 
rently works in off-line mode only, that is, the 
HTML files have to be created first and can then 
be used. Processing time depends very much on 
the size of the structures as well as other fac- 
tors beyond the immediate control of the author 
such as network traffic and overall processing 
load on the machine. On a Sun Ultra, process- 
ing typically takes between a few seconds for 
short sentences (converting the protocol for the 
sentence in Figure 9 takes about 3 seconds) and 
several minutes for very large and ambiguous 
structures. 
7 Conclusion 
In this paper, we have presented a tool for 
the visualization of a complex parse space by 
separating global information about the over- 
all structure of the parse space from detailed, 
local information while preserving the relation 
between both by means of hyperlinks and in- 
dexing. Our solution is anything but fancy: it 
is completely text-based and employs common- 
place, off-the-shelf tools for the display of hy- 
pertexts. This allows for ease of use and a high 
degree of portability, because no new software 
has to be installed at the user-end. Even though 
some compromises had to be made, for example 
by deviating from well-established forms of rep- 
resentation, the interface nevertheless provides 
appropriate functionality for the given purpose. 
In a word, HTML is used for what it was in- 
vented for -- the encoding of nonlinear infor- 
mation. 
8 Acknowledgements 
Gazelle is funded by the US Government under 
contract MDA904-96-C-1077. 
The use of unordered lists in HTML for the 
representation of feature structures was inspired 
by a similar approach taken in a web inter- 
face for the Gazelle system developed by Philipp 
KShn. 
I am very grateful to Kevin Knight, Ulf Her- 
mjakob and Daniel Marcu for various comments 
on earlier drafts of this paper. 

References 
Eduard Hovy and Kevin Knight. 1993. Moti- 
vating shared knowledge resources: An exam- 
ple from the Pangloss collaboration. In Pro- 
ceedings of the Workshop on Knowledge Shar- 
ing and Information Interchange (IJCAI). 
Kevin Knight and Vasileios Hatzivassiloglou. 
1995. Unification-based glossing. In Proceed- 
ings of the 14th International Joint Confer- 
ence on Artificial Intelligence. 
Kevin Knight and Steve K. Luk. 1994. Build- 
ing a large-scale knowledge base for machine 
translation. In Proceedings of the 12th Na- 
tional Conference on Artificial Intelligence 
(AAAI). 
Kevin Knight, Ishwar Chander, Matthew 
Haines, Vasileios Hatzivassiloglou, Eduard 
Hovy, Masayo Iida, Steve K. Luk, Richard 
Whitney, and Kenji Yamada. 1995. Filling 
knowledge gaps in a broad-coverage machine 
translation system. In Proceedings of the 14th 
International Joint Conference on Artificial 
Intelligence. 
Carl Pollard and Ivan A. Sag. 1994. Head- 
Driven Phrase Structure Grammar. Studies 
in Contemporary Linguistics. University of 
Chicago Press, Chicago. 
