APPENDIX H :
SCATTER PLOTS OF FINAL TEST RESULT S
This appendix contains selected scatter plots of much of the data found i n
appendix G . The data points in the scatter plots are labeled with abbreviations o f
the organization names, as follows :
BBN (BBN)
GE (GE)
GE-CM (GE and CMU )
HU (Hughes )
LSI (LSI)
MDC (MDESC)
MITR (MITRE)
NM-BR (NMSU and Brandeis)
NYU (NYU)
PMAX (Paramax )
PRC (PRC )
SRA (SRA)
SRI (SRI)
UM-CQ (UMBC and ConQuest)
UMA (UMass)
UMI (UMichigan )
USC (USC)
Most of the scatter plots are taken from the All Templates Row in the summar y
score report. The All Templates scores include severe penalties for missing an d
spurious data. See the paper on evaluation metrics by N . Chinchor in Part I fo r
further information .
0O
w0
0
0
w0
6)
CC
was a
sa
a"
L.
0
c)N
h
cc:
b
a)
U
a al
n s
N n ~ 7z
N n •u _
U n
s
U
i
N
¢Nn
z J nn 2
_
0
aa
~n cW7
S0
C=i 2n
z
m n
G
¢
n
N N2
aZ NJ
¢
I-
0U n Nu,
0 0°o 00 0 0l9
	
N
O. S W U— m- 0 z
0ON0 000 o00N0O
a R W U_ N _. 0 Z
rn
0
0
Q
C0CD
U
6)
N
CD
oV
ON
w F
OO
0
0 U
o
0N
w F
. 0
O'C
H— 2
y
_ O0
0
O
b
0
m
0
e
N
•
co
E• E
W c••
n % •
o c =p w
	
4i > 2o •
	
E n ° = mW
i m —* m ,¢ C n a m
n N 2
• m
i
0
	
0
	
0 0
O. C W 0 . CO O Z '
e
.5
w
iE
°
_
W >
.
i~ y -e
<
	
E . L, _mn 2
	
o —
nZ
a
p n~ y _
m•
a a w n° Q a
s = n
A
m
0
UU
C0
00
.°o UW
C
0
a
;4,o
	
Q
W . °
OD a)IA
LT.
0 O O OI .
	
0
	
b O oOl
O . C W U- V! . O Z
J
V
C
of
CO
a
H-3
OO
x
4n
i
. wV
S
N 11I
O • Z n N
Q)
	
V
N
m o
n
S
	
n ~ n _ n a
tUq
0
rn
aO
O
3
0
W
U0CO
Q )
0
Cd
7
0
O
Oco OmOO
b
0
RS
E0
0> W¢ Q W Z W IZ . H_ 0 Z
0
b . ON
0 alU
30
0
c:
<7f S
i
~C n N
JP Z
z
m
. J
cc
m
?y
~ <
oa,
.
na
d
c d
•V
N
0
2
0)
OOG0 OInOC ON
0 y w 0 W Z W¢ a H _ 0 Z
w
H-4
uV
n a
m
tz « "
~ 1
Q
4.4
C
N
r~
O
111
O
a
a
O
0 0
	
m N 0
	
O O
0 W ¢ C , Z W¢` I- - O Z
0
0
UU
~OW/J
J
v> 0
	
RSW
¢
F
O
jI
W0
- 0-4 tig d
N
z ~•'1 ¢N W
4
I
O
n
ff N
0 0 a 0 e U, 0 0
	
O O
Q ¢ W O _ 0 )—0Z
OO
U
4-
b
a
0
0
0I
O
J
a
W
H— 5
C
O
•
	
nn
o co')>1
	
- C
0 .
.La
O C a,„ col'' a y E
•
E
0
aas
Q
II
I I
• Cl
— 0
4
0
r 0o
C
a)y
C0
as
r^v/
J
W
U
O
m
• aa.
O O 0 0 0 0 0 0 0 0 0O
	
m
	
aD
	
1~
	
m
	
.n
	
O
	
CI
	
N
a Q W U N O 2
u
c
E .=
C7
01 E
mas m
tLZf
o =a ~ m
.N w• m
€ O0 0 m O0 0€ N
CL R W V— N— O Z
h
o 0
0N
O
0
0
W
0
0/
0
O
H—6
0O
O
O
U
U
0
n
v
	
n II N
0
0 O 0 cu v)
Q)
O O O 0O
	
0)
	
EO
	
I- 0 0Y
	
O)
	
N
axLU V _U)_0z
O0
0
0
•
a
m
z a
O O O 0 0 0 O 0 0 O OO
	
0)
	
Q)
	
n
	
f0
	
u)
	
w
	
N
	
-
a¢CU_tn_0 Z
H—7
OO
O0(
N
Ob
0
a
w w
0
U0
0
0N
0. W U _ V1 _ 0 z
a
3 0
a
T n ~Z ~ n cc
	
2N
	
6 n¢ 7
LLL
N
i
Z
m
z
= N
U
6
xU
0 n
0 a 0 0 a
	
0 0
	
O O
d¢ W U~ N_ 0 Z
m xu
2 HI ¢ ccr
n
u n
• C
• `
d
Z
S
Z
IS
c
u)
oa
2
f"il I
O
w w
H- 8
3 n 2a
zn n N
• aN n m
2 m
=UU
2 6
Ilk,
O0
n
0
0
0
	
0
	
0
	
0
	
Y
	
!7
	
O
	
O
	
a) o
a¢ W U_ N_ 0 Z
2
2 no
V0
q
n r
	
nz 2 . 2a
N
z
a n c, a z
O o
0 0 C,
	
O O
0
0
0
0
0
O
O
a Q W U_ N_ 0 Z w
w
Oa
w
LanO
C.)
H- 9
laW
Z
s
rN • a
m•
S
RN
2 I n
	
NJ
nn
6mm
_
VN 0U
CO
0CO CO 0R
d CC W 0— N— 0 Z
m
CC U
N
GC
J
0 aC.)W
N
C MM~IH
CI)
N
C CC0
0
a g
2U
Wv zz
n 10N n
2 II
x U
mm ~ n S
m vJ n ~u o n m
a 2S
0H O n =
21g
0N 00va0 0N0NC0 0m0OC
G. 2 W U— U) — 0 Z
0y
0
0m
Cm
CA
CC
U
0
CM
0
C0
H—1 0
W
2U < tI• 6
O
t
n
'laz x
nn
x
nazn
G O n n
n
N n
OU
2 •
o o
a¢ W 0— N— O Z
0
4UW
a
OO
Z n CU
O.
n
W
x
i
x
n
2z
_m
O
m
72
n
n
o02 F2
n
o
0
i
n
0O O0O O0 o 00 U, 0
CL of W C1 _ _ O Z
0
o
O
00
H—1 1
ro
= N ann
n
nm <
A.
n~61
Wv
CN nV
n
¢A.
n
oO
n : J
O z
o
oi n
0
a
W
O OO
N
O OON O
t_
O
N
Oa0
t7
a¢ W U— N_ o z
C
O A
CL)
...ac
C
U
. c
0U
O al
n z
nN
io n
_u
C7
	
Nn
<a
N
n
x<
~nn. Qt-
n
o n
m
oC
s
m n=
z N
_
aJ
b
V n
2
O
O
0
O
n
too
0 O000
0
0
a¢ w U— m —O z
C
N
O
L
C
CO
Gi,
H-1 2
w'
E
2
CC,
N n O i n
6
n Z n
Z
n m 7
n6
	
n KN
U0
S
cc N
J
Z ~
ry
0 V Qr
2
C)
C)
bn
In
0
n
0N
0
a¢ W 0 — (N 0 Z
0
01 0
C)
aCO
n
U
Wu• 6
5
C
6 < <
0°1 n Q. N y 1 =
Z GS _
U C'
r
in
		
0♦+O
		
mO CGU
G )
O QV
WC) '0.J ~
0
V
bn
~ I
F" n
en
N
	
O
	
O
	
ftS
V. . 4
b0
w a
O
a 2 W V— — 0 Z
H-1 3
I
n '
u Nn i
n
aN
n
z
x
n 2
,~ n
la. x
zm
m s rN
caw
n Vo O
d n~
~~r
i
OU
oN a
0 0N
Zg
u
.6 N
n n i-
N~ _ 'n i `a6
x •
n
amn a
0
J
4 V
2r
u u
n
mH
OOa0
o. w0_v,o z
nO
N bO.c
o
0
0
a
0
O
J
Uw
00
0rn
0m
0A
J
w
a
O
0O
Oa
m
H-1 4
aa))
o
u sz a.
N f
	
n 2
-
N
2= xU
II?.
m
z n
sr
rca. E'•n
o
I n
02r
w< m i
U
U n n
= a
CO =
y
z
— n
J
0
— IS
N
•0
U
0
co')
N
I
0
J
U
0
0N 00mO 0 0N
a¢ W U_ m_ o z
0rn 0(o
a¢ W U— C)) — 0 z
H—1 5
02
i
Uw
• a .
z0 }
0 z
a 2 Oct
a•
•U
°
m
z • ~~
C
r~U
ca r
0U
o
U ca
-
	
O
y
	
o
z
U
ti ~
Lr
W
as
U
>
0
0 O0a NO O On
tO
m
Q
WQ
d Q W o— v _ O z
co,
— 0O
s
•r
a
rt
2
f7
2U z
m /
U
ar J
n
_
2 2
n
z
2
U7 ~
n
2
0
0w
0
a
O
IL Q W O— N— O Z
a O
H-1 6
-
'0
0
0.
a)
0) as
SU
V
•
N.
n oor.
=
N
z
° n
.
zn
m . •N S
n
o.
U
oa
C
Z
U
•~ J
O
.
OU
r ~
01
O
O
On O
Co
ON ON O
a¢ W 0 co 0 z
y
CC
av
	
INW
C.)
21
Q N
n a
n
or. m n •
x<
n
n
• J
_ • oi m S
az
Oo
n
U
3
0)
d 2 W c) — N— 0 z
0
	
0
ON
0
0
V)
n
J
J
0
W
v
H-1 7
