APPENDIX F:
FINAL TEST SCORE SUMMARIE S
The tables in this appendix are the summary score reports produced b y
the scoring program . For information on the format of these reports, see th e
paper by Nancy Chinchor on the evaluation metrics in Part I of thi s
proceedings . For information on the distinction between the tables labele d
"scores-max-tradeoff.tst2" (which are for the required test runs) and th e
remaining tables (which are for the optional test runs), see appendix B on the
test procedures .
	
For specific information on the test setups for the optiona l
runs, see the papers in Part II by the individual sites .
SLOT POS ACT COR
	
PAR
	
INC ICR IPA SPU
	
MIS NON REC PRE OVG FAL
template-id 118 221 97
	
0
	
0 0 0 124
	
21 17 82 44 56
incident-date 114 0 0
	
0
	
0 0 0 0
	
114 4 0 * *
incident-type 118 97 75
	
11
	
11 0 11 0
	
21 0 68 83 0 0
category 83 0 0
	
0
	
0 0 0 0
	
83 35 0 * * 0
indiv-perps 97 0 0
	
0
	
0 0 0 0
	
97 54 0 * *
org-perps 70 0 0
	
0
	
0 0 0 0
	
70 56 0 *
perp-confidence 70 0 0
	
0
	
0 0 0 0
	
70 56 0 * 0
phys-target-ids 55 0 0
	
0
	
0 0 0 0
	
55 79 0 *
phys-target-num 38 0 0
	
0
	
0 0 0 0
	
38 80 0 * *
phys-target-types 55 0 0
	
0
	
0 0 0 0
	
55 79 0 * 0
human-target-ids 152 0 0
	
0
	
0 0 0 0
	
152 22 0 *
human-target-num 96 0 0
	
0
	
0 0 0 0
	
96 22 0
human-target-types 152 0 0
	
0
	
0 0 0 0
	
152 22 0 * * 0
target-nationality 18 0 0
	
0
	
0 0 0 0
	
18 105 0 * 0
instrument-types 25 0 0
	
0
	
0 0 0 0
	
25 93 0 * 0
incident-location 118 0 0
	
0
	
0 0 0 0
	
118 0 0
phys-effects 37 0 0
	
0
	
0 0 0 0
	
37 93 0 * 0
human-effects 56 0 0
	
0
	
0 0 0 0
	
56 83 0 * 0
MATCHED ONLY 1237 318 172
	
11
	
11 0 11 124
	
1043 761 14 56 39
MATCHED/MISSING 1472 318 172
	
11
	
11 0 11 124
	
1278 900 12 56 39
ALL TEMPLATES 1472 442 172
	
11
	
11 0 11 248
	
1278 2884 12 40 56
SET FILLS ONLY 614 97 75
	
11
	
11 0 11 0
	
517 566 13 83 0 0
Table Fl. ADS : scores-max-tradeoff .tst2
SLOT POS ACT COR PAR INC ICR IPA SPU MIS NON REC PRE OVG FAL
template-id 119 296 117 0 0 0 0 179 2 5 98 40 60
incident-date 115 111 68 22 21 0 22 0 4 4 69 71 0
incident-type 119 117 84 23 10 0 23 0 2 0 80 82 0 0
category 86 74 49 0 16 0 0 9 21 24 57 66 12 1 7
indiv-perps 101 60 22 0 26 10 0 12 53 41 22 37 20
org-perps 71 28 19 0 4 0 0 5 48 51 27 68 1 8
perp-confidence 71 28 10 2 11 0 2 5 48 51 15 39 18 2
phys-target-ids 55 29 12 0 13 11 0 4 30 76 22 41 14
phys-target-num 38 31 20 0 6 0 0 5 12 76 53 64 1 6
phys-target-types 55 29 10 2 13 5 2 4 30 76 20 38 14 1
human-target-ids 152 85 28 17 34 18 17 6 73 16 24 43 7
human-target-num 97 83 41 5 31 0 5 6 20 16 45 52 7
human-target-types 152 85 29 36 14 16 36 6 73 16 31 55 7 1
*target-nationality 18 0 0 0 0 0 0 0 18 106 0 * 0
instrument-types 25 50 11 6 4 0 0 29 4 71 56 28 58 2
incident-location 119 113 44 59 10 0 27 0 6 0 62 65 0
phys-effects 37 24 4 10 3 2 10 7 20 87 24 38 29 1
human-effects 55 20 1 7 5 1 7 7 42 77 8 22 35 1
MATCHED ONLY 1466 1263 569 189 221 63 151 284 487 780 45 52 22
MATCHED/MISSING 1485 1263 569 189 221 63 151 284 506 793 45 52 22
ALL TEMPLATES 1485 2530 569 189 221 63 151 1551 506 2574 45 26 6 1
SET FILLS ONLY 618 427 198 86 76 24 80 67 258 508 39 56 16 1
Table F2. BBN : scores-max-tradeoff .tst2
F—2
SLOT POS ACT COR PAR INC ICR
	
IPA SPU MIS NON REC PRE OVG FAL
template-id 118 184 107 0 0 0
	
0 77 11 12 91 58 42
incident-date 114 101 60 18 23 0
	
18 0 13 4 60 68 0
incident-type 118 107 82 12 13 0
	
12 0 11 0 74 82 0 0
category 85 65 42 0 17 0
	
0 6 26 27 49 65 9 1 5
indiv-perps 100 53 17 1 24 7
	
1 11 58 42 18 33 21
org-perps 69 25 16 0 7 0
	
0 2 46 55 23 64 8
perp-confidence 69 25 9 3 11 0
	
3 2 46 55 15 42 8 2
phys-target-ids 55 28 18 1 6 17
	
1 3 30 76 34 66 1 1
phys-target-num 38 30 20 0 6 0
	
0 4 12 76 53 67 1 3
phys-target-types 55 28 15 2 8 9
	
2 3 30 76 29 57 11 0
human-target-ids 148 85 29 14 35 20
	
14 7 70 15 24 42 8
human-target-num 96 84 42 6 29 0
	
6 7 19 15 47 54 8
human-target-types 148 85 30 33 15 16
	
33 7 70 15 31 55 8 1
target-nationality 16 0 0 0 0 0
	
0 0 16 106 0 * 0
instrument-types 25 21 8 2 2 0
	
0 9 13 87 36 43 43 0
incident-location 118 101 41 51 9 0
	
24 0 17 0 56 66 0
phys-effects 37 24 6 7 3 4
	
7 8 21 85 26 40 33 1
human-effects 56 17 2 7 3 2
	
7 5 44 78 10 32 29 1
MATCHED ONLY 1360 1063 544 157 211 75
	
128 151 448 739 46 58 1 4
MATCHED/MISSING 1465 1063 544 157 211 75
	
128 151 553 824 42 58 1 4
ALL TEMPLATES 1465 1659 544 157 211 75
	
128 747 553 1542 42 38 45
SET FILLS ONLY 609 372 194 66 72 31
	
64 40 277 529 37 61 11 0
Table F3. BB N : scores-cutoff-at-6 .tst2 (optional test run )
SLOT POS ACT COR PAR INC ICR IPA SPU MIS NON REC PRE OVG FAL
template-id 108 105 74 0 0 0 0 31 34 24 68 70 30
incident-date 104 72 52 11 9 0 11 0 32 4 55 80 0
incident-type 108 74 68 3 3 0 3 0 34 0 64 94 0 0
category 76 53 37 0 7 0 0 9 32 23 49 70 17 1 2
indiv-perps 94 44 18 2 11 1 2 13 63 42 20 43 30
org-perps 62 51 20 0 10 2 0 21 32 43 32 39 41
perp-confidence 62 51 7 10 13 2 10 21 32 43 19 24 41 5
phys-target-ids 54 39 21 2 4 4 2 12 27 70 41 56 3 1
phys-target-num 37 23 14 1 8 0 1 0 14 71 39 63 0
phys-target-types 54 39 17 3 7 2 3 12 27 70 34 47 31 1
human-target-ids 138 77 41 10 13 10 10 13 74 20 33 60 1 7
human-target-num 87 58 33 4 20 0 4 1 30 20 40 60 2
human-target-types 138 76 38 15 10 4 15 13 75 20 33 60 17 2
target-nationality 16 5 3 1 0 1 1 1 12 95 22 70 20 0
instrument-types 25 18 12 2 1 0 0 3 10 80 52 72 17 0
incident-location 108 74 50 22 2 0 0 0 34 0 56 82 0
phys-effects 36 18 8 1 2 2 1 7 25 79 24 47 39 1
human-effects 56 30 9 8 3 2 8 10 36 68 23 43 33 1
MATCHED ONLY 978 907 522 95 123 30 71 167 238 521 58 63 1 8
MATCHED/MISSING 1363 907 522 95 123 30 71 167 623 772 42 63 1 8
ALL TEMPLATES 1363 1236 522 95 123 30 71 496 623 1011 42 46 40
SET FILLS ONLY 571 364 199 43 46 13 41 76 283 478 39 60 21 1
Table F4 . GE : scores-max-tradeoff .tst2
F—3
_SLOT POS ACT COR PAR INC ICR IPA SPU
	
MIS
	
NON REC PRE OVG FAL
template-id 109 84 40 0 0 0 0 44
	
69
	
24 37 48 52incident-date
105 34 22 6 6 4 6 0
	
71
	
4 24 74 0
incident-type 109 40 24 14 2 0 14 0
	
69
	
0 28 78 0 0
category 77 40 15 0 13 0 0 12
	
49
	
20 19 38 30 1 8
indiv-perps 93 9 1 0 4 1 0 4
	
88
	
47 1 11 44
org-perps 62 4 2 0 1 0 0 1
	
59
	
53 3 50 25
perp-confidence 62 40 0 8 11 0 8 21
	
43
	
33 6 10 52 5phys-target-ids
56 8 0 1 2 0 1 5
	
53
	
66 1 6 62phys-target-num
38 19 2 0 2 0 0 15
	
34
	
56 5 10 79phys-target-types
56 2 0 0 0 0 0 2
	
56
	
68 0 0 100 0
human-target-ids 134 10 1 3 3 0 3 3
	
127
	
19 2 25 30
human-target-num 87 21 10 0 10 0 0 1
	
67
	
21 11 48 5
human-target-types 134 2 0 0 2 0 0 0
	
132
	
22 0 0 0 0
target-nationality 16 1 0 0 0 0 0 1
	
16
	
95 . 0 0 100 0
instrument-types 25 0 0 0 0 0 0 0
	
25
	
84 0 0
incident-location 109 22 11 5 6 0 1 0
	
87
	
0 12 61 0
phys-effects 36 3 0 1 0 0 1 2
	
35
	
82 1 17 67 0
human-effects 56 3 0 1 0 0 1 2
	
55
	
72 1 17 67 0
MATCHED ONLY 526 342 128 39 62 5 35 113
	
297
	
286 28 43 33
MATCHED/MISSING 1364 342 128 39 62 5 35 113
	
1135
	
766 11 43 33ALL TEMPLATES 1364 593
128 39 62 5 35 364
	
1135
	
1263 11 25 61
SET FILLS ONLY 571 131 39 24 28 0 24 40
	
480
	
476 9 39 30 0
Table F5. GTE : scores-max-tradeoff .tst2
SLOT POS ACT COR PAR INC ICR IPA SPU MIS NON REC PRE OVG FAL
template-id 116 163 86 0 0 0 0 77 30 0 74 53 47
incident-date 112 82 33 25 23 33 25 1 31 3 41 55 1
incident-type 116 86 45 18 23 0 18 0 30 0 46 63 0 1
category 85 86 47 0 21 0 0 18 17 13 55 55 21 27
indiv-perps 100 44 17 0 17 1 0 10 66 40 17 39 23
org-perps 69 58 20 1 18 6 1 19 30 36 30 35 33
perp-confidence 68 50 11 6 17 2 6 16 34 40 20 28 32 5
phys-target-ids 57 27 9 2 7 1 2 9 39 71 18 37 33
phys-target-num 39 17 7 0 :5 0 0 5 27 72 18 41 29
phys-target-types 57 27 7 1 10 3 1 9 39 71 13 28 33 1
human-target-ids 147 222 27 20 31 0 20 144 69 9 25 17 65
human-target-num 93 68 12 4 38 0 4 14 39 9 15 20 20
human-target-types 147 221 31 29 13 3 29 1 43 69 9 31 20 65 1 0
target-nationality 16 0 0 0 0 0 0 0 16 103 0 0
instrument-types 25 4 0 0 2 0 0 2 23 89 0 0 50 0
incident-location 116 79 15 55 9 0 20 0 37 0 37 54 0
phys-effects 37 20 6 2 2 2 2 10 27 84 19 35 50 2
human-effects 56 26 1 4 6 0 4 15 45 72 5 12 58 2
MATCHED ONLY 1109 1280 374 167 247 51 132 492 321 501 41 36 38
MATCHED/MISSING 1456 1280 374 167 247 51 132 492 668 721 31 36 38
ALL TEMPLATES 1456 2082 374 167 247 51 132 1294 668 1383 31 22 62
SET FILLS ONLY 607 520 148 60 99 10 60 213 300 481 29 34 41 2
Table F6 . HUGHES : scores-max-tradeoff .tst 2
F--4
SLOT POS ACT COR PAR INC ICR
	
IPA SPU MIS NON REC
	
PRE OVG FAL
template-id 109 79 58 0 0 0
	
0 21 51 24 53
	
73 26
incident-date 105 57 21 22 14 21
	
22 0 48 4 30
	
56 0
incident-type 109 58 39 17 2 0
	
17 0 51 0 44
	
82 0 0
category 78 58 37 0 7 0
	
0 14 34 17 47
	
64 24 1 5
indiv-perps 96 11 5 0 4 0
	
0 2 87 44 5
	
45 1 8
org-perps 62 1 0 0 1 0
	
0 0 61 54 0
	
0 0
perp-confidence 62 0 0 0 0 0
	
0 0 62 54 0 0
phys-target-ids 56 24 4 6 6 0
	
6 8 40 66 12
	
29 33
phys-target-num 39 17 9 1 4 0
	
1 3 25 67 24
	
56 1 8
phys-target-types 56 24 5 3 8 2
	
3 8 40 66 12
	
27 33 1
human-target-ids 135 30 10 9 5 1
	
9 6 111 21 11
	
48 20
human-target-num 86 25 10 0 13 0
	
0 2 63 21 12
	
40 8
human-target-types 135 30 10 6 8 4
	
6 6 111 21 10
	
43 20 1
target-nationality 16 0 0 0 0 0
	
0 0 16 97 0 0
instrument-types 25 0 0 0 0 0
	
0 0 25 84 0 0
incident-location 109 44 18 ' 22 4 0
	
7 0 65 0 27
	
66 0
phys-effects 38 14 4 4 0 0
	
4 6 30 80 16
	
43 43 1
human-effects 56 7 4 1 0 2
	
1 2 51 73 8
	
64 28 0
MATCHED ONLY 794 479 234 91 76 30
	
76 78 393 400 35
	
58 1 6
MATCHED/MISSING 1372 479 234 91 76 30
	
76 78 971 793 20
	
58 1 6
ALL TEMPLATES 1372 604 234 91 76 30
	
76 203 971 1031 20
	
46 34
SET FILLS ONLY 575 191 99 31 25 8
	
31 36 420 492 20
	
60 19 0
Table F7 . ITP : scores-max-tradeoff .tst2
SLOT POS ACT COR PAR INC ICR IPA SPU MIS NON REC PRE OVG FAL
template-id 106 81 52 0 0 0 0 29 54 34 49 64 36
incident-date 102 52 14 15 23 6 15 0 50 4 21 41 0
incident-type 106 52 38 7 7 0 7 0 54 0 39 80 0 0
category 78 52 31 0 1 0 0 20 46 8 40 60 38 1 6
indiv-perps 97 17 2 0 12 0 0 3 83 39 2 12 1 8
org-perps 59 4 3 0 1 3 0 0 55 54 5 75 0
perp-confidence 61 52 1 8 12 1 8 31 40 21 8 10 60 7
phys-target-ids 55 3 2 0 1 2 0 0 52 66 4 67 0
phys-target-num 39 12 6 0 6 0 0 0 27 67 15 50 0
phys-target-types 55 3 0 1. 2 0 1 0 52 66 1 17 0 0
human-target-ids 127 15 3 6 5 0 6 1 113 22 5 40 7
human-target-num 83 37 13 0 20 0 0 4 50 19 16 35 1 1
human-target-types 127 0 0 0 0 0 0 0 127 23 0 0
target-nationality 16 0 0 0 0 0 0 0 16 94 0 0
instrument-types 23 0 0 0 0 0 0 0 23 83 0 0
incident-location 106 44 15 27 2 0 0 0 62 0 27 65 0
phys-effects 38 11 0 7 2 0 7 2 29 78 9 32 18 1
human-effects 53 20 3 1 8 3 1 8 41 65 7 18 40 1
MATCHED ONLY 676 455 183 72 102 15 45 98 319 356 32 48 22
MATCHED/MISSING 1331 455 183 72 102 15 45 98 974 743 16 48 22
ALL TEMPLATES 1331 666 183 72 102 15 45 309 974 1025 16 33 46
SET FILLS ONLY 557 190 73 24 32 4 24 61 428 438 15 45 32 0
Table F8. LSI : scores-max-tradeoff .tst 2
F— 5
SLOT POS ACT COR PAR INC ICR IPA SPU MIS NON REC PRE OVG FAL
template-id 110 118 76 0 0 0 0 42 34 20 69 64 36
incident-date 106 71 41 14 16 0 14 0 35 4 45 68 0
incident-type 110 76 50 19 7 0 19 0 34 0 54 78 0 0
category 77 31 21 0 9 0 0 1 47 32 27 68 3 7
indiv-perps 94 27 10 0 7 2 0 10 77 41 11 37 37
org-perps 66 14 11 0 1 2 0 2 54 51 17 78 1 4
perp-confidence 65 13 6 0 5 1 0 2 54 51 9 46 15 1
phys-target-ids 52 19 6 6 4 3 6 3 36 73 17 47 1 6
phys-target-num 36 13 6 0 7 0 0 0 23 74 17 46 0
phys-target-types 52 19 7 4 5 4 4 3 36 73 17 47 16 0
human-target-ids 142 121 28 16 21 2 16 56 77 15 25 30 46
human-target-num 90 57 12 0 40 0 0 5 38 15 13 21 9
human-target-types 142 121 27 24 14 1 24 56 77 15 27 32 46 5
target-nationality 16 1 0 0 1 0 0 0 15 98 0 0 0 0
instrument-types 25 0 0 0 0 0 0 0 25 85 0 * 0
incident-location 110 70 36 29 5 0 4 0 40 0 46 72 0
phys-effects 35 0 0 0 0 0 0 0 35 87 0 * * 0
human-effects 55 1 0 1 0 0 1 0 54 76 1 50 0 0
MATCHED ONLY 1007 772 337 113 142 15 88 180 415 562 39 51 23
MATCHED/MISSING 1383 772 337 113 142 15 88 180 791 810 28 51 23
ALL TEMPLATES 1383 1063 337 113 142 15 88 471 791 1271 28 37 44
SET FILLS ONLY 577 262 111 48 41 6 48 62 377 517 23 52 24 0
Table F9. MDC : scores-max-tradeoff .tst2
SLOT POS ACT COR
	
PAR
	
INC ICR
	
IPA SPU
	
MIS NON REC PRE OVG FA L
template-id 112 187 97
	
0
	
0 0
	
0 90
	
15 9 87 52 48
incident-date 108 92 45
	
29
	
18 0
	
29 0
	
16 4 55 65 0
incident-type 112 97 67
	
27
	
3 0
	
27 0
	
15 0 72 83 0 0
category 80 69 43
	
0
	
14 0
	
0 12
	
23 20 54 62 17 1 8
indiv-perps 101 63 22
	
4
	
19 7
	
4 18
	
56 28 24 38 28
org-perps 66 55 26
	
3
	
,10 15
	
3 16
	
27 38 42 50 29
perp-confidence 66 25 8
	
2
	
11 5
	
2 4
	
45 50 14 36 16 2
phys-target-ids 54 28 19
	
2
	
5 17
	
2 2
	
28 74 37 71 7
phys-target-num 37 22 18
	
1
	
3 0
	
1 0
	
15 75 50 84 0
phys-target-types 54 28 18
	
1
	
7 12
	
1 2
	
28 74 34 66 7 0
human-target-ids 145 98 36
	
22
	
17 30
	
22 23
	
70 13 32 48 23
human-target-num 91 67 35
	
6
	
18 0
	
6 8
	
32 13 42 57 1 2
human-target-types 145 96 42
	
21
	
10 17
	
21 23
	
72 13 36 55 24 2
target-nationality 18 9 2
	
1
	
0 1
	
1 6
	
15 94 14 28 67 0
instrument-types 26 10 4
	
1
	
0 0
	
0 5
	
21 81 17 45 50 0
incident-location 112 95 35
	
43
	
17 0
	
4 0
	
17 0 50 59 0
phys-effects 36 19 9
	
3
	
1 6
	
3 6
	
23 85 29 55 32 1
human-effects 57 33 13
	
7
	
3 8
	
7 10
	
34 71 29 50 30 1
MATCHED ONLY 1266 1093 539
	
173
	
156 118
	
133 225
	
398 627 49 57 20
MATCHED/MISSING 1420 1093 539
	
173
	
156 118
	
133 225
	
552 742 44 57 20
ALL TEMPLATES 1420 1752 539
	
173
	
156 118
	
133 884
	
552 1673 44 36 50
SET FILLS ONLY 594 386 206
	
63
	
49 49
	
62 68
	
276 488 40 62 18 1
Table F10. NYU : scores-max-tradeoff .tst2
F—6
SLOT
	
POS ACT COR
	
PAR
	
INC ICR
	
IPA SPU
	
MIS
	
NON REC
	
PRE OVG FAL
template-id
	
109
	
118 83
	
0
	
0 0
	
0 35
	
26
	
24 76
	
70 30
incident-date
	
105
	
78 38
	
24
	
16 0
	
24 0
	
27
	
4 48
	
64 0
incident-type
	
109
	
83 62
	
17
	
4 0
	
17 0
	
26
	
0 65
	
85 0 0
category
	
76
	
57 34
	
0
	
11 0
	
0 12
	
31
	
21 45
	
60 21 1 6
indiv-perps
	
99
	
54 20
	
3
	
15 6
	
3 16
	
61
	
28 22
	
40 30
org-perps
	
64
	
45 20
	
2
	
9 11
	
2 14
	
33
	
39 33
	
47 31
perp-confidence
	
64
	
22 7
	
2
	
10 4
	
2 3
	
45
	
50 12
	
36 14 2
phys-target-ids
	
55
	
30 19
	
2
	
7 17
	
2 2
	
27
	
70 36
	
67 7
phys-target-num
	
38
	
24 20
	
1
	
3 0
	
1 0
	
14
	
71 54
	
85 0
phys-target-types
	
55
	
30 18
	
1
	
9 12
	
1 2
	
27
	
70 34
	
62 7 1
human-target-ids
	
140
	
94 34
	
21
	
19 28
	
21 20
	
66
	
14 32
	
47 21
human-target-num
	
87
	
68 35
	
6
	
19 0
	
6 8
	
27
	
14 44
	
56 1 2
human-target-types
	
140
	
92 40
	
22
	
10 15
	
22 20
	
68
	
14 36
	
55 22 2
target-nationality
	
18
	
9 2
	
1
	
0 1
	
1 6
	
15
	
91 14
	
28 67 0
instrument-types
	
25
	
8 4
	
1
	
0 0
	
0 3
	
20
	
81 18
	
56 38 0
incident-location
	
109
	
81 29
	
38
	
14 0
	
4 0
	
28
	
0 44
	
59 0
phys-effects
	
37
	
17 9
	
2
	
1 6
	
2 5
	
25
	
82 27
	
59 29 1
human-effects
	
55
	
29 11
	
5
	
3 6
	
5 10
	
36
	
68 24
	
46 34 1
MATCHED ONLY
	
1108
	
939 485
	
148
	
150 106
	
113 156
	
325
	
550 50
	
60 1 7
MATCHED/MISSING
	
1385
	
939 485
	
148
	
150 106
	
113 156
	
602
	
741 40
	
60 1 7
ALL TEMPLATES
	
1385
	
1249 485
	
148
	
150 106
	
113 466
	
602
	
1060 40
	
45 37
SET FILLS ONLY
	
579
	
347 187
	
51
	
48 44
	
50 61
	
293
	
477 37
	
61 18 1
Table F1.1. NYU :
	
scores-notarg .tst2
	
(optional
	
test
	
run )
SLOT
	
POS
	
ACT COR
	
PAR
	
INC ICR
	
IPA SPU
	
MIS
	
NON REC
	
PRE OVG FA L
template-id
	
111
	
139 91
	
0
	
0 0
	
0 48
	
20
	
21 82
	
65 34
incident-date
	
107
	
86 40
	
28
	
18 0
	
28 0
	
21
	
4 50
	
63 0
incident-type
	
111
	
91 63
	
25
	
3 0
	
25 0
	
20
	
0 68
	
83 0 0
category
	
79
	
64 40
	
0
	
12 0
	
0 12
	
27
	
20 51
	
62 19 1 7
indiv-perps
	
101
	
60 23
	
4
	
17 7
	
4 16
	
57
	
29 25
	
42 27
org-perps
	
66
	
51 25
	
2
	
9 15
	
2 15
	
30
	
38 39
	
51 29
perp-confidence
	
66
	
28 11
	
2
	
11 7
	
2 4
	
42
	
49 18
	
43 14 2
phys-target-ids
	
55
	
29 19
	
2
	
6 17
	
2 2
	
28
	
72 36
	
69 7
phys-target-num
	
38
	
23 19
	
1
	
3 0
	
1 0
	
15
	
73 51
	
85 0
phys-target-types
	
55
	
29 18
	
1
	
8 12
	
1 2
	
28
	
72 34
	
64 7 0
human-target-ids
	
142
	
92 34
	
20
	
19 28
	
20 19
	
69
	
15 31
	
48 2 1
human-target-num
	
89
	
66 36
	
6
	
17 0
	
6 7
	
30
	
15 44
	
59 1 1
human-target-types
	
142
	
90 40
	
21
	
10 15
	
21 19
	
71
	
15 36
	
56 21 2
target-nationality
	
18
	
9 2
	
1
	
0 1
	
1 6
	
15
	
93 14
	
28 67 0
instrument-types
	
25
	
10 4
	
1
	
0 0
	
0 5
	
20
	
81 18
	
45 50 0
incident-location
	
111
	
89 35
	
40
	
14 0
	
4 0
	
22
	
0 50
	
62 0
phys-effects
	
37
	
18 9
	
2
	
1 6
	
2 6
	
25
	
83 27
	
56 33 1
human-effects
	
54
	
29 11
	
5
	
2 6
	
5 11
	
36
	
70 25
	
46 38 1
MATCHED ONLY
	
1188
	
1003 520
	
161
	
150 114
	
124 172
	
357
	
604 50
	
60 1 7
MATCHED/MISSING
	
1407
	
1003 520
	
161
	
150 114
	
124 172
	
576
	
750 43
	
60 1 7
ALL TEMPLATES
	
1407
	
1405 520
	
161
	
150 114
	
124 574
	
576
	
1201 43
	
43 41
SET FILLS ONLY
	
587
	
368 198
	
58
	
47 47
	
57 65
	
284
	
483 39
	
62 18 0
Table F12. NYU : scores-noempties .tst2 (optional test run )
F— 7
SLOT POS ACT COR PAR INC ICR IPA SPU
	
MIS
	
NON REC
	
PRE
	
OVG FAL
template-id 120 174 106 0 0 0 0 68
	
14
	
16 88
	
61
	
39
incident-date 116 100 45 29 26 0 29 0
	
16
	
4 51
	
60
	
0
incident-type 120 106 79 7 20 0 7 0
	
14
	
0 69
	
78
	
0 1
category 90 106 61 0 20 0 0 25
	
9
	
5 68
	
58
	
24 30
indiv-perps 97 10 3 0 6 1 0 1
	
88
	
55 3
	
30
	
1 0
org-perps 70 1 1 0 0 0 0 0
	
69
	
58 1
	
100
	
0
perp-confidence 71 106 1 26 29 0 26 50
	
15
	
7 20
	
13
	
47 1 2
phys-target-ids 58 17 5 1 9 1 1 2
	
43
	
78 9
	
32
	
1 2
phys-target-num 40 16 10 1 4 0 1 1
	
25
	
79 26
	
66
	
6
phys-target-types 58 9 5 2 2 2 2 0
	
49
	
80 10
	
67
	
0 0
human-target-ids 149 6 1 3 2 1 3 0
	
143
	
23 2
	
42
	
0
human-target-num 97 4 4 0 0 0 0 0
	
93
	
23 4
	
100
	
0
human-target-types 149 5 2 3 0 1 3 0
	
144
	
23 2
	
70
	
0 0
target-nationality 18 1 1 0 0 0 0 0
	
17
	
106 6
	
100
	
0 0
instrument-types 26 0 0 0 0 0 0 0
	
26
	
94 0 0
incident-location 120 105 33 46 26 0 4 0
	
15
	
0 47
	
53
	
0
phys-effects 38 29 1 12 4 0 12 12
	
21
	
81 18
	
24
	
41 2
human-effects 57 14 0 6 6 0 6 2
	
45
	
82 5
	
21
	
14 1
MATCHED ONLY 1338 809 358 136 154 6 94 161
	
690
	
717 32
	
53
	
20
MATCHED/MISSING 1494 809 358 136 154 6 94 161
	
846
	
814 28
	
53
	
20
ALL TEMPLATES 1494 1 193 358 136 154 6 94 545
	
846
	
1586 28
	
36
	
46
SET FILLS ONLY 627 376 150 56 81 3 56 89
	
340
	
478 28
	
47
	
24 1
Table F13 . PRC : scores-rnax-tradeoff .tst2
SLOT POS ACT COR PAR INC ICR IPA SPU MIS NON REC PRE OVG FAL
template-id 105 83 56 0 0 0 0 27 49 28 53 67 32
incident-date 101 52 23 20 9 0 20 0 49 4 33 63 0
incident-type 105 56 41 12 3 0 12 0 49 0 45 84 0 0
category 72 26 17 0 5 0 0 4 50 29 24 65 15 7
indiv-perps 96 16 7 1 7 2 1 1 81 42 8 47 6
org-perps 60 15 6 1 5 1 1 3 48 51 11 43 20
perp-confidence 60 15 3 3 6 1 3 3 48 51 8 30 20 1
phys-target-ids 54 16 6 2 7 1 2 1 39 66 13 44 6
phys-target-num 37 16 13 0 2 0 0 1 22 67 35 81 6
phys-target-types 54 16 6 1 8 0 1 1 39 66 12 41 6 0
human-target-ids 132 46 22 12 9 2 12 3 89 20 21 61 6
human-target-num 84 37 26 3 7 0 3 1 48 20 33 74 3
human-target-types 132 46 21 15 7 1 15 3 89 20 22 62 6 1
target-nationality 16 3 1 2 C) 1 2 0 13 93 12 67 0 0
instrument-types 24 0 0 0 C) 0 0 0 24 81 0 0
incident-location 105 55 25 25 5 0 0 0 50 0 36 68 0
phys-effects 36 5 2 2 C) 2 2 1 32 80 8 60 20 0
human-effects 53 10 7 0 0 1 0 3 46 69 13 70 30 0
MATCHED ONLY 758 513 282 99 80 12 74 52 297 432 44 65 1 0
MATCHED/MISSING 1326 513 282 99 80 12 74 52 865 787 25 65 1 0
ALL TEMPLATES 1326 687 282 99 80 12 74 226 865 1075 25 48 33
SET FILLS ONLY 552 177 98 35 29 6 35 15 390 489 21 65 8 0
Table F14 . SRI : scores-max-tradeoff.tst2
F-8
SLOT
	
POS ACT COR
	
PAR
	
INC ICR
	
IPA SPU
	
MIS
	
NON REC
	
PRE
	
OVG FAL
template-id
	
103
	
63 32
	
0
	
0 0
	
0 31
	
71
	
34 31
	
51
	
49
incident-date
	
99
	
30 14
	
5
	
11 0
	
5 0
	
69
	
4 17
	
55
	
0
incident-type
	
103
	
32 19
	
1
	
12 0
	
1 0
	
71
	
0 19
	
61
	
0 0
category
	
74
	
32 18
	
0
	
5 0
	
0 9
	
51
	
20 24
	
56
	
28 1 1
indiv-perps
	
93
	
0 0
	
0
	
0 0
	
0 0
	
93
	
43 0
org-perps
	
61
	
0 0
	
0
	
0 0
	
0 0
	
61
	
49 0
perp-confidence
	
61
	
0 0
	
0
	
0 0
	
0 0
	
61
	
49 0 0
phys-target-ids
	
52
	
0 0
	
0
	
0 0
	
0 0
	
52
	
66 0
phys-target-num
	
36
	
0 0
	
0
	
0 0
	
0 0
	
36
	
67 0
phys-target-types
	
52
	
0 0
	
0
	
0 0
	
0 0
	
52
	
66 0 0
human-target-ids
	
133
	
2 2
	
0
	
0 0
	
0 0
	
131
	
20 2
	
100
	
0
human-target-num
	
83
	
0 0
	
0
	
0 0
	
0 0
	
83
	
20 0
human-target-types
	
133
	
0 0
	
0
	
0 0
	
0 0
	
133
	
20 0 0
target-nationality
	
16
	
0 0
	
0
	
0 0
	
0 0
	
16
	
91 0 0
instrument-types
	
23
	
0 0
	
0
	
0 0
	
0 0
	
23
	
80 0 0
incident-location
	
103
	
0 0
	
0
	
0 0
	
0 0
	
103
	
0 0
phys-effects
	
35
	
0 0
	
0
	
0 0
	
0 0
	
35
	
80 0 0
human-effects
	
53
	
0 0
	
0
	
0 0
	
0 0
	
53
	
70 0 0
MATCHED ONLY
	
495
	
159 85
	
6
	
28 0
	
6 40
	
376
	
246 18
	
55
	
25
MATCHED/MISSING
	
1313
	
159 85
	
6
	
28 0
	
6 40
	
1194
	
779 7
	
55
	
25
ALL TEMPLATES
	
1313
	
254 85
	
6
	
28 0
	
6 135
	
1194
	
1211 7
	
35
	
53
SET FILLS ONLY
	
550
	
64 37
	
1
	
17 0
	
1 9
	
495
	
476 7
	
58
	
14 0
Table F15. SYNCHRONETICS/UMD :
	
scores-max-tradeoff .tst2
SLOT
	
POS ACT COR
	
PAR
	
INC ICR
	
IPA SPU
	
MIS
	
NON REC
	
PRE
	
OVG FAL
template-id
	
104
	
47 38
	
0
	
0 0
	
0 9
	
66
	
36 36
	
81
	
1 9
incident-date
	
100
	
35 25
	
7
	
3 0
	
7 0
	
65
	
4 28
	
81
	
0
incident-type
	
104
	
38 34
	
4
	
0 0
	
4 0
	
66
	
0 35
	
95
	
0 0
category
	
71
	
28 15
	
0
	
7 0
	
0 6
	
49
	
27 21
	
54
	
21 1 0
indiv-perps
	
93
	
14 6
	
1
	
3 0
	
1 4
	
83
	
41 7
	
46
	
28
org-perps
	
60
	
41 11
	
1
	
6 2
	
1 23
	
42
	
37 19
	
28
	
56
perp-confidence
	
60
	
37 7
	
2
	
9 1
	
2 19
	
42
	
37 13
	
22
	
51 5
phys-target-ids
	
53
	
17 10
	
2
	
0 0
	
2 5
	
41
	
66 21
	
65
	
29
phys-target-num
	
37
	
11 7
	
0
	
4 0
	
0 0
	
26
	
67 19
	
64
	
0
phys-target-types
	
53
	
17 9
	
1
	
2 0
	
1 5
	
41
	
66 18
	
56
	
29 0
human-target-ids
	
127
	
37 12
	
7
	
6 3
	
7 12
	
102
	
19 12
	
42
	
32
human-target-num
	
83
	
25 10
	
1
	
12 0
	
1 2
	
60
	
19 13
	
42
	
8
human-target-types
	
127
	
37 15
	
7
	
3 3
	
7 12
	
102
	
19 14
	
50
	
32 1
target-nationality
	
16
	
3 0
	
0
	
0 0
	
0 3
	
16
	
89 0
	
0
	
100 0
instrument-types
	
23
	
13 3
	
1
	
0 0
	
0 9
	
19
	
77 15
	
27
	
69 0
incident-location
	
104
	
33 8
	
22
	
3 0
	
2 0
	
71
	
0 18
	
58
	
0
phys-effects
	
36
	
17 6
	
2
	
1 1
	
2 8
	
27
	
77 19
	
41
	
47 1
human-effects
	
53
	
1 0
	
0
	
1 0
	
0 0
	
52
	
71 0
	
0
	
0 0
MATCHED ONLY
	
519
	
451 216
	
58
	
60 10
	
37 117
	
185
	
274 47
	
54
	
26
MATCHED/MISSING
	
1304
	
451 216
	
58
	
60 10
	
37 117
	
970
	
752 19
	
54
	
26
ALL TEMPLATES
	
1304
	
558 216
	
58
	
60 10
	
37 224
	
970
	
818 19
	
44
	
40
SET FILLS ONLY
	
543
	
191 89
	
17
	
23 5
	
16 62
	
414
	
463 18
	
51
	
32 0
Table F16. UNISYS : scores-max-tradeoff .tst2
F— 9
SLOT POS ACT COR PAR INC ICR IPA SPU MIS NON REC PRE OVG FA L
template-id 113 215 107 0 0 0 0 108 6 17 95 50 50
incident-date 109 103 56 21 26 0 21 0 6 4 61 64 0
incident-type 113 107 77 20 10 0 20 0 6 0 77 81 0 0
category 81 67 55 0 8 0 0 4 18 28 68 82 6 8
indiv-perps 95 54 27 4 10 3 4 13 54 43 30 54 24
org-perps 68 51 35 0 6 0 0 10 27 45 51 69 20
perp-confidence 68 51 20 3 18 0 3 10 27 45 32 42 20 4
phys-target-ids 54 30 14 3 5 4 3 8 32 74 29 52 27
phys-target-num 37 20 13 0 6 0 0 1 18 75 35 65 5
phys-target-types 54 30 15 3 4 5 3 8 32 74 30 55 27 1
human-target-ids 144 95 50 14 17 4 14 14 63 16 40 60 1 5
human-target-num 92 76 45 1 25 0 1 5 21 16 49 60 6
human-target-types 144 95 54 21 6 2 21 14 63 16 45 68 15 1
target-nationality 18 6 4 1 0 3 1 1 13 99 25 75 17 0
instrument-types 25 11 6 0 1 0 0 4 18 84 24 54 36 0
incident-location 113 107 56 37 14 0 0 0 6 0 66 70 0
phys-effects 36 18 12 2 2 3 2 2 20 89 36 72 11 0
human-effects 55 34 14 7 2 3 7 11 32 72 32 51 32 1
MATCHED ONLY 1361 1170 660 137 160 27 100 213 404 751 54 62 1 8
MATCHED/MISSING 1419 1170 660 137 160 27 100 213 462 797 51 62 1 8
ALL TEMPLATES 1419 1929 660 137 160 27 100 972 462 1926 51 38 50
SET FILLS ONLY 594 419 257 57 51 16 57 54 229 507 48 68 13 0
Table F17 . UMASS : scores-max-tradeoff .tst2
SLOT POS ACT COR
	
PAR
	
INC ICR
	
IPA SPU
	
MIS
	
NON REC
	
PRE
	
OVG FA L
template-id 109 137 93
	
0
	
0 0
	
0 44
	
16
	
22 85
	
68
	
32
incident-date 105 89 51
	
18
	
20 0
	
18 0
	
16
	
4 57
	
67
	
0
incident-type 109 93 67
	
12
	
14 0
	
12 0
	
16
	
0 67
	
78
	
0 0
category 77 55 44
	
0
	
7 0
	
0 4
	
26
	
28 57
	
80
	
7 8
indiv-perps 94 53 26
	
4
	
9 3
	
4 14
	
55
	
39 30
	
53
	
26
org-perps 63 39 26
	
0
	
5 0
	
0 8
	
32
	
47 41
	
67
	
20
perp-confidence 63 39 13
	
3
	
15 0
	
3 8
	
32
	
47 23
	
37
	
20 4
phys-target-ids 54 30 14
	
3
	
5 4
	
3 8
	
32
	
70 29
	
52
	
27
phys-target-num 37 20 13
	
0
	
6 0
	
0 1
	
18
	
71 35
	
65
	
5
phys-target-types 54 30 15
	
4
	
3 5
	
4 8
	
32
	
70 31
	
57
	
27 1
human-target-ids 139 87 48
	
12
	
13 4
	
12 14
	
66
	
17 39
	
62
	
1 6
human-target-num 88 67 40
	
1
	
22 0
	
1 4
	
25
	
17 46
	
60
	
6
human-target-types 139 87 52
	
14
	
7 2
	
14 14
	
66
	
17 42
	
68
	
16 1
target-nationality 16 6 4
	
1
	
0 3
	
1 1
	
11
	
96 28
	
75
	
17 0
instrument-types 25 11 6
	
0
	
1 0
	
0 4
	
18
	
80 24
	
54
	
36 0
incident-location 109 93 50
	
31
	
12 0
	
0 0
	
16
	
0 60
	
70
	
0
phys-effects 36 19 11
	
3
	
2 4
	
3 3
	
20
	
84 35
	
66
	
16 1
human-effects 55 30 13
	
5
	
2 3
	
5 10
	
35
	
69 28
	
52
	
33 1
MATCHED ONLY 1154 985 586
	
111
	
143 28
	
80 145
	
314
	
670 56
	
65
	
1 5
MATCHED/MISSING 1372 985 586
	
111
	
143 28
	
80 145
	
532
	
778 47
	
65
	
1 5
ALL TEMPLATES 1372 1333 586
	
111
	
143 28
	
80 493
	
532
	
1217 47
	
48
	
37
SET FILLS ONLY 574 370 225
	
42
	
51 17
	
42 52
	
256
	
491 43
	
66
	
14 0
Table F18. UMASS : scores-no-cbr .tst2 (optional test run )
F–1 0
SLOT POS ACT COR PAR INC ICR IPA SPU MIS NON REC
	
PRE
	
OVG FAL
template-id 108 143 62 0 0 0 0 81 46 18 57
	
43
	
57
incident-date 104 0 0 0 0 0 0 0 104 4 0
incident-type 108 62 53 9 0 0 9 0 46 0 53
	
93
	
0 0
category 75 25 13 0 4 0 0 8 58 25 17
	
52
	
32 9indiv-perps 96
0 0 0 0 0 0 0 96 45 0
	
♦
org-perps 63 76 13 0 9 0 0 54 41 34 21
	
17
	
71
perp-confidence 64 38 0 5 12 0 5 21 47 30 4
	
6
	
55 5
phys-target-ids 56 0 0 0 0 0 0 0 56 68 0
phys-target-num 39 0 0 0 0 0 0 0 39 69 0
phys-target-types 56 10 0 0 3 0 0 7 53 66 0
	
0
	
70 0human-target-ids 134
0 0 0 0 0 0 0 134 23 0
	
♦
	
.
human-target-num 85 0 0 0 0 0 0 0 85 23 ♦0
human-target-types 134 44 0 19 7 0 19 18 108 16 7
	
22
	
41 2target-nationality 18 0 0
0 0 0 0 0 18 95 0 0
instrument-types 23 7 0 2 0 0 0 5 21 82 4
	
14
	
71 0
incident-location 108 102 4 38 0 0 8 60 66 0 21
	
22
	
59phys-effects
38 18 0 7 5 0 7 6 26 76 9
	
19
	
33 2human-effects
54 19 1 3 7 1 3 8 43 66 5
	
13
	
42 1
MATCHED ONLY 837 544 146 83 47 51 268 561 398 22
	
34
	
49MATCHED/MISSING
1363 544 146 83 47 51 268 1087 740 14
	
34
	
49ALL TEMPLATES 1363
1200 146 83 47 51 924 1087 1665 14
	
16
	
77
SET FILLS ONLY 570 223 67 45 38 43 73 420 456 16
	
40
	
33 1
Table F19. UNL/USL : scores-max-tradeoff .tst 2
SLOT POS ACT COR PAR INC ICR IPA SPU MIS NON REC PRE OVG FAL
template-id 104 63 37 0 0 0 0 26 67 27 36 59 4 1
incident-date 100 0 0 0 0 0 0 0 100 4 0 * ♦
incident-type 104 37 35 2 0 0 2 0 67 0 35 97 0 0
category 71 0 0 0 0 0 0 0 71 33 0 * * 0
indiv-perps 95 0 0 0 0 0 0 0 95 42 0
org-perps 61 61 9 0 8 0 0 44 44 38 15 15 72
perp-confidence 60 6 0 0 1 0 0 5 59 46 0 0 83 1
phys-target-ids 54 0 0 0 0 0 0 0 54 65 0
phys-target-num 38 0 0 0 0 0 0 0 38 66 0
phys-target-types 54 0 0 0 0 0 0 0 54 65 0 * 0
human-target-ids 130 0 0 0 0 0 0 0 130 22 0 ♦
human-target-num 82 0 0 0 0 0 0 0 82 22 0 ♦
human-target-types 130 9 0 6 0 0 6 3 124 19 2 33 33 0
target-nationality 16 0 0 0 0 0 0 0 16 92 0 * 0
instrument-types 22 0 0 0 0 0 0 0 22 82 0 • 0
incident-location 104 77 4 30 0 0 5 43 70 0 18 25 56
phys-effects 37 6 0 5 0 0 5 1 32 78 7 42 17 0
human-effects 53 3 0 1 2 0 1 0 50 71 1 17 0 0
MATCHED ONLY 535 262 85 44 11 0 19 122 395 283 20 41 46
MATCHED/MISSING 1315 262 85 44 11 0 19 122 1175 772 8 41 46
ALL TEMPLATES 1315 402 85 44 11 0 19 262 1175 1128 8 27 65
SET FILLS ONLY 547 61 35 14 3 0 14 9 495 486 8 69 15 0
Table F20. UNL/USL : scores-option-l .tst2 (optional test run )
F–l l
SLOT POS ACT COR PAR INC ICR IPA SPU MIS NON REC PRE OVG FAL
template-id 107 108 56 0 0 0 0 52 51 3 52 52 48
incident-date 103 0 0 0 0 0 0 0 103 4 0 `
incident-type 107 56 50 6 0 0 6 0 51 0 50 95 0 0
category 78 54 30 0 8 0 0 16 40 13 38 56 30 1 8
indiv-perps 93 0 0 0 0 0 0 0 93 47 0 '
org-perps 63 101 16 0 8 0 0 77 39 27 25 16 76
perp-confidence 62 26 0 5 7 0 5 14 50 38 4 10 54 3
phys-target-ids 54 0 0 0 0 0 0 0 54 69 0
phys-target-num 37 0 0 0 0 0 0 0 37 70 0 '
phys-target-types 54 1 0 1 0 0 1 0 53 69 1 50 0 0
human-target-ids 137 0 0 0 0 0 0 0 137 21 0
human-target-num 86 0 0 0 0 0 0 0 86 21 0
human-target-types 137 23 0 20 2 0 20 1 115 20 7 43 4 0
target-nationality 16 0 0 0 0 0 0 0 16 95 0 0
instrument-types 24 0 0 0 0 0 0 0 24 83 0 0
incident-location 107 105 7 40 0 0 11 58 60 0 25 26 55
phys-effects 36 12 0 8 2 0 8 2 26 81 11 33 17 0
human-effects 54 7 0 4 1 0 4 2 49 71 4 28 28 0
MATCHED ONLY 749 493 159 84 28 0 55 222 478 364 27 41 45
MATCHED/MISSING 1355 493 159 84 28 0 55 222 1084 732 15 41 45
ALL TEMPLATES 1355 888 159 84 28 0 55 617 1084 1353 15 23 69
SET FILLS ONLY 568 179 80 44 20 0 44 35 424 470 18 57 20 0
Table F21 UNL/USL : scores-option-4 .tst2 (optional test run )
