Statistics
Statistics |
|
|
|
|
Please have a look on our tutorial page to learn more about how the following statistics where generated. There you should also find some hints
on how to include your own indexing technique into the framework and so how to create new statistics including your own indexing technique. Please inform us about new results in this area. |
|
In the first table, the first two columns contain the name of the problem and the system that generated the problem. The third column contains the number of symbols in the signature of the problem, for example '3+4' means that the signature contains 3 nonconstant function symbols plus 4 constants. In the following 4 columns we indicate the total number of operations (Total in the table), insertions in and deletions from the index (Ins and Del in the table), and the maximal size of the index in number of terms (Max in the table) during the experiment. In the last four columns we show the average size and depth of the indexed and query terms, respectively. Here by size we mean the number of symbols in the term, and the depth is measured so that the depth of a constant is 0. |
| We ran each indexing data structure on each benchmark, i.e., we did 90 experiments. The results are given in the second table. We measured time spent and memory used by each system. In parentheses we put the time measured for index maintenance only (i.e. insertions and deletions). This time was measured in a second run of the 90 experiments, on the same benchmarks, but with all the retrieval requests removed. |
| problem and generator | sig | operations | indexed terms | query terms | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Total | Ins | Del | Max | Size | Depth | Size | Depth | |||
| BOO015-4 | wal | 3+4 | 275038 | 228 | 228 | 179 | 13.8 | 5.8 | 5.1 | 2.2 |
| CAT001-4 | vam | 3+4 | 2203880 | 18271 | 3023 | 16938 | 26.0 | 9.9 | 7.1 | 2.9 |
| CAT002-3 | vam | 4+4 | 2209777 | 12934 | 4181 | 11191 | 29.2 | 10.4 | 7.1 | 3.1 |
| CAT003-4 | vam | 3+4 | 2151408 | 18159 | 4476 | 16606 | 28.3 | 10.6 | 7.1 | 2.9 |
| CIV003-1 | vam | 6+14 | 3095080 | 70324 | 22757 | 47567 | 14.1 | 5.1 | 4.3 | 1.6 |
| COL002-5 | fie | 2+7 | 940127 | 13399 | 5329 | 8353 | 25.4 | 9.3 | 7.5 | 2.8 |
| COL004-3 | fie | 2+5 | 1176507 | 765 | 28 | 737 | 18.1 | 6.7 | 9.3 | 3.2 |
| COL079-2 | vam | 3+2 | 2143156 | 14236 | 4619 | 9633 | 38.8 | 11.8 | 7.3 | 2.7 |
| GRP024-5 | wal | 3+4 | 2686810 | 506 | 506 | 296 | 16.3 | 6.6 | 7.7 | 2.9 |
| GRP164-1 | fie | 5+4 | 11069073 | 53934 | 3871 | 50063 | 16.4 | 6.1 | 5.9 | 2.4 |
| GRP179-2 | fie | 5+2 | 10770018 | 52825 | 2955 | 49870 | 16.8 | 6.1 | 6.1 | 2.4 |
| GRP187-1 | wal | 4+3 | 9999990 | 2714 | 1327 | 1387 | 12.0 | 5.3 | 5.0 | 2.0 |
| GRP196-1 | fie | 2+2 | 18977144 | 3 | 0 | 3 | 26.3 | 7.0 | 10.8 | 4.8 |
| HEN011-2 | vam | 1+10 | 4313282 | 4408 | 439 | 3969 | 10.7 | 4.1 | 2.7 | 0.8 |
| LAT002-1 | vam | 2+6 | 2646466 | 26095 | 1789 | 24306 | 17.5 | 6.2 | 5.6 | 2.0 |
| LAT009-1 | wal | 2+3 | 2514005 | 596 | 596 | 291 | 19.2 | 7.4 | 7.8 | 2.8 |
| LAT020-1 | wal | 2+3 | 9999992 | 910 | 493 | 417 | 14.8 | 5.6 | 9.3 | 3.2 |
| LAT023-1 | fie | 3+4 | 822539 | 4088 | 2065 | 2499 | 18.4 | 6.6 | 6.0 | 2.3 |
| LAT026-1 | fie | 3+4 | 772413 | 6162 | 3509 | 4770 | 20.8 | 7.1 | 5.6 | 2.0 |
| LCL109-2 | fie | 3+4 | 312992 | 4465 | 519 | 3947 | 16.7 | 6.1 | 6.0 | 2.6 |
| LCL109-2 | wal | 2+1 | 463493 | 196 | 196 | 165 | 19.2 | 7.9 | 5.7 | 2.3 |
| LCL109-4 | vam | 4+3 | 1944335 | 40949 | 3135 | 37817 | 20.0 | 5.7 | 8.1 | 2.7 |
| RNG020-6 | fie | 6+6 | 2107343 | 4872 | 960 | 3912 | 17.4 | 5.4 | 6.3 | 2.4 |
| RNG028-5 | wal | 5+4 | 3221510 | 304 | 304 | 218 | 31.4 | 9.0 | 14.0 | 3.8 |
| RNG034-1 | vam | 4+4 | 2465088 | 15068 | 4589 | 11685 | 26.4 | 7.0 | 6.1 | 2.4 |
| RNG035-7 | wal | 3+5 | 5108975 | 482 | 482 | 360 | 21.7 | 9.0 | 13.9 | 4.7 |
| ROB006-2 | wal | 3+4 | 9999990 | 1182 | 34 | 1148 | 19.1 | 7.7 | 12.4 | 4.9 |
| ROB022-1 | fie | 3+4 | 922806 | 2166 | 826 | 1341 | 21.4 | 9.3 | 6.5 | 2.8 |
| ROB026-1 | wal | 2+4 | 9999991 | 648 | 15 | 633 | 16.9 | 7.9 | 12.6 | 5.0 |
| SET015-4 | vam | 4+6 | 3664777 | 3256 | 995 | 2261 | 16.3 | 5.8 | 3.6 | 1.4 |
| time(in seconds) | memory (in Kbytes) | ||||||
|---|---|---|---|---|---|---|---|
| problem | from | Vam | Wal | Con | Vam | Wal | Con |
| BOO015-4 | wal | 0.25 (0.00) | 0.31 (0.01) | 0.46 (0.01) | 11 | 575 | 11 |
| CAT001-4 | vam | 3.28 (0.34) | 5.74 (0.31) | 7.11 (0.41) | 3859 | 13786 | 3109 |
| CAT002-3 | vam | 2.90 (0.22) | 5.51 (0.31) | 6.67 (0.33) | 2483 | 9281 | 2021 |
| CAT003-4 | vam | 3.21 (0.34) | 5.82 (0.34) | 6.90 (0.46) | 3826 | 13595 | 3086 |
| CIV003-1 | vam | 7.57 (0.93) | 7.13 (0.65) | 15.57 (1.16) | 3754 | 22664 | 3081 |
| COL002-5 | fie | 1.30 (0.17) | 1.55 (0.21) | 2.61 (0.28) | 925 | 6090 | 922 |
| COL004-3 | fie | 0.96 (0.00) | 1.22 (0.00) | 2.39 (0.02) | 80 | 727 | 86 |
| COL079-2 | vam | 5.46 (0.29) | 8.41 (0.26) | 7.24 (0.43) | 2769 | 9158 | 2138 |
| GRP024-5 | wal | 3.54 (0.01) | 4.82 (0.00) | 7.44 (0.01) | 19 | 591 | 22 |
| GRP164-1 | fie | 17.60 (0.72) | 24.60 (0.61) | 32.06 (0.89) | 5823 | 28682 | 5352 |
| GRP179-2 | fie | 18.34 (0.71) | 24.25 (0.60) | 32.40 (0.87) | 5597 | 29181 | 5207 |
| GRP187-1 | wal | 10.44 (0.02) | 11.68 (0.03) | 17.64 (0.02) | 96 | 903 | 97 |
| GRP196-1 | fie | 6.96 (0.00) | 11.92 (0.00) | 15.45 (0.00) | 1 | 543 | 1 |
| HEN011-2 | vam | 3.36 (0.03) | 3.39 (0.02) | 5.18 (0.04) | 221 | 2069 | 211 |
| LAT002-1 | vam | 5.83 (0.32) | 7.72 (0.29) | 9.48 (0.44) | 3164 | 14603 | 2554 |
| LAT009-1 | wal | 3.78 (0.01) | 4.97 (0.01) | 5.97 (0.01) | 19 | 591 | 20 |
| LAT020-1 | wal | 17.73 (0.01) | 24.97 (0.01) | 29.87 (0.01) | 30 | 631 | 31 |
| LAT023-1 | fie | 1.10 (0.04) | 1.49 (0.03) | 1.92 (0.07) | 198 | 1646 | 210 |
| LAT026-1 | fie | 1.11 (0.09) | 1.49 (0.07) | 1.79 (0.12) | 373 | 2813 | 371 |
| LCL109-2 | fie | 0.47 (0.04) | 0.65 (0.06) | 0.80 (0.05) | 508 | 2285 | 466 |
| LCL109-2 | wal | 0.49 (0.00) | 0.66 (0.00) | 0.82 (0.00) | 16 | 591 | 15 |
| LCL109-4 | vam | 5.62 (0.70) | 7.65 (0.46) | 13.02 (0.72) | 6703 | 24403 | 4986 |
| RNG020-6 | fie | 2.25 (0.07) | 3.19 (0.05) | 5.33 (0.08) | 544 | 2435 | 517 |
| RNG028-5 | wal | 4.19 (0.01) | 6.66 (0.01) | 9.08 (0.01) | 28 | 607 | 29 |
| RNG034-1 | vam | 3.27 (0.33) | 4.95 (0.21) | 6.86 (0.34) | 2545 | 8330 | 2125 |
| RNG035-7 | wal | 8.19 (0.01) | 12.10 (0.01) | 18.55 (0.01) | 36 | 647 | 37 |
| ROB006-2 | wal | 9.88 (0.01) | 14.31 (0.02) | 21.60 (0.02) | 128 | 1142 | 116 |
| ROB022-1 | fie | 0.92 (0.03) | 1.20 (0.03) | 2.23 (0.03) | 119 | 1086 | 101 |
| ROB026-1 | wal | 8.52 (0.01) | 13.35 (0.01) | 17.34 (0.01) | 69 | 807 | 68 |
| SET015-4 | vam | 2.54 (0.02) | 2.69 (0.02) | 4.53 (0.05) | 314 | 1373 | 258 |
| total | total | ||||||
| 161.09 (5.48) | 224.39 (4.64) | 308.31 (6.90) | 44258 | 201835 | 37248 | ||