jDip 1.5 Performance

All tests were performed on a Pentium 4 (3.0 Ghz, 800 Mhz FSB, 1024 MB RAM, HT off) running Windows XP. Sun Java JDK 1.4.2_03 was used for all tests, without using any specific command-line options (unless otherwise indicated). Times are rounded to the nearest tenth of a second.

Real

Iterations	Total Orders	Time (s)	Orders per Second
10	1,010	0.13	8,080
100	10,100	0.36	28,133
1,000	101,000	1.2	82,854
25,000	2,525,000	21.0	120,329
50,000	5,050,000	41.5	121,686
100,000	10,100,000	82.8	122,054
"Real" [using Java 1.4.2 server VM]
Iterations	Total Orders	Time (s)	Orders per Second
25,000	2,525,000	17.8	142,005
50,000	5,050,000	29.2	173,111
100,000	10,100,000	56.0	180,055
150,000	15,150,000	81.7	185,391

DipAI

Iterations	Total Orders	Time (s)	Orders per Second
10	1,100	0.125	8,800
100	11,000	0.39	28,205
1,000	110,000	1.3	86,956
25,000	2,750,000	20.0	137,610
50,000	5,500,000	39.3	139,902
"DipAI" [using Java 1.4.2 server VM]
Iterations	Total Orders	Time (s)	Orders per Second
25,000	2,750,000	18.0	153,169
50,000	5,500,000	31.1	176,797
100,000	11,000,000	56.7	193,887

DATC

Iterations	Total Orders	Time (s)	Orders per Second
10	6,050	0.66	9,222
100	60,500	2.1	29,340
1,000	605,000	15.8	38,298
2,000	1,210,000	30.0	40,395
"DATC" [using Java 1.4.2 server VM]
Iterations	Total Orders	Time (s)	Orders per Second
1,000	605,000	21.4	28,262
2,000	1,210,000	31.6	38,279
4,000	2,420,000	52.4	46,149

Case Types

DATC: 162 cases, 605 orders, from section 6 of the January 23, 2004 DATC. A few additional test cases are included. These will adjudicate the most slowly because these cases contain difficult and edge cases with, for example, invalid orders, paradoxical situations, and NMR during adjustment phases.

DipAI: 9 cases, 110 orders total. A test set of orders from the DipAI interest group. Covers all phases (build, retreat, adjustment).

Real: 3 cases, 101 orders total. Three separate movement-only phases taken from the Describe game; game history is available here. Approximately 20 units are moving in each phase.

Case Evaluation

Cases are evaluated in sequence, and then repeated for each iteration. Thus for 4 iterations of 3 cases (A, B, C), the cases will be evaluated as such: ABC, ABC, ABC, ABC.

Included in the source distribution (downloadable in .zip or available through CVS) are all of the above cases, additional test cases, and the performance and regression testing tool (dip.misc.TestSuite).

The above findings show that the Java Server VM (java -server) can give a significant performance boost when evaluating very large numbers of orders. Furthermore, the adjudicator performs better with more iterations (due to Java Hotspot optimizations, which can be changed to allow higher performance with fewer iterations as well).

The adjudicator code path is most optimized for the movement phase. Situations that decrease performance are:

Successful circular movement, including swaps. Any failed circular movements or swaps will evaluate quickly.
Paradoxes (see DATC for more information)
NMR during the Adjustment phase when there are units to be removed (this is particularly slow)