Article Last Updated: August 15, 2015
Komodo 9 vs Stockfish 6 Chess Engine Tournament (32-bit single-core CPU benchmark)
Six months ago we published an article about top chess engines tournament in 32-bit single-core single-CPU arena on Windows XP setup. Follow-up of this article is our latest 64-bit chess engine tournament.
Classic Chess Board
Today, we will publish the new results of the same benchmark with new updated versions of top chess engines: komodo-9-32bit.exe and stockfish-6-32.exe, respectively. We will compare these two engines with the winner of the last tournament – Stockfish 5 (version stockfish_14053109_32bit.exe).
Again, same conditions for the match/tournament is selected (in fact, it was performed on the very same laptop computer), with one exception that this time hashtable memory allocation size was raised to 512 MB for each competing engine (that should not affect final results in any way).
ABOUT CONTESTANTS: UPDATED CHESS ENGINES
Stockfish 6 received some considerable updates, many additions and changes according to the authors and contributing team, but the same can be said for Komodo 9 versus Komodo 5, that we tested last time. So, will Komodo 9 now outperform free Stockfish 6 engine, or will Stockfish win again? Let us found out!
FIXED TIME RESULTS (3 seconds/move)
Download results table as image file (PNG format)
| Rank | Engine | Score | StockFish 6 | StockFish 5 | Komodo 9 | SB |
|---|---|---|---|---|---|---|
| 1 | Stockfish 6 Stockfish-6-32.exe |
38,0/60 | ===1===1==10=1= 0=1===1=1====1= |
1011101===11=0= 1=1=01=1111=1=0 |
986 | |
| 2 | Stockfish 5 Stockfish_14053109_32bit.exe |
27,0/60 | ===0===0==01=0= 1=0===0=0====0= |
==0=01=1000111= 01=100=11==010= |
831 | |
| 3 | Komodo 9 Komodo-9-32bit.exe |
25,0/60 | 0100010===00=1= 0=0=10=0000=0=1 |
==1=10=0111000= 10=011=00==101= |
785 |
* SB = Sonneborn Berger score
Legend:
0 … loss (0 points)
= … draw (0,5 points)
1 … win (1 point)
Total Games Played: 90 (30 games between each engine / total of 60 games per engine)
Maximum Theoretical Score: 60 points (60 wins)
Level: 3 seconds per move (fixed time)
Books: None
Ponder: OFF
Hash Table Size: 512 MB (per engine)
Komodo 9 Table Memory Size: 64 MB (factory default)
Hardware: Intel(R) Celeron(R) M CPU 430 @ 1.73GHz with 1,9 GB Memory
Operating System: Microsoft Windows XP Professional Service Pack 3 (Build 2600) 32 bit
TOURNAMENT TIME RESULTS (40 moves/2 minutes)
Download results table as image file (PNG format)
| Rank | Engine | Score | StockFish 6 | StockFish 5 | Komodo 9 | SB |
|---|---|---|---|---|---|---|
| 1 | Stockfish 6 Stockfish-6-32.exe |
44,0/60 | =11=101===1==== 1101==01==0==== |
10111111111=111 0111111111=1=11 |
949 | |
| 2 | Stockfish 5 Stockfish_14053109_32bit.exe |
30,0/60 | =00=010===0==== 0010==10==1==== |
=1=1110=1==1=== 1=0=11001=====0 |
830 | |
| 3 | Komodo 9 Komodo-9-32bit.exe |
16,0/60 | 01000000000=000 1000000000=0=00 |
=0=0001=0==0=== 0=1=00110=====1 |
529 |
* SB = Sonneborn Berger score
Legend:
0 … loss (0 points)
= … draw (0,5 points)
1 … win (1 point)
Total Games Played: 90 (30 games between each engine / total of 60 games per engine)
Maximum Theoretical Score: 60 points (60 wins)
Level: 40 moves in 2 minutes (tournament clock style)
Books: None
Ponder: OFF
Hash Table Size: 512 MB (per engine)
Komodo 9 Table Memory Size: 64 MB (factory default)
Hardware: Intel(R) Celeron(R) M CPU 430 @ 1.73GHz with 1,9 GB Memory
Operating System: Microsoft Windows XP Professional Service Pack 3 (Build 2600) 32 bit
CONCLUSION
Stockfish again completely destroyed Komodo (newest 32 bit editions). We did not expect this kind of results, really. All right, something is definitely “fishy” around here, we are not sure if Stockfish has some better Intel processor instruction set optimizations, but something obviously gives it a real edge on this system/machine/setup. More over, to prove that this was not some random luck, older Stockfish 5 engine version also beat Komodo 9 by a small, but noticeable margin.
However, not so fast! If you look carefully at the outcome of the 30 games played between Stockfish 5 and Komodo 9 in fixed time per move mode, you will see that they have scored evenly against each other (15 points)! The only reason why Stockfish 5 has 2 points more than Komodo in this tournament is because it gained few victories over Stockfish 6. So, we can safely assume that Komodo 9 is on the same level as older Stockfish 5 in this case, which is a progress that we were talking about in the article introduction.
On the other hand, in tournament clock version (40 moves in 2 minutes), Komodo 9 lost against both Stockfish editions. Obviously, some tuning with variable clock mode could be missing.
Stockfish 6 is considerably stronger than Stockfish 5 chess engine, which this tournament clearly shows. Stockfish 6 gained ~ 30% higher score than Stockfish 5 in fixed time mode, and even more in tournament mode. Quite an achievement!
And, as a final reminder, please take into consideration that this test was done on a 32-bit single core / single processor machine. Multi-processor / multi-core processing capabilities of chess engines architecture and sub-routines were not taken into account.
DOWNLOAD
To stand to our claims and increase the credibility of the article, we provide a PGN files of the entire tournament for download (files can be opened with Notepad or similar text editor, or imported in chess clients): StockFish 6 vs Komodo 9 Chess Engine Tournament x32 (click to download zip archive)
Comments
Post A Comment