Aquarium Engine Matches and Tournaments, Part One 06 October 2011

Engine matches and tournaments got a very noticeable update in Aquarium 4.0.6. The “Match properties” and “Engine properties” dialog boxes were redesigned to increase consistency in naming and options. Several useful options were also added: ● Unified and improved dialogs for creating matches and tournaments ● Openings can be played from a PGN file ● Prevent unloading of engines from memory between games ● Display game count and duration estimate for a new match/tournament ● Tournament/match settings can be saved and play started later ● Engines can be added to an ongoing tournament ● The board can be automatically inverted to follow the stronger side ● Elo difference is displayed in engine matches ● The time used for individual moves can be displayed in the notation ● Number of endgame tablebase hits can be displayed in the notation ● The games are now stored in CDP (Chess Assistant) database format ● Asymmetric time controls in engine matches are tied to engine instead of color

This month I describe engine competitions in Aquarium in general and cover engine matches in detail. Next month I will describe how to create and run different types of engine tournaments.

Then Engine Competitions View

One of the advantages of Aquarium for new users is that you can eliminate all features of the program from view, except for those which are needed for the task that you want to perform. This applies to engine competitions. After clicking the Engine Competitions button in the sidebar, the Aquarium screen looks as follows.

The screen looks like it comes from a small program dedicated to playing engine competitions. Here is what you see in this image: 1. The Ribbon is at the top of the image. It shows commands that are available for engine competitions, such as creating a new match. Note that commands that can’t be used here are not displayed, so you don’t need to fumble around in dozens of unrelated menu items and buttons to find what you are looking for. 2. The sidebar (or Navigation Pane) is on the left side of the window. At the top of the sidebar there are links to the views that you have opened. Each view is shown on a separate line. As an example, you can open a view for observing the current game. Below that, there is a list of custom actions related to engine competitions. Finally, the big blue buttons at the bottom are used to switch to a different mode. For instance, you may switch to Sandbox if you want to view a game while an engine match is in progress.

Engine Competitions view

3. The Events list window shows a list of all events (matches and tournaments) that are still in progress. When an event finishes, it is moved to the Finished event list, which can be displayed by clicking the tab at the bottom of the Events list window.

4. When you select an event in the Events list window, the Tournament table window displays the table for the event. If you select a running tournament (shown with a green background in the Event list window), the tournament table is updated after every game, so you can get a quick overview of the current standings. 5. The Current game window displays the game that is in progress along with a diagram showing the current position. If you want more details about what is going on in the game, there is an option to open a new view showing just the game with the click of a button. The windows marked three through five are displayed in the working area of Aquarium and can be rearranged any way you like.

The Event List Window

One of the unique features of Aquarium is that it keeps a list of your current and finished engine matches and tournaments. This means that you can quickly look up the results and games of any event.

	The following columns are displayed in the Event list window: ID is a unique identifier that Aquarium assigns automatically to each event. Name is a name that you give to the event when it’s created. The name is always prefixed by the ID of the event in square brackets.
The Event list window

Type shows the type of contest. “Match” is a normal two-engine match or a Rybka Randomizer (Monte Carlo) match. “Round” indicates a round-robin tournament and “Swiss” or “Gauntlet” are displayed for the corresponding type of tournament. The number of cycles played in “Round,” “Swiss,” and “Gauntlet” tournaments is shown with a number in front of the tournament type. For example, a double round-robin tournament is displayed as “2/Round.” Time shows the time control of the event. As an example, “1min/game” means that each engine will get sixty seconds to play the entire game. Games shows how many games have been played in the tournament and the total number of games. As an example, an eight player round-robin tournament consists of twenty-eight games. If twenty games have been completed, the “Games” column will display “20/28.” You have the option to define matches with an infinite number of games, which means that the match will be continued until you stop it manually. In this case the total number of games is displayed as zero. As an example, after twenty games the “Games” column would show “20/0.” Standings is only displayed for matches. It shows the number of wins, draws and losses of the first engine. As an example, “+2=3-1” means that the first engine has won two games, drawn three and lost one. Started shows the date and time when the event started. Status can be either “Running” or “Stopped.” An event can be paused whenever needed and continued at a later time. If you right click an event, the following menu will be displayed.

	View all games opens the engine_games database (where all engine games are stored) and displays the games from the selected event. You can view the games even while the event is being played. In that case all the games preceding the current one will be displayed. Build statistics tree creates a statistics tree from the games of the selectedevent.
The Event list right-click menu

The Finished event list shows a list of completed events.

	You can switch between the two lists by selecting the corresponding tab at the bottom of the list window as shown in this image. The Finished event list works exactly like the Event list. Their only difference is the completion status of the events.
The Finished event list

There are two buttons in the Ribbon for managing the event lists:

Finish marks an event in the Event list as finished. Aquarium will move it to the Finished event list. Events are also moved automatically to the Finished event list when the last game is completed. This applies, for instance, to matches with a limited number of games, round-robin tournaments, etc. Remove deletes the event permanently from the list. You can use this button if you want to remove old or unimportant events.

Finishing or removing events

The Tournament Table Window

The Tournament table window displays the tournament that is currently selected in the event list: Right-clicking a result in the table displays the following menu items (see also the screenshot to the left): Load. Selecting this item displays the corresponding game. View all games displays a list of all finished tournament games. It works exactly like View all games in the Event list window. The “Sort by” drop-down list allows you to display the engines in alphabetical order or ordered by current standings.

The Current Game Window

The Current game window displays the game that is currently being played in the tournament. The tournament name is displayed at the top of the window. Below that there is a white square, indicating the engine playing the white pieces. In this example, Rybka 3 w32 has the white pieces and it has ten seconds (0:00:10) left on its clock. The black square shows that Brutus is playing the black pieces and has five seconds left. The red-colored type indicates that Brutus’ clock is running. The diagram shows the current position in the game and at the bottom of the window the moves of the game are displayed.

The Current Game Window

Creating a New Engine Match

There are different ways to start an engine match. You can, for instance, start a match from the engine list. Click the Engines button in the sidebar to switch to the engines list.

Here is what you are seeing in this screen-shot. First I selected Stockfish and then while holding down the CTRL key, I selected Rybka 4 w32. This highlights both engines. If I want to start a match between them, I click Start Play in the Ribbon as shown. This will bring up the Match properties dialog box, which is described below.Note that I can use this method to mark as many engines as I like. If more than two engines are selected, Start Play displays Tournament properties instead of Match properties.

Starting an event from the engine list

A second method to start a match is by clicking the Aquarium Button in the upper-left corner of the application window.

Starting an event from the Aquarium Menu

The Aquarium Menu will be displayed. Select Engines and then “Start engine match” from the menu. The Match properties will be displayed. As the menu shows, you can also use this method to start an engine tournament.

The third and most common method is to start an engine match from the Engine Competitions view. If you haven’t already done so, click the Engine Competitions button in the sidebar.

	The Create New button in the Home-Competitions tab allows you to create a new event. Select Match to create an engine match. Next month we’ll look at the two other menu options, Tournament and Rybka Randomizer match.
Starting a match from the Ribbon

Selecting Match from the menu displays the Match properties dialog box shown below.

Here you can select the parameters for the match, such as the engines, hash size, opening book, time control, etc. Related options are grouped together and we’ll have a look at each of them below. Even though I’m focusing on engine matches here, almost the same parameter settings apply to both engine tournaments and Randomizer matches. When you start a new match, the settings you choose become the default for the next match that you create. This means that if you are running a series of matches under similar conditions you may only need to change a couple of settings.

Match properties

Event. The first thing you need to decide is the name of the event. You can either type the name directly into the Event field or click the Edit name button in the upper-right corner for more advanced options.

Clicking the Edit name button displays the Match name dialog box. Instead of using a fixed text for the name of the event, you can use patterns that are replaced with actual values by Aquarium. The patterns are displayed in two columns in the Available Patterns pane. The first column contains patterns related to the starting date of the event: ● $dd – Day of month (two digits; zero padded if less than ten). ● $d – Same as $dd, except there is no zero padding (e.g. “1” instead of “01”). ● $mm – Month, displayed as two digits (e.g. “01”). ● $m – Same as $mm, but without zero padding. ● $yy – The last two digits of the year. ● $yyyy – Year (four digits).

Patterns can be used in event names

The second column allows you to specify patterns that will be replaced with the engine names: ● $E1 – Primary engine. This is the first engine specified (directly below the event name). ● $E2 – Secondary engine. This is the second engine (specified below the first one). Clicking the Default button generates the pattern shown to the right of the button. It shows the engine names and the date when the match was started.

Engine Settings.

There are two identical Engine Settings panes, one for each engine playing the match. The left side has drop-down lists for selecting an engine and opening book. You can select any installed engine from the engine list. Here I have selected Rybka 4.1 w32. I have also selected Jiri’s Rybka 4 book as an opening book for this engine. You can select different opening books for the engines if you like. Besides its native books, Aquarium also supports the CTG opening book format. The right-hand side of the Engine Settings contains five different parameters. Hash Size determines how much memory the engine uses for its hash tables in the match. You must make sure that the hash tables of both engines fit into available memory at the same time. It is recommended setting the hash size to about one-quarter of the memory size (RAM) of the computer. That way the total memory usage will be less than half the available memory. Note that Rybka allocates memory in powers of two, so you should set its hash size accordingly (2MB, 4MB, 8MB, 16MB, 32MB etc.). If you set the hash size to zero, the current hash size as specified in the Engines view will be used. Max book moves can be used to limit the number of moves played from the opening book. Many opening books are very deep, sometimes reaching into the endgame. If you prefer to let the engines play a greater part of the game on their own, you can set this parameter to a low value. As an example, setting it to eight will ensure that no more than the first eight moves of each game will be played from the opening book. Note that some lines in the book can be shorter than eight moves and in such cases the engines will be switched on earlier. Ponder. If this option is selected, then the engine will think on its opponent’s time. You must be careful if you use this option. Let’s say that you are running the match on a dual core computer. If both engines use two cores with ponder on, then they start competing for the two cores and the results will be unpredictable. In order to run such a match successfully, you would need a quad computer (four cores). You should not let the engines ponder unless the total number of cores required by the engines is less than the number of cores in your computer. No auto-unloading. Selecting this option loads the engine at the start of the match and keeps it loaded until the end of the match. No flag loss means that you don’t want the engines to be flagged when they run out of time. Instead the game will continue. Match Settings. These settings determine which color each side plays and the number of games. Primary engine plays. The first engine (specified below the Event field) will get the color chosen in this option. Number of games is the maximum number of games that will be played in the match. If you, for instance, set it to 100 then 100 games will be played unless you decide to end the match early. If you set this parameter to zero, the match will continue until you stop it manually. Of course, you can pause a match at any time and continue later. Switch sides. If you select this option, the engines will switch sides after every game. In the screen-shot above, “Primary engine plays” is set to white, so the primary engine plays the white pieces in the first game. If you don’t select Switch sides, then it will continue to have white in all the games. Otherwise it would have black in the second game, etc. Note that “Play each opening twice” (described below) will always force the engines to switch sides. Start Position(s). Aquarium offers great flexibility in specifying the starting positions and variations to be played in engine games. You can use opening books in either Aquarium format or CTG format as was mentioned above. The Start Position (s) pane offers additional methods of deciding the starting moves or starting positions of engine games. Initial. This is the most commonly selected option. All the games will start from the normal starting position in chess. FEN String. The games will start from the same position that is specified by the FEN string. You can paste the FEN (or EPD) string directly into the text box. You can also click the button on the right-hand side to bring up the “Setup position” dialog where you can set up the position on a chessboard. EPD File. The games will start from positions stored in EPD format in a file. Click the ellipses button to choose a file. The first game starts from the first EPD record and Aquarium goes sequentially through the positions in the file until one game has been played from each position. If the number of games is greater than the number of positions in the file, then you will get more than one game starting from each position. Note, however, that if you select “Play each opening twice” (described below) each starting position will be played twice (with switched colors), so you will need twice the number of games to go through all the positions. PGN File. The games will start from game fragments stored in a PGN file. Typically, these fragments would contain the first few moves of the games. The only difference between using an EPD file and a PGN file is that in the latter case you can get complete games, played from the normal starting position. Note that these options are not limited to specifying typical opening positions. You can, for instance, start games from a middlegame position or an endgame position. Such engine matches and tournaments are sometimes of interest to correspondence chess players. All these Start Position(s) settings can be combined with the use of an opening book, if you like. Therefore, you must remember to switch the opening books off if you want the engines to play on their own from the given positions. Play each opening twice and engine ratings. The following options can be found in the lower left corner of the Match properties dialog box. Play each opening twice. If this option is selected, each opening will be played twice. The engine that has the white pieces in the first game will play the black side in the second game. Change engines’ rating. Select this option if you want Aquarium to update the ratings of the engines. Below these two options, there is a section that shows some statistics about the event. The number of games is shown (if a limited number of games is specified), the number of position (if you start from a FEN string, an EPD file or a PGN file) and the estimated time required to play the match. If you click the small green arrows in the lower right corner, the statistics will be updated. Adjudge. Sometimes the result of an engine game becomes clear long before it is finished, but UCI chess engines have no way of resigning or agreeing to a draw. All games are played until they end by the rules of chess; e.g., with a checkmate. Sometimes that’s exactly what you want, but at other times it can become tiresome, especially at long time controls. Here you can tell Aquarium to adjudicate games as drawn or won depending on the following two conditions. Adjudicate as draw after ___ moves. If neither side has managed to win after the specified number of moves, Aquarium will stop the game and record it as a draw. Victory if score is ___ or more centipawns. If the evaluation of both engines is above this limit for three consecutive moves, Aquarium stops the game and awards the win to the side with the advantage. Note that the evaluation is specified in centipawns. If you want the games adjudicated when the evaluation goes above six pawns (or below minus six pawns), you would enter 600 here. Time Control. Aquarium offers a wide variety of time controls for engine games: Blitz, classical tournament time controls, time/move and fixed depth. On top of that you can choose asymmetrical time controls, thereby handicapping one of the engines. Of course you can also set up material handicaps by using the Start Position(s) options described above. The Time control panel is shown in the image above. The Blitz tab is currently visible, but there are three more tabs for other types of time controls: Tournament, Time/move and Fixed Depth. Looking at the Blitz tab you see a drop-down list, Predefined controls, at the top of the panel. This list contains the most frequently used blitz settings for one to five minute games (without an increment). You can either choose one of these options or adjust the settings manually. The White panel sets the time control for the first engine. In blitz chess, the game must be completed within a certain time. You can set the time in the “minutes” box. In this example it is set to one minute. The “seconds increment” is set to zero, so we are talking about bullet games. If you set the “seconds increment” to a positive value, the increment will be added to the engine’s remaining time after each move it makes. Normally, you would select “Same time for both players,” which assigns the same time control to both sides. However, there are situations where you may want to choose different time controls, in which case you would clear the check box. This opens up the Black panel and allows you to enter different time settings there. When running engine matches with asymmetric time controls, the White time control is tied to the primary engine (the first engine specified in the Match properties window) and the Black time control is tied to the second engine. This means that even if the engines switch colors after every game, they will always use the same time control as they had in the first game. Note that this is different from engine tournaments where the time controls are tied to the color; i.e., the time control specified in the White panel is always used for the engine playing white and the time control in the Black panel is used for the engine having the black pieces. The Tournament tab offers a very flexible method of specifying time controls. The time control can have up to eight stages. Each stage has the same structure and you can change the parameters by clicking the underlined links. The example in the screen-shot shows the FIDE time control consisting of two stages. In the first stage the players get ninety minutes (shown as “1 hrs 30 mins” in the example) to complete the first forty moves (“for first 40 moves”). There is no increment (“+ 0 mins /move”). After that (“then”) they get thirty minutes (“0 hrs 30 mins”) to complete the game (“for game”) with a thirty second increment after each move (“+ 30 secs / move”). If you click one of the underlined numbers, you can modify them (e.g., change “1 hrs” to “2 hrs”). The menus for the other underlined items are shown in the next graphic. When you click the links, the menus with the white background are displayed and you can choose one of the options. If you select “for game” instead of “for first,” this will become the final time control stage. Note that in the following lines the “for first” menu item is replaced with “for next.” The increment can either be in minutes (“mins”) or seconds (“secs”). If the final stage of the time control is repeated until the game ends, you should select “repeat” instead of “then,” which indicates that another stage will follow and a new stage is automatically added. Note that this option disappears if you select “for game,” as it already indicates that this will be the final time control stage. The Time/move tab looks similar to the Blitz tab, but there are important differences. As before, you can choose from various “Predefined controls” or define your own time control. In this example the time control is “0 minutes 5 seconds per move,” which is displayed as 0′ + 5” in the Predefined controls. Note that unlike the Blitz tab there is no “+” indicating an increment here. This simply means that the engine gets five seconds to come up with a move. Unused time is carried forward to the next move. Therefore, you should not be surprised to see the engine have fifty seconds to think about its tenth move if the first nine moves were played from an opening book. When using this type of time control it’s important to select “No flag loss” (described above in the section about Engines Settings). The reason is that chess engines often overstep such a time limit by a few milliseconds, causing a loss on time. Fixed depth is a simple time control, but it is not actually time limited. Therefore, the clocks will start from zero and count up instead of down. The only parameter you need to set is Depth. The higher it is, the deeper the engine will analyze the position before making a move. Of course, higher depth also means that the engines will take longer to make their moves. If you clear “Same for both players,” you can set different depths for the two engines. In that case the depths are tied to the engine (and not the color), just like in blitz and time/move games. After you have set the parameters for the match, you have two options. You can either start the match immediately or store the settings and start the match later. Clicking OK starts the first game immediately, but Save Only stores the match and you can start the games at a later time. In both cases the match will be added to the Event list window described above. When a match is running you can follow the current game by clicking the Observe button in the Ribbon. This brings up the view shown below. We’ll have a closer look at this screen next month as well as the options that are available while engine games are in progress.

Conclusion

We have seen that Aquarium has great support for engine matches and the settings you need are available in a single dialog box, the Match properties. Next month we’ll discuss round-robin, Swiss, and gauntlet tournaments, as well as Randomizer matches and how to follow and manage engine competitions in progress. Due to the improvements in Aquarium 4.0.6, where the dialogs for engine competitions were unified, you will see that most of what you have learned here also applies to tournaments and Randomizer matches. Dadi Jonsson.