Interactive Deep Analysis (IDeA) With Rybka Aquarium 2010 17 November 2009
One of the biggest changes in Rybka Aquarium 2010 is the greatly improved Interactive Deep Analysis (IDeA). Many players were happy with the possibilities offered by IDeA in Rybka 3 Aquarium, but I’m afraid that it will look like a dinosaur compared to the new version. It provides better automatic analysis and much more effective ways to control the analysis process and the shape of the analysis tree. I’m sure that many serious players will find that the new and improved IDeA is an essential tool to stay competitive. Analyzing with many engines. The original IDeA could only use one engine for the analysis, but the new version can split the analysis between many engines. You can, for instance, use four instances of single core Rybka for the analysis on a quad computer, essentially reducing the multiprocessor overhead to zero. It’s like getting one extra core for free compared to running Rybka in 4-CPU mode. Analyzing on a network of computers. This feature is a dream come true for power users, analysts, and serious players who want to take their analysis to a new level. If you own two computers, you can of course run Aquarium on one of them and use it for the analysis. In the new version you can also add engines running on your other computer to the pool of IDeA analysis engines. Using this method I have tested up to thirty-five engines analyzing simultaneously. The engines were running on various computers, some on my local network while others were located in different countries and connected to my Aquarium over the Internet. IDeA takes care of splitting the analysis between the available engines. Multiple IDeA projects analyzed in parallel. You can define many analysis projects, each with its own analysis and tree settings. IDeA can analyze any number of those projects in parallel. The status of each project is saved between sessions, so you can manage many different IDeA projects much easier than before. Multiple analysis “roots” for a single project. The starting position for IDeA analysis is called a “root.” If you are analyzing a position in IDeA when another interesting position shows up in the tree, you can make it a new root, without even stopping the analysis. You can remove the original root or let IDeA analyze from both roots. You can use this feature to focus IDeA on selected positions without restarting it. IDeA can keep a list of all roots that have been used in a project. Shaping the analysis tree. The former single “variativity” parameter was not very effective and didn’t affect IDeA sufficiently. Now there are a number of new settings that have a significant influence on the shape of the IDeA analysis tree.

• The “Tree width” parameter determines how wide the resulting IDeA tree will be. Although similar to “variativity,” this new parameter is much more effective.
• The maximum length of variations can be defined. This allows you to make the tree denser near the root and avoid very long variations.
• Analysis can be limited to positions within a certain evaluation range. Positions outside the specified range will not be explored further.
• The number of alternatives generated for White/Black can be limited.
• A “preferred side” can be set; e.g., for developing your opening repertoire.
• A list of “thematic moves” can be given. You can use them to ensure that certain moves are always analyzed. For instance, in an attacking game you might want to make sure that the moves Bxh7+ and Ng5 are considered.
• “False alarm check” immediately performs additional analysis of new moves that look “too good to be true,” preventing IDeA from spending more time on them if they turn out to be worse than they first looked.

All of these settings can be changed on the fly during the IDeA session, further emphasizing that IDeA is designed to be used as an interactive tool. Move repetition detection. IDeA tasks no longer consist of a single position, but a full line that leads from the root node to the position to be analyzed. This enables Rybka to report 0.00 scores for twofold repetitions, and is generally more correct. This also leads to the fifty-move rule being observed. Automatic backup of analysis trees. Besides more robust tree handling in IDeA, the analysis trees are now automatically backed up. Not only does it prevent data loss, but it may also prove useful in some other cases too. Browsing the analysis queue. You can see a list of all tasks for the current stage; which tasks were automatically scheduled by IDeA and which ones you added manually. The corresponding positions can be displayed on the board and you can see the engine’s output for the selected task, prioritize the tasks, and delete unnecessary ones. If there are tasks waiting in the queue when you stop the analysis, they will be stored and processed the next time you start the analysis, even if you restart Aquarium. About IDeA IDeA is one of the most advanced chess analysis methods available. The purpose of IDeA is to analyze a position deeply and return as much information about it as possible to enable the user to get a better understanding of the nuances of any position. IDeA keeps a permanent record of its analysis in a tree structure, which is unlimited in size. The user can browse the analysis tree at will, both while the analysis is in progress and after it has finished. The user can direct the analysis into the positions that are of most interest to him by excluding or adding positions and variations to the analysis. IDeA is an incremental analysis method, which means that it can be stopped at any time and the next time it is started, it resumes where it left off. Users can exchange analysis trees, work together on analysis projects, combine their results etc. In short IDeA is is highly selective search, controlled by Aquarium (and the user). Interesting lines are analyzed deeply, but weak moves are only considered briefly or not at all. Don’t forget that the “I” in IDeA stands for “Interactive” and your involvement in the analysis process is the key to understanding and improving the analysis. Note that the following is based on a beta version of Rybka Aquarium 2010. Some implementation details and screen designs may look different in the final version. Getting Started If you want to get started as quickly as possible, open a game, place the cursor on the position you want to analyze and click the IDeA button. IDeA will start analyzing the current position, using the default project and the default analysis parameters, which of course you can change at any time. Some of the information you see while IDeA is running will be familiar, but some of it will be completely new even if you used the original IDeA version. While IDeA is running, the Ribbon will display the “Interactive” group, which is similar to the original IDeA version, although you will also find some new features there. All of this is explained below. Move coloring is unchanged from the old IDeA version, so you can color moves green to focus the analysis on those moves or red to stop further analysis. The two leftmost buttons are used to stop the analysis. The “Project Active” button is highlighted, which means that this project will be analyzed when IDeA is running. If you click it, IDeA will stop analyzing the project, but continue analysis of other active projects. Click “Stop IDeA” to stop the analysis of all projects. If you use this same method to send a second position to the default project, the first root position will be removed. Therefore, you probably don’t want to use this method for large-scale analysis, although it’s helpful for quick analysis and to explore some of the new IDeA features. If you want to change the options for the default project, select “Options” from the IDeA button menu. Some of the options are familiar from the previous IDeA version, but others are new and will be explained later in this column. “Analyze in Project” allows you to choose a different project (or create a new one) instead of the default project for the analysis. “Show Control Center” is one of the methods to switch to IDeA view. Most experienced users will probably prefer to use the IDeA view to define their projects and run IDeA. Switching to IDeA View Not surprisingly, IDeA has been promoted in Aquarium and now has its own view. Click the IDeA button in the sidebar to switch to IDeA. The first time you do that you will see the IDeA Control Center. As mentioned above, you can also switch to IDeA by selecting “Show Control Center” from the IDeA button menu on the ribbon. The IDeA Control Center IDeA can keep track of multiple analysis projects. You can analyze one or more projects at the same time and revisit older projects and restart them where you left off. Each project has its own settings and the same project can even have multiple root positions, allowing you to analyze different positions in the same tree at the same time. Older projects can be deleted when they are no longer needed. IDeA can be used both for analyzing a set of positions (without creating additional nodes in the tree; e.g., a list of EPD records) and for traditional IDeA analysis, based on expanding the tree from the root position(s). A project setting determines which type of analysis the project will perform. The IDeA Control Center keeps track of all the projects you have defined. It consists of the three windows shown in this screenshot. The Project List window lists all available projects and some basic information about them. The Project Options shows options defined for the project that is currently selected in the Project List window. The Project Analysis displays the current analysis status of the selected project. Each of these windows is described in more detail below. The Project List Window The IDeA Project List window displays one line for each project that you have defined. Project Name is the name that you chose for the project. Type can be either “Local” or “Remote.” Local projects are run on the same computer as Aquarium, but in remote projects the analysis is run on a different computer. Note, however, that a local project can use a mixture of local and remote UCI engines. Status can be “Inactive,” “Active,” or set to a certain CPU usage percentage. When IDeA starts analyzing, it only analyzes projects that are marked as ‘Active’ or show a CPU usage percentage. Current shows if the project is “Idle,” “Analyzing,” “Generating Tasks,” “Minimaxing,” or “Waiting for tasks.” Created shows when the project was created. It’s only a reminder for the user. Modified shows when the project was last modified. The “New Project” and “Delete Project” buttons at the bottom of the window add a new project or delete the currently selected project. If you right-click over one of the projects, the following menu is displayed. View opens the selected project view. It’s equivalent to double-clicking the project. Pressing Enter also opens the highlighted project. Edit… allows you edit the project parameters. Make Default makes the project the default target for certain operations, such as sending positions from the Sandbox to IDeA for analysis. See the “Getting Started” section above. Make Active changes the project’s status to “Active.” This means that it will be included in the analysis the next time that IDeA is started. If IDeA is running when a project is made active it is started immediately. The Project Analysis Window The Project Analysis window displays the status of the project that is currently selected in the project list. It also allows you to add new tasks and analysis results to the project and minimax the project tree. Here we see the Project Analysis window for a project that is currently being analyzed. The “Project status” shows that eight tasks are being analyzed simultaneously. This means that at least eight instances of the analysis engine are active in IDeA. “Tasks in queue” shows two numbers: eight (displayed in green color) and ten. The green number shows the number of tasks that are currently being analyzed (corresponds to the number in “Project status”). The number after the “+” sign shows the number of tasks in the queue that are waiting to be analyzed (ten in this example). In case you have manually added tasks to the queue, they will be shown in red. The “Add tasks” link let’s you add tasks (EPD records) from an EPD file to the analysis queue. The “Add results” link is similar to “Add tasks,” but in this case the EPD records have already been analyzed and the positions and the analysis results are added to the tree. The “Minimax now” link allows you to minimax the project tree any time you like. “Elapsed time for this stage” shows how much time has been used for the current stage in the project, and the “Estimated time left” is also shown. The Project Options Window The Project Options window let’s you change the options for the currently selected IDeA project. These options are initially set when you create the project, but you can use the Project Options window to quickly change them at any time after that. The changes take effect the next time you analyze the project or at the start of the next analysis stage if you change the options while the project is being analyzed (except that switching to another “Analysis tree” takes effect immediately). You can see how the options are split into two panels: Three basic options (the tree, the root positions and the analysis quality) and several ‘Tree generation options’ that can greatly influence the shape of the analysis tree. All the options available in the Project Options window are described in the next section about creating projects. Creating a project When you want to create a project, click the “Add” button in the Projects group on the Ribbon or the “New Project” button at the bottom of “Project List” window. The “IDeA Project Properties” dialog box will be brought up. As you can see in the image below, the “Project Properties” dialog has two tabs: “Common” and IDeA. We are looking at the “Common” tab. Project description is a text area where you can choose any name you like to identify your project. The name will be displayed in the “Project Name” column of the project list window. Project type can be either local or remote. Local projects are analyzed on the computer where Aquarium is running, but remote projects are intended for analysis on a different computer or computers by exchanging EPD files. While you are learning how to use the new features of IDeA it is recommended that you stick to local projects. CPU usage. You can select one of three values from this drop-down list: “Inactive,” “Active,” and “Set CPU usage.” This field corresponds to the “Status” column in the project list and determines if the project is included in the IDeA analysis or not. When set to “Active” it will be analyzed the next time IDeA is started or immediately, if IDeA was running when the status was changed. “Inactive” projects are not analyzed. If you select the third option, “Set CPU usage” a new field is displayed where you can enter a CPU usage percentage. Here I have selected 30%, which means that IDeA will allocate thirty percent of its analysis time to this particular project. If the total percentage for all active projects is greater than 100, Aquarium scales the percentages so that they add up to 100. In that case other projects will not be given any CPU time, even if they are marked as “Active.” Analysis tree specifies the Aquarium tree where IDeA will store the analysis for this project. Clicking the link with the tree name shows the “Open” dialog box where you can either select an existing tree or type in the name of a new tree. Root positions let’s you add one or more root positions to the project. Each root serves as a starting position for expanding the tree. If more than one root is specified, IDeA splits the project’s analysis time equally between them. When you click the link (in this example “1 root position(s)”) the “Root Node List” dialog box appears. The example shown below is taken from a project that has three root nodes. The root positions are specified by the EPD strings in the “Positions” pane. The position corresponding to the highlighted EPD string is displayed on the diagram. You can add more root positions to the project by clicking the “Add” button, which displays the familiar position setup dialog box. You can also add a new position by pasting it (Ctrl+V) instead of clicking the “Add” button. If you paste a PGN game, the final position of the game is added to the root node list. The “Edit” button (or double-clicking the EPD string) displays the position setup dialog and allows you to modify the selected position. If you want to delete a root position from the list, highlight it and click the “Remove” button. The check-boxes to the left of the EPD strings allow you to exclude root nodes from the project without deleting them from the list. In this example, the first position doesn’t have a check-mark next to it, so it will not be a root node in the analysis. Full minimaxing of score and position count is done for the first active root node in the list, but only the score is minimaxed for other active roots. You can order the root node list any way you like by using the up/down arrows to the right of the list. Analysis quality defines for how long or how deeply each position will be analyzed. When you click the link, the “Analysis Quality Settings” dialog is displayed. You can specify any combination of time and depth. With the parameters specified as in the screenshot, each position will be analyzed for exactly thirty seconds. Specifying “30 seconds AND 1 plies” ensures that each position will be analyzed for at least thirty seconds. If the specified depth hasn’t been reached by that time (highly unlikely for depth 1), the analysis continues. “Max. time” ensures that the analysis time will not exceed thirty seconds. When “Wait for next depth” is selected, IDeA will not finish the analysis when the required combination of seconds and plies has been reached. Instead it will continue until the engine has completed the search at the current depth (or exceeded “Max. time”) and returned the first move at the next higher depth. If the score doesn’t change much, the analysis stops. Otherwise it will continue until a possible fail high/low has been resolved. Note that when you select “Wait for next depth” it’s essential to set “Max. time” much higher than the standard time, as completing the additional analysis can easily take two or three times longer or even more. Tree configuration let’s you specify a tree configuration to be used for displaying the analysis. You would normally create a new tree configuration for the analysis in which case you should click the “New…” button. The new tree configuration wizard will suggest a tree configuration with the same name as the analysis tree. Click “Finish” to accept the name or type a new one into the “Configuration name” field. Next, switch to the IDeA tab. Here you can set parameters that affect the shape of the analysis tree. Automatic tree expansion (IDeA). When this option is selected, normal IDeA analysis will be performed, expanding the analysis tree from the root positions. If it is not selected, IDeA will only analyze the positions given to it (e.g., from an EPD file) and not expand them. It only adds the positions with their evaluation to the analysis tree. Limit White alternatives to ensures that IDeA will not generate more alternatives for White in a position where the specified number of alternatives has already been reached. Limit Black alternatives to works in the same way for Black. Tree width affects the shape of the analysis tree. Increasing the tree width will cause more varied alternatives to be considered and create a “wider” tree. False alarm check immediately investigates further a new alternative that seems to be better than previously explored alternatives. It often happens that a new move looks very good when it is first evaluated, but further analysis quickly finds a refutation. If a new alternative has a better evaluation than the best move found so far (based on its minimaxed score), then the new alternative is immediately extended by the number of plies specified in this option. If this option is not selected, IDeA may need to pass through a few stages before the new move is refuted. Preferred side can be set to “None” (which disables it), “White,” “Black,” or “Both.” If it is set to “None,” IDeA creates a comprehensive analysis of the root position and the white and black sides are treated the same. This can be very useful in many cases, but in others you may prefer more focused analysis. A typical situation is a player preparing an opening repertoire. When analyzing a variation that he will only play as white, he will analyze differently than when preparing for playing the black side of the same variation. Basically, preparing for the white side, he is only interested in finding (and playing) White’s best moves against all reasonable Black moves. Of course it may require the analysis of several White moves before the best move can be determined, so this is not the same as analyzing only a single White move in every position. This is also how Preferred side works and setting Preferred side to “White” in this situation can result in considerable savings in analysis time. Thematic moves allows you to list one or more moves separated by commas (e.g., Bxh7, Ng5), which will always be tried by IDeA. Limit variation length prevents IDeA from extending variations beyond the specified number of plies. This is a very useful option that creates a denser tree if it is set fairly low at the beginning. You can then gradually increase the variation length as the analysis progresses. Project score bounds limits the analysis to variations that have a centipawn evaluation within the specified bounds. This option can significantly reduce the time used for analyzing dubious lines (although it also has other applications). If you are analyzing a quiet position, an interval of 100-200 centipawns between the lower and upper bound often works well. Starting IDeA After setting all the options for a project, you can start IDeA from the IDeA Control Center. First you must decide the engine configuration to use for the analysis by clicking the “Engines” button, which opens the “IDeA Engines Setup” dialog box shown below. It displays a list of the engines that IDeA will use for the analysis. Use the “Change” link to choose a different engine and “Add engine” to add more engines. The “#” column shows how many instances of the engines will be used in the analysis. You can manually change the number of instances for the first engine in the list if it’s a local engine by double-clicking the cell. The number cannot be changed for other engines. The check mark in the “Enable” column enables/disables the corresponding engine. After completing the engine setup, click the ‘Start’ button on the Ribbon to start the analysis. Running IDeA While IDeA is running, you can view the details of your active project(s) by opening the project view as was explained above. IDeA gives you excellent tools to monitor the progress; which positions are being analyzed; the evaluation, time and PV for the positions, etc. You can also jump straight to one of the positions being analyzed, view it on the chessboard and see the variation leading to the position in the notation. When you want to stop IDeA, click the Stop button on the Ribbon in the IDeA Control Center. You can switch between the project view and the IDeA Control Center by selecting them in the sidebar. The project view has five windows: the familiar board, notation and tree windows and two IDeA specific windows: Project Status and Stage Status. An overview of the project windows is shown in the image below. These windows can be used for interacting with the IDeA analysis at the task, stage, or project level. IDeA displays the results of its analysis in the tree window. The notation window can be used for browsing the tree, adding new positions to the analysis, etc. The Project Status Window The Project Status window gives an overview of the current project settings, the number of tasks that have been analyzed and the CPU time used. Note that the displayed CPU time takes the number of engines into account, so it can be much higher than the elapsed time. Analysis settings correspond to the Analysis Quality Settings that are accessible from the IDeA Control Center (Project Options window and the IDeA Project Properties). IDeA settings show some of the settings that determine the shape of the analysis tree. Clicking the link displays the IDeA Tree Expand Options, which allows you to view and modify all the settings. These options are the same as you saw on the IDeA tab of the IDeA Project Properties dialog box described above. Root nodes displays the number of root nodes in the project. Clicking the link brings up the Root Node List described above. The second link “(show)” displays the project’s root node positions on the chessboard. If the project has multiple root nodes, clicking the link repeatedly cycles through the root nodes. The Stage Status Window The Stage Status window shows an overview of the current analysis stage and the tasks that belong to the stage. Stage name displays the current stage (“Search for alternatives” in this example). Tasks shows the same information as “Tasks in queue” in the Project Analysis window in the IDeA Control Center: The number of tasks currently being analyzed (green) and the number of tasks in the queue waiting to be analyzed (red: manually added; black: automatically generated). Below the Tasks, all the tasks generated for this stage are displayed as colored squares. Gray squares, marked with an “f,” are finished tasks. The green squares show tasks that are currently being analyzed. In this case we see eight green squares, which correspond to the green number in the Tasks. The white squares are tasks that are waiting in the queue. Manually added tasks are displayed as red squares while they wait in the queue. They have a higher priority than automatically generated tasks. As you can see by its thicker border, one of the green squares has been selected, either by clicking it or by jumping to the corresponding position in the notation. The ‘a2’ means that it is a search for the second alternative in the position. The lower half of the window displays information about the selected task. Current analysis shows that the total analysis time for this position is eleven seconds. Note that a single task may require the analysis of multiple positions. Below that, the PV from the engine is displayed. In the square brackets at the start of the line, we see that the current evaluation is +0.00 at depth 25. The displayed evaluation and depth were recorded after ten seconds of analysis (10s). After that the PV itself follows and in the next line there are three links: Finish immediately finishes the analysis of the current task. Delete deletes the current task from the queue. Infinite switches to infinite analysis of the position. The last line shows the Task settings for the selected task. Note that there may be different settings for different tasks in the queue. The Tree Window and Move Colors While IDeA is running, it updates the tree window regularly, allowing you to browse the results of the analysis. You can focus the analysis on certain moves or variations by coloring them green (using the right-click menu) or exclude moves from further analysis by coloring them red. The role of the tree window hasn’t changed much in the new version, so I refer you to my June 2008 column for further information. Although move colors work the same as in the original IDeA, it’s likely that the introduction of multiple root nodes will affect their usage in the new version. Adding Positions to the Analysis Queue The move coloring method only allows you to mark moves that are already in the tree. If you want to request analysis of positions that are not there, you can add your own moves to the analysis queue and specify what type of analysis should be performed. When you browse the tree, the moves are automatically added to the notation window. If you run into a position that you find interesting, you can start experimenting by making moves on the board; those moves are also added to the notation window. There are several options to choose from when you request analysis of a position in the notation window. The next image shows the five buttons in the project view that are used to create analysis tasks and send them to IDeA. Current Position. The position on the board is scheduled for analysis. It is analyzed in the same way as if IDeA had selected the position automatically. Auto-play. IDeA lets the engine play against itself, starting from the current position and stores the results in the analysis tree. The user decides how many moves should be played and can also set an evaluation interval to stop the auto-play earlier if the evaluation falls outside the interval. Alternative. IDeA searches for a new alternative in the current position. All positions. You are not limited to adding a single position to the analysis. You can add as many moves and variations to the notation window as you wish and then send them all at once to the analysis queue by clicking “All Positions.” If you need to add several thousand positions to the analysis, it’s more practical to use some of the methods described below. Custom Task is a very flexible way of specifying an analysis task. When you click the button, the Custom Task Properties dialog box is displayed where you can define the parameters for the task. This option did not exist in the original IDeA version. You can also add a position to the root node list of the project by right-clicking in the notation and selecting “Add As a New Root” from the menu as shown in the next image. New positions can be added to the analysis queue while viewing a game by using the buttons in the “Send to IDeA Project” group. You can select the project that the positions are sent to by clicking the tool button in the lower right corner (highlighted in the image). Most of the other buttons are familiar, but sending a position to an IDeA project highlights the corresponding move in the notation and “Clear Marks” can be used to remove the highlighting. I have already mentioned above that you can add positions to IDeA projects by loading them from EPD files. That’s what the “Add tasks” and “Add results” links in the Project Analysis window do; the second one adds positions that have already been analyzed. The menu displayed by the “Manage” link in the Stage Status window (see the screenshot below) also has an “Add Tasks…” item that allows you to load an EPD file and add the positions to the project. An EPD file with evaluations can also be added to the project by choosing the last item on this menu, “Add Results.” Finally, there is a very powerful option to send positions from a collection of games to an IDeA project. First, mark the games that you want to process in the game list and then right-click and select “Send Games to IDeA”. A dialog box will be displayed where you can select a move interval and if variations should be included. You can also select all moves, or limit them to “White,” “Black,” “Winning,” or “Losing” side. The most powerful feature is the tree filter that can be used to exclude positions that exist in the selected trees. One obvious option is to use the project’s analysis tree to filter the games, so that only new positions are sent to the project, but there are many other uses for the tree filter. Conclusion If you are feeling overwhelmed by all the new features and think that you will need years to learn how to use them, don’t worry, just take the first steps and you will find that IDeA can work more or less automatically. After seeing how it works in automatic mode, you may want more control over the analysis. At that point, just re-read the relevant sections of this article or ask questions on the Rybka Forum and step-by-step you will master all the new IDeA tools. I want to emphasize that every IDeA option is based on a practical need and many of them were pointed out by Aquarium users. The implementation of the analysis and the new features were carefully discussed and then tested by beta testers. Only the features that were shown to improve the analysis were included in the final Aquarium release. Step-by-step you will find your favorite settings and greatly increase the efficiency of your analysis. Good luck!