Starting a New IDeA Project 19 September 2010
IDeA is meant to be used interactively and your interaction with IDeA should start before you let it analyze the first position of a project! When you create a new IDeA project, you start the analysis from a position that is of particular interest to you. It could be a critical position in your favorite opening, the first move out of the opening in a correspondence game, or just an interesting position that you would like to understand better with the help of IDeA. After creating a new IDeA project for a position, there are two things that you should aim for in the initial stages of the analysis: 1. Make sure that all important moves are covered. Some of the methods you can use for that purpose are described in this column. As with any other analysis method, missing an important move in the root position, or near the root, may render your analysis worthless. 2. Quick build-up of main variations. The sooner the main variations are established the more focused and productive the analysis becomes. A quick initial build-up of the analysis tree is also important if you are using many engine instances. By interacting with IDeA, your chances of achieving those two goals will be greatly improved. In this column I discuss interacting with your IDeA project even before the analysis starts. There are various sources of information that can help you at that stage, besides creative use of chess engines, such as game databases, chess books, magazines, and online resources. Creating a New IDeA Project The first question when you analyze a new position is if you should create a new IDeA project for the analysis or add it to an existing project. As an example, a correspondence player should create a new project for each game he plays. The advantage is that when a new move is made, the analysis tree already contains considerable analysis of the new position. However, it would be a mistake to use the same project to analyze all his ongoing games. The reason is that the analysis tree would become huge and performance would be worse than if the games were analyzed in separate projects using one analysis tree for each of them. My advice is to create a new project with a new analysis tree, unless you have a good reason to do otherwise. When you have decided to create a new project it is best to switch to IDeA by clicking the big IDeA button in the sidebar. Here you click the “Add” button in the ribbon and follow the instructions that were given in ” Interactive Deep Analysis (IDeA) With Rybka Aquarium 2010 .” The project settings can be changed at any time, even while analysis is in progress, and it is worth giving the initial settings a little thought. You may want to choose different settings while the project is being launched and then adjust them later when the tree starts taking shape. Let’s say we want the tree to be dense and well explored near the root position. We want lots of alternatives to be considered and each alternative to be extended a few plies. We can achieve this with the settings shown below. Even if I want IDeA to examine a wide variety of moves near the root, I still set “Limit White alternatives to” ten and the same for Black alternatives. This means that in no position will IDeA generate more than ten alternatives. If I hadn’t selected these options, there would be no limitation on the number of possible alternatives. I think that in most positions having up to ten alternatives is more than enough and this condition will only rarely have any effect. In other words, these settings are of little importance for what we are trying to achieve, but may save a little time by omitting alternatives that would be of little interest. I set “Tree width” to 85%, which is very high. It means that IDeA will consider alternatives even if they seem much worse than the best move. This can be very useful in positions with deep tactics. In such positions, chess engines sometimes overlook or take a long time to find the best move, unless they are forced to analyze it. The tree width should definitely be lowered after the initial stages of the analysis. Note that in this case, I have partially neutralized the effect of the high tree width by setting “Project score bounds” to -100 to +100 centipawns. This means that after IDeA generates an alternative that is outside these bounds, no further alternatives will be generated in that position. Let’s say that the third alternative in a position gives an evaluation of +150, then that will be the final alternative in the position. The same applies if IDeA extends a line and the evaluation of the new move is outside the limits. Setting “Project score bounds” is OK in positions that are strategic in nature, but you have to be more careful in tactical positions where a move may get a very bad evaluation when it is first encountered, but improves greatly after the line has been extended. Of course, you must take the approximate evaluation of the root position into account when you set the project score bounds. Otherwise, you risk excluding potentially interesting variations. Below the “Tree width” I have set “False alarm check” to four. This setting (along with other settings in this dialog box) was explained in ” Interactive Deep Analysis (IDeA) With Rybka Aquarium 2010 .” You might want to set it a little higher when starting a new project, but I usually set it to four. One experienced IDeA analyst likes to have it set to six, but there have been not tests to compare the effect of different “False alarm check” settings. “Preferred side” is set to “None,” which is the right choice if you want a balanced analysis of the position where both sides have the same priority. Later on, after you have seen how the analysis is developing, you may choose to change this setting. If you are, for instance, working on your white repertoire you could save some analysis time by setting preferred side to “White.” I’m not using “Thematic moves” in this example, but they can be quite useful when beginning a new project. One example is a tactical position where several sacrifices are possible. Each of them can be added as a thematic move to force IDeA to look at them. In an opening where you have an isolated pawn on d4 you could set d4-d5 as a thematic move to ensure that it is always tried and evaluated by the engine. Several of the above settings affect how wide and dense the tree is, but “Limit variation length” sets an explicit limit on how deep it is; i.e., how long the variations in the tree can become. You can use this setting to focus the analysis on moves near the root. In this example I have set it to a very low value: six plies. This means that IDeA will not extend variations beyond that point. All variations in our new project will start at the root and a variation will not be extended if it is already three moves long (six half-moves or plies). Clearly a low value is only appropriate when starting a new project, as we are doing here, and of course you must increase the ply limit or turn this option off later on. Finally, let’s see what is hiding under the “Advanced” link. When starting a new IDeA project you can increase “Percentage of leaf nodes to prolongate” to 100% while the analysis tree is still small. If you are using many engine instances for the analysis, you may want to keep it at 100% for the whole project, but if you are using a single engine the default of 20% is better when the analysis tree gets bigger. This setting controls how many analysis tasks IDeA generates at each stage. The higher you set it, the more tasks will be generated and the longer the analysis of each stage will take. Just like all the other settings you can modify it at any time. One side effect of increasing this setting is that you get a better coverage of a wider range of moves. IDeA gives preference to the best variations, so the additional analysis tasks resulting from a higher percentage setting will come from extending variations that currently are scoring worse. This effect is particularly noticeable in new projects while the analysis tree is still small. In some cases it can lead to more accurate evaluation of little analyzed moves. As an example, in some tactical positions the best move will be found much faster than it otherwise would be. Note that tree width also affects the number of tasks generated at each stage. With a wide tree (85% in our example) and “Percentage of leaf nodes to prolongate” set to 100% almost all variations within the “Project score bounds” would be extended. That’s why I set the “Project score bounds” to cut off variations that are not of interest in the project. Seeding a New Project With Positions and Lines Now that we have defined the project and set its parameters with an eye on our initial goals it is time to start looking for ideas to try out. Of course the easiest way forward is to start the analysis immediately. However, for serious analysis it may be less productive than spending some minutes studying the position, looking for good continuations and seeding IDeA with the resulting positions. If you are going to analyze a position from the opening stage of a game, you can probably find some games, analysis, or books with useful information about the position. You can use moves and variations from these sources both to ensure that the most important alternatives get IDeA’s attention right from the start and also to speed up the initial stage of the analysis if you are using several engine instances. If you are going to analyze a middlegame position, you have fewer options. It is unlikely that the same position has occurred in another game or has been analyzed before. In such cases an experienced player may know typical plans for the position and based on that knowledge he can suggest plausible continuations. A less experienced player, who is eager to improve and wants to learn from the analysis, can check his library for a discussion of similar positions, pawn structure, etc. In the early middlegame it can also be useful to look over games that started from the same opening variation. When analyzing an endgame, you are in a similar situation as when analyzing a middlegame position. Also here, the stronger and more experienced player can use his knowledge to come up with promising continuations, while the less experienced player can consult endgame books to get a better understanding of the position. Regardless of the type of position and the strength of the player, chess engines are indispensible for serious analysis. In addition to the moves and variations that you have determined to be important for the position, you can use chess engines to generate further ideas. You can use the moves and variations that you come up with to seed the new project with the positions that you want to be analyzed. IDeA always gives highest priority to user entered positions, so it will start by analyzing the positions that you are most interested in. There are two basic methods of seeding IDeA with positions to analyze: 1. Add the positions to the notation pane in IDeA and create a game with variations, starting from the position you are going to analyze. You will probably end up with a game with lots of variations, but when you are finished add all the positions to the analysis queue. 2. Add the positions directly to the analysis queue from a game you are viewing. You can use this method to add selected positions while viewing a game or you can select a list of games and send positions from all of them in one operation; e.g., a range of moves. Below I explain how to use these methods and different sources of information to let IDeA focus on the moves you are interested in. Sending a Game Fragment to the IDeA Queue When you have created a new IDeA project you can manually send positions from a game you are viewing to the project for analysis. We’ll start by seeing how you can use the raw game notation to create positions for IDeA to analyze. While viewing a game, switch to the Analysis tab and click the tool button in the lower right corner of the “Send to IDeA project” group as shown below. In this example I called the new project A29. I have selected it from the drop-down list and after I click the OK button, its name will be displayed below the buttons as shown here. Let’s say I’m analyzing the position after White’s tenth move in this game: 1.c4 e5 2.Nc3 Nf6 3.Nf3 Nc6 4.g3 Bb4 5.Bg2 O-O 6.O-O e4 7.Ne1 Bxc3 8. dxc3 h6 9.Nc2 Re8 10.Ne3 We will use this position in most of the examples that follow. The game continued: 10…b6 11.Qc2 Bb7 12.b3 Ne7 13.a4 a5 14.Ba3 d6 15.Rad1 Qc8, etc. I want IDeA to analyze these moves, so I add them to the analysis queue. I click “Other” in the “Send to” group and select “Send Game” from the menu. The “Send positions to IDeA” dialog box will be displayed. Here you only need to specify “From move” and “to move” and make sure that you are sending the positions to the correct IDeA project, “A29” in this case. After that, click OK. If you switch to IDeA now and open the A29 project, you will see that twelve tasks have been created and are waiting in the analysis queue. The task color shows that these are manually added tasks. Note that we haven’t started the IDeA analysis yet. As you can see, there are many options to choose from when sending game positions to the project queue, although some of them are not important when you are starting a new project as we are doing here. For an explanation of other fields see ” Getting More Out of IDeA in Aquarium 2010 .” Sending Several Games to the IDeA Queue Although sending positions from a single game can be useful for a new project, the more common scenario is that you have several games where your position has occurred and you want to select positions from all of them for analysis. In this case I am using a database of correspondence games. Start by searching for the position you are interested in (I used the position shown in the diagram above) and mark all the games (Right-click > Mark All). Right-click over the game list and select “Send games to IDeA.” The “Send positions to IDeA” dialog box that we saw above will be brought up. Use the same parameters as above and click OK. When we switch to IDeA and look at our project we will see that instead of just twelve tasks, we now have 200 tasks coming from thirty-two different correspondence games. Of course the same positions occur in many of those games, but Aquarium removes the duplicates before adding the positions to the queue. Therefore, we have 200 unique positions added to the queue. Using Additional Resources The above examples showed how you can send positions based on raw database games to IDeA. However, you can combine ideas from many different resources to create a game with lots of analysis and interesting variations before sending it to IDeA. After that, you can either send the positions directly to the project queue or copy all your analysis into the IDeA notation window and use the “All Positions” button to send the positions to IDeA. Some examples are shown below. Adding Infinite Analysis Results Aquarium has some very nice infinite analysis features (see ” Infinite Analysis with Rybka Aquarium .” Of course you can do “normal” single engine analysis, but ” Analysis Presets ” are a great feature when preparing for a new IDeA project. It becomes more useful as the number of available CPUs increases. A single analysis preset can combine engines running on different computers. One of my favorite presets runs eight single core engines as shown below. The positions being analyzed here is the one shown in the diagram above. You learn a lot about the differences between the chess engines when you see them analyzing side by side as shown here. Let’s take a closer look at the analysis results: The engines agree that the position is equal or slightly better for White. Rybka 4, Komodo 1.2, Deep Shredder 12, Stockfish 1.8 and Deep Sjeng 3.0 all like 10…d6. Naum 4.1 prefers 10…Ne5. HIARCS 13.1 sees 10…Ne7 as the best move Finally, Deep Junior 11.2 suggests 10…b6. As you can see, analyzing with several engines can point out different methods of playing the position and give you material for trying out in IDeA. When you see a variation in the analysis window that you would like to analyze further with IDeA, click on it and press the Ins (or Insert) key. It will be added to the notation. You can also use the right-click menu in the analysis window. Another good source of possibilities is to run some engines in multi-variation mode and of course you can use classical single engine analysis. Regardless of the configuration you choose, interacting with infinite analysis should always be considered. When you are done with the infinite analysis and have added all interesting variations to the notation, you can use the method described above to send all the positions to the IDeA queue. Some users prefer to copy the notation and paste it into the IDeA notation window and then click “All Positions” to add the positions to the queue. If you are planning on adding variations from other sources, you might want to wait until you have added all the variations you want to send to IDeA. Note that the next update of Aquarium (version 4.0.6) includes a check that prevents duplicate positions from being added to the analysis queue, even if the positions are sent in separate requests. Adding Variations From Opening Books Assuming that you are analyzing an opening position, you can add variations from opening books to the analysis queue. I assume that the position you are going to analyze is currently selected in the Sandbox or a database game that you are viewing. Start by switching to the opening book you want to use as a source. In the image below, I am still using the position discussed above and the opening book is Jiri Dufek’s new opening book for Aquarium. Here we can see the move that most chess engines liked, 10…d6, is also the most popular move in engine games, over-the-board games, and correspondence games (for more information about Jiri’s opening book see the two previous ChessOK Cafe columns). It looks like a natural and good move, but in some cases you may want to take your opponent out of book early or try to find a non-obvious improvement. We also see that although 10…b6 (suggested by Deep Junior above) isn’t very popular it has been doing quite well, so it will be interesting to analyze it further with IDeA. Finally, based on the CAP evaluations, 10…a5 might be interesting too, although none of the chess engines suggested it. The same goes for 10…Ne5, which was also suggested by Naum 4.1 above. After picking out the moves that you would like to analyze with IDeA, select the first one in the tree window and play through the variations with the arrow moves. All the moves you make will be automatically added to the notation pane, if they are not already there. Do the same for the second move, etc. Using this method you can pick up interesting moves from the opening book. Of course, you should only copy the most interesting moves and variations. It won’t help IDeA much to fill the analysis queue with second-rate moves. After going through the Rybka 4 Aquarium book, you can switch to another opening book and repeat the process if you believe it has some good analysis of the position. As before, you might want to wait a little longer before sending the variations to IDeA if you are going to examine other sources. ChessOK Opening Tree Mode The ChessOK Opening Tree Mode is a web-based game tree, based on over four million games. In spite of the name, the positions in the tree cover the games from beginning to end. This makes it a great resource for ideas when analyzing the opening or the early middlegame. You don’t need your chess software to browse the tree and you can access it anywhere you have an Internet connection. One nice feature of the ChessOK Opening Tree is that is gives you a quick overview of the draw rate for each alternative. The database that the tree was built from (HugeBase) contains mostly overthe-board games, with relatively few correspondence games, and a very limited number of chess engine games. You can get statistics for a position either by entering a FEN-string (or EPD-string) into the FEN field or by pasting the moves leading up to the position into the PGN field. Here is an example, once more using the position we discussed above, illustrating the statistics and other available features. Clear board resets the board displayed to the left of this table to the normal starting position. Takeback can be used when you have pasted a game in PGN format into the field to the right of the Takeback button. It can also be used when you have been browsing the tree and want to take back your last move. You can also use the left and right arrows to browse the moves you have made. Clicking a move takes you to that position. Load PGN is used after pasting a PGN game into the text field to the left of the button. It parses the game and displays the final position. Instead of using PGN format, you can paste a position in FEN or EPD format into the FEN field and then press Load FEN to jump to the position in the tree. That’s what I did in the example shown in the screenshot. The PGN field below the FEN field shows the moves leading to the current position, either from the starting position or from the position given by the FEN string. In this case I used a FEN string and haven’t made any moves yet, so the field is empty. There are nine columns in the statistics table and one line for each move: Move displays move itself. Eval. is the CAP evaluation of the position. For information about CAP, see Trees and Opening Books in the Aquarium FAQ. Games shows the number of games where the move has been played. When you reach a position that has only occurred in one game, information about the players, when and where it was played and the result is displayed. W/won is the percentage of games won by White. B/won is the percentage of games won by Black. Draws shows the draw percentage in games where the move was played. It gives you a quick overview of the draw rate for each move. Note that this column is not available in Aquarium trees. W/Elo shows the Elo rating of the highest rated player who had the white pieces in a game where the move played. B/Elo shows the Elo rating of the highest rated player who had the black pieces in a game where this move occurred. Year shows when the most recent game with this move was played. The moves shown in this position in the ChessOK Opening Tree Mode are already familiar to us: 10…d6, 10…b6, 10…Ne5, and 10…a5. Clearly these are the moves that are worth examining in this position. After feeding IDeA with the most interesting continuations, assembled from the sources we have examined I started the analysis using the settings shown at the beginning of this column. After some initial analysis, I changed “Limit variation length” to twenty, lowered tree width to 20% and let the analysis run overnight with analysis quality set to 120 seconds and depth eighteen. The screenshot below shows the analysis results in the root positions. The results confirm that 10…d6 is a good choice, but if you are looking for good alternatives, you might want to take a closer look at some of the other moves, such as 10…b6. A Middlegame Position When you are analyzing a middlegame position you have fewer options. If it is a position from one of your own games, you probably wouldn’t find anything about it in your databases, opening book, chess literature, etc. Here is an interesting position from a correspondence game in 2004 between a chess engine and Arno Nickel, who had the black pieces. I start by running the analysis preset with eight single core engines, each with a 256MB hash size. * DeepShredder12UCIx64 [-0.03] d=17 1.b3 a5 (0:00:38) * Hiarcs13.1MP [+0.24] d=18 1.Rh3 Nd7 (0:03:50) * DeepJunior11.2.x64 [+0.02] d=19 1.g4 fxg4 (0:03:50) * DeepSjeng30_x64 [+0.51] d=17 1.Rh3 Nd7 (0:02:55) * komodo-12-64-ja [-0.25] d=21 1.b3 a5 (0:03:56) * stockfish-18-64-ja [+0.76] d=26 1.Rh4 c5 (0:03:43) * Naum 4.1 x64 [+0.24] d=19 1.Rh3 Nd7 (0:01:37) * Rybka 4 x64 [+0.29] d=17 1.Rh3 c5 (0:03:31) 16552kN I’m only showing the first move of each variation. It seems that many engines like the natural 1.Rh3 here and Deep Junior’s 1.g4 is an interesting move. I’m not too keen on the passive 1.b3 and for a quick check with IDeA I decided to start by sending IDeA the first four moves of the variations given by Rybka and Deep Junior. Stockfish gave a short variation (1.Rh4 c5) that I also added to the mix. After adding them to the notation and cutting them down to the first four moves, the notation window in the Sandbox looks like this. Next I send these positions to IDeA using the method illustrated above. For the IDeA analysis I used the initial settings shown at the beginning of this column, except I disabled “Limit variation length” and “Project score bounds.” For this run I set analysis quality to sixty seconds and depth sixteen. After activating the project and starting the analysis, IDeA first analyzes these nineteen positions. Already at that stage 1.g4 looks promising. It is at the top of the list with a +0.72 score. Examining the variations a few minutes later, I see that the line starting with 1.g4 is more or less forced; i.e., the alternatives generated along the mainline have a much worse evaluation than the mainline itself. Therefore, I decided to lower the tree width to 35%. I also notice that the mainline follows the variation given by Deep Junior, so I add the rest of the variation to IDeA by first copying it from the analysis window, pasting it into the IDeA notation and clicking “All Positions.” I left the analysis running for about an hour and then I had a nice tree of variations. These results look good. Besides showing that 1.g4 is the best move by far, I also get analysis of all the moves suggested by the chess engines before I started IDeA. What I like when looking at this table is that most of the analysis time has been spent on the best moves. If we hadn’t seeded IDeA with 1.g4, there would probably have been some time until it was “discovered” and therefore it would have been much less analyzed by this time. It is interesting to compare the evaluation of the chess engines in infinite analysis with the IDeA results. The method I used here to seed IDeA with positions, although effective in this case, was rather simple. Using analysis presets, you can combine chess engines in many different ways when searching for ideas to analyze. Conclusion IDeA was designed to be used interactively, as many users know. Fewer know that you can interact with IDeA even before you begin the analysis of a new project. In this column I have shown how you can seed IDeA with interesting positions before the analysis starts. The ideas for positions to analyze can come from many different sources. A well chosen set of positions will lead to faster and more balanced results where more time will be spent on analyzing the best variations.