Houdini 2 Aquarium 01 December 2011
In an announcement earlier this month, ChessOK introduced a new member of the Aquarium family: Houdini 2 Aquarium. It’s actually a group of products comprising Houdini and Aquarium, available both in a single package and as separate products. As chess players know by now, Houdini is currently the highest rated chess engine and the new Aquarium version is an even better analysis tool than previous versions. Best of all, the new Aquarium version will be available as a free update for current Aquarium 2011 users. Houdini 2.0b Houdini 2.0b is an update to Houdini 2 with some minor bug fixes. More importantly, this version adds Nalimov EGTB support. This means that both Gaviota and Nalimov endgame databases are now supported. Houdini 2.0b comes in three versions: Houdini 2 UCI Standard. The standard version supports up to 6 cores and 4 GB of hash. This should be sufficient for most users. Houdini 2 UCI Professional. The professional version supports up to 32 cores, 32 GB of hash and so-called NUMA-architecture. This version is for high-end users with powerful hardware and is more expensive than the standard version. Houdini 2 UCI for IDeA. This is a single-core version, specifically intended for use in Aquarium’s IDeA analysis. I’m sure that this version will be appreciated by IDeA users who prefer running multiple single-core engines. It is included with the other versions mentioned above in the Houdini 2 Aquarium (Pro) package. All Houdini engines come in 32-bit and 64-bit versions and will run on any recent Windows version. A 64-bit Windows version is required to run 64-bit Houdini. Besides Windows, Houdini can also be run on Linux under Wine. If you are one of the few who are not already familiar with Houdini, here are a few items of interest: ● It is currently the strongest available chess engine, leading all major rating lists. ● When playing against Houdini, you can set its skill level anywhere from zero (beginner) to 100 (full strength). ● “Position learning” automatically saves analysis results in a learning database that Houdini will use to speed up future analysis. ● The complete hash table can be saved to a file and reloaded at a later time when you want to continue the analysis. ● Houdini supports FischerRandom chess (Chess960). Houdini 2 Aquarium Houdini 2 Aquarium is comprised of the following software, game databases, and trees (position databases): 1. Improved Aquarium 2011. The improvements in this version are described below. If you are already an Aquarium 2011 customer, you will get this version for free. 2. Houdini 2.0b. Houdini 2 Aquarium Pro comes with the Houdini Pro version and Houdini 2 Aquarium Standard includes the standard Houdini version. 3. Hugebase. This is a database of 4.7 million, mostly human, games. The database has been updated to include games up to October 1st, 2011. 4. Corr2011. A database containing over 500,000 correspondence games. 5. HugeStatistics (d2m). A statistics tree based on Hugebase with over 15 million positions. 6. CAP evaluations. The CAP tree grows faster than ever and now contains over 55 million analyzed positions The minimaxed chess engine evaluations are displayed in the CAP column in the tree window. 7. A weekly download of over 2,000 new games. Aquarium Improvements Improved web export If you like to publish games and analysis on the web, you might want to try Aquarium’s web export, which has been improved in the new version. Here you can see an example of what the blog export looks like in the new version. Note the new navigation buttons below the board. You can also use the left and right keyboard arrows to navigate through the game. Databases Aquarium can now read PGN files up to 4GB in size (instead of 2GB before). However, it is recommended that you convert big PGN files to a CDP database if you need to modify them. There are now template files for game header fields in AquariumData\Config\Templates. These files can be edited by advanced users; e.g., to add more header fields. A problem with deleting games from PGN files has been solved. Two new features related to game deletion were added. The new Compress and Undelete All buttons are located on the Home – List tab. The first one compresses the database and permanently removes all deleted games. Undelete All can recover games that you have deleted, as long as you haven’t compressed the database. Infinite analysis When you insert a variation from infinite analysis into the notation, a comment has been added indicating in which position the analysis was performed. The moves with the green background were the result of infinite analysis, which was run in the position after 24.Qxe5. The first moves (24…fxe5 25. Rf1) suggested by the engine coincided with the actual moves of the game, but instead of continuing with 25…Rc8 as in the game, the engine variation continued with the moves shown in the variation starting with 25…Rc7. The comment after the variation, “24.Qxe5” (highlighted), shows the actual starting point of the analysis. If you only want to insert the engine evaluation (and not the full variation) into the notation when running infinite analysis, you can use the Ctrl-E keyboard shortcut. You can also have the evaluation inserted automatically when you stop the analysis (or move to a different position). Switch to the Analysis tab and click the tool button (dialog box launcher) in the bottom right corner of the Infinite Analysis group. The following dialog box will be displayed. Time before marking analysed position with Engine Analysis style automatically marks moves that you have analyzed for the number of seconds specified. This lets you keep track of positions that you have analyzed, without cluttering the display with positions that have only been analyzed briefly. In this example, I have specified thirty seconds, so a position will only be marked if it has been analyzed for half-a-minute or more. If you select After this time, insert the evaluation automatically, Aquarium will insert the engine evaluation into the notation when you stop the analysis or move to a new position, provided it has been analyzed for at least thirty seconds (or whatever time you have specified). The next screen-shot shows an example of what the results look like. There are two positions that have been analyzed for more than thirty seconds: 13…f6 and 15…Kf7. We see that the evaluation after 13…f6 is +0.71 after seven minutes and fifteen seconds of analysis and the analysis reached depth 20. One of the new features of Aquarium 2011 is engine specific analysis trees. These were explained in Aquarium 2011, Part One. You can connect each engine (or a group of engines) to a specific position tree which stores the analysis of that engine. I explained how you can modify any tree configuration to display the engine trees in Aquarium 2011, Part Two. It is best to explain how this works with a screen-shot: Here you can see the analysis results of five different engines displayed in the tree window. Besides the evaluation of each engine you can also see the analysis depth, which is a new feature. The deepest analysis is displayed in bold so you can quickly locate it. As you can see, the engines didn’t agree on the best move here. Critter and Shredder prefer 32.Be4; HIARCS and Stockfish like 32.Bg1; and Rybka likes 32.Bb6. Looking at the rightmost column in the image you can see that Rybka preferred 32.Kg2 up to depth 11. After that it switched to 32.Be4, but after reaching depth 21 it gave the highest evaluation to 32.Be3 for two iterations. Finally, Rybka switched to 32.Bb6 at depth 24 with an evaluation of -0.41. This was the final iteration of the analysis so it is displayed in bold. After trying out this new feature you will probably find out that it is a really good way to compare the analysis of chess engines and learn about their differences, strengths, and weaknesses. Additionally, it is a great method to populate an IDeA tree as was explained in Sending Games and Position to IDeA. Pasting game fragments Here is a nice little feature that can be helpful when you have a game which doesn’t start from the standard opening position and you want to add the missing moves to the game. The following game fragment starts at move six: [Event “Test”] [Site “MyPlace”] [Date “2009.11.04”] [Round “?”] [White “WPlayer”] [Black “BPlayer”] [Result “1-0”] [FEN “rnbqkb1r/pp1p1ppp/4p3/3nP3/1p6/P4N2/2PP1PPP/RNBQKB1R wKQkq – 0 6”] [SetUp “1”] 6.axb4 Bxb4 7.Na3 Nc6 8.c4 Nde7 9.Nc2 O-O 10.Nxb4 Nxb4 11.Ba3 Nec6 12.Qb1 a5 13.Bd3 f5 14.Be2 d5 15.exd6 Qxd6 16.d4 Bd7 17.Qb2 Be8 18.OO Bh5 19.Rfd1 Rae8 20.Rd2 b6 1-0 If you paste this PGN into a new game in Aquarium and want to add the missing moves, just post the following move sequence into the same game and choose “Join” when asked: 1.e4 c5 2.b4 cxb4 3.Nf3 Nf6 4.e5 Nd5 5.a3 e6 Since the final position of this move sequence matches the starting position of the game in Aquarium, you will get a complete game, starting from move one. IDeA improvements Increasingly, IDeA users are taking advantage of master trees to store their analysis (see IDeA Master Trees and IDeA Game Links). Now you can save analysis from all your projects into their corresponding master trees with one click, instead of having to go through each project and and export it to the master tree. This feature is a new menu item under the Export button on the Master Tree tab. Select “From all projects” and Aquarium will go through all projects that have a master tree defined and update the master tree with new project analysis. The All Positions button in the Home – IDeA tab sends all the positions in the notation to IDeA for analysis. This is a popular feature, but some users have been surprised when they see that it works exactly as advertised and actually sends every position, including the starting moves of the game, which are of no interest in the IDeA project. The new Aquarium version adds two new options to this button (see image below): “From current position” and “Up to current position.” “From current position” avoids the matter mentioned above by only sending positions after the current position in the game. “Up to current position” does the opposite and sends all positions from the start of the game up to the current position. Saving the best for last, the most important improvement to IDeA in the new Aquarium version is much better control over the shape of the analysis tree. The part of IDeA that decides which lines are extended has been completely rewritten to address issues reported by Aquarium users. This means that even with the default settings you will see improvements in this area. You will no longer see all the analysis go into the best move, while a slightly worse move is more or less neglected. In the past it would often turn out that the “second best” move was actually the best one, when you started analyzing it in more detail. You choose the tree shape from a drop-down list on the IDeA tab in IDeA Project Properties. This dialog box can be invoked from the Analysis group in the Home – IDeA tab on the Ribbon. As the image shows, you can choose from seven different tree shapes. ● Default. This is the shape that should be compared to IDeA in previous versions. However, it gives better coverage of alternatives, so you are less likely to see “neglected” variations just because they are slightly worse than the best one. ● Wider. This option generates more alternatives than the default setting. ● Longer generates fewer alternatives than the default and therefore longer variations. ● Longest generates the fewest alternatives of all the tree shapes and focuses on extending the best lines. ● Tactical is intended for tactical positions. It generates a very wide tree at the root and gives low penalties for bad moves. ● Aggressive creates an extremely wide tree and has low penalties for bad moves. This can, for instance, be a good option where a series of deep tactical moves need to be found. ● Positional creates a wide tree with high penalties for bad moves. This is a good choice in positions where there are no deep tactical surprises. Although the new IDeA algorithm and the selection of tree shape will give better results, even in automatic mode, it is still recommended to use infinite analysis as well, especially since its now so closely integrated with IDeA. Here is one example of how the tree shape can be used depending on the type of position you are going to analyze. The kings have castled on opposite wings, the h-file is open, White has wellpositioned pieces and Black is behind in development. White clearly stands better, but he must play aggressively or else his advantage will evaporate. I created an IDeA project for this positions and selected “Aggressive” tree shape and one minute analysis time for each position. I set a limit of three alternatives for Black and score bound from -200 to +200 centipawns, relative to the root score. I used six instances of Houdini 2 for IDeA for the analysis. After a couple of hours the analysis tree looked as follows in the position shown in the diagram: “Aggressive” clearly lives up to its name and finds 1.g4!!. This is the key move although there are many complications even after that move has been found. What is even more interesting for experienced IDeA users is to look at the “N” column, which shows how many positions have been examined in the variations starting with each move in the “Move” column. It looks completely different from what you would expect in older Aquarium versions. Even though the scores of the moves vary widely (from -0.22 to +0.76) the number of analyzed positions varies surprisingly little. This is the result of choosing “Aggressive” tree shape. It examines many moves deeply and has a good chance of finding quite deep tactical moves as in this example. Since 1.g4 has the fewest number of positions, it is clear that it was found to be a good move late in the process. I let the analysis run overnight and in the morning it seemed clear that 1.g4!! was actually the best move in the position. The main variation was 1.g4!! fxg4! 2.Rxh7!! Qxh7 3.Bxg6 Qd7 4.Rh1 Re7 5. Be5 Bg7 6.Nf4! Na6 7.Bh7+ Kf8 8.Ng6+ Ke8 9.Nxe7 Qxe7 10.Qg6+ Kd7 11. Bxg7 Nc7 12.Bf6 Qf8. This variation agrees with the analysis by Roland Rösler, which he published on the Rybka forum. As you can see from this example, the new IDeA version can create tree shapes that are very different from those we used to see in previous Aquarium versions. Here one example of a position that has been considered very difficult for chess engines.

 

I created an IDeA project with this position that arose in the game Nils Grandelius – Danny Raznikov in Bulgaria last month. Black played 49…Rb5 and lost since rook is trapped on b4 after 50.Nd3. For this project I used the “Default” tree shape and only thirty seconds/position. A chess engine needs much more time to see that the rook is actually trapped, so it is interesting to see how the “effective depth” of IDeA is much greater than the analysis depth of individual positions in the tree.

 

 

 

 

 

After an hour of analysis with six single-core Houdini engines, IDeA had seen that 49…Rb5 leads to problems for Black (+0.93), as the screen-shot shows. Instead of 49…Rb5, it would have been better to play 49…Rc4. Conclusion The big news in this Aquarium release is, of course, that Houdini and Aquarium have joined forces. On top of that Aquarium introduces new features that are important for chess players and analysts, both those who use infinite analysis and those who use IDeA (most of which will also use infinite analysis, of course). The most important of these are improved infinite analysis trees for chess engines and the redesigned IDeA algorithm, along with a convenient selection of the shape of the analysis tree. Dadi Jonnson