Reports | June 06, 2007 23:23

[lang_nl]Uit die openingsbibliotheken![/lang_nl][lang_en]Turn off those opening libraries![/lang_en]

[lang_nl]Vandaag zijn niet alleen de finales van de Kandidatenmatches in Elista begonnen, maar ook de "Ultimate Computer Challenge" computermatch tussen Deep Fritz en Deep Junior. Kirsan Iljoemzjinov, de president van de FIDE, schijnt de match persoonlijk te sponsoren. Da's natuurlijk zeer nobel van hem, maar vermoed mag worden dat Chessbase anders wel had willen bijspringen. Iljoemzjinov noch Chessbase heeft overigens gereageerd op de uitdaging van Rybka waarover wij eerder berichtten. Of dit verrassend is, laat ik graag aan de verbeelding van de lezer over.[/lang_nl][lang_en]Today not only the finals of the Candidate matches in Elista have begun, but also the "Ultimate Computer Challenge" computer match between Deep Fritz and Deep Junior. Kirsan Ilyumzhinov, president of the FIDE, appears to sponor the match himself. Of course this is all very moving, but I suspect that Chessbase would otherwise have donated the money. By the way, neither Ilyumzhinov nor Chessbase have responded to Rybka's challenge about which we wrote before. Whether this is really surprising, I leave to the reader's imagination.[/lang_en]

[lang_nl]Als schaakliefhebber wil ik echter nog een ander een beroep doen op Iljoemzjinov en de programmeurs van Deep Fritz en Deep Junior. Het heeft te maken met het gebruik van openingsbibliotheken.
De eerste matchpartij is alweer afgelopen: hij eindigde in een bloedeloze remise na 32 zetten, waarvan de eerste zestien al bekend waren uit een partij die in de Chessbase database te vinden is. De vraag dringt zich op: waarom toch?

Waarom die openingsbibliotheken? Hoe kunnen we de werkelijke kracht van schaakprogramma's ooit inschatten - zeker als ze tegen elkaar spelen - als de programma's domweg openingszetten kopi?ɬ´ren die door mensen bedacht zijn? Wat is daar leuk aan? Hoe kunnen de programmeurs dat zien als een uitdaging? Of zijn de programmeurs helemaal niet uit op een uitdaging en willen ze gewoon geld vangen?
Willen ze niet weten wat hun programma vindt van de beginstelling? Denken Chessbase en Iljoemzjinov dat wij, het publiek, dat niet willen weten? Denkt men dat het publiek zit te wachten op twee machines die bestaande kennis oplepelen uit hun geheugen? Dan hebben ze het mis. Niemand zit daarop te wachten. Het is een aanfluiting, een wanvertoning, en bovendien een belediging van de oprechte interesse en intelligentie van alle schaakliefhebbers.

Computerschaak - we zullen ermee moeten leren leven. Maar computers die om geld tegen elkaar spelen en daarbij gebruik mogen maken van door mensenhanden in honderden jaren opgebouwde openingskennis vol overbekende oude koek, in plaats van hun eigen opvattingen en de eventuele gebreken (?ɬ®n mogelijk nieuwe inzichten) van de software te tonen - dat is een lachertje.

President Iljoemzjinov, u bent zelf een schaakliefhebber. U maakt mij niet wijs dat u meer geinteresseerd bent in het feit dat twee computers grote hoeveelheden data kunnen opslaan en reproduceren, dan in de spannende mogelijkheid dat computers nieuwe inzichten kunnen verschaffen in de beginstelling, de eerste prille zetten van het schaakspel zelf uitvinden, of onze eeuwenoude opvattingen over schaaktheorie omver werpen. Laat computers vanaf het begin zelf denken, en laten we d?ɬ°n eens kijken welke engine daar beter in is.

Programmeurs, zet uit die openingsbibliotheken! Als jullie durven.[/lang_nl][lang_en]As a chess lover I would like to make another kind of appeal to Ilyumzhiov and the programmers of Deep Fritz and Deep Junior. It has to do with the usage of opening libraries.
The first match game has finished already: it ended in a bloodless draw after 32 moves, of which the first sixteen were already known from a game in the Chessbase database. The question arises: why, for Heaven's sake?

Why these opening libraries? How can we ever measure the true strength of chess programs - especially when they're playing against each other - when the programs simply copy opening moves made by humans? What's the fun in that? How can the programmers see this as a challenge? Or are they simply not interested in a challenge and do they just want the cash?
Don't they want to know what their program thinks of the beginning position? Don't we want to know this? Do Chessbase and Ilyumzhinov think that we, the public, don't want to know this? Do they think that the public wants two machines copying existing knowledge from their memory? Well, they're wrong then. Nobody in their right mind is waiting for this. It is a ridiculous event, and on top of that an insult to the true interest and intelligence of all chess lovers.

Computer chess - we're gonna have to live with it. But computers playing each other for money and using opening knowledge accumulated by man over the centuries, rather than displaying their own opinions and possible limitations (and new insights) of the software - that's a joke.

President Ilyumzhinov, you're a chess lover yourself. Surely you're not telling me you're more interested in the fact that two computers can store large amounts of data and reproduce it, than in the exciting possibility that computers can give us new insights in the beginning position, the possibility that they could re-invent the first moves of chess, or shake our age-old beliefs about chess theory. Let computers think from the first move, and then let's see which engine is better.

Programmers, turn your opening libraries off! If you dare.[/lang_en]

Arne Moll's picture
Author: Arne Moll

Chess.com

Comments

Alkelele's picture

"It?¢‚Ǩ‚Ñ¢s the human drama, the usual human dillemma?¢‚Ǩ‚Ñ¢s, human doubts and fears (should I or should I not play this line against this guy? Did I really calculate everything correctly? Am I thinking too long for this move? Are my intuitions correct? etc.)

Computers don?¢‚Ǩ‚Ñ¢t feel any of these these doubts, these fears, in short, they don?¢‚Ǩ‚Ñ¢t experience the OTB drama."

No, but the book-makers behind the engines do! ;-)

To be serious, I can understand your reasons for finding human games interesting. I can also understand your interest in seeing what engines would have to say about chess, without "booking-help".

Perhaps you can also learn to find engine+book matches interesting!? Seriously, I can recognize all these "human doubts" in the engine+book setting -- it's just the book-maker and the engine authors who are having them. Interestingly enough, there are many people in the engine-room on playchess who also find this book-making exciting (and there is MUCH more to it than just copying human games).

It's like the question: What openings is Deep Blue going to play against Kasparov? Now, THAT is an interesting question, even if Deep Blue has no emotions and doesn't feel the drama. For me, the question: "What openings is the Junior team going to play against Fritz?", can be interesting in the same way. And the games resulting from this can also be interesting for me. I simply like to see state-of-the-art chess being displayed, knowing that a lot of thoughts were put into making strong opening choices.

peter's picture

This is from today's Chess Today:

"The score in the match between
computer programs "Deep Fritz" and
"Deep Junior" (which is being held in
Elista together with the 2nd round of
the Candidates' matches) is equal so
far ?¢‚Ǩ‚Äú two draws. I'm not sure that it
deserves to be reported on at all
though. One could have spoken that
this match indeed had a puprose to
determine the best computer chess
program in the world, if: a) One of the
participants were "Rybka", obviously
the strongest chess engine nowadays;
and b) The organizers clearly
indicated which hardware the
programs were run at. But none of
these conditions are present.
Moreover, not only have the
organizers not been concerned to
indicate the number of processors
used by each engine, but even the
versions of the programs! That's why
the match looks somewhat ridiculous."

Alkelele's picture

Well, very interesting discussion! :-)

I would like to point to game 3. It shows that very interesting games can very well be had also when books are used. To me, games like these make a match like this exciting. Maybe not for some of you, who may be more interested in other aspects. But, without books, we would never see topical and spectacular games like this. Anyway, enjoy:

[Event "President's Cup"]
[Site "Elista"]
[Date "2007.06.08"]
[Round "3"]
[White "Deep Junior"]
[Black "Deep Fritz"]
[Result "1-0"]
[ECO "B97"]
[PlyCount "169"]
[EventDate "2007.??.??"]

1. e4 c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 Nf6 5. Nc3 a6 6. Bg5 e6 7. f4 Qb6 8. Qd2
Qxb2 9. Rb1 Qa3 10. e5 h6 11. Bh4 dxe5 12. fxe5 Nfd7 13. Ne4 Qxa2 14. Rd1 Qd5
15. Qe3 Qxe5 16. Be2 Bc5 17. Bg3 Bxd4 18. Rxd4 Qa5+ 19. Rd2 O-O 20. Bd6 Re8 21.
O-O f5 22. Qg3 Nc6 23. Qg6 Qd8 24. Bc4 Kh8 25. Ng5 Qxg5 26. Qxe8+ Kh7 27. Bf4
Qg6 28. Qxg6+ Kxg6 29. Bxe6 Nf6 30. Bc4 Kh7 31. Bc7 a5 32. Ra1 a4 33. Bb5 Ne4
34. Rd3 Nb4 35. Rd8 a3 36. Bd7 Nxc2 37. Rf1 Bxd7 38. Rxa8 Be6 39. Be5 Ng5 40.
Rf2 Bb3 41. Rxf5 Kg6 42. Rf1 a2 43. Ra7 Ne3 44. Rxb7 Bd5 45. Rxg7+ Kh5 46. Rc1
Nxg2 47. Kf2 Nh3+ 48. Ke2 Nh4 49. Ba1 Be4 50. Rc4 Bb1 51. Rg3 Ng5 52. Rb3 Nf5
53. Kd1 Ne4 54. Kc1 Kg5 55. Rf3 Kg4 56. Ra3 Kf4 57. Rc8 Nf2 58. Rcc3 Ne4 59.
Rf3+ Kg4 60. h3+ Kg5 61. Ra5 Ned6 62. Ra4 Ne4 63. Rb4 Nfd6 64. Kb2 Nf5 65. Rb5
Ned6 66. Rc5 Nb7 67. Rc4 Nbd6 68. Rg4+ Kh5 69. Rgf4 Kg6 70. Kc1 Kg5 71. Rg4+
Kh5 72. Ra4 Kg6 73. Rff4 Nc8 74. Ra5 Ncd6 75. h4 Kf7 76. Re5 Kg6 77. Kb2 Kf6
78. Kb3 Kf7 79. Ra4 Kg6 80. Ra6 Kf7 81. h5 Kf6 82. Kb4 Kf7 83. Ra7+ Kf6 84. Rd7
Kg5 85. Rxd6 1-0

And the stem game:

[Event "Bundesliga 0607"]
[Site "Germany"]
[Date "2006.10.28"]
[Round "15"]
[White "Shirov,Alexei"]
[Black "Ftacnik,Lubomir"]
[Result "1/2"]
[Eco "B97"]
1.e4 c5 2.Nf3 d6 3.d4 cxd4 4.Nxd4 Nf6 5.Nc3 a6 6.Bg5 e6 7.f4 Qb6 8.Qd2 Qxb2 9.Rb1 Qa3 10.e5 h6 11.Bh4 dxe5 12.fxe5 Nfd7 13.Ne4 Qxa2 14.Rd1 Qd5 15.Qe3 Qxe5 16.Be2 Bc5 17.Bg3 Bxd4 18.Rxd4 Qa5+ 19.Rd2 0-0 20.Bd6 Re8 21.0-0 f5 22.Qg3 Nc6 23.Qg6 Qd8 24.Bc4 Kh8 25.Ng5 Qxg5 26.Qxe8+ Kh7 27.Qxe6 Nf6 28.Qe2 Ne4 29.Rd3 Qg6 30.Ba3 Ne5 31.Rd4 Qb6 32.Rfd1 Nc6 33.Qe3 Nxd4 34.Qxd4 Qxd4+ 35.Rxd4 b5 36.Bd3 Ra7 37.Rd8 Be6 38.Re8 Bc4 39.Bxe4 fxe4 40.Rxe4 Rd7 41.Bb4 Kg6 42.Re7 Rd1+ 43.Re1 Rxe1+ 44.Bxe1 Kf5 45.Ba5 h5 46.g3 Ke4 47.Kf2 g5 48.c3 g4 49.Bb4 Kd3 50.Ba5 Bf7 51.Bb4 Bg6 52.Ba5 Kc4 53.Ke3 Kb3 54.Kd2 Ka4 55.Bc7 a5 56.Bd8 b4 57.cxb4 axb4 58.Bf6 Kb3 59.Bg7 Kc4 1/2

Titu's picture

Hi Arne,

I actually disagree with you about the opening books.

But just to answer your call, the Rybka team has announced a match vs GM Ehlvest, FIDE rating 2643. Rybka will play without a proper opening book and will be black in all games.

http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=1183

Regards,
Titu

arne's picture

Wow, that's cool news, Titu. Thanks for the tip, I will be following this match with extreme interest!

Jeroen's picture

Also the comment that computers should not use openingbooks is not very interesting. They would play the same chess as the guys in the 1820's that had no clue how to start a game properly.

All we know now about opening theory is based on >100 years of human experience, testing lines, failures, successes and so on. Would be weird if your opponent before a game asks you to switch off that knowledge, wouldn't it!?

Instead of reacting furiously at these computers you could also put it into another perspective: the computer to help you to improve your repertoire. To point out your analysis mistakes. Your guide to distrust the claims in most of the written books.

Don't forget Rybka on a fast PC is over 3000 Elo, NO author of a written openingbook has that playing strength!

Frank Str?ɬ§ter's picture

The problem with turning off opening books is that you get identical games when there is no "tweaking" (code, hardware) between games.

arne's picture

@Alkele. Thanks for your extensive and thoughtful reply. I particulary like your remark about testing groups: I *would* in fact be highly interested in seeing these results and progressions. But I suspect most of these are not public at all, are they?

In my view, a computer-computer can by definition only be interesting if we get to see 100% of the moves made by computers - not 50% by humans followed by a few obligatory moves from the computers themselves. That's not a match between computers, that's a match between opening experts who program the opening library. And although I'm sure that these opening rivalries can also be quite interesting - as interesting as, say, opening discussions or surveys or whatever - I still think it's extremely misleading and cheap to advertise this as a challenge between two computers.

As you rightly point out, this is more a challenge between two teams trying to out-wit each other by all (well, all?) means possible. Fair enough, but in my opinion it has nothing to do with the fascination that I myself - and I suspect a lot of other laymen - do have for the potential power of chess computers and what they can teach us, and for artificial intelligence in general.

Marvol's picture

At least the teams behind the engines have had an active part in creating the books.

What I find more disturbing is that the engines run on different hardware. What kind of a comparison is that? The winner may be the inferior program using an inferior book but running on the best machine. What!?

Coen's picture

The computerized search for opening perfection has caused the sporting element of improving chess theory to disappear. I fail to be impressed anymore by opening improvements, knowing these can be produced by letting some computers evaluate opening lines.

What makes chess a sport that is fun to play and watch is that each player tries to make the best decision constrained by of his knowledge, preparation, experience, talent, time management, character, ...

If you look at the comments on ICC for example you see that a lot of computer people fail to appreciate this. They are blaming players for missing easy wins, because their computers tell them the evaluation drops from +2.45 to -0.2.

Eddie's picture

I have to agree with Arne, the match is for me uterly uninteresting. IMO a computer match without opening books and on equal hardware would be more interesting. But I can see how others have interest in these battles. Of course there is a race to design the strongest program, and it's only natural that the programs are getting ever stronger. But why can't these 2900+ programs teach me how to play better chess? It should not be too hard, I'm quite a weak player. Yes the they can help analys my games, they do a fine job calculating difficult positions. But when it comes to really evaluating my play, they sure suck. Why is that? How come this is such uninteresting aspect? Is this not sort of an ultimate challenge for the computer, to really teach the human to improve!?

arne's picture

@Nelson, Jeroen. As I said before, I've written extensively about the problems of computer chess already (see above for the link). I know there are different angles to the subject. But this is not the point of this article.
The point of my article, which I maintain, is first of all that the use of an extensive opening book is in my opinion too 'human' to be an equal (50%!) part of 'computer chess'. It has nothing to do with measuring the strength of computers. And secondly, that it would really be much more interesting to give computers the chance to let the public see what they make of our 500 year knowledge of existing opening theory by 'inventing' or 'improving' it themselves.
I agree that computers can find fascinating stuff in human opening knowledge, and that they play incredibly strong, but these two aspects are in my opinion really unbalanced in the commercial computer matches being held these days. That's why I will follow the Rybka-Ehlvest match with more than the usual interest!

Jeroen's picture

@ Tom

No problem! Let ChessBase hire the GM's, find novelties for 6 months. I welcome that. Preparation is part of a chessgame. Not only in top GM games, but also in computer games.

BTW, the moves in an openingbook are the result of human knowledge and computer assistance. Especially the last few years the computer part has become bigger. When a GM finds a super novelty, it is most likely the engine has found it and he only needed to work out the details.

The opening is a vital part of a chess game and for me it is simple: the one with the best openingpreparation has the advantage over other players and is therefore increasing his winning chances. So the current developments are increasing the level of chess.

Arjon's picture

The reason opening books exist is that the opening moves exert knowledge of strategy instead of knowledge of tactics. In theory, a computer can not think far enough ahead to make sense of 'knights before bishops' yet. Maybe, in the future it could.

The facts is that computers play a totally different game than humans when playing chess. There are more than enough differences to note. Endgames are a more obvious example, how many times has your engine given 1.0 to a totally drawn position in the past? Humans can't copy the tactical insight of chess computers, but chess computers can copy the insight of humans so far as openings and theoretical endgames are concerned.

Therefore, computers can't make any useful moves in the first 10 moves, most of the time. They won't be bad moves, they won't give away pieces and will probably end up with a position with heavy pieces focussed on the enemy king, but they can not yet be proven to be the theoretical best moves and are thus no more exciting than the existing opening moves.

Nelson Hernandez's picture

Jeroen, you are too venerable a book-builder to bother answering this. Let me try to take care of this problem.

Arne, in case you don't know, Jeroen is one of the most senior and well-known book-builders in the world, with over 20 (or is it over 25?) years of experience. He is the head of book-building for Rybka, the strongest chess engine in the world. Think about that before you reply.

That said, there is no question that the foundation of opening theory is human play. Humans played chess and recorded moves long before computers came on the scene. What Jeroen is saying is that computers, taking human theory as a foundation, have gone far, far beyond what human GMs have played over the board. And this theory development is accelerating every year as technology moves ahead. If you stop and think that the top engines on quad-core machines can now play at 3000-3150 ELOs you ought to ask yourself how that is possible if opening theory development wasn't an important part of the story.

arne's picture

@Alkelele, Nelson. I don't dispute that computers can find amazing novelties, that this is extremely interesting, and that we can all learn something from this. All I'm saying is that in a commercial match, especially with so much publicity and money at stake, there are also other things to be taken into consideration. In matches, we're interested in much more than cool novelties: we're interested in the very foundations of chess. Either one admits that computers are only 3150 rated if we give them huge opening libraries (and without them, they're much lower rated), and in this case one is misleading the public -or one really trusts the engine's strength and then one can have a nice marketing commercial about how strong computers are already.

The point is simply this: I've learned that many of my objections can be checked or tested daily on Playchess, on testing sites, etc. Great! But why organize a match then? Why donate this kind of money if we can all see thousands of computer matches all day long? What's the point?
The only reason to do this, in my opinion, would be if computers had something fundamental to say. Then it's really news! Otherwise it's just one of the thousands of matches that take place on a daily basis: so why the media coverage?
It's only fair that when so much money is at stake, the public gets to see the real deal, 100% computer-made moves, not 50%, and including all weaknesses and flaws of computer chess, and not just the pearls of it.

Bert de Bruut's picture

Jeroen, stating "In no way a GM game can reach that level" is, taken literally, crap. Books and databases are packed with flawless games by WorldChampions, GM's and even lesser gods, games that will stand the test of time -and engines- since no improvement is possible...

If you meant to say, more modestly, "In no way a GM can consistently play on a 3000+ level" you might have had a point, albeit still a minor one, since they can - with unlimited time to ponder, as correspondence GM Arno Nickel has demonstrated not too long ago by whitewashing Hydra.

arne's picture

You're right of course, Bert, but really, the only 'perfect game' I would be really interested in, is the game which is perfect from the first move. And this is exactly the point: we will never know this, or even be able to scratch the surface of it, as long as we keep forcing human (and therefore perhaps flawed) opening theory down computer's throats.
In my opinion, the search for true perfection, I mean mathematical perfection (meaning the Najdorf could appear a very good opening to us, but could mathematically speaking be losing) can only be attained if we constitently to let computers give their opinions on human opening moves, starting from move 1 or 2.
For all we know, 1.a3 is winning, and 1.e4 is losing. There have been much stranger surprises in science already. Why couldn't chess opening theory be another one?
Anyway, I've written about such questions already on this site. No need to repeat it here ;-)

Arjon's picture

All first moves are drawn anyway, a perfect game from move one would be the most boring game ever. The only reason computer chess is really interesting is their analysis of human chess. The nice positions that computer vs. computer matches have come up with were only interesting _because_ they began with normal opening theory.

Besides, if you look in deeper how computers analyse game positions you will have even more reason not to care about their analysis from the first move. Their evaluation heuristics (!!) are not perfect ergo there will always be differences in gameplay between engines. This means that 'the perfect game from move 1' does not exist, as there are multiple answers that all get the same 0.0000 evaluation, but which these lines are will depend on the engine!!! These will be games with (my guess) about 50 moves, as they will play to the end. Take the usual 36 moves a board and you reach 36^50 (if you think all the mates before the 50th move will be a problem, let's take 36^25 just for the heck of it). This number is so large and so littered with drawing lines that all these perfect games are boring. All that differs between engines is which of the boring lines they call the best.

Bottom line is, you can't get the perfect game, as you can not seperate one 0.000 game from another 0.000 game and which set of 0.000 lines an engine chooses to be the best is at the hand of the _human_ evaluation function.

BRUZ_LEE's picture

You're completely right. Humans are not allowed to use Opening Books during tournament play, so why should computer programs be allowed to use stored opening theory data during play.
Imagine a human player showing up with the volumes of the Encyclopedia of Chess Openings for a tournament game!

Alkelele's picture

I have to kindly disagree with you on this one. I think opening books are absolutely essential to keep this a match of this kind interesting.

Let me address a few of your points:

1) As to finding out which commercial engine is strongest, these matches serve little purpose. They are far too short. Instead, the testing groups are investigating this question in thousands of games, and we can already look up the results.

2) To find out how the engines play the opening stage themselves, users can also just turn on their own version, or he can consult the games from the testing groups (they usually play with limited books, so that the engines are on their own from move 6-12 or something like that).

3) The first game of this F-J match was dull, but that was, as I see it, mainly a result of a conservative choice of opening variation. For example, white can play instead 7.g4 when there is still lots to be explored, and the engines would have been allowed to express themselves.

IMHO, this kind of match is interesting since it pits some potentially extremely strong chess entities. The teams behind them bring everything they have: Extreme hardware, opening books with killer novelties worked out by experts, the newest and yet-to-be released beta of their program etc.

All these things cannot be "copied" in advance by testing groups. They are something unique, for a unique and limited event.

I think that's how one should look at these events: As highlights in the evolution of automated chess machines. And, playing pre-programmed moves from book is a valid part of a chess machine.

True, to get excitement, we would need opening book authors who are able to steer the games into interesting waters. My point is that, WITHOUT this cutting-edge opening battle being a big part of a match, the match is largely redundant -- we could just as well look at already available games and results from the testing groups.

Marvol's picture

I just thought this up (so it may be rubbish :P), but here goes:

It is obvious that a direct comparison between the ways a human on the one hand, and a chess engine on the other, play the opening. A human has no physically seperable memory to turn off or remove, an engine has. Humans can never play without their memories; engines can.

One solution is to play Chess960, but this does not appeal to everyone.

How about setting an (arbitrary) limit to the number of positions any engine may use in a game or tournament? The number would have to be carefully balanced to allow a fairly substantial book but without being exhaustive.

This would create a strong incentive for programmers to create a book that 'launches' the program into positions it can play well, without being able to go straight to a drawn middlegame or giving the engine all solutions on a silver platter. In short, the program would have to do substantially more thinking.

It would simultaneously reward finding 'holes' in the opponent's opening knowledge (whereas now there are unlikely to be any holes to be exploited), very similar to the way humans go about this.

arne's picture

@Marvol. I'm sure there are all sorts of degrees of 'unfairness' about the current match.
My point is, however, that apart from this unfairness, it's simply *uninteresting* to see computers reproduce existing, mainly man-made, book moves up till move 20 or 25. That alone is my argument for not allowing it. Why not let the machine decide for itself whether he considers 1...c5 is a good reply to 1.e4? Isn't that much more fascinating?

Alkelele's picture

@arne. Indeed, the testing groups are quite open about their games and results :-) (They have to in order to be considered reliable).

The best I can give is a link to Rybka's site:

http://www.rybkachess.com/

Here, under "Independent testing", there are links to the main testing groups, and you can download lots and lots of games here. For starters, you can also go to the CEGT games replay zone where there are quite a lot of high quality games (long time, strong hardware).

A word on the opening books CCRL and CEGT use: They usually use limited opening books (tailor-made for testing purposes), and they always give white and black the same book (unlike this, the SSDF group uses the programs' books, if available). Then they play a match between two engines, and add the results to the database. They are very careful to avoid any statistically flawed methods.

So have at it and enjoy :-)

As for the discussion raised by your post, I think it mainly comes down to differences in interest. I would just like to point that, often enough, we should see some interesting ENGINE battles even when they play with specialised opening books. The first game was not interesting, but it's probably like human chess: Sometimes you see a non-game where the players safely manage to enter a dull and drawish position right after the opening stage.

arne's picture

@Frank. Yes, I have thought about that too. Perhaps the 'learning ability' of the engine should be limited to the games that the engine played without book, so that it can evaluate a certain position from a previous game and then decide whether or not to repeat it?

Matt Helfst's picture

I agree with you Arne. It would be great for the spectators to see the computers play the openings for themselves and as you stated it could revolutionize human openings for the future. It would be great to see some openings other than 1. e4, 1. d4, 1 c4, and 1. Nf3

Uri Blass's picture

BRUZ_LEE said:

1)"Humans are not allowed to use Opening Books during tournament play, so why should computer programs be allowed to use stored opening theory data during play."

2)"Imagine a human player showing up with the volumes of the Encyclopedia of Chess Openings for a tournament game!"

My response:

1)Computers do not use opening Books during tournament.

They only use data that is in their memory.

2)Humans have the full right to memorize material from the Encyclopedia of Chess Openings and use the moves that they remember in tournament games.

If you think that computers are not allowed to remember because they have better memory than humans then you can also forbid using hash tables by the same logic because you can say that it is the same as humans who use notes in their search to remember what they thought earlier.

Uri

Jeroen's picture

Peter,

Turn off openingbooks!? Are you kidding? Of course not! If you would only know how some of the computer openingbooks are developing....

Computers are inventing new theory. Computers are killing existing theory. Just take a written openingbook made by humans, run though it with Rybka and after a few pages you know that you cannot trust the book at all!

There is only one thing to do: the combined efforts of human thinking and the computer analyis power of today will tell you what the exact truth is of the current opening theory. Don't believe any book. Trust your own findings with the help of Rybka.

As for the Junior-Fritz match: I agree with Alkele here, the Bd3-line is completely dull and g4! instead is much more interesting. But Fritz and Junior are not the top programs at this time and so are their books. I don't expect any interesting novelty to pop up in this match.

Jeroen

Nelson Hernandez's picture

Some misconceptions need clarification. Arne says the opening books being used are merely reproductions of human moves. Nothing could be more incorrect. This was true perhaps six, eight years ago, but no longer. Cutting-edge opening theory is now produced every day in considerable profusion by computer vs. computer games played on chess servers the world over. For the most part, computer vs. computer games are superior to human games in that they contain fewer blunders and inaccuracies than the majority of GM-level human games. More to the point, every single day there are thousands of such games played, as opposed to the relative trickle coming from GMs and IMs.

I suspect that the opening books being used contain some blend of human and computer games that is, year by year, tilting more and more toward the computer.

Finally I wish to echo Dagh Nielsen's remark about this being a competitive exhibition where anything goes. The contestants are not trying to scientifically prove which engine is stronger--that is done by CCRL, CEGT, SSDF and other websites. The contestants are trying to win the match and garner favorable publicity. In order to enhance your chance of winning you must have a competitive opening book. If you, as a human, come to a tournament or match unprepared in your memorized openings you will very often lose. Computers can simply be considered to have vastly superior resources in the field of opening theory and if they don't utilize all the weapons available they are needlessly handicapping themselves.

arne's picture

@Uri, you might be interested in my essay about the question you raise in your points here: http://www.chessvibes.com/?p=352&lp_lang_view=en
There is much to say about this subject, but as you can see from the title ('thoughts about an unequal battle') I agree with you that you cannot, as we say in Dutch 'compare apples with pears'. But still, when two computers are playing against each other, this becomes somewhat easier, don't you agree?

arne's picture

Well, Jeroen (by the way, it was me who wrote the article, not Peter!) and Nelson, thanks for your interesting replies. I knew I would learn something after writing the article! ;-)

@Jeroen. Of course it's interesting to know if a computer has found deep, amazing novelties in some topical lines of the Najdorf or the Slav defence. You're looking at opening theory then from the perspective of the practical player. Perhaps I'm too philosophical for that. To me, the only interesting aspect of these strong computers, and especially their public display, is to find out if they can find some deep, amazing novelties at move 2 or 3. Or perhaps at move 1. After all, who says the Najdorf is the most critical line in chess? Who decides that? It surely is the experience of humans over the past decades, but who knows what computers will find out? Look at what Deep Fritz did against Kramnik in that last match game (the Sicilian with Re3). That was a move that was produced (I assume! perhaps I'm wrong about this) because the computer was 'out of book'! Still, it was the most interesting move of the match, a completely new, to humans ridiculous-looking opening *idea* (note the difference with a single *move*) as early as move 10! Why should this be a priori impossible at move 2 or 3? Would this be boring? I don't think so!

@Nelson. Yes, you're right of course - you're just more realistic (and, perhaps, more cynical) about it. I just don't like this whole idea of publicity: in my opinion it's misleading and only possible because of the idee fixe of the big public that we're still seeing something to do with real, scientific 'research', something to do with A.I., with the progression of mankind in the grand scheme of things, etc. But of course, we're just watching a funny competition. I agree you can shurg your shoulders about it, but at least it wouldn't generate such an interesting discussion! ;-)

arne's picture

@Jeroen, I was thinking a bit more about what you said in your comments. I have to admit that I quicky became very confused...

First, you write: "Computers are inventing new theory. Computers are killing existing theory. Just take a written openingbook made by humans, run though it with Rybka and after a few pages you know that you cannot trust the book at all!"
Then, however, you say: "[Without opening book] computers would play the same chess as the guys in the 1820?¢‚Ǩ‚Ñ¢s that had no clue how to start a game properly."

This all seems utterly contradictory. How can computers be killing existing theory, and at the same time play chess like the guys in the 1820s without existing knowledge?
You can't have it both ways: either computers have very interesting fundamental things to say about chess openings - and in this case, what's more natural than to start with this on move 1? - or they *don't* have anything fundamental to say about chess openings, and in fact then we have to agree that it's not the computers that are killing opening theory. It's humans making smart use of computers (and I completely agree that definitely is a very nice thing for humans). It's a whole different story.

Let me put it another way. You mention these guys from the 1800's not having a clue how to start a game properly. Well, let's take some openings these guys actually played. What do we see? Sicilian, French, Queen's Gambit, Ruy Lopez ... etc.etc. How can you say these guys didn't have a clue? And do you really mean to say that these openings lead to uninteresting games when played by computers?

Well, I guess you don't mean that. But what do you mean, then? Do you mean that, currently, chess computers actually can *not* find out these openings themselves, like those 1800's guys did? Well, OK, but then this is a clear weakness in computer strength and their opening understanding, and why not expose and investigate it fully and try to improved it, rather than 'hide' it by letting them find cool novelties in the Marshall or the Najdorf defence at move 20? Isn't that, from a scientific point of view, a much more interesting task to take on?

The fact that after 1.e4 my Rybka doesn't list 1...c5 among the top 5 (!) alternatives even after a considerable thought, is perhaps an indication that Rybka doesn't think so highly of the Sicilian defence. It tells programmers perhaps something about how it evaluates the centre, or whatever, I'm no chess computer expert.

But why force the poor program to play this opening in computer matches? Apparently this shows a lack of trust in the enigine's own findings. Isn't this strange? Instead of ridiculing human weaknesses by saying that they write bad opening books and that they're full of mistakes, perhaps it's better to focus on the weaknesses of the engines. Anyway, looking at chess openings only from a human, practical perspective seems to me a rather small-minded approach to the challenges of computer chess and A.I in general.
But then again, perhaps I'm just being too philosophical about all this :-)

Alkelele's picture

@arne. Well, I am afraid that if you make engines play without opening books, you will 99/100 times get uninteresting opening play (this can be verified in the engine-room on Playchess). Usually, they will sooner or later make what is already known to be an inaccuracy, and the game and any post-game analysis will usually confirm this.

I suspect Jeroen's and Hernandez' point is that, in order to get interesting opening play, you have to use whatever data is already available, and then build on top of that. So the book-makers do exactly that -- they analyse new opening ideas, using both engines and their own human creativity in the process.

The end result can be extremely interesting games. As an example, I can point to the recent 6th Freestyle qualifier. There are some games and novelties there that are simply mind-boggling, and the ratio of "interesting opening discussions" taking place there is probably much bigger than in human tournaments.

So, the point is, to give the spectators some interesting and new stuff to enjoy and wonder about, it is most probably necessary to use books made by humans (+their engines). Otherwise we would just see the deep plate reinvented over and over.

Please note, when you want to use engines and engine matches to provide us new revolutionary ideas, this is already happening -- these engine ideas are noted and checked by the book-maker (or Freestyle player) and he fills his book with them, in case they are any good.

I am definitely not saying that the first F-J game was interesting. I just don't see that removing engine books will solve THAT problem. A better way to solve it would be for the F and J teams to try some more interesting opening lines in the next game, and perhaps even reveal a new idea or two :-)

Finally, I do very much agree that it is very interesting to find out what new ideas and insights the engines can give us about old opening positions. I just fear that we will not see much of this if the engines have to play without book from move 1.

Dom77's picture

Just limit book opening to 10 moves just like Hydra has done in the past then we get some decent novelties not these 20 odd opening moves zzzzzzzzzzzzzzzzzz

peter's picture

From Budapest I'm following this discussion with interest. To some extend I agree both with Arne and with the Nelson/Jeroen side. Arne has a good point in his general claim that there's something wrong with a match being played with so much money and media attention involved, where for a very big part of the games the computers aren't actually playing themselves. And I agree with the Nelson/Jeroen side that it's very interesting to see computers cracking existing opening theory, BUT this is different thing, and in my opinion not the main idea of this computer match. The idea is to see which program is playing the best chess. So I'd say: give them exactly the same hardware, no opening theory, no tablebases, just the software that tells them how to play chess. But well, I'm not a computer chess expert either. Perhaps people aren't interested in comparing the software at all, perhaps different things are at stake, I don't know.

@Nelson
I can inform you that Arne knows very well who Jeroen is. Your "Think about that before you reply" really sounds like an argumentum ad verecundiam to me...

Bert de Bruut's picture

Some of the engine-fanboys posting in this thread obviously have a grossly exaggerated opinion about the "theory-building" by engines. Indeed, engines may sometimes find strong novelties - but only after twenty or so "human" moves have been played first... Left to their own devices, engines will have a very hard time to, say, ever slay a Dragon or "re-invent" the Benk?ɬ? gambit.

So when looking for directions how to play the Najdorf or Marshall, engine builders obviously are obliged to turn to the games of Vishi Anand and correspondence chess players like Joop van Oosterom. Immodest claims that human theory is made redundant by engines on a daily basis, are a flagrant misrepresentation of what is actually going on: that the (re-)evaluation of opening theory is still in human hands, much helped nowadays by the calculating powers of engines indeed, but hardly if at all directed by them.

Nelson Hernandez's picture

@Bert: I am not saying human involvement in theory is obsolete or redundant. All I am saying is that modern books are the sum of all human and engine-developed theory, and empirical engine outcomes are coming to play an ever-bigger part in opening theory.

It would be very interesting to see an Anand play openings from memory and face a book put together by Jeroen. The empirical criteria would not be the final outcome of the each game, but rather the first computer evaluation following Jeroen exiting his book. I'm willing to wager that over the span of 100 games Jeroen would establish a clear evaluative advantage with both white and black, looking at median outcomes only. It might only be +0.15 or +0.20 but that is pretty significant out of the opening when you're playing with Rybka and using top hardware.

Nisse's picture

hi.

I agree with the writer here, it would be much more interesting with a program vs program match that didnt involve opening books. or actually I think that the most interesting thing would be a long match, 24 games or so, with half of the games being played with opening books and half without.

the weak point in all chess programs is their ability to "think" long term, or rather their inability to. Im guessing the position calculating has to be directed somehow for the program to be able to calculate in directions, and that is its strategic ability. so in a way using no opening book is the ultimate test of a chess program, on the area where it is at its weakest and is in no way like a human brain.

but then it is also interesting to compare chess programs in terms of raw calculating power. then its fitting that the computer has the opening book to get a helpful push in the right direction, basicly all the way into the middle game I guess. for that the hardware obviously has to be the same, otherwise there will be no point.

if nothing else it will be much more fun to watch a program vs program match that is without opening books. or well, its not fun to watch the same games over and over if that is what it sums up to, but maybe limit the opening books to 3-4 moves with its colour? or even better, like someone here suggested, allow them to memorize the games played previously in the match, if that would be possible from a programming point of view.

arne's picture

@Alkelele. Yes, let's give a kiss to the book programmer who analysed 27.Qxe6 and put it in Junior's library.
So, we found out that the Poisoned Pawn can produce exciting games. What else is new? We can also find exciting games between 1700-rated amateurs who play the same line.
But why is this suddenly worth 100,000 dollars? And what does this have to do with computer chess? Maybe you think I'm sarcastic, but in fact I'm dead serious: come on, 59 moves, 27 of which are in the database. How low can you go?

Alkelele's picture

Hmm, Junior played 27.Bf4, improving on Shirov's 27.Qxe6. I doubt Junior was in book at move 27, but anyway, it is easy (for some engines) to see that 27.Bf4 is better.

So, there you have it! An engine or an engine+book produced an interesting novelty!

And the rest of the game was very exciting, an unbalanced ending where it was not clear if white could win, despite being two (!) exchanges up. High quality chess in a theoretically interesting line, what more can you ask for?

I have a serious question for you: All the points of criticism that you aim at engine matches involving books, are they not also valid for human games? Why give 100,000 dollars or more to humans, when all they do is pretty much repeating old lines/games and insights (like, Poisened Pawn being able to produce exciting games...)?

Why is the Shirov-Ftacnik game more interesting than the Junior-Fritz game? (If you answer that it was the first one, then, does that mean that Motylev-Anand and Anand-van Wely were boring?)

I'm fine with you finding it interesting to see how engines would play without book -- I am just trying to point out that other people find it interesting to see how engines play WITH book. It doesn't have to be mutually exclusive.

arne's picture

Simple answer, Alkelele. Humans actually think that these lines are critical, so they play them. What's more natural?
But engines don't think that at all: it's the programmers that decide it for them. If it was up to the engine, it would never play a Sicilian, let alone sacrifice a pawn on b2.(You can easily test this at home.)
Therefore, it's completely unnatural that we see computer games in these lines. That's the difference.
And yes, I do think the Shirov game is more exciting. Why? Well, it has to do with a non-chess aspect of the game: it's the human drama, the usual human dillemma's, human doubts and fears (should I or should I not play this line against this guy? Did I really calculate everything correctly? Am I thinking too long for this move? Are my intuitions correct? etc.)
Computers don't feel any of these these doubts, these fears, in short, they don't experience the OTB drama. It's all very cold and sterile. I guess knowing that there were no emotions involved, no doubts, etc., just kills looking at chess games for me.

Nelson Hernandez's picture

I honestly don't understand this discussion. Arne is criticizing computers for being computers. Don't you see, computers are a human creation, and they use human knowledge in every aspect of their hardware and software? Their principal attribute is the ability to follow complex rules and calculate at speeds that humans cannot match. So what exactly is this argument about, anyway?

It's funny, this morning on the Chessbase site they describe a move in the second Fritz-Junior game as "a novelty" when in my database I see the position having been played over 50 times, always by chess engines, and the line that was followed extends several moves further, oddly enough transposing into a human game between Chinese players. That is what I am saying when I talk about chess theory being advanced by computers. Opening books are the blending of human and computer games, and advanced opening books further blend in analysis, novelty moves and other features.

What I am saying is that a chess engine is not just a stand-alone program. It is a program working in conjunction with an opening book and endgame tablebases. These things working together are what give computers huge advantages over humans. There is no reason to disaggregate them.

Jeroen's picture

Frankly speaking I am always very amused by (chess) people being sarcastic about computer chess. It's a kind of 'denying' which is very humanlike, by the way. Instead of accepting the facts and try to make positive use of them, they get verbally agressive and try to put things in the negative. In one century they acted the same way when there appeared cars for the first time :-).

The thing that makes computer chess AND tournaments like Freestyle so interesting, is that the combination of a superbly prepared openingbook plus computer program (and even added a human) produces >3000 Elo chess. And yes, with blunders and weak moves also. This is a totally new dimension in chess. In no way a GM game can reach that level.

And guys, I am not AGAINST human games, on the contrary. But there is a very rich field available that can contribute to rise the level of chess, the level of opening preparation. It's called computer chess. Face the fact, accept it. Use it! It is as easy as that.

tom's picture

well how would for instance the rybka team find it very honest if chessbase would accept the challenge and then would hire a GM-team that for 6 months would research all kind of novelties (with whatever engine they like). Then it is about how deep your pockets are, not about how strong your engine is (and yes I'm aware even then Rybka could win but fritz of whoever would probably do beter that normal due to something that has nothing to do with their strength).
A match would be interesting if both engines would be allowed a modest openingbook which allow them to get into the game without any completely stupid positions. Or, if possible each team could be allowed to insert x games they find relevant (or perhaps give boh computers the same games) and let the program use this information.

For me it's about who the stronger program not about who has the better support.

Latest articles