Columns | April 05, 2010 16:45

Kasparov, Donner and the infinite regress of knowing

Infinite regressLast week, in an important team match, my opponent played an innocent sideline against my pet-opening. I had prepared quite well for this encounter, but had not looked at this particular variation since it is so rarely played. Over the board, I knew I had once looked at this line in detail, and had also found a nice way of dealing with it - but to my chagrin I couldn't remember it now.

Slighly altered version of the Homunculus Objection adapted by Dave Cantrell, originally published in Smithsonian 16 (1) (April 1985):97.

For some minutes I tried to retrieve the lost knowledge from the depths of my memory, but to no avail, so I decided to be practical and play a natural move instead. Further on in the game, in a normal middlegame position, instead of just playing some natural move, I suddenly saw a funny little intermediate move, weakening my kingside but weirdly complicating the position. I briefly started calculating the consequences of the move and I started to like it more and more. After some time, since I couldn't find a clear refutation, I played it.

The funny thing is that all the time while I was contemplating this crazy little move, I was totally aware of its 'ugliness', of how utterly 'unnatural' it looked and how unpositional its foundations really were. Despite this, I decided it was worth the risk and offered interesting fighting changes. Unfortunately, it turned out the move was rather easily refuted and I was left with a wrecked kingside, resulting in a zero for the team. (Though we did win the match.)

These situations are examples of a phenomenon called metacognition, or 'knowing about knowing': I knew I knew a particular opening line, but I just couldn't remember it. Likewise, in the middlegame, I knew my weird idea was against all positional rules, but I played it anyway.

A famous chess-related anecdote involving metacognition was once described by Tim Krabbé is his story A Walk with Kasparov. In this story, Krabbé describes how Garry Kasparov, having just lost to Jeroen Piket in the last round of the VSB tournament in Amsterdam, starts talking to him after the game. Kasparov tells Krabbé he had prepared the novelty he played in the game, but then to his horror couldn't remember the lines:

I ask him: 'Do you mean to say that Re4 might be in your computer and you forgot about it?'
'Maybe, maybe,' he says. 'I'm just curious to know. Are you curious too?'
'Yes I am.'
'Then come to my hotel and we'll check.'

Krabbé joins him and Yuri Dokhoian to Kasparov's hotel room, and watches as Kasparov opens his laptop, expecting to find the idea behind the novelty 19...Re4 hidden in his database. But even now, Kasparov can't find his forgotten home preparation, which Piket refuted with the reply 20.Bg3!

He is desolated, but he cannot find 19.Na4, let alone 19...Re4. He's absolutely sure Na4 is in one of his computers somewhere, but it doesn't seem to be in this one.


Garry Kasparov in September 2009 in Valencia

Interestingly, Krabbé kind of suggests that Kasparov may have been confusing lines and that he never analysed the move 19...Re4 in this position in the first place. Put differently, Kasparov was wrong about being sure he'd forgotten what he once knew! Of course, the interesting question is why, if he didn't remember the exact line, Kasparov had played the move anyway? When he played 19...Re4 he obviously felt the idea behind it must have been 'at the tip of his fingers', but how could he be sure he would retrieve this lost knowledge during the game?

There are several possibilities. Perhaps Kasparov had some experience in retrieving lost knowledge (after all, he does had a photographic memory) within reasonable thinking time. Or he just had to remember the precise move order and everything would be allright again. Or maybe a little distraction by walking around would be sufficient to get his mind back on track again. In my own experience, such 'aha!' moments usually come only days or even weeks after the actual event - not minutes.

In a recent blogpost on metacognition, Johan Lehrer of The Frontal Cortex describes the familiar 'tip-of-the-tongue' moment when you just can't think of the name of an acquaintance:

What's interesting about this mental hiccup is that, even though the mind can't remember the information, it's convinced that it knows it, which is why we devote so many mental resources to trying to recover the missing word. (...) But here's the mystery: If we've forgotten a person's name, then why are we so convinced that we remember it? What does it mean to know something without being able to access it?

The larger question is how the mind decides what to think about. After all, if we really don't know the name - it's nowhere inside our head - then it's a waste of time trying to find it. This is where metacognition, or thinking about thinking, comes in handy. At any given moment, we automatically monitor the flux of thoughts, emotions and errata flowing in the stream of consciousness. As a result, when a name goes missing we immediately analyze the likelihood of being able to remember it. Do we know the first letter of the name? Can we remember other facts about the person? Are we able to remember the first names of other acquaintances from high school? Based on the answer to these questions, we can then make an informed guess about whether or not it's worth trying to retrieve the misplaced memory.

The interesting thing in the Kasparov example is, in my view, that Kasparov did not remember correctly. As said, he was sure of something he shouldn't have been sure about. He was too confident of himself, perhaps: after all, someone like Kasparov has to remember such an awful lot of variations that inevitably, something goes wrong in his head from time to time. Maybe he would have been helped by something Jonathan Rowson describes in his book Chess for Zebras (2005): unlearning. Say what?


Jonathan Rowson in November 2008 at the Dresden Olympiad

Unlearning is really a way of of constantly looking at the baggage you bring to chess positions ans trying to work on the baggage that is most obviously problematic. It is also a way of trying to look at chess positions with fresh eyes, as free as possible from prejudices. When you succceed in doing this, you start to see the prejudices as prejudices, and not as absolute truths, and that's when real improvement becomes easier.

Basically, according to Rowson, Kasparov should have looked at the position after 19.Na4 with 'fresh eyes' and realized that even if he had once analysed 19...Re4, he didn't like it after 20.Bg3. He should have unlearned what he thought he had learned, so to say. He should have ignored the voice in his head telling him to play the 'analyzed' move Re4.

But wait a minute ... does ignoring the voice in your head also imply that in my own above mentioned game, I was actually right in playing that ugly-looking move', ignoring my intuition and going for concrete, 'fresh' complications? Stubbornly, consciously going against your intuition and your natural sense of reality was often subject of the chess writings of J.H. Donner. Here's how he described his thoughts during a game against Milic in 1950:

Until this moment I hadn't used much time to think, just fifteen minutes on my 19th move. Everything went fine, I saw everything and felt contempt for my opponent. But now a paralysing doubt overwhelmed me. Suddenly, I saw white pieces coming from all sides, while I was clearly aware that White didn't have a realistic chance. 'Stay calm,' I said to myself. 'You're winning.' But it didn't help. I couldn't calm down. I did see the best move in the position, h5, and I wanted to play it, but I touched my rook and played Rg8.


J.H. Donner (6/7/1927 - 27/11/1988)

Whenever I read fragments like this (Donner describes many of them), or whenever I experience them myself, I become very pessimstic about books teaching you how to play chess. If even players like Donner and Kasparov (occasionally) couldn't handle the metacognitive voices in their heads telling them to play moves they only thought they remembered, or something they knew was bad, how does improving your chess knowledge by reading chess books actually help during a game?

During a game, you may be perfectly aware that a certain move violates some deep positional principle, but if another voice in your head tells you to 'unlearn' that learned principle and look at the position without preconceived knowledge (a phenomenon John Watson calls 'rule independence'), the result is the exact opposite. How to decide which voice is right? How to decide when to follow a rule and when to violate it? The problem of improving chess players is not that they don't know the definition of a weak square, but when the weak square is relevant and when not.

And even if, like me, you know that you're vulnerable to this kind of metacognitive confusion, it isn't easy to 'switch off' this mode of thinking just like that: after all, this only hands the question over to the next meta-level. Now you're ignoring the fact that you're ignoring that voice in your head telling you that you know a certain move must be played... And once you become aware of this, it starts all over again. How to avoid an infinite regress of metacognitive levels, leading straight to insanity?

I'm sure you'll say that the answer is to simply stop worrying about these different levels of cognition. Just focus on the chess! Does a super talent like Anish Giri think about metacognition? Of course he doesn't! He just plays the right move and that's the end of it! But most chess players are not super talents, and even a self-conscious grandmaster like Donner found it easier said than done. Heck, even Kasparov sometimes couldn't avoid this confusion in his head. At least for some of us, the problem is real.

Jonathan Rowson wrote that "if you want to become a better player, you need better habits, and you cultivate better habits through training", echoing Harvard psychologist Jerome Bruner's observation that "knowledge helps only when it descends into habits." Maybe they are right. Maybe training is the key to everything. Big like most chess players, I have never received any formal form of training, nor do I have time for it now. How to compensate this? Will we amateurs ever be able to make up for this lack of excercise? Or will any chess improvement book we read only increase our confusion?

Perhaps it is our fate that we'll always feel a bit like Faust, who during his quest for worldly knowledge, said:

Two souls, alas, are housed within my breast,
and each will wrestle for the mastery there.

If you don't recognize this feeling, make sure you'll forget this article right away - before it's too late.

Share |
Arne Moll's picture
Author: Arne Moll


Thomas's picture

Arne, you write that you couldn't remember your preparation (for this specific sideline). The report on your club homepage - to which you link - states "Inspired by his own preparation, Arne played - g5 ... which had more disadvantages than advantages and Pieter won easily." [Geinspireerd door de eigen voorbereiding speelde Arne g5 , hetgeen meer na als voordelen met zich meebracht waarna Pieter de partij bekwaam naar zich toe trok.]
Who is right?

Jo snow's picture

You must have unlearned how to write in the past few weeks. Last few articles have been superb!

Nemozyne's picture

Much effort expended in rewinning misplaced games through metacommentary post factum.

Rob Witt's picture

Great article and a brilliant last sentence!

bas1191's picture

An intriguing essay on metacognition as a snake biting its own tail which, I agree with Arne, eventually spirals into insanity - or rather the inability to conclude and decide. Maybe the thought patterns you described have to do with what people use to call intuition? I guess it's no coincidence that Donner, whom you quoted, in his writings focused so much on intuition, which he argued to be the fundaments of chess playing capacity. The interesting point in this respect being that Donner in one of his other early columns advocated rejecting all efforts to learn chess (the essence of chess is that there are no rules to be followed). In other words: denying the usefullness of (rationally) training chess. Which ofcourse himself he effectively never did :-)

Jens Kristiansen's picture

Yes, Rini Luyks. And Bent Larsen was once (for who know which time) by a jourrnalist asked if to him chess was an art, a sport or a science? His answer: "None of them - it´s a game!"

Rini Luyks's picture

I agree with David Bronstein, who once said (incredible, but he did!): "It's just a game!"

Frans's picture

Every good professional (at least in art or sport) must have at least 1000 hours of good practicing and training. In that way the brain will not be distracted anymore by thinking about thinking, or by disturbing emotions. It then just does what it has to do.

I see no dilemma in this anyhow. And what has Faust to do with it? He had other and more important problems than yours. Or have you sold your soul too?

test's picture

In that way the brain will not be distracted anymore by thinking about thinking, or by disturbing emotions. It then just does what it has to do.

I think it's the other way around as Arne Moll illustrated with the example of Kasparov. We can't escape our brains and the way our brains work so the distraction is unavoidable, it's more a matter of how much we let it affect us.

I had to google what Faust had to do with it: Faust, a medieval physician, lawyer and theologian who has become frustrated with the emptiness of conventional learning and religion, and who longs for a more direct communion with the real knowledge of nature. Conventional studies having failed him, he turns to magic. He invokes the Spirit of Earth, who shows him even more clearly his divided divine/earthly nature.

"Two souls, alas, are housed within my breast,
And each will wrestle for the mastery there,
The one has passion's craving crude for love,
And hugs a world where sweet the senses rage;
The other longs for pastures fair above,
Leaving the murk for lofty heritage."

Rick Massimo's picture

One of Rowson's deadly chess sins is "thinking." He quotes a youth coach (I don't have the name because I don't have my copy with me) who says something along the lines of "When I go over one of my students' losses, I eventually come to a bad move, and I ask, 'Why did you play this move?' Invariably, the answer starts with 'Well, I thought ...' and that's when I say 'Don't think. Look.'"

Whenever you're remembering at the board, you're not even thinking, much less looking.

When I pick up a new opening, there comes a point where I may "know" it better than I did before, but not only do I not score as well as I did when I didn't "know" it as well, but I score better when I take a punt and play an opening I "don't know." Because I'm looking instead of thinking and remembering.

You fill your head with as much knowledge as possible, and during the game you let it go and just look. It's a slow process, but eventually, your "nonthinking" brain is much stronger than your "nonthinking" brain was a few months ago. Kind of like exercising, where you're weaker right after a workout than you were before, but eventually the muscles build.

shane's picture

Agree with Jo snow

test's picture

I only play dubious moves if I have a plan in mind that compensates for it. If I don't see the compensation, I don't play it. I don't have trouble not playing a move even if I'm sure it's possible if I have forgotten the follow-up.

In general.

Chess rules are based on statistics. The chance of a random move being correct will be greater if that random move happens to adhere to certain general rules.
For example bringing the queen out very early will be bad in the majority of cases. So instead you play a more sound move like developing a knight to f3 and your odds of having avoided catastrophe will have greatly improved.

Risky, unsound, dubious looking moves are only possible if you know the correct follow-up. (Usually an extremely narrow path between many losing alternatives.) So if you don't know or see the correct follow-up you are taking huge chances, placing bad bets.

But you're always juggling odds against each other. For example in a bad position after the sound move the odds of the outcome may be 50% draw, 50% lose. After the risky move the odds may be 33% draw, 33% lose, 33% win. (Playing for three results instead of two.) So you added the possibility of winning and decreased the odds of losing. But those odds might be different: 75% lose, 25% win. Then what do you do? (It doesn't help that you can't be sure of these exact numbers even though they point in the correct general direction.)

And as an amateur I am also weighting two other values against each other: wanting to win and wanting to have fun. Playing a boring move may increase my odds of winning but will decrease my odds of having fun. So the less serious the game, the less compensation I need to play unsound moves. I'll take much bigger chances in blitz just for the fun of it. :)

chandler's picture

Arne, you probably looks like you need a break from playing, reading and writing... that last is just a tad below your usual level; but it's been some time like that.

chandler's picture

and reading my sentence, I probably shouldn't write at all....

test's picture

I'm always fascinated by the way our brain works and how our perception of reality is just a flawed interpretation by our brain beyond our conscious control. Good enough to survive, but not accurate by any stretch of the imagination.

Of course the whole subject of "reality" is not clear cut. From Wikipedia:

... there simply and literally is no reality beyond the perceptions or beliefs we each have about reality. Such attitudes are summarized in the popular statement, "Perception is reality" or "Life is how you perceive reality" or "reality is what you can get away with" (Robert Anton Wilson), and they indicate anti-realism — that is, the view that there is no objective reality, whether acknowledged explicitly or not.

Optical illusions perfectly illustrate this:

This is Not a Spiral.

Where is the white knight?

Peter Doggers's picture

Agree with your comment, but I'd leave out the last five words. ;-)

Arne Moll's picture

@Thomas, well it's both true: I saw that idea during my preparation, but in a totally different variation! So yes, it was inspired by my preparation, but it was far from actual, concrete preparation.

@test: If you haven't read Douglas Hofstadter's Gödel, Escher, Bach and I Am A Strange Loop yet, go buy them now!

T. Goto's picture

Interesting as always. My humble opinion of this matter is that we need the synthesis of both thinking and experience. (Maybe it is not a coincidence that GM Rawson is from the country of Hume.) And thus,

'Concept without experience is empty; experience without concept is blind' (Kant)

Peter Doggers's picture

"The problem of improving chess players is not that they don’t know the definition of a weak square, but when the weak square is relevant and when not." This is exactly the problem I mentioned to Mark Dvoretsky, last Wednesday in Amsterdam. I asked something along the lines of "So often do we recognize different little rules, like 'don't move your pawns on the side where you are weaker', or 'when being up material, exchange pieces, when being down, exchange pawns', but how on earth do we decide which mini-rule is the most important in a specific position?" The answer of the great trainer was both astonishingly simple and amazingly helpful: "When training, think about rules. When playing, don't. Just try to find good moves." He added that these good moves should be found and played based on your intuition, and your intuition should be improved by training - playing through games analysed by great players, and reading about all those mini-rules. Just forget about all that when you're sitting behind the board.

Thomas's picture

@Arne: Did -g5 work in the other variation? I presume that you analysed it more thoroughly, maybe with help from engines ... . But maybe you don't want to say too much, as this wasn't the last time you play your pet-opening :)

Eduardo's picture

somebody can put the Kasparov's game here ?

Arne Moll's picture

@Peter: Interesting, though it sounds suspiciously like the 'don't think of a pink elephant!' advice. I think when your entire evaluation system is based on rules, it's hard not to think about them, but who knows.

@Eduardo: You can see the game here.

Peter Doggers's picture

I'm convinced that it's about "switching on a tactical modus" as soon as you sit behind the board. Think in moves and variations only. I understand it's very difficult for us amateurs, but I also think it's good advice in the long run, also for someone like Merijn, who understands chess very well but probably mainly fails tactically against GMs. (Not sure, just an impression.) It's about getting into the flow of not thinking about that pink elephant, but just about moves. It's the same flow as not thinking about how (well) you're serving during a tennis match - no, just serve. It would be an interesting experiment to try to maintain such a flow as long as possible and perhaps an "interval training", like I'm doing at the moment with running, could be a way to deal with it. Focus on moves for a minute, then relax a minute, then focus on moves for a minute, etc. At some point one should be able to concentrate (another way to describe it) for ten minutes in a row and solve very difficult problems!?

Arne Moll's picture

Peter, perhaps I'm missing something here, but in the example of my game, that is exactly what I did! I was completely in tactical modus and I did only think in moves (g5!) and variations only. My conclusion (based on calculations - albeit flawed ones) was that it was OK but I missed a simple reply after which it turned out my position was in ruins from a positional point of view. Somehow there's an intermediate step which I missed, but I'm afraid this step turns out to be 'well, that's to be able to calculate variations correctly, and evaluate them correctly, and that's just talent!' ;-)

noyb's picture

I believe that what lends to the situation described is the fact that in some positions there is more than one move that leads to victory, and in other positions there is more than one move that leads to just a draw, and still others where there are several moves, all of which only ultimately lose. This situation creates confusion!

T. Goto's picture


I think Kramnik would agree with Dvoretsky. I remember Kramnik saying: to become a good player, you must somehow absorb the whole of chess history. I was struck by particular expressions he used: 'somehow' and 'absorb'. My guess is that Kramnik consciously and purposefully studied and trained by studying great games and principles, yet, as a great payer, these knowledge became his second nature. It is not to say that his second nature is not infallible, or he no longer need to train and study. He is known to be a hard worker. A great player like Kramnik demonstrate the synthesis of conscious part and unconscious part in greater degree. I think that's the difference of mastery. I think the quoted comment by Dcoretsky points to that. Thank you for your input. It's nice to think about those things.

Luca's picture

Great article!
The same happens in finance when you trade markets. You know you must take a position but then you can't remember why and you second guess yourself.

This also has something to do to with the "paradox of choice" (see the book by Barry Schwartz). Our brain is a 10000 yr BC brain while our tech is 2000 AD tech (See Nassim Taleb's example on Homo Sapiens vs. Lion) . We cannot work out so many informations , this is when our metacognition comes into play.

After all having so many variations and choices is not a proxy for happines or best decision taking.

Think the site needs more articles with this interdiscipline attitude.


Peter Grønborg's picture

Another great article, Arne! As I've said before, I like it very much when the concept of improving players is touched upon and the whole meta-thing is intrigueing me wildly :-)

Ted Teodoro's picture

I wanted to say something, but I couldn't find the words and I ultimately forgot my point. I know it was good though.

Latest articles