Reports | January 17, 2013 10:40

More on the alleged cheating case at the Zadar Open

In a week when the biggest story in the mainstream media is about Lance Armstrong possibly admitting the use of doping to Oprah, the chess world has its own cheating story that seems to be dragging on. Associate Professor of Computer Science and Engineering Ken Regan made an analysis and wrote a piece that addresses the question "What constitutes evidence of cheating?" Meanwhile, the Association of Chess Professionals (ACP) has started an open petition against cheating.

The discussion about cheating in chess has a long history, and seems to be showing highs and lows like waves in the sea. After the alleged cheating incident at the Zadar Open the subject is on a lot of people's minds again.

To remind you: the story was about Borislav Ivanov, an untitled player rated 2227, who scored 6/9 and a 2697 rating performance which included victories against GMs Bojan Kurajica, Robert Zelcic, Zdenko Kozul and Ivan Saric. Arbiters searched Ivanov but no evidence for direct or indirect help by a computer chess program was found.

It's clear that many chess fans feel strongly about this topic: the article has generated 264 comments already. In the very first, the famous chess columnist Leonard Barden wrote:

Either:

1 Borislav Ivanov is probably the first adult (as opposed to a junior talent) with a confirmed low rating ever to achieve a 2600+ GM norm performance in an event of nine rounds or more, playing highly rated opponents throughout. (...)

Or:

2 Borislav Ivanov is the first player ever to successfully cheat at a major tournament over multiple rounds without the cheating mechanism being detected.

In the comments that followed, many seemed to agree with Barden and some of them pointed out a high correlation between moves suggested by strong engines and the ones played by Ivanov.

Five days after we reported about this, the story was picked up by Chessbase as well, and this prompted Bulgarian IM Valeri Lilov to do his own analysis. He posted his work on YouTube – a video of over an hour – in which he speaks of "the cheating incident in Zadar". Lilov also comes to the conclusion that too many moves are similar to the computer's suggestions. (A few days ago Lilov posted another video in which he reacts to feedback.)

No proof?

The problem here is of course that there is no hard proof. Ivanov was searched, but nothing was found. Therefore, the big question is: what constitutes evidence of cheating, when no electronic device is found and someone plays 400+ Elo points above his level?

This is not a new question either, and in fact it's something that Ken Regan, Associate Professor at the Department of Computer Science and Engineering of the University at Buffalo, has been paying attention to for years. Regan has created a model that falls broadly into the category of predictive analytics. As explained here, the model aims to calculate probabilities for the moves — based on the skill of the player making them — that enable predicting means and variances for large enough aggregates of moves. The rate of agreements with a computer is one stat he can predict — for a non-cheating player. The rate of blunders is another.

The Regan test

After he found out about the Ivanov case, Regan decided to run tests. It took him two days to run the main test with two top programs, Rybka 3 and Houdini 3, run several supporting analyses, and then run his statistical analyzer. He spent another week writing a report (PDF here), which he sent to two officials of the Association of Chess Professionals (ACP), Emil Sutovsky and Bartlomiej Macieja. Regan concludes:

My model projects that for a 2300 player to achieve the high computer  correspondence shown in the nine tested games, the odds against are almost a million-to-one. The control data and bases for comparison, which are wholly factual, show several respects in which the performance is exceptional even for a 2700-player, and virtually unprecedented for an untitled player.

Regan addressed the ACP with two questions:

1. What procedures should be instituted for carrying out statistical tests for cheating with computers at chess and for disseminating their results? Under whose jurisdiction should they be maintained?
2. How should the results of such tests be valued? Under what conditions can they be regarded as primary evidence? What standards should there be for informing different stages of both investigative and judicial processes?

Enough proof?

The danger with analysis like Regan's is that we might be damaging our sport even more than the cheaters are doing. At some point nobody can score an exceptional result anymore without being frowned upon or simply being accused of cheating. This point was also mentioned in the long thread on Slashdot about this story:

Lance Armstrong was initially judged by the USADA to have used PED based not on testing results, but on the testimony of former teammates, some of whom failed their own tests, and may have had an ax to grind. First, because they feel they may have been singled out because of their assocation with Armstrong, second because they may have been pressured by Armstrong or the relationship to use PED, third because they may actually have witnessed Armstrong either taking PEDs or encouraging it, and fourth ALL of the above. The end result is that no one in cycling at the international level will be able to withstand the mere accusations. Non-analytical positives will become the norm. Every champion will be suspect, unless 100% testing is done, and then, as in Armstrong's case, new tests will be conducted on previosuly collected samples, in effect finding athletes guilty in arrears for using PEDs not yet known. Eventually coffee and Gatorade will be banned. And this will stain cycling to the point that fans like myself will turn away.

Chess will go this route. No Master of any rank will be allowed to exceed their 'reasonable' ability. Analysis will be conducted, perhaps electronic surviellance will be used to both check for transmissions and as forensics to be subjected to detailed analysis, suspects will be accused, strip-searched, imaged, run through the metal detectors, scrutinized, and judged guilty based on non-analytical positives. Chess will devolve into the meanest of states, blood sport not for the winners, but for the losers. I expect past upsets to be scrutinized for problems and winners discredited, even posthumously.

Regan wrote about this aspect:

I share the worry of many that a few cases of people “being clever” may ruin much pleasure.

In his letter to the ACP he added:

The point of approaching ACP is to determine how the contexts and rules should be set for chess. The goals, shared by Haworth and others I have discussed this with, include:

(a) To deter prospective cheaters by reducing expectations of being able to get away with it.
(b) To test accurately such cases as arise, whether in supporting or primary role, as part of uniform procedures recognized by all as fair.
(c) To educate the playing public about the incidence of deviations that arise by chance, and their dependence on factors such as the forcing or non-forcing quality of their games.
(d) To achieve transparency and reduce the frequency of improper accusations.
(e) Finally, hopefully to avert the need for measures, more extreme than commonly recognized ones, that would tangibly detract from the enjoyment of our game by players and sponsors and fans alike.

Meanwhile, the ACP has started an open petition against cheating.

We, the undersigned chess professionals and regular competitors in FIDE rated events, share the view that computer-assisted cheating is a major problem in chess and ask the ACP to address FIDE in order to take all necessary steps for fighting this plague.

You can sign the petition here.

 

Peter Doggers's picture
Author: Peter Doggers

Founder and editor-in-chief of ChessVibes.com, Peter is responsible for most of the chess news and tournament reports. Often visiting top events, he also provides photos and videos for the site. He's a 1.e4 player himself, likes Thai food and the Stones.

Chess.com

Comments

Jacob Aagaard's picture

I had a student who worked hard for six months and went from around 2100 level to 2300 level. He scored 5/5 in a weekend tournament with 2200+ average. He was instantly accused on computer cheating and treated appaulingly. He got fed up with the whole thing and as far as I know, have not returned to chess since. We need to be very careful here; especially as we have no great evidence of this being a major problem in our sport.

Mike's picture

Yes but you are not getting to the point. Ivanov played 99% Houdini- moves and beat 4-5 2600 GMs. That`s something completly different to an average of 2200+

Anonymous's picture

Now with massive amounts of opennign analysis and GM games available, there is no reason an ordinary master can't make a huge leap in results if they significantly increase the amount of time they are studying (memorizing) the royal game, especially including his upcoming opponents.

Nigelp's picture

Complete nonsense.

Anonymous's picture

Isn't it weird no one seems to ask if Boris left the playing area during his games?....Yet the stats seem to be the major preoccupation. Bizarre.

Nigelp's picture

Exactly

Anonymous's picture

Ah, but there IS evidence of a major cheating problem. Google Sebastian Feller. Read the articles about cheating on chessbase and on this site.

Coco Loco's picture

That's a very good point. Chess at the club level is a hobby, and a very pleasurable one at that. If we club players decide to devote some precious few hours every week (or month, or ...) to this hobby, the goal is not to win the local 4-round patzer tourney, but to have a good time engaging in an activity we enjoy. Lack of sportsmanship seems to me to be a much bigger problem than cheating at this level.
Yes, Ivanov was playing in a "real money" tournament, and I agree the scrutiny should be much higher for such events. But spreading the hysteria to the local level would be terrible for our game. People need to keep their egos in check; the likelihood that your opponent used a computer to beat you is essentially nil.

Kronsteen's picture

For those who (just) love the game, the world of "organized" chess is increasingly inhospitable. What a shame.

Chris's picture

That is a point.

rolpol's picture

Are there ways of managing the potential risk of cheating, such as delays on live broadcasts and a ban on phones/electronic devices whilst the game clock is ticking, that would make it virtually impossible for anyone to be accused of this?

It's becoming increasingly obvious that malicious allegations could become more of an issue than cheating itself.

Frans's picture

Its prety anoying how these so called "experts" test the collaration between the player's rating and that of the performance. If u closely look at moves made by Kramnik, they are more computer like, but that may not be regarded as cheating because of his rating, how stupit for a professor to imply that a 2000 rated player for example can not spot a wining move in complex game without being accused of cheating?? They should really scrap this short sighted ideas, because they are doing more harm to our beloved game man!!

ETD's picture

I agree that everyone should be very careful about using this sort of study to conclude cheating. But since you call Ken Regan an "expert" in scare quotes, I think it's worth pointing out that he is a truly first-rate theoretical computer scientist, as well as an IM in chess, and he's done as much serious research on this particular subject as anyone.

Anonymous's picture

Guess Fischer, with his perf of +inf and definitely matching comp's lines must've cheated against Larsen and Taimanov ... What do the omniscient stats say about that?

Anonymous's picture

Excellent point...what was the statistical probability that Bobby would win all his games against Bent and Mark? Approximately .000000000000000000000001%. The statisticians should have no role whatsoever is the final judgement whether or not someone cheated.

Simple Stat Man's picture

Uh, no. Even if the events were all independent and the odds on Fischer's winning any particular game were only .5, the odds would be .5^12 = 0.000244 or about one in 4000. Long odds to be sure, but not one in a million. But Fischer was a big rating favorite against Taimanov and a significant one against Larsen, so it wasn't a 50-50 proposition each game. Nor was each event (each game) wholly independent: as the score grew increasingly lopsided Taimanov and Larsen were increasingly affected; Larsen in particular began playing recklessly and went downhill fast.

Simple Stat Man's picture

"One in a million" refers to the approximate odds Regan gave for Ivanov, not to the crazy number in the comment this is a reply to.

Anonymous's picture

Uh, no. You forgot about draws! About 37% of all games are won by White, and about 30% by Black (at least in my database). With these percentages, assuming equal players and independent games, the odds of a player winning 12 straight games, 6 with White and 6 with Black is less than 2 in a million (rather than your 2 in 10,000).

Anonymous's picture

Technically you are correct...my point was what are the odds of an event occurring that has never occurred before in the long history of chess...never.

Horscht's picture

Perhaps for some tournaments the instalment of this simple rule could help: if you win against an opponent rated let's say 150 points higher than you, you are obliged to come to the commentary room immediately after the game and explain the game. (A player could even be rewarded for this, as it is interesting for the spectators, and of course most genuine players will not mind showing a game if they just beat a strong opponent.) This will reveal cheaters, as they will not be able to explain why they made certain computer moves (or they will give vague, strange reasons) and why they rejected certain other more human candidate moves. Let them face the firing squad of commentators and chess journalists in the audience!

Louis van Meegeren's picture

Now there is an excellent idea!
On second thought, wasn't Anand a bit vague in his post mortem last tuesday?

Anonymous's picture

I have a "feeling" he cheated, don't you?

jsy's picture

Yes, but Anand hasn't been accused of anything, so he needs not say anything.

Claude's picture

There is an excellent in-depth article (in german) from Thomas Richter on thi subject on the schachwelt blog : http://www.schach-welt.de/BLOG/Blog/Betrugsverdacht

At least IM Lilov's analysis doesn't seem to stand up as being very convincing.

I find it interesting that Thomas Richter cites a completly different part of Regan's analysis:

"Despite my large Single-PV data set, we don't know enough to say "beyond reasonable doubt" in a judicial process at this time. But it is possible that with more data we can say so, as happens in other fraud-detection fields, and we need to start preparing for this eventuality."

Thomas's picture

Thanks for mentioning this so I don't have to do it myself :) - I was away from the Internet (in Wijk aan Zee) earlier today. Technically, the quote isn't part of Regan's analysis. As I mentioned in the article, Regan's open letter seemed so diplomatic (IMO) that I didn't understand what exactly he concludes - hence I sent him an email. The quote is from his reply and (as he knows German) he also checked and authorized my German translation.

Regan took his time - first for the analysis, then for writing up his results - while Lilov jumped to conclusions much more quickly. Lilov may not be all wrong, but the examples I gave (shouting "first line of Houdini!" for known opening moves and a very obvious move) suggest to me that he wasn't unbiased from the very start.

At the end, my article also has an "interview" (summary of an email exchange) with Emil Sutovsky on the ACP petition.

AAR's picture

Its not fair to accuse someone of cheating without having valid proof.

In future, any upsets in tournaments may be viewed with suspicion.

People expect a player to climb slowly across the rating ladder. They will not accept any jumps in points range. Can't better training cause a jump?
Thank God these people are not policing the NYSE, else Apple would have taken another 10 years to reach 700$ share price.

Ortenix's picture

But there is valid proof! The proof is that Ivanov played computer-generated moves. (Lots of them!) That he played exquisitely or even immaculately in some rounds, while in round 8 (after move 15 when the transmission was turned off) he made very human-like moves with many mediocre moves or even mistakes, and he was soon crushed be a GM, as it would be expected. In "his" other games, on the other hand, he crushed similarly strong GMs, as perhaps not even Carlsen wouldn't be able to do.
The difference in Ivanov's play is blatant. It's really not the case as some people in the previous disscussion suggested that he had some good games and then not such good games.

"People expect a player to climb slowly across the rating ladder. They will not accept any jumps in points range."

No one says players can't have (relatively) fast improvement in their performance. Players have had this and will have in future. But suddenly playing at a level of 2800, immitating (or rather following) computer moves is something completely different.

Why can't some people discussing here see this difference?

mdamien's picture

I watched Lilov's video and then stepped through the suspected games with Houdini myself, and tend to agree that Ivanov was probably cheating. To the question of proof though, the evidence falls short. Clearly, we would agree that there is no deductive proof, but if anything an inductive one -- a question of percentages and preponderance of evidence. But at what point do these percentages become sufficient? It should be noted that correlation to computer moves, in itself, is hardly condemning since in many positions and games the computer moves are natural and/or attainable with some thought. In the case against Ivanov, the condemnation comes from human assessment of those positions where the correlation seems particularly unlikely: look at the depth of complications behind this odd-looking move, look at how he moved the "wrong" rook to the open file, from a human standpoint. The fact that straightforward computer-move-correlation analysis requires such human intervention leaves us with a problem in authority: if Ivanov is in great form and playing at a 2800 level, is Lilov qualified to assess his analysis, his style, his intuition even?

I tend to agree with Lilov, but it's troubling that Ivanov seemed damned if he did, and damned if he didn't in every instance. If he played good moves, it was supporting evidence because the computer liked the moves; if he played bad moves it was supporting evidence because it showed how he played when transmissions were turned off. At other points where Ivanov makes second-rate moves, points that could be considered counter-indicative, Lilov glosses over them or provides an explanation that supports the cheating conclusion.

So the difference is that yes, I for one believe he was probably cheating based on the high correlation of computer moves in positions where Lilov and I both find the moves unusual for a human, but that our opinions are insufficient to say that it's a proof of cheating.

elgransenor1's picture

2200s don't play at 2800 level, whatever "good form" they are in.

mdamien's picture

Some established 2200 can certainly play at a 2800 level; it's a question of how long they can sustain such elevated play. Glance back a few articles here on CV to Andreas Weber, if you're feeling stubborn about your assertions. Incidentally, Ivanov's performance rating for the tournament was 2697.

Anonymous's picture

Statistics are not "proof".

Chris's picture

there were not valid proof. the comparison analysis has to be done for many GM's playing chess and then the results published. then we can tell how similar are playing programs an people.
May be programs are playing like people not people like programs.

Frans's picture

Does that mean now, when Kamsky jumped from 2400+ - 2700 in a single year he was cheating? Please understand that there mant factors that contribute to a player's performance, that range from: sufficient training, preparedness for the match, prior preparation against opponents and tools being use for prepation.

I'm not there could be no cheating, but basing one's allegation on an amature's win agaist a strong GM doesn't hold water, because they just lost and because they hate upsets, they come up with these stupid allegations.

Thomas's picture

One thing which was for some reason mostly ignored or forgotten to mention: the live transmission was also off (at least for Ivanov's game) in round 9!
http://chessbase.de/nachrichten.asp?newsid=13794
"Auch in der neunten und letzten Runde wurde Ivanovs Partie gegen GM Ivan Saric (2626) nicht live übertragen, doch dieses Mal gewann Ivanov" (playing his most complex and double-edged game of the entire event)

Anonymous's picture

I agree.

Owen's picture

The most worrying part is that players like Ivanov are the least of our worries... What if smart people decide to use smarter, far more subtle ways to cheat?

Ortenix's picture

Well, this is a question I asked myself reading throughout the previous discussion on this topic. What if there are players already using "helping devices", but clever enough to cover it shrewdly?

This is all quite sad, because the majority of players indeed play fair, and they enjoy the game. Full point.

But it doesn't mean at the same time that there aren't players who won't hesitate to cheat, whatever their motivation is. And FIDE and tournaments' organizers has to deal with this situation, I'm afraid. Some measures have been already suggested in the disscussion.

Of course, this is all casting a shadow on chess. Unfortunately. But precautions and exact rules how to deal with such situations are strongly needed. Ok, the arbiters didn't find anything Ivanov might have had on himself, but just that he took off his shirt and emptied his pocket is NO proof that he didn't have a device either. Rather, it looks like pure amaterism, I'm sorry to say that. And I'm not saying the arbiters did a bad job or anything. I'm talking about insufficient rules and insufficient measures. Common - with all the cutting edge technology nowadays!

And the comparision with Armtrong? Of course Armstrong did a very bad job to cycling. But be aware - don't think that there won't be any "Armstrongs" in chess!

The very sad thought is though, that Armstrong was able to cheat despite all the antidoping controls. If this is going to be the case in chess as well that cheaters will have the upper hand, then it looks quite sad and like a never-ending story (like in other sports), never-ending battle between people who cheat and the federations/arbiters/commissionars trying to purify the sport. But it really doesn't mean that we should rely on the presumption that "most players play fair, so it's ok." With ever advanced technology and ever stronger computer engines, this attitude is no longer sustainable.

Creemer's picture

So the chances were a million to one. With millions of games being played and this being the first result of its kind, statistically, this was to be expected. Thus: hard evidence is needed. As long as it is not there, go study endgames, catch up on your theory, do the dishes, do anything that furthers your own development.

Anonymous's picture

The chances are 1 million to 1 to have such performance in 9 games so not only 1 game !! Averagely if 9 million games are played and all of them in 9 game tournaments then you can expect such performance once.

However the megadatabase has only 5,4 Million games registered over the whole history of chess (several centuries) with many games not played in 9 game tournaments. This gives food to think, no ?

slaven's picture

Eventhough there is no hard proof here, I am 100% convinced that he cheated. It is possible for 2200 player to beat GM, but only if he has bad day or blunders. But to beat four strong GMs in that way, you must be around 2800 player. Not to mention the moves corresponding 99% to chess engine moves.
The real problem here is to find out how he cheated, And I agree, without proof we can not accuse him.
For nwt, it looks like perfect crime:-).

Alex T's picture

It is a real minefield. In England we had a player in their mid-20s rise from 2100-odd to 2500 within a space of roughly five years, but there is no suggestion he cheated. It's obvious he's devoted extensive time to studying the game and has had success in rapid tournaments, too. So surprising leaps in chess strength can and do happen.
But my instinct here is that this player probably has cheated. The moves often defy conventional logic and are often fearless in a way a human wouldn't normally play. The problem is that it's possible he hasn't cheated, and in this case he's been tarnished for no reason.

With no hard and fast rules possible we have to take these incidents on a case-by-case basis. My view, and perhaps I'm naive here, is that most people wouldn't want to cheat. There is no real achievement in pushing your rating up without doing the actual work.

Soviet School's picture

There are lots of players cheating ! It is only te blatant ones who are caught, the more subtle ones are taking advantage of people's naively as viewed by many of Ivanov,s supporters on this thread.

I have come across people obviously using computers in unrated games on Playchess

Alex T's picture

It is a real minefield. In England we had a player in their mid-20s rise from 2100-odd to 2500 within a space of roughly five years, but there is no suggestion he cheated. It's obvious he's devoted extensive time to studying the game and has had success in rapid tournaments, too. So surprising leaps in chess strength can and do happen.
But my instinct here is that this player probably has cheated. The moves often defy conventional logic and are often fearless in a way a human wouldn't normally play. The problem is that it's possible he hasn't cheated, and in this case he's been tarnished for no reason.

With no hard and fast rules possible we have to take these incidents on a case-by-case basis. My view, and perhaps I'm naive here, is that most people wouldn't want to cheat. There is no real achievement in pushing your rating up without doing the actual work.

Bertil's picture

I second Valeri Lilovs suggestions about device to interfere with radio signals. The arguments against are not really valid. In this specific case, Borislav Ivanov will have no opponents in the future, regardless of anything else. I believe this will resolve the issue.

Anonymous's picture

Just published on http://whychess.com/node/6864
A brilliant interview with Borislav Ivanov.

Anonymous's picture

Thanks for the link...Seeing his picture with the two hot girls, I now think I understand why the anger toward him. It has nothing to do with a talented chessplayer (rating 2200) taking down a couple of GM non entities.

Coco Loco's picture

Two hot girls? One yes, the other... You should get out more.

Anonymous's picture

At my age they both looked hot...Get out more?...My wife expects me home by five PM.

Anonymous's picture

Now with massive amounts of opening analysis and GM games available, there is no reason an ordinary master can't make a huge leap in results if they significantly increase the amount of time they are studying (memorizing) the royal game, especially including his upcoming opponents.

KING's picture

Read his transcript. Answer to below question was that he beat those monster computers 10-0 each???? Are you kidding me, even the top grandmasters I am sure will be happy with par score with Rybka,Houdini

"How did you manage to beat the Croatian grandmasters?

I dont think there was anything special in my games at all. The Croatian GMs made some horrible mistakes during our games. Of course I practiced a lot with the computer, and after beating Rybka and Houdini by 10-0 each, i was absolutelly sure that no-one was gonna stop me winning. After all the arbiters were kind of polite and intelligent, you know, the other players weren't though...."

Pages

Latest articles