Reports | March 29, 2010 18:54

Chess engine controversy

Controversy over chess enginesIn this week's The New Yorker, James Surowiecki calls Apple's launch, next week, of the expensive iPad 'a fundamental gamble, namely that people will pay for quality'. In today's world, free (digital) stuff is indeed so easy to get that it's increasingly difficult for companies to make money with new technology. Think downloading movies and music; think installing a cracked version of Office or Windows 7. But what if something of equal or better quality can be obtained legally and for free? Like... a chess engine?

Rybka

Like almost all serious chess lovers, here at ChessVibes headquarters we're big fans of the Rybka software, which is relatively cheap and will for sure be your strongest chess coach ever. Recently, however, a chess engine was released on the internet which is claimed to be even stronger than Rybka's latest release... and it's totally free. Welcome to the world of open source software, and its many complicated discussions and controversies.

Firebird

Firebird is an open source chess engine developed by a team of anonymous Russian programmers who call themselves the Decembrists, after the well-known uprising in Russia in 1825. It's part of a whole family of chess engines called IPPOLIT. It was released in October 2009 with its source code. In other words, the guys who made the program didn't care if others found out how they did it - they share their 'code' with the whole world. (Update: See Vincent Diepeveen's lengthy comment for more details on the background on the Firebird-programmers. As Russian translator Paul Janse notes, Ippolit Matveevich Vorobyanov is the name of the anti-hero in Ilf and Petrov's famous novel The Twelve Chairs.) Firebird is slowly but steadily gaining in popularity, not only with hardcore chess computer fans but also with strong chess professionals. The reason? Not only is it completely free, but it's also allegedly stronger than Rybka. Various comparisons on the internet between Rybka and Firebird have suggested that Firebird may be some kind of improved Rybka, although nothing is very clear here, either. (According to a small survey among the ChessVibes editors, we found that Firebird does seem to reach deep ply levels a bit faster.) At the recent Amber tournament in Nice, ChessVibes asked two leading Dutch grandmasters, Jan Smeets and Erwin L'Ami (who happen to be on Veselin Topalov's team), whether they knew about the Rybka/Firebird developments, and what they thought of it. Here's how they see it:

Smeets: I read on some forums that such a clone existed, I think this was in October last year. I think I read it at the Rybka Forum. However, there it wasn't allowed to mention the name, but if you Googled it it was easy to find. At that point it was Ippolit. First it was Ippolit, then Robbolito, then Igorit. Robbolito was a one-core engine, but quite a strong one-core engine, so you could run it simultaneously with other engines. Igorit was the first multi-processor engine but I think that one was a failure. And then came Firebird, which was a combination of Robbolito and Igorit, and this one was good. I use many engines, because it's good to vary. L'Ami: One day I entered his hotel room, during the Corus tournament, and I saw all these strange, crazy names I had never heard of. But now I know that currently everybody uses them. Smeets: Yes, many, many players use them. It's also a popular subject on the ICC for example. But it doesn't make a big difference, you know. These programs are so strong these days. I think they're all playing at 3300, 3400 level so fifty points weaker or stronger doesn't really matter.

Controversy

So what's the deal with all these new engines? Is one really stronger than the other, and how does it matter? Well, here's where the controversy begins. Right afer Firebird was released, a statement appeared on the Rybka forum site (Smeets also refers to it) in which Rybka's creator, Vasik Rajlich, officially declared:

There was an open-"source" (using the term loosely) clone of Rybka 3 released in the spring. Unlike the last time, there was no real attempt to hide the cloning - the hackers were even kind enough to keep me updated via email.

Rajlich alludes to a previous confrontation with Russian open source progammers - or 'pirates' as Rajlich calls them - back in 2007, when the open source engine Strelka was released, which was, according to the Rybka team, suspiciously similar to many features of the Rybka 1.0 engine. Here's the relevant background from the Wikipedia-article on Rybka:

There were allegations that Strelka was a clone of Rybka 1.0 beta, in the sense that it was a reverse-engineered and slightly modified version of Rybka. Several players found Strelka to yield identical analysis to Rybka in a variety of different situations, even having the same bugs and weaknesses in some cases. Osipov, however, stated repeatedly on discussion boards that Strelka was based on Fruit, not Rybka, and that any similarities was either because Rybka also was based on Fruit, or because he had tuned the evaluation function to be as close to Rybka as possible. With the release of Strelka 2.0 beta, source code was included. Rajlich stated that the source made it "obvious" that Strelka 2.0 beta was indeed a Rybka 1.0 beta clone, although not without some improvements in certain areas.

Rybka & Fruit

An important aspect of the whole argument is yet another accusation, this time from the Russians, namely that the first release of Rybka was itself largely based on the open source engine Fruit, which was released in 2006 and is now a so-called freeware program (not to be confused with open source software!). In a lengthy, very interesting video interview Rajlich gave to Nelson Hernandez last year, he didn't really answer the question as to which open source programs initially influenced the development of Rybka the most:

Hernandez: What chess engines in public domain, when you got started, had the biggest influence on the earliest versions of Rybka? Rajlich: Well, actually I started in a kind of strange way. I printed out just about every single paper there was to print out about computer chess; all these academic papers. A lot of them are interesting, a lot of them are just really relevant, actually. (...) So I kind of started to work through that, that was how I started. Probably it's not the most efficient way to do it. Probably the most efficient way is to take an open source program - at that point it would have been Crafty - and just kind of go through that. And I gradually worked around through that. (...)

Those interested in the gory details of the allegations might also want to read the IPPOLIT Wikipage, which includes statements such as:

  • Rybka's piece square tables are generated from the same code as Fruit's.
  • Rybka's pawn evaluation is virtually identical to Fruit
  • Rybka's "pattern" evaluation is virtually identical to Fruit's

These are, well, interesting claims, which suggest deep code-researching, but unfortunately, the website contains mostly stuff like:

In the this the prominence of the Revolution versus unto the Capitalists obliges with the stroboscopic clarity unto the final victory. For the Revolution: anonymous for with the philosophics. For the Capitalists: anonymous or plus known (too), in with the conveniencings.

Such incoherent nonsense makes it considerably more problematic to take the claims from the IPPOLIT team serious, and it's probably one of the reasons why Rajlich is so fed up with these guys. Anyway, the net result of all this is that the IPPOLIT-article on Wikipedia has now been deleted and accusations of censure and even some far-fetched global conspiracy theories are suddenly all over the internet. On the Rybka forum and even on other chess computer sites, all discussion of IPPOLIT software are deleted or banned, causing even more anger with the 'Decembrists' and their supporters. And they seem to have a point, as this aborted discussion on Talkchess.com shows.

Evidence

One thing that's clearly lacking is concrete evidence from the Rybka team that Firebird is, in fact, a true clone of Rybka - something that is, of course, required in the case of any serious accusation. But so far, the evidence has not been presented in a coherent way. In an intriguing and generally polite discussion on the forum of Chess.com, one defender of IPPOLIT react as follows:

[Rajlich] claims that the authors were in correspondence with him the whole time they were doing this. Show us the correspondence and maybe I will believe. BUT, (and thats a big but) reverse engineering is not illegal. For years there have been forums that have been trying to figure out how Rybka works by its playing style. That is also reverse engineering. All [Rajlich] is saying is that [IPPOLIT] uses ideas similar to Rybka. He can't or won't prove that these ideas are even in Rybka. And even if they are, Rybka is a five year old program.

The Rybka team itself apparently doesn't want to spend much time about it. In a brief reaction, Rajlich wrote to me: "These are all just decompiled Rybka 3 clones. It's pretty obvious from the Ippolit sources, any programmer will tell you the same thing." As a programmer myself, I must say I find his point of view understandable, because I know how hard it is to make good code, and how proud a well-written script can make one. Rajlich is also, obviously, tied to a highly successful commercial product with links to other companies and sellers. (You've guessed it - here's where the conspirary theories start to unfold.) On the other hand, it would enormously help resolve the controversy if some real evidence was presented by the Rybka team. This could be 'code snippets' (relevant fragments of code), or other striking silimarities in design, or even, as the above commenter suggested, quotes from correspondence with the IPPOLIT programmers. The problem, I assume, is that Rybka's code is not open source, and showing it as part of evidence against pirates may in turn compromise its integrity - and this time, it wouldn't be stealing. This puts Rybka in an unpleasant Catch-22 situation, which was no doubt gleefully foreseen by the Decembrists. But even apart from any technical discussions - what if Rajlich is right and Firebird is simply a Rybka clone - a product of piracy, that is - only stronger? Should we all stop downloading it just because it wasn't manufactured in an entirely 'fair' way? As another commenter on the Chess.com forum muses:

If they absolutely ripped off Rybka, then I would be happy to remove Firebird from my computer and purchase Rybka 3. No big deal. I've got the money and want to support software developers, as I have always done in the past in the field of music. Music software is far more expensive. Several of my music programs cost between $250.00 to $500.00. One product requires a $100.00 upgrade fee each year to stay current. I don't use cracked software.

Not worth it

That sounds very noble, but how realistic is it? Perhaps hardcore computer programmers have some sense of professional ethics, but what about pro chess players like Jan Smeets and Erwin L'Ami? Can they expected to be that honest as well? Aren't they right to be just interested in the best available chess engine and compare them, use them all to their own advantage? Here at ChessVibes headquarters, we're in serious doubt. We're very sympathetic of Rybka's cause, simply because it's such an outstanding and cheap product. Rajlich wrote to me he very strongly believes in having a positive message:

Rybka 3 doesn't even have normal copy protection. Future versions of Rybka will be available over the internet - users will log into PlayChess or ChessPlanet [or ICC or FICS - ed.], and their analysis will run on remote machines, like with cloud computing. This has a lot of nice properties - continually updated Rybka versions, possibilities for shared analysis, hardware power available from traveling devices like Pocket PCs, iPhones, flexible hardware availability, etc. It also has the side effect that it stops all software piracy - these now-ancient problems from two years ago won't be repeated.

That's great news (and the idea of online engines is controversial in its own right) but it doesn't answer the moral dillemma whether we should use potential clones or not. We'd really like to give Rybka the benefit of the doubt, but at the same time we think that as long as Rybka's accusations are not based on concrete evidence, using Firebird as an interesting alternative isn't morally wrong necessarily. After all, as Surowiecki writes in the same New Yorker article, information (or, in our case, evidence) is also an aspect of quality - increasingly so, especially in our modern digital world. Perhaps most importantly, Jan Smeets makes an excellent point when he says it really doesn't make that much difference, unless you're going to hold matches between the two programs just for academic purposes. In practice, who cares if a 3300 rated engine or an 3350 one is assisting you in analysing your games? Such a trivial difference is simply not worth it to award potential pirates and mistreat the original programmer. We hope the whole matter will be resolved soon.

Arne Moll's picture
Author: Arne Moll

Chess.com

Comments

Creemer's picture

To me the most interesting aspect of this situation is the fact that the source code is freely accessible. No matter how clever one team of programmers is, the open source-method virtually guarantees a better end result in the long run.

I think in a few years all software, so chess software too, will be free. Companies will make money by selling services, not products.
And chess engines will become so strong that the art of programming will be to make analysis understandable for human beings.

Ladies and gentlemen, we are about to lose control.

buri's picture

How is it that you install these chess engines? I downloaded it, but where do I go from there? Can anyone please help me?

Ed Collins's picture

Buri, you need an interface (GUI) to use Rybka, Houdini, Stockfish, Firebird, and dozens of the other chess engines that are available.

WinBoard, Arena, and the Fritz software are three popular interfaces. There are many others.

Arne Moll's picture

There you go, buri. One excellent reason to buy official software is that you get neat manuals, support and guarantatees. With open source software, you have to figure things out for yourself :-)

Arjo's picture

@ buri: i got the same problem... most programs are packed by winzip, 7zip (the .7z extension) or simular programs... just type the extension (.7z or .rar f.e.) in google and you will find the rigst program to unpack and then install the chess engine

unknown's picture

Download and install Arena 2.0.1 from: http://www.playwitharena.com/

Then "Engines" --> "Install new engine"...

Jarvis's picture

Which of these programs are compatible with Mac OSX? Would be happy to hear someone knowledgeable expound on the subject of chess engines for Mac computers!

brabo's picture

Firebird can very well be quicker and still be a clone of Rybka. By tweeking some of the heavy time consuming trees you can make a program easily much faster. On fast timecontrols this will often lead to better performances but on the slower normally this should give a disadvantage if the programmed knowledge in the timeconsuming trees is well done.

Personally I find it incredibly unlikely that one can build a superprogram in such short timeframe. As an experienced progammer I know how much time it takes to make a proper program from scratch. One has to start from some other source and today there aren't much available supercodes available so an illegal copy is the most likely scenario. It can be Rybka but also Naum and Deep Shredder are valid choices to start from.

I am strongly against copying software. Programmers need to spend countless hours in coding, testing,... so I find it justice that if you use their labour that you pay for it. If everybody decides that you don't need to pay then there will be very little new software still made.

The whole matter is going on for a very long time as can be seen on the different chessforums (also indicated by Smeets). The number of illegal downloads and illegal copies has only increased the last decade. I don't see a fully trustable solution for the nearby future.

Felix Kling's picture

From legal point it's rather clear to me: Those engines are heavily based on Rybka code, therefore the license they use is not valid, as copyrighted material was used and publishing code from someone else with just another license is not legal of course.

These are the reasons why the cloners don't use their real names. Only Norman Schmitt, who is known for releasing clones as new engines and trying to make money with it, uses his real name for his version of the clone (afaik).

It's a bit sad that people use those clones and some even think it's right to do so. About the playing strength: The only way to really test it is playing a lot of games. There the clones are apparently slightly stronger at fast time controls and weaker or as strong as Rybka at slow time controls. It looks like the clones don't use all the Rybka code and are some kind of crippled Rybka that is slightly faster therefore.

To summarize: Downloading those clones is almost the same as pirating Rybka, which was already very common apparently. It damages all engine authors, since the sales for the legal (ie. original) engines will most probably drop and leads to a different situation on the market, so that authors probably won't publish their engines as "local" versions anymore to protect the code.

About this article: It's questionable if linking to such material is right from legal and moral point of view.

Felix Kling
(Rybka webmaster)

Arne Moll's picture

@test: that is also one of the points I express in the article, but perhaps Vincent Diepeveen can answer it in more detail, since I suspect he is the biggest expert on this issue that has commented on it so far.

Vincent Diepeveen's picture

Look, there is no question about these facts, as more than 1 expert is either proving it or showing it:

a) all those clones are derived from Rybka's source code. Mentionned here gets firebird, but there is in fact a few dozen of others.
b) they did not get derived from ippolit, though ippolit also was derived from rybka
c) total sales of Rybka is less than 1 year salary, similar to what someone makes who works at the supermarket; whereas it's obvious Vasik isn't the programmer of it; which isn't relevant further here, as it's obvious he 'owns' somehow Rybka. Yet this was already clear a few days after he launched rybka at 5th of december 2005 (which in The Netherlands is a day at which we give gifts for Santa Claus); realize most MiT guys have a salary of around a $150k a year, far above total sales of Rybka.
d) A british team studied some of the clones and has proven that the codes of the clones are somehow from the rybka source code; again those codes have more than the ippolit code, so we know 2 things at the same time then:
d1) it's derived from Rybka
d2) it's derived from a later sourcecode branch not published in any form
by the rybka team
e) to improve all those individual clones is really big expert work, you don't manage this just at home. You also need very advanced, as google calls it 'bookwriting programming skills' to do all this, besides big hardware to tune it
f) Rybka has indeed been banned by ICGA in meantime, as they stole some tables from Fruit, and didn't defend themselves towards ICGA
g) The real evidence is sometimes easier than you guess - if it takes a big supercomputer to tune all this, all the 'source codes' published on the internet, as far as some got published share, including Stockfish, Rybka, Crafty and all those engines, they all share 1 thing: the actual code to mathematically TUNE the parameters doesn't get displayed. Now it's pretty obvious what Crafty is doing, the others are just lying around about what happens. It's a fulltime job however
h) from several of those programs different authors have shown indications and evidence that many programmers worked at the actual code, even when officially just 1 or 2 names get displayed (and the original author claiming he didn't work actively lately anymore)
i) this is all professionals whom you need to PAY to get something done
j) some of the so called 'authors' of clones you can find an adress from, a physical one where they live; and that's provable guys who use source codes illegal
j) so someone CAN claim money. Be it Vasik be it others who own the code.
k) 0 courtcases have started however, that's weird you know.
l) so we know that a lot happens 'sneaky'. We know big hardware is needed. We know somehow sources get stolen, source that are *not* on the internet. We know there is no courtcases at all (which is surprising, you can really claim BIG cash in fact). We know sales actually are really neglectible, so all that effort is not for the sales obviously as there hardly aren't. We know many nationalities are toying around in this field, under which some Sheikh's from the Middle East which run a few projects computerchess related; whereas in reality 95% of all experts live in a circle around The Netherlands. None of the expert programmers have been hired by the different Sheikh's. The Sheikh's know what's worth money and what isn't, and the computer chess engines aren't worth money, so they don't offer much cash, or nothing at all, so no one COULD get hired.
m) so where the arabs do not offer money, sneaky somehow some of those engines are real strong; in fact that strong that only now, nearly 7 years after Rybka came out of the blue sky, directly at space shuttle strength stronger than any other engine, it's obvious projects somehow get funded with 0 courtcases getting started; so there seems some sort of 'central control'.
n) Please realize what happens: there are strong chess engines, it takes 10 years to become a big expert in computer chess and only a few can do that, it's not something you do within a few months - too many math skills needed for that (algorithmic skills), out of the blue sky all the existing chessprograms, some which took 20 years to build, they get blown out of the water by some newer software, from which no one knows who built it, yet there is the educated guess of supercomputers getting used to test out problems and tune it well, and now by 2010-2012, slowly the 'known' programs get stronger again, yet it's start 2012 now and only now *maybe* one of those programs might get stronger than one of the rybka clones (latest clone that tops the charts now is called houdini, another obvious rybka derivate). Who threw in all that cash, as you don't beat the existing strong software 'just like that' of course. Who threw in the BIG cash? If we do some calculations on a small paper you're looking at a 9 digit number in dollars, if we just include computing time.
We should maybe look to just before 2005, which ORGANISATIONS suddenly got a lot more funding, so much that they didn't know out of silliness what to do with it?
m) i see Nelson Hernandez gets quoted, a computer chess enthusiast. I have another quote from Nelson Hernandez, as you might know he's a retired captain of the US army: "i live 3 miles away from CIA headquarters"

mk's picture

That post from Vincent Diepeveen is SO much nonsense. The authors of Rybka say it was reverse engineered -- the suggestion that it was taken from stolen unpublished source code is ridiculous.

chessintervention's picture

Lol indeed, i also downloaded Firebird...

so i organized a lil ' match between Rybka 3 32bit & Firebird 1.2 32w,
i dont know if i messed up w/ the parameters(though it looked fair) but Rybka got raped in blitz in this match...

So i too will switch cause i dont spend hours on analysis, and the quicker the better in this case...

Vincent Diepeveen's picture

I give you the space shuttle and ask you: "please improve it and kick our old shuttle with the new shuttle". Will you manage?

First thing you will notice is that you need all kind of expensive toys and tools to do so. But will you manage to improve it?

Roo Bookaroo's picture

In complete agreement with "Jarvis".
The matter of compatibility of these new chess engines with Mac OS X is important, the more so that the millions of users of IMacs, and more so, iPhones, IPod touch, and the coming iPads would hugely expand the use of such engines, and benefit the popularity of chess, which is so uniquely well adapted to displays on the screens of all Apple hardware.

nep's picture

There are lots of discussion about the Rybka/Fruit "cloning" (using cloning loosely) at the talkchess board, but all these discussions were moved to a forum only visible to subscribers - and this evidence could be found also on the rybka forum, but again, in a subscribers only subforum.

About the Rybka cloning Fruit (and Fruit 2.1 is still open-source and can be easily found), the evidence provided showed that Rajlich was not honest when asked about how much of the strength of Rybka came from Fruit (he said very little, but it seems to be a lot), however, about using Fruit code (which would characterise copyright infringement), the case was not so clear, and some people believe there was code copying, some don't.

Regarding Firebird, it is NOT open source, and the "developer" (again, using the term loosely) has been exposed twice already cloning chess engines - this time, cloning in a strict sense, using the source code of open source engines (Toga and Viper), and releasing as if they were his own, closed source engines.

Vincent Diepeveen's picture

As a chess programmer i studied Ivanhoe/Firebird. First of all it's total different from IPPOLIT. Ippolit is a lot weaker, has a worse evaluation function and is not using all processors in the system if you have more than 1.

The changes, especially the parallel ones, are total non trivial even for big experts. Maybe half a dozen programmers on the planet is capable of carrying this out on their own.

It's also several programmers who worked at it, and it's obviously no Russians, as not a single Russian is capable of strong parallel programming on his own in this field. That's just a cover for western programmers.

From the styleguide used we can see it is a mix of NASA styleguide with a more senior programmer who worked in the code who is just doing whatever he likes to do.

Russian styleguide you can see well in the first released clone of rybka 1.0 which is strelka. Russians use total other styleguide, which confirms what the russian programmers posted on the forums - that they have nothing to do with this.

Some of the programmers names listed, if you translate them to English also are rather suspected and translate to western names like Robert Fischer.

These styleguides get used also in Germany and Netherlands. Not so much elsewhere. This doesn't come as a surprise to me if you realize where those experts live. Please note that NASA type organisations in the US have no persons who are expert in this field. They're all living in and around Netherlands. With 1 of them living just over the border in France and another one furthest away lives in Muenchen.

The real question is who pays for it. So much nontrivial work which requires good parallel programming skills (which again only half a dozen of experts in game tree search on this planet have) and another one for evaluation and another one for search. Then you need testing and management.

So the first time to create all this was really years of work.

When i tried counting i came to 5 programmers who can carry it out. If i subtract myself from that it's 4. I know them all by name and speak with most of them regurarly, so i can assure you that if one of them has been doing this, it was a fulltime paid contract.

The alternative for this is hire half NSA which costs millions as you need a far bigger team then to get it done. Obviously that's not what anyone would do, just hire an expert to get it done.

I do not know who pays for this, but all these guys are commercial guys and this is total fulltime work to get accomplished. You don't do this in 'your spare time'. Also it requires extensive testing at big hardware, nearly no one has such machines at home.

It is a blind guess who paid for this. Maybe some burocrat wrote down a signature too much somewhere. From government viewpoint of course throwing a few million away is no big deal, that's their core job. I wouldn't know a single company which would do this.

Yet it again gives computerchess more attention of course, be it in the wrong manner. I've already seen a dozen arabs around here now. Not sure why. Maybe they like seeing source codes.

This is not open source at all. Be warned. Using this code can have serious consequences as it is all copyrighted code.

I want to make clear however that i have nothing to do with all this. I didn't get hired for this project. Seems they pay for this sort of work also a very low fee, $30 an hour. Now of course dozens of people all together have been involved in this, so i doubt any of those 4 others has clean hands here, they all love to get hired on a pay by hour basis.

Most likely they all got hired.

Even very tiny details have been 'fixed', which is incredible really. We already saw this in Rybka code of course; the material evaluation which already was really great, was suddenly rewritten by yet another clown using a neural network.

Total independant part of the code it is obviously.

Some 'similar' engines do not share this code, i wonder why. I guess only 1 contractor paid for it and it was carried out by 1 specific person (could be many programmers who did do the job, though the person who qualifies most also posted publicly on the internet he has been busy tuning material with a neural network and that it was years of fulltime work and "only" brought 20 elo, which IMHO is a lot for just 1 small yet important part of the evaluation).

So where they worked real professional everywhere, of course it doesn't have really a lot of chess knowledge yet.

It's all based upon tuning. That tuningscode you can find nowhere, they keep that part secret, even the method how it gets done is secret, ensuring them to get hired again if another organisation is interested.

So you can toy at home sirs, but you don't have that huge computational power that is needed to tune it well. You realize how much energy i burn at home for that, and i still don't have that much like what was used for this.

Either you put in a lot of chessknowledge in the chess engine, like i do, or you are really dependant upon huge supercomputing power from some very big government organisation, as tuning has to compensate for lack of knowledge.

We don't have that hardware in the Netherlands even needed for this and Netherlands has really big hardware.

So forget about Russia or any other nation without huge supercomputers that scientists can toy with just for fun.

Thanks,
Vincent

The Riddler's picture

Your writing is so huge that I only analize two of your statements:

"The changes, especially the parallel ones, are total non trivial even for big experts. Maybe half a dozen programmers on the planet is capable of carrying this out on their own." Let us put it in this way: Why only half a dozen? Where do you get this? Are you or anyone make any study in this topic? If not this is pure talking.

"It’s also several programmers who worked at it, and it’s obviously no Russians, as not a single Russian is capable of strong parallel programming on his own in this field. That’s just a cover for western programmers.". This sound like you know all the Russians ¿You're very sociable person or kind a God? This is the type of fallacy that experts usually use to sustain his "reasoning". Show us that this is true, if you can't this is false and all your reasoning too.

The generalizations like you made are so false if you're not prove it or at least give it some true support! If you can't support your statements, you should say that is your opinion and not a true fact. And of course, your opininion maybe right or wrong!

Arne Moll's picture

@Felix: agreed about the direct linking, so I removed this. However, I don't see what's wrong with linking to wikispace or wikipedia pages.

@nep: discussions are fine with me, as long as they're polite and deal with arguments and not insults. In fact, I think discussions are an important and vital source of information, especially in such obscure areas as open source and software cloning. If it gets us closer to the truth, all the better!

Jaques of London's picture

what a great site. our family really loves the game of chess and we love your blog

test's picture

"Personally I find it incredibly unlikely that one can build a superprogram in such short timeframe."

Yes, but the same can be said about Rybka. The Fritz, Shredder, Junior programmers had been working on their programs for years and years, then suddenly out of nowhere comes Rybka and leaves them all in the dust. Equally hard to believe.

If these new engines are reverse engineered from Rybka that's wrong. If Rybka was based on some other open source program that was freely available, I don't see how this is wrong.

On the moral issue for the consumer: it's clear you are doing something you are not supposed to when you install a copied version of Microsoft Office for example.
But if it is unclear whether some program is an illegal clone, I suppose the user could be excused. If it's just word against word, who do you believe?

(The average user is not a programmer. I for one just have to take other people's word for whether or not some program is a clone or not.
To be fair from reading all this I do get the impression that these new programs are clones, but again; ultimately it's word against word and I wouldn't put it above the Rybka team to make these allegations just to protect their product.)

The problem with the moral issue in general is that even though there are certain laws in place, that does not mean the consumer has to agree with them. Laws are passed far above the consumers' head. Many laws are unjust. I don't feel bad when I break a law that is morally unjust.

Then there is the matter of unethical behavior by companies and corporations. As an example: I don't feel bad installing a ripped off Windows because Microsoft is know to have used unethical business behavior to force this abomination down our throats and is basically extorting us by forcing us to buy a new version every other year.

Vincent Diepeveen's picture

If you want to buy a new program or even a copy of an existing program under a different name, that's not a problem. Just costs money. It's obvious Vasik did not program rybka version 1.0. Forget about all the other rybka's. Just version 1.0 is interesting, after 1.0 you see how rybka slowly changes from American type engineering to European type engineering, and more and more parts of later rybka's are easy to figure out which experts most likely made it. Some hardly do secret about it, they even mention that they managed to improve thing XYZ, they just don't quote the rybka name. That's all legal so far. Just business. Someone wants a software toy, his name is Vasik, he can get it of course. NO PROBLEM.

All so far legal, this has happened 100 times before and will happen 100 times in the future as well.

Just 1 problem. It played way way stronger than anything prior to it. Copying something from another program is illegal of course, that says more about the type of engineers working on it, and is further not relevant here.

The problem is that Rybka won games because of how it values for example a chess piece. You can give it the value of 3.675 which is what i had it at, back in 2005, or you can give it 4.06, the value Fruit had it at. It appears it really matters in computerchess which value you assign to each 'parameter'. There is thousands of such parameters. In 90s many tried to do this automatic. As far as we knew back then they all failed.

It appears now we just didn't have the computing power to get it done. With enough computational power you can do this.

There is however 1 problem. Such tuningssoftware, as far as it exists, it's top secret and usually state owned.

How do you intend to use top secret software for a chess engine product which has an expected income of maybe a few thousands of dollars within a few years time?

The real question is: how much did government organisations fund him to do that? And also the tunings software, which is top secret software, i have a statement about that.

I asked a Dutch guy, working for the Dutch government whether it would be possible to use NSA classified software to tune a chess engine. The answer was: "no that is impossible in The Netherlands, using secret software to modify parameters of a program that somehow makes it later on the internet, that is illegal".

See, THERE is the big problem.

Did i explain the problem more clear now than some years ago?

a) Einstein level guys tried in 90s and failed (as far as we knew back then)
b) nowadays it's possible with total superior forms of calculation power
c) the software to tune gets kept secret in all these projects, and the few statements we have are obvious lies, besides for example the StockFish team is doing just total contradictary statements; even from crafty we don't have any
tunings code nor scripts, though it is the only project where we can more or
less see what happens by looking at how bad the parameters actually are tuned
in crafty as compared to rybka and clones.

We had a case of a Cuban team starting up a clone; within days they were discovered, banned everywhere, and by now everyone has forgotten about their 2010 attempt

So we have a lot of illegal clones, from which the authors do not get caught somehow, if someone from Cuba tries it, he's caught within a day, everything has been proven about what program he actually cloned and how. Every statement he did do has been analysed to the fullest, and he has been banned directly.

In fact such things happened from 2007 to 2010 several times, yet it took them 5 years to ban Rybka, though by start 2007 i heard several chess programmers complain very LOUD, about rybka having stolen something illegal from Fruit, the first indication of that being there just weeks after its release at 5 december 2005.

The only interesting project to study is Rybka 1.0, the rest is total irrelevant; sure i try to beat them, but basically it runs parallel and searches a tad deeper and has a tad more chessknowledge (just a few patterns, though i need to mention this is very HARD to make) - no big deal, as just showing some big cash gets you all that.

The real amazing thing is rybka 1.0 and nothing else.

What happened before rybka 1.0, that's what i am interested at and nothing else,
because once someone opens his big wallet and pays for everything of course the rest is pricey, yet it'll get done :)

S's picture

You should correct the link to the decemberists.
The band isn't bad, but still.

About the engine. Maybe they use some ideas of other engines, and add a bit of their own, just like writers do in their articles. But I will surely use Robolitto until I see some proof of copyright infringement.

Arne Moll's picture

@S: Oops, corrected :-)

@Vincent Diepeveen: Thanks for the information! I've updated the article with a reference to your comment.

chess fan's picture

for noobies like me: download the engine, unzip it and copy the 'exe(cutable)' file and paste it in the engine folder. For me it was under C > Program Files > Chessbase > Engines.UCI and paste the .exe file here. Then just start your interface (for me it was Fritz11) and go to 'Engine > New UCI engine et voilà

noyb's picture

Facinating discussion and information, thanks Arne! Excellent job as always.

My feeling is that it's best to stick with reputable companies and individuals. While it's a bit seductive dealing with mavericks (i.e., Neo in the "Matrix" movies), at the end of the day there's a lot to be said for paying for a service/product and the support that comes with it (i.e., legal obligations defined, etc.).

I'd rather pay the cost of a good or service and know that it's back by reasonable guarantees of support, rather than get something for free and "take my chances." Also, I very much enjoy being paid for my work, and I want to fairly compensate those who provide for me.

In short, "Quid pro quo".

Ayumi's picture

Some of the reactions here are truly breathtaking.

The (not so) underlying idea that (open source == piracy) just show an outstanding lack of understanding bringing us back to the good old 2000s when Microsoft equated GNU/Linux with some kind of communist cancer. And indeed, some of the comments here (and parts of the otherwise interesting article) are on that same level. Next thing you'll tell me if that I'm a dangerous pirate because I'm using an open-source operating system.

Never heard of Firebird before (well I did, but it's a open-source database product http://www.firebirdsql.org/). That said, it must be stressed that reverse-engineering is not illegal (not everywhere at least). People saying that somehow the russian guys are stealing copyrighted material obviously have no idea how code decompiling works (if indeed only reverse engineering was used here, and not true code stealing).

That doesn't mean that it's really ethical though and yes, I understand that it is infuriating. However, if the article is correct, it may well be that nothing illegal is going on.

Ayumi

Ayumi's picture

Is it some kind of april's day fool?

Arne Moli, I don't think Vincent's comment is credible. It's full of fantasies and visibly full of hate too (see the comments about the arabs). I think linking to his comment in your article is not a very good thing as I think it is enough to completely discredit it.

The only explantation I can think of is that it's some kind of joke...

Ayumi

Arne Moll's picture

Ayumi, I don't believe commenters here think open source = piracy, as this distinction is clear to anyone who read the article (or followed the hyperlink to the article on smashingmagazine.com). As you say yourself, the question is whether this particular open source software is based on hacking or not. I don't see what's unclear about this.

ops's picture

stockfish chess engine is also free and strong as rybka 3

unknown's picture

Stockfish is also new Chessbomb engine...

Onestone's picture

The article incorrectly states that Firebird is an open-source engine. It's not - the author has never released its source code. Firebird is based on the Ippolit/Igorrit/RobboLito/IvanHoe sources, but this doesn't make it open-source itself.

Felix Kling's picture

@Ayumi: I'm using Ubuntu and a lot of other open source software. But the problem with licenses like the GPL is that some people may publish code under it while not being the authors of the code.

About Rybka and Fruit: Rybka is influenced by Fruit, but not based on it. Vas' code is 100% original. There are some techniques from Fruit that are in any chess engine now, so it's just part of the development of chess engines. Actually spreading ideas is one of the ideas of making code open source, it speeds up development.

@Arne: The wikispaces page you are linking to offers direct download links for those clones. It also contains obviously wrong "information", like fake names for the hackers and so on. I don't see why it makes a difference if such a page is a wiki page or not.

About stockfish: It is a strong and afaik completely original engine, so it's good to use it. But it's not as strong as Rybka 3 (yet), see e.g. http://computerchess.org.uk/ccrl/4040/

chris's picture

As a GNU/Linux user, I know there are a lot of very strong free open source chess engines available, & I feel it is morally better to use them (though I have bought a copy of Rybka) & Scid (again I have bought ChessBase twice), & GNU/ Linux (again I have bought versions of XP, Vista & 7).

I don't like the idea of programs that usually originate in other people's ideas being fenced off by commercial interests that claim that they own them so the public can be forced to pay for them.

Glaurung is over 3000 Elo & its author Tord Romstad says that he refuses to commercialise it,in part because :

"Another important factor is that trying to sell my engine would probably force me to close my source code in order to keep a competitive advantage towards other chess programmers. Because I think sharing and cooperation in computer chess leads to more rapid progress than secrecy and competition, closing my source code is a very unattractive option to me."

He also makes the point : "It could also be argued that the cause of cloning is not the existence of open source chess programs, but the very fact that releasing a chess program without source code is considered acceptable. If all chess programs were released with full source code, cloning a chess engine and getting away with it would be impossible."

Long live Richard Stallman !

ops's picture

http://wiki.winehq.org/MacOSX/Installing

i dont know if with it, you can run windows programs on osx

test's picture

from http://www.chess.com/forum/view/general/fre-chessengine-firebird-better-...

"by myou27:
It is not a rumor that Rybka comes from Fruit. The parsing code and the time management are nearly direct in the copying of code. The chess stuff is also a lot of the same, but the numbres are more tuned in the evaluation. The search changes the pruning and the material tables are also added. The base of it looks still like Fruit. The hash tables are about the same though Rybka has changed those in later versions. The other "major change" from Fruit (Letouzey) to Rybka (Rajlich) was to rewrite it for bitboards to make it 3 times faster on 64 bit CPU, but a programmer said that takes maybe 3 hours once you having a working implementation of both Fruit and bitboards around. Rybka just uses Crafty bitboards so making them is easy. Also the node counts and depth were <> by "Vas" likely to try to coverup this, plus the sales pitch was that Rybka was a <> engine rather than a fast one. Mr. Letouzey appears unminding because Rybka only copied ideas and not GPL code."

I take from this that everybody copies from everybody else. So Rajlich taking the moral high ground here sounds a bit hypocritical to me.
Still, rules are rules; one shouldn't clone a commercial closed source program and release it in the wild. (IF that is what actually happened.)

From the video interview with Rajlich I take it that the cloud computing thing is a godsend as it allows Rajlich to keep making improvements without having to worry about everybody else copying him and catching up for free.

IanO's picture

@Jarvis & @Roo Bookaroo:

Robbolito was easy to compile for Mac OS X. However, the later branches (Igorrit, IvanHoe) start using Windows-specific multithreading, and so are not suitable for the Mac. You might be able to run them under WINE, though.

Despite the single threading, I found Robbolito to be stronger at fast time controls than other Mac engines: Stockfish, HIARCS, Rybka 2.2n2, Toga II, and Fruit.

What a strange situation; I'd be perfectly willing to pay for a version of Rybka which works seamlessly on my Mac (not the microwine hack). Instead I have to use Robbolito for free because no one is claiming authorship!

noyb's picture

After further reflection upon this and other articles in the past several months, I'd have to say that Arne certainly is doing the best job of interesting chess journalism in recent memory. Keep of the good work Arne!

test's picture

From the Talkchess.com link above in the article:

"by myou27:
It is not a rumor that Rybka comes from Fruit. The parsing code and the time management are nearly direct in the copying of code. The chess stuff is also a lot of the same, but the numbres are more tuned in the evaluation. The search changes the pruning and the material tables are also added. The base of it looks still like Fruit. The hash tables are about the same though Rybka has changed those in later versions. The other "major change" from Fruit (Letouzey) to Rybka (Rajlich) was to rewrite it for bitboards to make it 3 times faster on 64 bit CPU, but a programmer said that takes maybe 3 hours once you having a working implementation of both Fruit and bitboards around. Rybka just uses Crafty bitboards so making them is easy. Also the node counts and depth were <> by "Vas" likely to try to coverup this, plus the sales pitch was that Rybka was a <> engine rather than a fast one. Mr. Letouzey appears unminding because Rybka only copied ideas and not GPL code."

I take from this that everybody copies from everybody else. So Rajlich taking the moral high ground here sounds a bit hypocritical to me.

Of course it depends on what exactly we are talking about. It seems clear to me that everybody steals from everybody else. But making an exact copy of a commercial closed source program and then releasing it into the wild under another name for free is wrong. (IF that is what actually happened.)

From the video interview with I take it that the cloud computing thing is a godsend for software developers as it enables them to make improvements without having to worry about everybody else copying them. (Of course it also allows programmers to copy stuff themselves without having to worry about anybody else finding out.)

Arne Moll's picture

@Onestone: Rajlich himself calls the programs open source, so now I'm confused myself. I'm no expert but if the source isn't open how can the source even be compared with other programs such as Rybka?

@Felix: It's really impossible to remove all links to sites that potentially link to the download pages. You really can't expect people to check this all the time - we are not living in China! As Jan Smeets notes, a simple Google search will suffice. Surely you're not implying we should stop mentioning Google as well? I think it's just the nature of the internet that pages are linked to each other and create a, well, a web. I don't see how it's possible to fight this and what's the point.
By the way, I did write in my article that the wikispace-page contains 'incoherent nonsense', so readers are warned. Isn't this enough?

Felix Kling's picture

Arne, afaik this page is the main website of that clone. If it would just link to some other page, this would be something different.

Antichrist's picture

Downloaded Firebird and Ivanhoe today. Rybka just died on me so I switched from Little Fish to Big Bird - ABC, simple as Sesame

Onestone's picture

@Arne: Rajlich called an unspecified engine "open-source" (presumably the original Ippolit). Firebird came several months later. Do you see Firebird's source code available anywhere on its site? I don't. This TalkChess thread discusses that further: http://talkchess.com/forum/viewtopic.php?t=33139 (registration is required)

CAL|Daniel's picture

Rajlich's idea to have future versions of rybka login from servers is the dumbest thing I've ever heard. I wouldn't pay 2 cents to use that bs. If I can't install a copy on my computer, you can forget the whole thing. In this way, he is PUSHING users to find other computers like these supposed 'clones'

Chris Falter's picture

@ test (If Rybka was based on some other open source program that was freely available, I don’t see how this is wrong.)

I do. The license of open source programs generally prohibits use of the code in commercial products. So if Rybka programmers lifted and used Fruit source code, it's quite possibly illegal.

Now the Rybka folks say that they didn't actually use the source code, they just reverse engineered it. (They studied it carefully and emulated it, which is the same thing.) Thereby the Rybka team acknowledges that reverse engineering is acceptable. The Decembrists obviously didn't have Rybka source code; the most they could have done is reverse engineer. So how could Rybka accuse the Decembrists of wrong-doing? They themselves did the same thing.

test's picture

Cloud computing has a lot of advantages. Google Docs is an example and presumably quite popular.
It also has disadvantages. You need an internet connection. And you do not own a copy of the application. So if the owner of the application decides "no more soup for you" then that's it, you are out of luck. Another aspect is privacy. For examle as a company I would not trust my documents into the hands of some other corporation like Google, no matter how "Don't be evil" they pretend to be.

For a chess engine it makes a lot of sense. It's cost effective or cheaper if you are interested in always having the latest hardware and the latest version available, or as Rajlich put it always having the strongest version in existence.

Jo's picture

Bravo Arne! Awesome article

test's picture

On the CCRL Discussion Board they say they are not testing these new engines because and I quote: "We will not test any engines or derivatives of engines that have anonymous authors."

They could have deleted the part "or derivatives of engines" because it is redundant. They don't test because the authors are anonymous.

I agree that it's funny that they remain anonymous, but it does not prove anything. Does anybody have any concrete proof about the supposed cloning or whatever it is btw? I haven't seen it yet while googling around about this and am about to stop now. ;)

redwhitechess's picture

I will just jump in here without reading long post above.

Usually I found free software is OK only for light working. I used OpenOffice,but if you think MS Office is suck, this free Openoffice will be more frustrated you. Same goes to LINUX, I've been waiting 10 years+ to see how Linux can answer my profesional work, but till now I better buy proprietary software because they are realiable. You spend money for the reason. If you compare free software like GIMP vs Photoshop that is a wide differences that explain why Photoshop is not free.

For Chess, since most users need no / less ergonomic, can stand more crashes, the free software written by non commercial programmer is still realiable .

appleby's picture

Out of the ippolit family Ivanhoe is the strongest and has the most cutting edge release cycle.
Ivanhoe's version numbers are slightly confusing, The lower the version number the newer the release.
i.e Ivanhoev73 is older than Ivanhoe63 (current)
Whats most striking is that Ivanhoev63 is at least 100 elo points stronger at longer time controls also ( 50/3 ) on a 8 core i7 970

I agree with the article, since there is allot of people who want the truth of a chess position regardless of the chess engines name and are bored by the red-tape antics of the commercial enterprises.

In my opinion we have a right to use whatever gives the most accurate analysis, be it commercial or GPL

Here are some links to the latest version:

http://depositfiles.com/files/pfe3ele52
http://www.mediafire.com/?jzt0mmz3zmz
http://speedshare.org/download.php?id=4764FD4711
http://www.zshare.net/download/7429917678ad0939/

/R

Pages

Your comment

By posting a comment you are agreeing to abide our Terms & Conditions