Topic Rybka Support & Discussion / Rybka Discussion / Shredder is Computerchess Software world champion in Japan
1
2
Win to 8 : 9 Points !!!!!!!!!
![[de]](/mwf/v12/flags/de.png "Germany")
there's no such title :)
![[no]](/mwf/v12/flags/no.png "Norway")
mmmh ok. but here http://www.grappa.univ-lille3.fr/icga/tournament.php?id=210 it's not called like that :) I'm confused :)
I also don't get why this should be a software championship - obviously cluster capability is not rewarded there :)
I also don't get why this should be a software championship - obviously cluster capability is not rewarded there :)
> mmmh ok. but here http://www.grappa.univ-lille3.fr/icga/tournament.php?id=210 it's not called like that :) I'm confused :)
But here it is: http://www.grappa.univ-lille3.fr/icga/event_info.php?id=41
> I also don't get why this should be a software championship - obviously cluster capability is not rewarded there :)
I would think to determine the best chess software. To bad Rybka was not one of the participants.
eh to determine the best chess software can't be the goal of a tournament with 8 rounds :) that's what the rating lists are for.
What was the goal of WCCC then? No, seriously.
the goal of the WCCC is to have the best engines competing against each other (that's fun for spectators and interesting for programmers). It shows what is possible in computer chess today.
It's easy to really find out which engine is the strongest, that's what rating lists are for. but that's a completely different thing.
It's easy to really find out which engine is the strongest, that's what rating lists are for. but that's a completely different thing.
By your logic, the world chess champion would also be chosen from the ratings lists, rather than by a head to head match. This is a fallacy of course. There is a difference between performance against the masses and performance against a select group.
It is perfectly legitimate for the winner of the equal hardware tournament to trumpet his victory in this format. This is the most relevant tournament for the 99.9999% of players who don't have a cluster or cluster software. I am guessing that Team Rybka did not compete in this because they no longer care about non-cluster Rybka.
It is perfectly legitimate for the winner of the equal hardware tournament to trumpet his victory in this format. This is the most relevant tournament for the 99.9999% of players who don't have a cluster or cluster software. I am guessing that Team Rybka did not compete in this because they no longer care about non-cluster Rybka.
well, yes, a big test would be logical if that understanding of the title (the best engine) is applied. you could also make a rating list just for the selected group.
I don't think the qual hardware tournament has any relevance just like the main tournament has no relevance for a normal chess player. Ask the rating list guys, they will confirm that in 9 games anything can happen and drawing conclusions about the engine's strength then is totally wrong.
I don't think the qual hardware tournament has any relevance just like the main tournament has no relevance for a normal chess player. Ask the rating list guys, they will confirm that in 9 games anything can happen and drawing conclusions about the engine's strength then is totally wrong.
The variance in nine games is very high, but this type of contest is used all of the time in related venues, like human competition, over arguably more accurate methods such as use of ratings lists.
I consider the use of custom books to be an advantage in these type of tournaments when compared to the ratings lists, as they allow the engine developer to generate an opening book that capitalizes on an engine's strengths and avoids its weaknesses, rather than being judged on what is in some generic opening book.
Ultimately, the best way to test would be to run something like the current hardware restricted tournament, but in a round robin format with a much larger number of games. This would be impractical for a human tournament, but could be easily done for an engine tournament by leasing server time and running many games in parallel.
Conversely, the ultimate hardware test, which I personally find very interesting, is the least educational when it comes to relative engine strength, as nothing that is used has any solid relationship with what is commercially available. I liken this to being a fan of the Can Am series races, where one could be entertained by Jim Hall's outrageously innovative cars which had absolutely no bearing on the reality of ordinary people. Sorry to use an example from decades before you were born...
I consider the use of custom books to be an advantage in these type of tournaments when compared to the ratings lists, as they allow the engine developer to generate an opening book that capitalizes on an engine's strengths and avoids its weaknesses, rather than being judged on what is in some generic opening book.
Ultimately, the best way to test would be to run something like the current hardware restricted tournament, but in a round robin format with a much larger number of games. This would be impractical for a human tournament, but could be easily done for an engine tournament by leasing server time and running many games in parallel.
Conversely, the ultimate hardware test, which I personally find very interesting, is the least educational when it comes to relative engine strength, as nothing that is used has any solid relationship with what is commercially available. I liken this to being a fan of the Can Am series races, where one could be entertained by Jim Hall's outrageously innovative cars which had absolutely no bearing on the reality of ordinary people. Sorry to use an example from decades before you were born...
![[us]](/mwf/v12/flags/us.png "United States")
Your statement is correct about humans and competition. And I think its also true that given exact (or nearly the exact) same hardware, a computer software tounament with 9 rounds does give bragging rights to the winner and 2nd/3rd place even.
As far as it not being overly accurate (as pointed out by other posts), I think bragging rights in a tournament + engine rating lists should both be used by people wanting to purchase chess software. Even if a particular software doesn't have the highest engine rating, it would still be considered a worthy engine if it can win tournaments.
I think the world chess software championship has more relevence for many people (over the open hardware "world computer chess championship"), if only because the software tournament shows which engines can do very well on a home computer.
I'm not particularly impressed by Rybka's performance in the World Computer Chess Champtionship, because very few (are there any?) people want to put together a 200 core cluster computer to play chess.
(Ya, I know, Rybka Cluster isn't available for purchase anyway. It would be almost impossible to sell it, since it so highly tuned for a specific hardware system, among other items)
Don't get me wrong, I think having cluster chess computers is a great thing, but more from a analytical setting, or in an engine room were the cluster plays many games against a large variety other engines, rather than in a head to head shootout competition with X number of rounds to decided the winner.
As far as it not being overly accurate (as pointed out by other posts), I think bragging rights in a tournament + engine rating lists should both be used by people wanting to purchase chess software. Even if a particular software doesn't have the highest engine rating, it would still be considered a worthy engine if it can win tournaments.
I think the world chess software championship has more relevence for many people (over the open hardware "world computer chess championship"), if only because the software tournament shows which engines can do very well on a home computer.
I'm not particularly impressed by Rybka's performance in the World Computer Chess Champtionship, because very few (are there any?) people want to put together a 200 core cluster computer to play chess.
(Ya, I know, Rybka Cluster isn't available for purchase anyway. It would be almost impossible to sell it, since it so highly tuned for a specific hardware system, among other items)
Don't get me wrong, I think having cluster chess computers is a great thing, but more from a analytical setting, or in an engine room were the cluster plays many games against a large variety other engines, rather than in a head to head shootout competition with X number of rounds to decided the winner.
I think one argument is that you can be interested in two seperate things:
1) Which engine is strongest on equal hardware, scientifically speaking.
2) Which automated chess entity is the current king of the hill in a sporting event (like, formula 1 for racing cars).
My point of view is that a world championship or a chess match is a sporting event. You are not seeking any scientific answers, but you are rather organizing an event to allow someone to call himself the current "king of the hill". You attach specific significance to a designated event. Just like Spain is king of the hill in soccer because they won the WC (and not some series of obscure training matches). As such, you are interested in PEAK performances, the best of the best on a given day (or weak, or month).
While on the other hand, you have the scientific curiosity where you are interested in finding out which entity is the best one in principle, that is, you play enough games to rule out randomness, and you enforce equal hardware and openings (or in the case of soccer, you enforce neutral fields and you play over so long a period so that injured players are cancelled out of the equation).
In my view, you have two excellent options for answering these two questions. One is a WC where everything is allowed, so that you get the PEAK performances. The other one is the tests performed by the rating groups where they play countless games on equal hardware and openings.
From this point of view, an equal hardware WC adds nothing significant to the whole equation. It is practically redundant, since it only goes towards answering a question that the rating groups are answering anyway within the first week of their extensive testing procedure lasting many months.
On a more aggressive note, you could say that an equal hardware WC actually undermines the "real" WC (where PEAK performance is on the table) and as such is counterproductive. It takes away value from the "real" WC.
So that could be an argument against participating in an "equal hardware" WC.
Please note that I have absolutely no insight into the background of Rybka not participating in the equal hardware WC.
I just find that, in general, the arguments put forward for "being interested" in an equal hardware WC are not particularly good, since the answers sought from this thinking are answered 100 times better by the rating groups anyway.
1) Which engine is strongest on equal hardware, scientifically speaking.
2) Which automated chess entity is the current king of the hill in a sporting event (like, formula 1 for racing cars).
My point of view is that a world championship or a chess match is a sporting event. You are not seeking any scientific answers, but you are rather organizing an event to allow someone to call himself the current "king of the hill". You attach specific significance to a designated event. Just like Spain is king of the hill in soccer because they won the WC (and not some series of obscure training matches). As such, you are interested in PEAK performances, the best of the best on a given day (or weak, or month).
While on the other hand, you have the scientific curiosity where you are interested in finding out which entity is the best one in principle, that is, you play enough games to rule out randomness, and you enforce equal hardware and openings (or in the case of soccer, you enforce neutral fields and you play over so long a period so that injured players are cancelled out of the equation).
In my view, you have two excellent options for answering these two questions. One is a WC where everything is allowed, so that you get the PEAK performances. The other one is the tests performed by the rating groups where they play countless games on equal hardware and openings.
From this point of view, an equal hardware WC adds nothing significant to the whole equation. It is practically redundant, since it only goes towards answering a question that the rating groups are answering anyway within the first week of their extensive testing procedure lasting many months.
On a more aggressive note, you could say that an equal hardware WC actually undermines the "real" WC (where PEAK performance is on the table) and as such is counterproductive. It takes away value from the "real" WC.
So that could be an argument against participating in an "equal hardware" WC.
Please note that I have absolutely no insight into the background of Rybka not participating in the equal hardware WC.
I just find that, in general, the arguments put forward for "being interested" in an equal hardware WC are not particularly good, since the answers sought from this thinking are answered 100 times better by the rating groups anyway.
> I just find that, in general, the arguments put forward for "being interested" in an equal hardware WC are not particularly good, since the answers sought from this thinking are answered 100 times better by the rating groups anyway.
But for me they are not. I find the sporting element to be more exciting than ratings tests. And for me I find the sporting element of an equal hardware even to be even more exciting and interesting to follow. For in the back of my mind I am not wondering if hardware had anything to do with it so I can follow the event as an even sporting event.
Best engines? Stockfish, Naum, Komodo, Spark, Critter, Deep Sjeng, Hiarcs the strongest engines according to rating lists (using your definition of the strongest) were not there. Does it make Rybka's title less worth now? I don't think so and same applies to Shredder's title as well.
Second, the engines in the tournament are not the same engines that are tested and listed by the various rating lists.
Second, the engines in the tournament are not the same engines that are tested and listed by the various rating lists.
Well, the best engine was playing and Rondo is private afaik, but shouldn't be that weak. However, it would have been even better with the other strong engines also competing (like Sjeng).
Those engines that played aren't listed in rating lists, sure. If you would want to know the "real" strength of them, you would have to make a 1000 games test or something like that. Otherwise the rating lists could stop engines tests after 10 games :)
Those engines that played aren't listed in rating lists, sure. If you would want to know the "real" strength of them, you would have to make a 1000 games test or something like that. Otherwise the rating lists could stop engines tests after 10 games :)
It's a competition, World Championship. The winner, strongest or not it doesn't really matter, is the World Champion. Same is with human chess. Anand is number two on the list (unofficial live list), nevertheless he is the Word Champion. True, Rybka is probably the strongest engine, but it's irrelevant here. Rybka team chose not to participate and have to live with Shredder being a World Champion this year. Sad, but a fact.
Just out of curiosity, do you consider the German football team best because they are the best on paper, or the Spanish team because they won the World Cup?
I bet the rankings were rather different before the results of the World Cup were factored in...
Sure, but not regarding Spain vs. Germany. Though, I do think Germany is better, Spain nevertheless was and still is higher ranked.
EDIT: The original list was missing a few countries.
November 2009 - February 2010
1. Spain
2. Brazil
3. Netherlands
4. Italy
5. Portugal
6. Germany
March 2010
1. Spain
2. Brazil
3. Netherlands
4. Portugal
5. Italy
6. Germany
April 2010 - June 2010
1. Brazil
2. Spain
3. Portugal
4. Netherlands
5. Italy
6. Germany
July 2010
1. Spain
2. Netherlands
3. Brazil
4. Germany
8. Portugal
11. Italy
EDIT: The original list was missing a few countries.
November 2009 - February 2010
1. Spain
2. Brazil
3. Netherlands
4. Italy
5. Portugal
6. Germany
March 2010
1. Spain
2. Brazil
3. Netherlands
4. Portugal
5. Italy
6. Germany
April 2010 - June 2010
1. Brazil
2. Spain
3. Portugal
4. Netherlands
5. Italy
6. Germany
July 2010
1. Spain
2. Netherlands
3. Brazil
4. Germany
8. Portugal
11. Italy
I am at a severe disadvantage here, discussing football, but I thought I read an article about how good the German team should have been, based on the team members, as opposed to how well they actually performed on the field. The rankings would be expected to reflect the teams performance on the field...
In any event, the ratings you've provided seen to be very well correlated with WC results, and after seeing these lists, I am much less impressed by Paul the Octopus.
In any event, the ratings you've provided seen to be very well correlated with WC results, and after seeing these lists, I am much less impressed by Paul the Octopus.
I think the rating list depends on team's past results, hence the tight correlation. But frankly, I don't believe it represents team's true strength.
Norway at 14th place and in 1995 we even were 2nd. As much as I'd love it to be true... we're nowhere near the top.
Norway at 14th place and in 1995 we even were 2nd.
As much as I'd love it to be true... we're nowhere near the top.
Norway seems to have a number of structural disadvantages that would make it difficult to reach the top:
- Small population
- Weather isn't conducive to practicing 365 days a year
- People are too polite
There's also the possibility the country is too prosperous. I suspect that in Brazil, you get a fair number of great players who grew up very poor with football being their only chance for a good life (same as here in the US with basketball). Maybe in Spain and Portugal and even Italy as well.
- Small population
- Weather isn't conducive to practicing 365 days a year
- People are too polite
There's also the possibility the country is too prosperous. I suspect that in Brazil, you get a fair number of great players who grew up very poor with football being their only chance for a good life (same as here in the US with basketball). Maybe in Spain and Portugal and even Italy as well.
![[gb]](/mwf/v12/flags/gb.png "United Kingdom")
and lets keep in mind spain lost their first game vs the mighty switzerland. thus it isnt wise to read too much into one result.
![[es]](/mwf/v12/flags/es.png "Spain")
> Maybe in Spain and Portugal and even Italy as well
As for Spain, that was 60 years ago
...and then again, maybe things were better 60 years ago! Spain might have been poor under the Generalissimo, but it didn't have 20% unemployment, zombie banks, a busted housing market, etc., etc., with more to follow. (Don't think for a minute I think things are different over here.)
you are right here. Current situation is quite sad and difficult in my country. Let's see if we are able to recover...
and if you are talking about how good a team should be on paper, well that is why england were one of the big fav's going into the last WC with players like rooney and gerrard etc (who never got it going, after a long and tiring english season methinks), facing the unknown -21 kids of germany. thats how we saw it anyway. just shows how different real life is from how it looks like on paper...eh
the football ratings are valid as from oct 2010, which is why england have moved one position up after their last few convincing wins in euro qual's. how germany would ever be at top of any football ratings over past 20 years is beyond me. i mean, a few freek results in the world cup (a tourney very much like the computer chess WC in that the best teams from the ratings list are not the main competitors) after facing a couple of underprep'd teams, and having a clear goal disallowed v england - who had started to get on top of them and had just levellled with a great goal!! does not make a #1 team. and holland #2 is just a joke as well. one swallow does not make a summer, as they say.
regarding the title: ok, apparently I'm wrong :)
..and isn't it 7/8? :)
![[nl]](/mwf/v12/flags/nl.png "Netherlands")
but nobody from Rybka team has yet clarified why Rybka did not participate in the software section with Rybka 4
a) forgot
b) afraid to loose
c) irrelevant tournament
d) let's give the runners up a shot at a price as well
seriously, why??
a) forgot
b) afraid to loose
c) irrelevant tournament
d) let's give the runners up a shot at a price as well
seriously, why??
e) the Rybka team team are all volunteeers, were incredibly busy, they all have day jobs, with the time difference in Japan, it was simply asking too much of them ?
Even with Rybka's Elo advantage, with a short tournament there was a good chance they wouldn't have won without the hardware advantage.
My guess is b+c
Rybka can only lose status in these events even when it wins there is always the question about the hardware factor.
The positive news from this event was Rondo. I'm looking forward to playing with this engine in the future.
Rybka can only lose status in these events even when it wins there is always the question about the hardware factor.
The positive news from this event was Rondo. I'm looking forward to playing with this engine in the future.
how about shredder?
guess this was a significantly improved version over ds12 and wonder when it will be released.
guess this was a significantly improved version over ds12 and wonder when it will be released.
Well, I already own Shredder :)
maybe vas didn't want to let the private Rybka version run on those computers :)
> maybe vas didn't want to let the private Rybka version run on those computers :)
Was he afraid all the new and improved bugs in the private Rybka would crash those computers??
He could have played with Rybka4. It's stronger than Shredder right?
Are you sure the Shredder that played at WCCC was the same as what is currently being sold? Perhaps SMK had an epiphany and the Shredder Beta is 100 Elo stronger! OK, time for me to wake up...
I am quite sure this shredder version is much improved.
Good point. No way Vas would let his latest and greatest run out in the wild.
![[mx]](/mwf/v12/flags/mx.png "Mexico")
My money was on c but this apparently is the official reason, Vas afraid of someone hacking the extraneous computer, or otherwise getting the Rybka executable, which would doom his remote model, and even Rybka 5, etc. Too many risks, reward too small.
> My money was on c but this apparently is the official reason, Vas afraid of someone hacking the extraneous computer, or otherwise getting the Rybka executable, which would doom his remote model, and even Rybka 5, etc. Too many risks, reward too small.
I agree, wasn't worth the risk for him
It still doesn't answer the question: "So why didn't they play on the WSCS with some stock commercial Rybka 4"?
Right there, nothing to risk, Rybka would have won easily, so my money again is on c.
Right there, nothing to risk, Rybka would have won easily, so my money again is on c.
The probability of Rybka 4 winning a short tournament on the given hardware was pretty high, but remember that Vas is selling to a different group now and wouldn't want to tarnish the reputation of his cluster software if by chance R4 didn't win the same hardware competition.
That's very few trust on his commercial product's quality on his side.
So we have "the tourney was irrelevant" as one option, "very few confidence on the performance on the commercial version on small hardware" as another, perhaps even "we never thought about the possibility of using commercial Rybka instead of the last development version".
Paul commented that "Rybka lost by not joining", are we again on "the risk was too high if Rybka joined and lost"? Because I feel the Cluster's reputation is being damaged more by the games it has been drawing against supposedly weaker opponents, making me doubt if commercial Rybka 4 on a Quad would have done worse than the Cluster.
So we have "the tourney was irrelevant" as one option, "very few confidence on the performance on the commercial version on small hardware" as another, perhaps even "we never thought about the possibility of using commercial Rybka instead of the last development version".
Paul commented that "Rybka lost by not joining", are we again on "the risk was too high if Rybka joined and lost"? Because I feel the Cluster's reputation is being damaged more by the games it has been drawing against supposedly weaker opponents, making me doubt if commercial Rybka 4 on a Quad would have done worse than the Cluster.
I think the days of Rybka UCI are over. I wouldn't be surprised if going forward, Rybka competed only in events that allowed running on the cluster. Vas doesn't seem like the kind of guy who would be happy competing with a much weaker engine, like R4.
Topic Rybka Support & Discussion / Rybka Discussion / Shredder is Computerchess Software world champion in Japan
1
2
Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill