Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / In retrospect (locked)
1 28 9 10 11 Previous Next  
Parent - - By bob (Gold) Date 2011-07-15 18:47
Our report did not address "functional equivalence" regarding the evaluation.  Mark took it term by term, and described how each program did the evaluations, what numbers were used, etc.  Your "evaluation number" comparison can suggest copying if two programs produce nearly identical matches.  But just because they don't, that does _not_ imply that either is original, for reasons that are both obvious, and previously explained here.
Parent - - By Banned for Life (Gold) Date 2011-07-15 19:06
No code is provided in the comparison for reasons that are both obvious and previously explained. Lying about this won't change anything.
Parent - - By bob (Gold) Date 2011-07-15 19:26
No code is provided for the evaluation because the data structures make it much more complicated to compare.  But mark's point by point comparison, using text rather than code, is easy enough for anyone to understand.  If they actually read the report, that is...  I will point out that Zach's report does include some code comparisons...  That was a part of the overall report...
Parent - - By Banned for Life (Gold) Date 2011-07-15 19:45
No code comparison is provided for the evaluation. Mark shows similarities in order, data structures, and values for some features. It is easy enough to understand, and doesn't show code copying. Zach's code contains commented assembler from two very old rybka versions with no tie ins to fruit, and a comparison of an old Rybka's UCI parser and time management to Fruit, showing reuse. An honest assessment would not have to exaggerate, or embellish the facts.
Parent - - By bob (Gold) Date 2011-07-15 19:56
If you do the same things, in the same order, using the same scoring terms, and the same tests on when to apply them, what do _you_ assume?  serendipity?  You buy into that 0.000000000001% probability stuff?  How many lotteries have you won, btw???

Those of us that actually _write_ complex code (and I have written assemblers, compilers and operating systems for many years, in addition to chess programs) know that two different people won't do the same things, in the same way, in the same order, using the same values, accidentally.  Just doesn't happen.  Doesn't happen in 100 line student assignments where they _are_ given the same software requirements.  Doesn't happen in a 2000-5000 line evaluation either.
Parent - - By Banned for Life (Gold) Date 2011-07-15 20:20
There has never been a claim that Rybka was written without knowledge of what was done in Fruit. On the contrary, it has always been acknowledged that Fruit was studied carefully. There is no claim of independence here, and no claim that the similarity to Fruit is coincidental.

The proper statement for your student assignment would be: You give your students an assignment to write. Then you show them what you consider to be a great solution from the same assignment given in a previous year. They are allowed to study it and take notes, but can't hold on to it. The students go away, complete the assignment, and turn it in. Would you be surprised if the assignments had similar structure to the model you provided?

This happens frequently when answering solicitations, where sponsors frequently provide a model proposal (from a previous contract). Not surprisingly, all the proposals are likely to have many characteristics in common with the model proposal. Copying or even paraphrasing text would be inappropriate and unlawful. Copying the structure is fine.
Parent - - By bob (Gold) Date 2011-07-15 22:08
Yes I would be surprised.  Because in one class, the book we use has source code that is close to some assignments.  I warn them, and they write their own original code that meets the assignment requirements, without copying a thing, and without writing code that looks anything like anybody else's...
Parent - - By Banned for Life (Gold) Date 2011-07-15 22:13
OK, so your students aren't smart enough to work from a template and they value originality above productivity. I hope they stay in academia because this is a sure-fire loser in the commercial world.
Parent - - By bob (Gold) Date 2011-07-15 22:17
Pretty funny since students of mine seem to excel in the software world.  Not everyone is a copy-n-paster.  Some actually have original ideas that can push progress along a bit.  Others only know how to hit the "copy" button on the xerox machine.  Most computer science students are not in that camp.
Parent - - By Banned for Life (Gold) Date 2011-07-15 22:22
It might be pretty funny to hear your idea of excelling. These people couldn't possibly work as part of a large software team if they value originality over everything else. Only an academic could espouse this nonsense with a straight face.
Parent - - By zwegner (***) Date 2011-07-15 22:27
Nonsense.
Parent - - By Banned for Life (Gold) Date 2011-07-15 23:08
Hi Zach,

I apologize for any collateral damage that has resulted from my joust with Bob.

That said, as you no doubt are aware, a high percentage of the dollar volume in software engineering is wrapped up with modifying existing software. In the general case, that software is likely to have been written by someone else. If the software was written by someone else in the same company, there are no rights issues, and the faster the software team or individual can make the specified changes, the more valuable it/he/she is (where 400 production lines of code a month is considered very high productivity). If the software was written by someone at another company, it's make-buy decision time. The best way to go is frequently to buy the rights from the owner and proceed as before (I do this kind of thing all the time). The alternative is to recode whatever it is. There are all kinds of tools for estimating how much this will cost, but (at least in my experience) they generally aren't any more accurate than a back-of-the-envelope calculation based on Software Engineers estimate of time to complete X 3 (optimism scale factor) X loaded rate (generally ~$200 an hour for a senior software engineer). I've never met a Software Engineer yet who wouldn't prefer to roll his own, but this is almost always the more expensive solution.

If you think about it, modifying someone existing engine code is not terribly dissimilar to the "preferred" method of development described above. So, while I understand why engine developers don't like to see code reuse, engine developers should also understand why reusing code whenever possible makes sense commercially. Your feedback is of course appreciated.
Parent - - By zwegner (***) Date 2011-07-15 23:55
Hi Alan,

> I apologize for any collateral damage that has resulted from my joust with Bob.


No problem--I figured it was hyperbole.

From my experience, using third party code is usually a bad thing in the long term. As you are probably aware, by far the vast majority of software is horribly written. Our group at work is rather notorious for throwing out other peoples' software (even those from other groups in the same company) and rolling our own. I suppose this sort of thing is different everywhere, depending on the project--there isn't really that much third party code that is applicable to what we do, and we are pretty hardcore about code quality. For the standard business software, I'm sure there's millions of lines of "rote" code that I'd rather quit my job than write myself.

Copying source from other engines isn't a bad thing IMO (I released the source to ZCT for a reason). My problem is when it is done dishonestly. But I really think that code reuse for a chess engine is pretty much useless, unless you copy the whole engine. A chess engine is very much "more than the sum of its parts". It needs to have a simple, solid, foundation, with every component working in tight synergy. It needs to be programmed in such a way that all of these components are easily changeable, and IMO no open source software comes close to this.

I guess you haven't realized this yet, but if any engine developer was really concerned with what makes sense commercially, they wouldn't be doing computer chess. :)
Parent - - By Banned for Life (Gold) Date 2011-07-16 07:47
Interesting response on many different levels. I've jotted down some random thoughts. Sorry I'm so long winded...

From my experience, using third party code is usually a bad thing in the long term. As you are probably aware, by far the vast majority of software is horribly written. Our group at work is rather notorious for throwing out other peoples' software (even those from other groups in the same company) and rolling our own. I suppose this sort of thing is different everywhere, depending on the project--there isn't really that much third party code that is applicable to what we do, and we are pretty hardcore about code quality.

"Design it once, sell it many times" has always been my ideal. I'm not sure who came up with this, but it's crucial to productive engineering. The stuff I build consists of electronics and software. Building the electronics is a marvel of efficiency. The hardware people have no problem reusing other people's stuff, and the hardware development process has become incredibly efficient over the past thirty years. Software development is a very different story.

On the electronics side and there have been huge advances over the last thirty years. Building a finished PCB thirty years ago was a complicated process that required an engineer to design the circuit, a draftsman to draw it up, then the design had to be physically transported to a company that built PCBs. They would manually set up the process and produce your boards which would arrive a few months later. Meanwhile, you would have ordered and received your parts. The boards would then go to a production line where young ladies would solder in the parts. The stuffed boards then made their way to a technician for testing, and after he gave his OK, the boards were ready.

Today the same process can be done by anybody with a PC downloading PCB design software, designing and laying out their circuit with free software. Other free software can be used to perform PSPICE analysis to ensure the circuit works as intended and to optimize its parameters. The design can then be sent to a PCB house that will build the boards, order the parts, stuff the boards, and test the boards at low cost and with very little human intervention, all this in a two-week time-frame. Because the process is so efficient, this work can be done here in the US competitively with anywhere else in the world.

Meanwhile, software productivity has gone nowhere. As you point out, the vast majority of software is horribly written. No software person or group wants to reuse others code. Everyone is Picasso trying to develop a masterpiece, but generally with abysmal results. This does not bode well for the future. It takes way too long to get things done, and it makes it attractive to outsource software development to places with lower labor rates, e.g. India. I would be very worried about this if I were a young software developer in the US.

I really think that code reuse for a chess engine is pretty much useless, unless you copy the whole engine. A chess engine is very much "more than the sum of its parts". It needs to have a simple, solid, foundation, with every component working in tight synergy. It needs to be programmed in such a way that all of these components are easily changeable, and IMO no open source software comes close to this.

I've always been struck by the heavy emphasis in the chess engine community on programming, rather than on algorithm development and selection. There is probably some historical explanation for this, but it doesn't seem to closely reflect what you have in the commercial world where you would generally have algorithms designed, modeled, and documented first, before even thinking about cranking out code. One of the important things to characterize at the onset would be interaction between algorithms. As an admittedly very simplistic example, this would show if a chess engine's search function's speed was increased, how much more of of a bottleneck would result from the time required to run the evaluation function.

With the typical target based software model used for chess engine testing, you have the advantage of ultimate realism, but you don't get the visibility you would get from a higher level model. So you end up seeing "I tried such and such, but it didn't work for me" a lot, with the implication that the developer didn't really have a clue what was happening, but only saw that it didn't improve competitive results.

I guess you haven't realized this yet, but if any engine developer was really concerned with what makes sense commercially, they wouldn't be doing computer chess. :)

Computer chess, hell chess in general, is an interesting hobby. There are really only a very small number of people making a good living from it. But as is frequently the case, people get really passionate about the not-so-important stuff. As I like to point out, nobody gets very excited when they hear that Apple might have filched some of Kodak's IP in the iPhone they use every day. But start talking about IP infringement in engine software, and people are ready to kill each other...
Parent - - By bob (Gold) Date 2011-07-16 15:14
A chess program is not designed like any other software in the world.  In a chess engine, there are things that must be done, but programmers spend literally _years_ trying to write the code as efficiently as possible, because speed is critical.  We do some things incrementally, we compute some things on the fly.   Crafty is currently right at 45,000 lines of code.  It has had as much effort expended as a 450,000 line program, and most likely more.  The final goal is not "something that takes the given inputs, and produces a correct output."  The goal is much more than that.  All of us have spent 1000x as much time optimizing and rewriting pieces to be as fast as possible, as we have expended on the basic design of any single part of the engine.

So comparing this to other software projects doesn't fly.  I remember a comment by someone up at Cray many years ago, saying "Bob/Harry, this is the _first_ piece of code that does anything useful, that has zero wait cycles from top to bottom."  (a comment on how well optimized it was, that any memory access was done enough cycles in advance of where it was actually needed, that there were no delays.  That no register was used until enough cycles had elapsed so that the value was ready.  etc.)  How big was the code?  maybe 50 lines of fortran.  just over 900 lines of Cray Assembly Language.  Several years of tweaking to get it to the "perfect" performance stage.

And that same effort is applied everywhere.  That's one of many reasons why we consider "copying is verbotten..."
Parent - - By Banned for Life (Gold) Date 2011-07-16 18:56
A chess program is not designed like any other software in the world.

The only sense in which this is true is that there is no real market for this software, and it doesn't really affect people's lives in a significant way.

In a chess engine, there are things that must be done, but programmers spend literally _years_ trying to write the code as efficiently as possible, because speed is critical.

Come on Bob. The resources expended on chess engines are insignificant. I would guess that at least 10,000 times more effort has been expended on different algorithms in my cell phone: Downmix, Viterbi Decoder, Demodulation, Interference Rejection, etc, just to pull a few algorithms from the front end processing. Each of these has been optimized to reduce the end cost by as little as a fraction of a penny, befitting the fact that a billion are made each year. I could give thousands of other examples. If people were willing to pay tens of billions of dollars a year for chess engines, the engineering effort would be a lot more structured. They're not, so the effort is seat-of-the-pants.

So comparing this to other software projects doesn't fly.  I remember a comment by someone up at Cray many years ago, saying "Bob/Harry, this is the _first_ piece of code that does anything useful, that has zero wait cycles from top to bottom."  (a comment on how well optimized it was, that any memory access was done enough cycles in advance of where it was actually needed, that there were no delays.  That no register was used until enough cycles had elapsed so that the value was ready.  etc.)  How big was the code?  maybe 50 lines of fortran.  just over 900 lines of Cray Assembly Language.  Several years of tweaking to get it to the "perfect" performance stage.

A statement by anyone claiming that a chess engine does anything useful is immediately suspect, but that's beside the point. People decide to purchase one HPC rather than another based on work that may require finite element analysis, large array corner turning operations, computation fluid dynamics, weather prediction, etc. One would hope that Cray and other HPC venders put orders of magnitude more effort into these important tasks than you put into your chess engine.
Parent - - By bob (Gold) Date 2011-07-16 19:15
You might suspect that your statements are true, but they are not.  Many of those algorithms you mention are far larger than a chess program.  Look at the effort to tune/optimize any single piece that is maybe 10K lines of code (the actual engine) and you might see that chess wins hands down.  I've been tuning/optimizing Crafty since 1995.  It came from Cray Blitz which dates back to Blitz which goes back to 1968.  And a major part of the effort has been what do I do, when do I do it (so that I can defer and possibly never have to do it if the branch gets pruned later), and how do I do it with as few cycles as possible.  To understand the issue, spend 6 months writing a basic program that can play chess.  Then let's compare speed.  Mine will certainly be faster.  Then lets play a game to highlight exactly what you left out, and what I included but in a way that makes it extremely fast.

There is much more optimization in chess than you have any idea about.  I've done modulation/demodulation code in modems algorithms I fooled with back in the days when CODEX and 2400 bps SYNCH was common.  Nothing like this.  The algorithms are not that complicated.  Optimizing a straightforward algorithm and optimizing a computer chess program are on completely opposite ends of the spectrum.  You might contact my Cray Blitz partner, Harry Nelson, he's in the phone book in Livermore CA.  He optimized _everything_ that ran on a Cray.  Fission and Fusion simulations for the LLNL/LANL weapons programs and DOE energy projects, weather forecasting codes, and of course Chess.  Ask him about the challenges and which he thinks was the more complex task, a 7M line Fusion simulation or Cray Blitz.  You might be surprised.

Being exposed to other technological issues does _not_ prepare you for computer chess at all...

As far as the Cray personnel comment, there are pieces of code that will issue as many instructions per clock as a specific processor can do.  Such have been done for X86, in fact.  But they do not do useful computation, they just try to reach the theoretical max somehow.  We reached it in exactly one function, which was used in a real program.  That's where the comment came from.

One can always hope for anything.  I know what kind of effort I have put into optimization over the years.  From the simple temporal/spatial locality issues that influence cache, to memory layout to assist pre-fetching, to the operations I use and in what order to avoid pipeline stalls, to the design of the program so that I always strive to practice procrastination.  In chess-speak, never do now what you can put off until later, because with pruning, sometimes later _never_ comes.  But you have to have walked the walk before you talk the talk.  Writing a chess engine is not a daunting task.  Writing a chess engine that is competitive, by being both fast and smart, _is_ a really daunting task.  But no one realizes how daunting until they actually pull it off.  Then they "get it".  And they realize the those that copy are really cutting out many years of development time they should have had to spend to get to that point...

I could tell you stories about how much Cray helped us, to a point where right before a WCCC, the vice-president called me and asked if it would be possible to borrow a couple of their system people to fix a system problem that was causing problems elsewhere.  Each time she asked one of the group if they could look at this, they responded, "I am helping Bob and Harry by trying to fix a MP library problem that is causing the C library to hang in a rare case..."  So Pat called me and said "since all these guys think they are working for you, can you ask one of them..."  All in fun of course, but there was massive man-hours invested in CB, and in Crafty.  More than you might imagine could be spent on a program whose kernel is around 10K lines of code or so.
Parent - - By Banned for Life (Gold) Date 2011-07-16 22:02
Your argument has several serious disconnects:

Look at the effort to tune/optimize any single piece that is maybe 10K lines of code (the actual engine) and you might see that chess wins hands down.  I've been tuning/optimizing Crafty since 1995.  It came from Cray Blitz which dates back to Blitz which goes back to 1968.  And a major part of the effort has been what do I do, when do I do it (so that I can defer and possibly never have to do it if the branch gets pruned later), and how do I do it with as few cycles as possible.  To understand the issue, spend 6 months writing a basic program that can play chess.  Then let's compare speed.  Mine will certainly be faster.  Then lets play a game to highlight exactly what you left out, and what I included but in a way that makes it extremely fast.

1) The objective of a chess engine isn't to be fast, it's to be strong. The people writing the strongest engines tend to be young men, rather than old men, and frequently they don't have that many years of chess engine experience. Let's look at today's top developers. In no particular order I would include:

- Robert Houdinart
- Richard
- Anthony and Zach
- Vas
- Tord

I'm not sure that any of these guys have ever worked on an HPC, let alone done micro-instruction level optimization, so this appears to be irrelevant. In fact, recent experience indicates that the quickest path to greatest strength is through making modifications to existing baselines, whether it was their own or someone elses.

The algorithms in chess are not as simple as a straight downmix stage in a receiver, but they are orders of magnitude less complex than the CFD algorithms used to solve for the time-varying pressure under a helicopter's rotor. This would be neither here nor there except that I can't fail to mention that the CFD market is many orders of magnitude bigger than the chess engine market.

Being exposed to other technological issues does _not_ prepare you for computer chess at all...

I'm not buying it. The top engine developers are no doubt very smart guys, but for every very smart guy working on a chess engine, there are tens of thousands of very smart guys working on cell phones, tvs, airplanes, and all the other items that we need and use everyday. There is absolutely no reason to believe that the world's smartest people have all gravitated toward working on one of the world's least important problems.

I could tell you stories about how much Cray helped us

Back then, computer chess was a valuable marketing tool. This was also IBM's rational for pouring so much money into the matches with Kasparov. That was then. The current situation is much different, and there is very little interest in the field.

but there was massive man-hours invested in CB, and in Crafty.

And the grand total is much less time than is spent on cell phone development in a single day.
Parent - - By bob (Gold) Date 2011-07-16 22:25
When you add "I'm not sure..." your point flies out the window.  We _all_ spend a lot of time optimizing the code.  Particularly when we get an idea to add something, and our goal is to make it "free" if possible (no computational cost).  Another key point.  If you take a given program, and make it faster, you _always_ make it stronger, assuming you know what you are doing and didn't break anything.  Speed is the most important part of a chess engine.  Otherwise why is everyone always buying new hardware to run their favorite chess engine on?

BTW, I've done my fair share of CFD work.  The chess algorithms _are_ more difficult, by far.  CFD is a computationally intensive problem, but it is one with some very precise mathematics and physics behind it.  Mississippi State University has been one of the leaders in this field, having an NSF Engineering Research Center there that specializes in CFD along with some world-class CFD people.  Chess is anything but precise, which makes it much more difficult to work with.  Only beginners equate "lines of code" with "program complexity and difficulty to write them."  I have written, by myself, several much larger software programs, well beyond 200K lines of code for each.  One to precisely simulate the hardware for a Xerox Sigma 9 computer system, including interrupts, virtual memory hardware, and I/O hardware including typical I/O delays.  I used that to do a major modification to the xerox operating system where I need to boot the system and debug it while the normal O/S was running (a VM-type application.)  I wrote a micro-program compiler for an HP2100A (a machine that was microprogrammable so that you could create your own assembly language instruction set architecture.  I wrote a basic interpreter for that same machine that allowed one to do process control programming, with simulated A/D, D/A and D/D converters, and some "virtual" hardware that one could control (a virtual sawmill, satellite tracking system, etc...)  A load and go assembler for the IBM/360 so that students could compile and run asm programs instantly, rather than using the standard IBM assembler/linkage-editor that would take 2-3 minutes per program in the early 70's.  That is enough for the discussion here.  And not a _one_ of those was even close to the complexity of a chess program.

As far as the time spent on cell phone development in a single day?  Nowhere near the equivalent of 40+ man-years, and that only counts one person.  I had Harry on Cray Blitz, and now a group of 3 helping with Crafty.  It is way more complex than you imagine.  Complexity and size, or effort expended and size have nothing to do with each other in this particular domain of programming.
Parent - - By Banned for Life (Gold) Date 2011-07-17 01:06
When you add "I'm not sure..." your point flies out the window.

I guess this is an oblique insult of some kind but being so sure of yourself is definitely not a character strength.

We _all_ spend a lot of time optimizing the code.

Sure. But the strongest engines belong to a few guys who are relative newcomers to the field and work on their engines part time.

Particularly when we get an idea to add something, and our goal is to make it "free" if possible (no computational cost).

Exactly. If your engine is much faster than the rest, but also much weaker, then the other guys must be coming up with more good ideas than you are. In the general case, speed is important, but having better algorithms is much more important.

Speed is the most important part of a chess engine.

If this were true, Crafty would be number one. It would be interesting to see what kind of time handicap Crafty would need to go 50% vs Houdini with both running on equal hardware.

Otherwise why is everyone always buying new hardware to run their favorite chess engine on?

Faster hardware provides a linear order speed increase to a problem that is exponential order. Better algorithms and heuristics are not limited in this manner. But buying faster hardware is very, very easy, so everyone does it.

As far as the time spent on cell phone development in a single day?  Nowhere near the equivalent of 40+ man-years, and that only counts one person.

There is way more than 40+ man years being spent on cell phone development every day.
Parent - - By bob (Gold) Date 2011-07-17 01:46
You only have to read.  Speed across two different programs doesn't mean a thing, any more than tachometer readings from two different cars tells you how fast you are going, or what your fuel economy is.  But for _any_ program, if you make it faster, you make it strong, if all you change is the speed.  Every time.  no exceptions.  And that is what I explained...


There is way more than 40+ man years being spent on cell phone development every day.


So every day you believe that there are 40 * 365 * 3 (assuming 8 hour shifts) or 43,800 people working on that one issue?  not on chip design.  RF radiation/reception issues, but just on the internal software?  Get real.
Parent - - By Banned for Life (Gold) Date 2011-07-17 02:37
So every day you believe that there are 40 * 365 * 3 (assuming 8 hour shifts) or 43,800 people working on that one issue?

First lets correct your arithmetic. A man year consists of approximately 240 days, less for those with more than two weeks vacation. So I am claiming that there are far more than 9600 people working on cell phone development (but there are far more than 43,800 people working on cell phone development).

  not on chip design.  RF radiation/reception issues?

Due to the scale of cell phones, custom chips are used everywhere possible. The development of these chips is certainly part of the cell phone development process and are the mainstay of companies like Qualcomm which makes most of its money from selling chip sets. Samsung, LG, Sony, Motorola, Nokia, RIM, and a long, long list of other companies are involved in developing these chip sets.

, but just on the internal software

If we were just going to include the internal software, we would be looking at huge OS development efforts by Apple, Microsoft, and Google, to support smart phones, and we would also be looking at hundreds of thousands of applications developed by tens of thousands of companies. Easily more than 43,800 people, and we haven't even mentioned all the people providing infrastructure services around the world.
Parent - - By bob (Gold) Date 2011-07-17 04:03
No, I am specifically taking your statement about the phone _internals_.  You do tend to keep shifting the argument...  You did not mention the O/S which makes no sense as I don't count that as part of the chess development process either...
Parent - By Banned for Life (Gold) Date 2011-07-17 05:42
You don't need to look at the OS if you don't want to, although a phone OS is a part of a cell phone in a way that a PC OS is not (unless you are a stinking Mac user). There are already hundreds of thousands of smart phone apps, even a few that play chess. The work on these alone dwarf your 40+ years. Or you can look at the time spent on the custom chip sets, which show what happens with hardware and firmware when you are selling a billion units a year. This is a real and competitive business where smart people are drawn by the massive potential. Chess engines are a nice hobby for mutants...
Parent - - By Banned for Life (Gold) Date 2011-07-17 02:40
Speed across two different programs doesn't mean a thing, any more than tachometer readings from two different cars tells you how fast you are going, or what your fuel economy is.  But for _any_ program, if you make it faster, you make it strong, if all you change is the speed.  Every time.  no exceptions.  And that is what I explained...

Of course any third grader would understand this concept. But the reality is that there are far better engines than crafty that have been put together by guys working a few years part time. These engines are either faster, or have better algorithms/heuristics, or are better tuned, or all three. Which one is it?
Parent - - By bob (Gold) Date 2011-07-17 04:01
Which engines with a "few years part time"???

Stockfish has been around for far longer than "a few years."  We do have some robo derivatives that have not been around very long, but that is hardly comparable.

I knew 15 years ago, that when I decided to go open-source, competing was going to be difficult.  I accepted that, for reasons discussed many times previously.

As far as your final question goes, I am unable to read the author's minds so I can not answer that.  That kind of answer seems to be your forte'...
Parent - By Banned for Life (Gold) Date 2011-07-17 05:51
Which engines with a "few years part time"???

The strongest readily available engine right now is Houdini. Its author has promised a new, significantly stronger release later this year. Critter is catching up fast. Many feel this is a truly original engine developed by a guy working part time who doesn't have illusions of doing it as a business.

I knew 15 years ago, that when I decided to go open-source, competing was going to be difficult.

At one point, people were making heave use of Crafty. Later they made heavy use of Fruit. Now they can make heavy use of Stockfish and Robo. So you're no longer competing with yourself.

As far as your final question goes, I am unable to read the author's minds so I can not answer that.  That kind of answer seems to be your forte'...

I think you're trying to insult me again, but it's such a weak attempt that I'm not completely sure...
Parent - - By AWRIST (****) Date 2011-07-17 10:32
Bob, just a short interruption:

coming from Crray and Blitz and Harry Houdini, you have a biased perception somewhat.

Could you specify how many light years Fabien had been working on his Fruit code? 2 years? 5 years? 10 years or 50 years?

Last time I looked Fabien rather looked like a young man, so he might have borrowed his code from his father?

Any short answer will do. Thanks. BTW also Banned for Life, from your intense debate I am slowly understanding better what this is all about. Thanks.
Parent - By bob (Gold) Date 2011-07-17 15:48
I actually do not know.  The primary "point" for fruit was very simple everything.  Could have been only a year or two.  Could have been longer.  But if you look at history, most have a significant "history".  Can't say "all" do.
Parent - - By zwegner (***) Date 2011-07-16 18:24
I agree about the hardware/software gap. While I can't see any a priori reason why this should be the case, hardware design is a much more developed science. Software engineering is such a young field, and really hardly anybody knows what they are talking about. I have my own ideas for the "proper" ways to develop software, but they mostly come from my rather limited experience and from intuition.

There are a few problems I see:
--The people who are researchers in software engineering itself are usually idiots. These people usually become consultants that get paid out the ears. They waste a lot of time trying to quantify productivity and code quality, ending up with things like the tools you mentioned earlier (which are basically useless). They come up with lots of buzzwordy methodologies (RAPID, scrum, all that crap) and try to push it on managers, who then proceed to force all the engineers to use them against their will.

--For some reason, software seems to have a much-too-long lifetime. At least, we still have to deal with bad decisions made in software many years ago. There has just been so much accumulation of cruft, it needs to be thrown away. While I thought he was crazy at first, I have come to agree with Tord Romstad that software just needs to be rewritten periodically. Relatedly, I don't think any of the programming languages or operating systems available now are all that great.

--There is an enormous amount of software that needs to be written (or at least somebody thinks it needs to be written), and very few people are capable of writing it properly. The average CS RCG is utterly incompetent. Ever hear of "fizz buzz"? If not, look it up, it is very saddening.

Coming back to computer chess, really, the most important thing is cranking out tons of code changes and being able to test them as rapidly as possible. If you spend much time trying to work out an algorithm, in most cases it's going to be a waste of time. Most changes are just going to be rejected. The sort of high-level algorithm design you talk about has mostly been collectively worked out (alpha beta, LMR, etc.), and all that's left is the details. Most of these details are far too hard to be evaluated by a human, so we just throw a bunch of shit at a wall and see what sticks.
Parent - By bob (Gold) Date 2011-07-16 19:25
I can certainly state, with absolute certainty, that graduates that have taken courses under me can deal with the fizzbuzz type design questions easily and quickly.  Because they have written almost a dozen programs dealing with sockets and processes and connections and such, or simulating virtual memory replacement algorithms, or to determine cache characteristics, not by using CPUID, but by actually "breaking cache performance intentionally".  For example, how many levels of cache?  For each level, how bit is it.  What is the N in N-way set associative?  what is the blocksize/linesize?  Is it inclusive or exclusive.  So _some_ CS graduates are capable of coding.  I won't disagree that some are not...
Parent - By Banned for Life (Gold) Date 2011-07-17 01:39
The people who are researchers in software engineering itself are usually idiots.

I'm not so sure. In some businesses, the way to maximize revenue is to get people to believe that they need to do things differently every year or two. Think French Fashion. I suspect the researchers may be afraid that consistency will lead to a lack of recurring income. This is different in the hardware field, where progress quickly obsoletes what you have, be it computers, cell phones, TVs, whatever. This is a major driver for hardware. Nobody even wants to think about the nightmare where people keep their stuff for years and say "Why should I replace it?".

They come up with lots of buzzwordy methodologies (RAPID, scrum, all that crap) and try to push it on managers, who then proceed to force all the engineers to use them against their will.

You're probably too young to remember TQM, where they forced everybody at many large companies to spend a week playing with colored balls and tinker toys (I'm not making this up!). These things change names every few years, but they never go away. They afflict businesses with low productivity growth. They are especially prevalent in the government, where productivity is generally less than zero.

--There is an enormous amount of software that needs to be written (or at least somebody thinks it needs to be written), and very few people are capable of writing it properly. The average CS RCG is utterly incompetent. Ever hear of "fizz buzz"? If not, look it up, it is very saddening.

Once again, I hope some of you young guy figure out how to dramatically improve software productivity. If you don't, I suspect that when you're my age, you'll be thinking nostalgically about the times when software was written in the US.

Coming back to computer chess, really, the most important thing is cranking out tons of code changes and being able to test them as rapidly as possible. If you spend much time trying to work out an algorithm, in most cases it's going to be a waste of time. Most changes are just going to be rejected. The sort of high-level algorithm design you talk about has mostly been collectively worked out (alpha beta, LMR, etc.), and all that's left is the details. Most of these details are far too hard to be evaluated by a human, so we just throw a bunch of shit at a wall and see what sticks.

You can't focus on new algorithm development like you can on tuning. But you never know what algorithms are out there undiscovered. If someone had thought up something as important as null move pruning or LMR and kept it secret, think about the advantage they would have accrued. Of course in today's environment, someone would discover the advantage via RE, but maybe cloud computing will catch on and this will no longer be possible. This might be actually be the road back to originality in chess engines...
Parent - - By Rebel (****) Date 2011-07-18 10:15
Bob Hyatt:
Those of us that actually _write_ complex code (and I have written assemblers, compilers and operating systems for many years, in addition to chess programs)

Interesting....

Can you elaborate?

Names, companies, periods. Perhaps an URL?
Parent - - By bob (Gold) Date 2011-07-18 14:48
Assembler for the IBM 360.  Best reference is Hyatt, R.M. and Burge, C.D., "The USM Load and Go Assembler", Proceedings, ACM Southeast Regional Conference, Chattanooga, TN, June, 1972 (383-394).

Assembler for the HP2100a that handled both normal assembly language and the HP microprogramming assembly language so that you could change the basic instruction set behavior.  This included a simulator that perfectly emulated the HP2100a so that students could do the programming stuff without having to fight over the single box that we had while debugging their code:  39. Hyatt, R.M. and Burge, C.D., "A microprogramming Simulation Package for theHP2100", Proceedings, Xerox Exchange Meeting, Atlanta, GA, May, 1974 (143-159).

A complete basic interpreter, plus support for A/D, D/A and D/D, plus 10 simulated "things" (external processes that needed to be controlled):  37. Burge, C.D. and Hyatt, R.M., "A Digital Simulation Technique for Teaching Process Control Concepts", IEEE Southeast Regional Conference, Orlando, FL, April, 1974 (415-427).

A complete VM that ran on a xerox computer, which perfectly emulated that hardware, so that I could boot and debug the operating system, while others were using the system.  Slow to be sure, but it was the only way to do the development project I was working on:  38. Hyatt, R.M. and Burge, C.D., "Hardware Simulation Timing Algorithms", Proceedings, ACM Southeast Regional Conference, Nashville, TN, April, 1974 (272-282).  Hyatt, R.M., "A Sigma 9 Hardware Simulation Package", Proceedings, Xerox Exchange Meeting, Atlanta, GA, 1978 (189-203).

Next is a complete system, real-time operating system for the TI 960b minicomputer, plus software to remotely sample EKG data, compress it on the fly, send over a 2400bps synch modem connection (multiplexing up to 8 patient's data over that info stream) reconstructing it at the central cardiology monitoring, and displaying it on CRT devices so that cardiologists in a central hospital could diagnose / treat patients in remote areas.  We even made this work from a moving ambulance after we were able to get Chrysler's "project starpack" declassified so that we could use their omnidirectional sat antenna (not a dish) on an ambulance.   Burge, C.D. and Hyatt, R.M., "A Digital Approach to Remote Cardiac Monitoring", Proceedings, ACM Southeast Regional Conference, Birmingham, AL, April, 1976 (241-249).

A non-published example would be stripping the communication out of the xerox CP-5/CP-6 operating system, and replacing that by designing and writing the interface code to let the sigma talk to a front-end communication processor that offloaded all the character-by-character interrupt handling which was a context-switching nightmare, even on what would still today be the best context-switching architecture in existence (machine had multiple sets of registers, one for user programs, one for interrupt handler so that no register saving/restoring was needed when handling an interrupt, etc.  This was done for Telefile computer products in Irvine California.  They gave us (USM) the front-end processor, in return for my removing all the communication software from the operating system, adding support for the front-end, and providing them the source code (all in xerox assembly language, some 250,000 lines of code total.  Project was done by yours truly, as were all of the above projects coding efforts.

That covers the first 10 years that I worked at USM.  Do I need to go farther?
Parent - By Rebel (****) Date 2011-07-19 08:12
How interesting those early days. A lot of wild-west was going on. I had a fascination myself for the low-level-stuff. I worked for a company that ran its administration on NCR Century 200 computer. I rewrote lots of the crappy utilities that were part of the operating system into something easy to use.

But the craziest thing I have passed through was a project we did for a (I believe) bank. There was this new promising mini-computer (forgot the name) but the release of the assembler software was delayed and delayed. The bank became impatience and hired us to start the programming anyhow. So we had to program in raw machine code, typing op-codes followed by operands. The real fun (read horror) was of course when making a change you had to go through the whole program fixing all the jmp offsets that got out of sync, same with calls. We found some ways around it by writing very small sub-routines surrounded by lots of NOP's but all in all progress was so slow the bank decided to wait for the assembler software after all. We never heard of them again, nor from the promising mini-computer.

It was my first experience with assembler, quite a baptism.
Parent - - By bob (Gold) Date 2011-07-15 17:16
Notice the "did not work..."  Think that means "not in crafty"???

I'm still waiting for you to take the two versions you cited specifically, 23.2 and 23.3 and show me what ideas were copied to reduce the EBF.  I even published the diff of the two searches to make your job simpler.  To date, not one response from you...

BTW, I only tested the idea because of a discussion on CCC about its use in Stockfish and Komodo, because I did not believe the idea was a sensible one.  But in the discussion, I decided to test it to determine if my intuition was correct or not.
Parent - - By Rebel (****) Date 2011-07-16 07:36

>Notice the "did not work..."  Think that means "not in crafty"???


The framework of the discussion has always been your super aggressiveness towards Rybka while in the meantime using parts of Rybka's legacy for your own benefit. Now that you finally admitted you tried a Rybka idea I think I made my point sufficiently. The fact the idea "did not work" is irrelevant. You tried, the intent was there. But maybe you want to convince your audience that in the case the idea was a "working one" after all you never would use it because it's origin is Rybka? If so good luck.
Parent - - By bob (Gold) Date 2011-07-16 14:56
Sorry, I admitted I tried a "Stockfish" idea.  That was eventually revealed to have come from Robolito.  Which _might_ have come from Rybka.  I simply saw the discussion on CCC, thought "that doesn't seem very reasonable to me" and used the cluster tools I have access to to test the idea.

I like the back-pedaling however.  You talk about our flawed process, and ignore yours.  You asked "when are you going to delete the Rybka code or ideas from my engine?"  I challenged you to back that up.  How did that go?  Find any Rybka ideas or code in Crafty?  You then specifically claimed that the branching factor reduction from 23.2 to 23.3 was from Rybka ideas.  I even diff'ed the search code and gave you the output.  See any Rybka ideas?  Any _hints_ of Rybka ideas?

And _our_ process was flawed?  I think that's pretty funny.  And then you use that "if you want to convince..." silliness.  It would seem to me that once you make a claim, you have to back it up or retract it.  Backed up the claim that I copied ideas yet?  Retracted it?  So who wants to make this personal?  And who is unwilling to confront the truth?  And you talk about _us_ (on the panel) as being biased/unethical/etc?  Pot, kettle.
Parent - - By Rebel (****) Date 2011-07-16 17:37
On open-chess you said:

the _only_ "Rybka idea" I have tested was the tt-singular extension.

You are retracting that now ?
Parent - - By bob (Gold) Date 2011-07-16 17:44
Do you see "Rybka Idea"???  Do you understand why someone puts an obvious expression in quotes?

get serious.

In the tt-se thread about stockfish, someone pointed out it was in one (or all) of the ip*/robo* family.  And there is suspicion that ip*/robo* is based on rybka.  A suspicion that has not been confirmed.  Hence "Rybka idea" means "this term is being used in a different way than normal"  or in my case "as a statement of potential suspicion" rather than "a statement of fact."

Again, you made the claim that rybka ideas were _IN_ Crafty.  Now you are resorting to the "but at least you tested one rybka idea"  So we go from "some rybka ideas are _in_ your program" to "you did test one idea even if you rejected it."  See any sort of "agenda" in moving from the former statement to the latter?  The challenge is _still_ pending for you to show exactly one idea that is in Rybka, and which magically appears in Crafty-23.3 but was not present in version 23.2 or earlier versions.  So far, I've seen nothing but hot air and backpedaling, with zero evidence offered.  Isn't it about time to fix that???  You are clearly backed into a corner.  There is only one way out.  Grown men would take it.
Parent - - By Rebel (****) Date 2011-07-17 09:09

> Again, you made the claim that rybka ideas were _IN_ Crafty


Sure there are.

How long is it now that the Strelka sources (=Rybka 1.0 beta) became available? 3 years? 4 years? And you want to uphold the myth you were not influenced by it ?

Why this denial of the obvious ?

Bob, you can fool the non-chess-programmers, perhaps even the junior-chess-programmer but not the experienced ones who know every corner of our little world. I have been around 30 years, you even longer. Just stop.

You may convince some of the non-chess-programmers here and elsewhere but not the experienced ones. Your luck is there are only a few chess programmers that are willing to challenge you every time you are being dishonest, as in this case. I can't blame them for being smarter.

In the mean time I received 2 mails from collegue chess programmers chosing my side in this debate.

Typical...........
Parent - - By AWRIST (****) Date 2011-07-17 13:00
Thanks Ed, for you insisting. You know, the falsehood is basical in sciences for a bad reason. ou always meet twice with certain guys and then they will hold against you, what you had supported against someone else. This isnt honest but understandable.

A famous case from another field. At the time JFK has been assassinated he was brought to the Dalla Hospital, almost dead because of the incurable losses of mere brain tissue through the shot that exploded almost all his cerebellum which is the part on the back and important for especially the raw tasks of the brain. Now the exact wounds were important for the direction from where the bullets were shot.

So first thing they fiddled was hiding the huge wound in the rear of the head.

But as always they couldnt completely fake the whole environment including the medical doctors at the site.
Now there was a young assistant called Cranshaw. He saw the whole mess of Kennedy's head. He called livelyx what he has see, not heard by third persons. But he wrote his report not until he was shortly before his own retreat. He had made a fabulous career in that same hospital. Now in his momories he exactlyx described the truth. Then he was asked why he didnt tell before. His answer goes into the direction I've mentioned above. He said, if I had told the truth I could have forgotten about my own career. Perhaps you remember that the SS controlled everything in the cover up. The next doctors all were taken under oath of the military Law. So they knew what to tell in public.

I wanna thank you for speaking out the evident. If a prosecutor like Bob is lying about his own activities and that he once took something in the same manner he is always accusing Vasik now, this does normally take away all innocence from Bob. A prosecutor must be clean and Bob isnt clean at all. This is what you have proven. Of course this will now be used against you too. Because you told the truth that wasnt authorized as being allowed to tell. Good to know that you are now out of the active race. Otherwise they would now tear you into parts. What was the last insult. I read something about a turkey full of something.

There is a lesson out of your example that especially the younger should remind for their own future. The truth is something that is not always easy to tell. Because like in the cosa nostra most of the time everybody has done something wrong once in a while. We are all humans and nothing but humans. But it's still so important to tell the truth, also if it means that you have a hard time just through this telling the truth. But in such momentss you can see if an indicidual is honest and more if he's man enough to go through the fire of those who accuse you of breaking the somewhat criminal consence. Fallacy is that normally you are not sober either. But is that a surprise. Only those who have done nothing at all they could claim of not having made a single mistake.

What should be the moral out of your statement?

Someone like Bob Hyatt, everytime when he shows something wrong in Vas, should realise that he should better use a more relativated speech which would consider possible explanation instead of working with character assassinating terms. His whole blabla about the time periods being saved, is relative in regard of the bright ideas Vas had in mind when he completed his Rybka. Wouldnt it be a pity if he couldnt have done something like Rybka because he was stuck on some necessary tech that was already invented but were not allowed to be taken?

And you, Ed, couldnt you clarify that compared to the profit you could make in the past with these table entities of chess computers you made much more money than what is actually possible with software stuff? Here you can also see how hypocrit Bob is because he never had to work in a market and sport for having a safe income. He should just shut up with his insinuations. For me Vas is innocent. Because many might have done what he did. And this is not excusing something. It just should be weighed in proportions of the relevance of the field. If someone has the talents like Vas, he should take the shortest cut to get to the essentials of his work. Just my opinion.

And all the best to you.
Parent - - By Rebel (****) Date 2011-07-17 15:01
Hi Rolf,

Long time no see, good to notice you are well.

I wanna thank you for speaking out the evident. If a prosecutor like Bob is lying about his own activities and that he once took something in the same manner he is always accusing Vasik now, this does normally take away all innocence from Bob. A prosecutor must be clean and Bob isnt clean at all. This is what you have proven. Of course this will now be used against you too. Because you told the truth that wasnt authorized as being allowed to tell. Good to know that you are now out of the active race. Otherwise they would now tear you into parts. What was the last insult. I read something about a turkey full of something.

One minute, this is not accurate. You can't compare the Rybka case with Crafty. Bob did nothing wrong. He only denies something he should not.

What should be the moral out of your statement?

You already gave the answer. The mote and beam thing.
Parent - - By Ugh (*****) Date 2011-07-17 15:56
Perhaps you two could compromise and, instead of Rolf's "lying" in relation to Bob, instead use the term "dissembling". Ironically rather like the relationship between "copy" and "plagiarise", well, maybe not ;-)

"Dissembling" describes the presentation of facts in a way that is literally true, but intentionally misleading.
Parent - - By AWRIST (****) Date 2011-07-17 16:57
I understood Ed the way that Bob took ideas from Rybka, tested ot in their code expression and then saw it didnt work and then threw it out again. Ed claimed that Bob would deny this.

Ed argued that Bob would criticise Vas for something, that he himself would do too, the same on a level of understanding of the intention in his mind, to take something although it's forbidden and then working with it for some time. Ed saw this as similar to what Vas did. Or did I misunderstand something?

So I used the term lying. But perhaps this might be completely false in the light of what really happened. I would be incapable of adding anything to that problem.

Could you explain what a lay should understand in the controvery between Vas and the rest of the World led by Bob?

BTW I still am against the following:

Doubts exist for a certain program but it's closed and the programmer doesnt explain. Now some anonymous figures RE the source code of the program and build their own cheated program and then publish its source code. But because it mainly consists of the same formerly closed program. That is a dirty way of blackmailing IMO when the sole interest is to cheat on the other programmer (here Vas Rajlich).
Parent - By Ugh (*****) Date 2011-07-17 17:49
Ed is correct.

On your second question, what is behind the controversy? I see a parallel between the frontier running out as they reached the Pacific coast, the decline of America as a great power and the rise of other powers (BRICS). We have computer chess running out of point after Kasparov was reached, the decline of American chess programming as leaders, partly represented by Hyatt, and the simultaneous rise in the East, first of Vas, then of the anarcho-programmers from Russia or wherever. American/Western programming is behind and unable to catch up, let alone understand how the eastern "terrorists" are beating them. Result is Hyatt thrashing around like a demented bull elephant, destroying himself and everything around him.

Quatsch? Maybe ....
Parent - By bob (Gold) Date 2011-07-17 15:58
"This is what he has proven"???

I'm waiting on that "proof" myself.  Not seen anything but innuendo, "you must have" and other such nonsense.  Every version of my code is publicly available and has been as each version was released.  He could even take the easy road and read through the version descriptions in main.c to find that "strelka/robo/rybka" idea that he is _so_ certain was copied.

Or not...
Parent - - By bob (Gold) Date 2011-07-17 15:56
I made a factual statement.  No ideas were taken from strelka, or ip*.  I challenged you to take the source and point out just one.  You failed miserably.  But you keep on insisting, without doing due diligence.  Fortunately, the investigation into Rybka's origin was a bit better handled.

All you have to do is show some proof.  I mean, my source is public.  If you can identify exactly when Strelka showed up, I can limit your "search" to the specific version of crafty that was available at that time and beyond only, as my ftp box has the dates that anyone can see.  Just offer the proof.  You don't have to reverse-engineer anything.  You can start with the version of Crafty that was available just prior to strelka, and diff the newer versions one by one.  Is that too hard?  Only, of course, if there is nothing to find.

Up to you.

And "typical" is a good word here.  I don't care about "2 private emails."  Nor Tarot cards.  Nor palm readers.  Just some cold, hard facts to prove your statement.  Or mine.

waiting.  As I have been doing since you first made the silly claim.  Just prove it.  And I can't hide like some do, my source is _still_ public for anyone to see...

You could even just start reading in main.c.  We know that Rybka 1.6.1 was a copy of a Crafty version after 19.8.  So Strelka was much later than that, which limits the scope of your search significantly.  Either get busy or shut up.
Parent - - By Prima (****) Date 2011-07-17 22:08
Bob, in a situation like this especially after Ed failed to prove his accusations despite the fact all Crafty source codes are publicly available for comparison, to ignore him. But this would give Ed an easy way out.

It's very obvious Ed's anger and emotions got the best  of him to lash out with such unfounded defamatory accusations & lies. Even sadder is the fact Ed doesn't know when to stop these insanity, retract his lies, and make amends. Not that the apology might do any good at this point. He's clearly passed that point of offering an apology.

I, and I'm sure others, are watching him assassinate his very own character and integrity.
Parent - - By Rebel (****) Date 2011-07-18 10:57
Did not realize I had an integrity :eek:

So thanks.

May you should try to get one of your own :razz:
Up Topic Rybka Support & Discussion / Rybka Discussion / In retrospect (locked)
1 28 9 10 11 Previous Next  

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill