Author: Uri Blass
Date: 10:25:20 09/09/00
Go up one level in this thread
On September 09, 2000 at 09:53:07, Vincent Diepeveen wrote: >On September 09, 2000 at 09:11:27, Uri Blass wrote: > >>I understand now that in most of the cases the beta testers get the commercial >>version of the program that they test so they get some payment. >>When I replied the opinion poll question I thought only about getting the beta >>version without getting the commercial version. >> >>I think that it is better to pay them money for their job when they will have to >>buy the package of beta+ commercial version. >>The reason for it is that not all the beta testers do the same job and I think >>that it is not correct to give the same payement for different jobs. > >What a weird poll. Betatesters deliver very few comments usual that's useful >for a programmer. Most things i see when i am myself following games of DIEP. > >If i'd pay for that then i'd not be only wasting money, but also i would >be bankrupt within days. Handing out versions (beta or whatever) is >something different. You can't get comment back without people having your >program! It depends on how much you pay for them I think that beta testers who are not productive should pay for the package of the beta and the commercial version. If they do not give a lot of information (for example if they give only 3 tournament time control games with the beta) you should pay them less than the price of the package. If they give you more important information you can pay them more money. > >Over the years i've had with DIEP about 100 betatesters or so, but only >a very few, usually those who ran/run it at the internet, have delivered >good comments on it. > >Usual someone offers to 'betatest' in order to get a version of a program. > >Right now i'm spreading the winboard version of DIEP quite easily around, >and so far the useful info i get back is real pathetic. If a betatester >concludes that there sucks something bigtime in diep's eval because it >plays g4? then i'm already real happy. As i can fix *something* then. > >considering the thousands of fixes i do a year, that's only 0.001 part of >the fixes at most. > >Most comment is like: > "diep sucks in closed positions". Well i knew that already! These were not the kind of comments that I gave as a tester of Junior. Example for a comment that I gave about Junior is: The new version of Junior needs a lot of time to solve the fail high in a position that I read in the newspaper(I know that he looked at my emails because in one case when I sent him the wrong position he asked for the right position). I also gave comments about things to fix in the piece square table or ideas to build new tables or ideas to do more extensions(I think that out of it only something about the piece square tables was changed because of my ideas). I gave these comments based on watching games and I do not know much about the source code of Junior. There was one case when Amir told me that something about the rules when to evaluate draws without search(it was about KBP vs K draws when the bishop controls the wrong corner). Junior did not use tablebases at that time(It was nhear the time that Junior5 came) and tablebases cannot solve this problem completely because of the fact that the defender side may have also pawns) I told him to change < to<= in the source code and he did it but I had a better idea that I suggested (using new tables of distance to the corner and he did not use my idea) I sent Amir Ban more than 100 emails with ideas and comments and unfortunately he ignored most of them and I know that Junior does not use most of my ideas. I worked for Amir because I am from Israel and I wanted to support a program from Israel but I decided to stop doing it and one of the reasons was the fact that he did not reply most of my emails and did not use most of my ideas. > >The best 'bugfixes' are comments usual from other programmers, like >the comment of Steffen recently. Also Bob has helped me over the years >bigtime, especially when talking about the parallellisation of DIEP. > >There are exceptions of course on this betatester picture, who *do* >regurarly give useful comments. It's hard to find such betatesters. > >Even harder as that is to find people who want to auto232 play your program >against other programs like Shredder, Fritz & others. Just 1 or 2 games >is not what i mean then. > >Someone who wants to run DIEP at an auto232 player and send me back >results of the auto232 player can ring me any time of >the day! > >>Here is an example. >>Suppose beta testers are asked to give games from some known positions against >>known opponents at 1 hour/game time control. >>Suppose one of them gives 100 games and another one gives only 10 games(nobody >>of them gives more information). >> >>I think that the tester who gave 100 games should get better payment. > >Oh well. What to do with 10000 games of 1 0 at the icc? I think that you can tell the testers the time control that you are interested in and I did not suggest to pay for something that you did not ask for. > >Also what if 90 of those 100 games are exaclty the same bookline? This is the reason that I said from some known positions and you can tell the testers the positions. > >Secondly, what if those games are all played with an older version, >despite you sending already a new version to the betatester? In this case the testers should not get money for their job because they did not do what you asked them for. > >Or what if you already fixed the bug, or what if the 'bug' is not a bug >but a feature which is interpreted wrong? > >Now we didn't talk about the many silly reports, which i don't mind as >i'm happy that there is a *small* chance that such a report can be good. > >The average betatester doesn't even have 2000 rating or something, and >even if it's 2200 then still this means he's tactical insane compared to >the program, so the average 'this went wrong' report needs at least >to be carefully checked whether it's true in advance. > >Apart from that see the many posts of mate in 2 or 4 or something here at >icc. > >I don't care whether DIEP solves a mate in 4 in 0.001 or 0.01 seconds or in 2 >seconds. > >It's just not interesting! > >>If you tell all the testers to give the same computer time you cannot use some >>testers who have less time and cannot use efficiently some testers who have more >>computer time to give(because they have more computers or because their >>computers are not busy in other tasks). > >if i'd had to pay a betatester i would sign a contract and i would >assume he'd be 12 hours a day busy >testing my program. So i would expect daily reports then with things >being wrong. Because if i do somethign like that myself with my own program, >then each 5 minutes i remember something that's wrong in the program! I think that the contract should be based on the success of the program in the market. I do not think that beta testers should earn money when the programmer does not earn from it. I also think that 12 hours a day is not needed but the money that the tester earns should be dependent on the question if the tester is productive for the program. Testers who almost give nothing should earn less than the price of the program and it means that they should pay for the testing. Testers who are more productive should earn from the testing. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.