CCRL Testing/Contributing

[Andrew Matthews]

Hi All,

I had the pleasure of having a connect with Graham, who very patiently walked me through the CCRL processes and framework.

Having had a detailed conversation with Graham, I am very keep on joining the group and contribute to the best of my ability.

I will be contributing most of my CPU time towards bringing as many new releases up on the 1CPU list as quickly as possible, followed by more comprehensive 4CPU and 8CPU runs to follow at the 40/15 time controls. As the longer time control needs longer time to complete, more hands on deck here would help!

Given the above motivation, in today's world, I also need to explicitly say this, I am NOT interested in testing or incorporating any SF derivatives. Major version releases on the main branch are the only ones I will test, just like Ice Cream, my choice here is Pure Vanilla, Original SF Only. I believe there are extremely talented, hard working engine authors out there, who's creations deserve far more of my attention.

Again, just want to take the opportunity to thank Graham for being the kindest and most patient guide in me figuring it out!

Andy

[Andrew Matthews]

For the above, here is a list of computers/specifications that I will run:

1. Primary : Ryzen 9 5950X (16core) w 48GB RAM, 5-man syzygy (all on SSD)

2. Secondary : Ryzen 9 5800HX(8core) 16G RAM, 5-man syzygy (all on SSD)

I will also be dedicating some more compute time from my older machines, giving them a new purpose in life!

[Ray]

Nice hardware there.

I stick to 5-men syzygy tablebases but some blitz testers use 6-men. Memory usage with 6-men can climb alarmingly at longer time controls as the O/S caches them. Unlikely to be an issue on the 5950X with 48GB RAM but could definitely be a problem on the other machine. 5-men definitely safer at 40/15 in my view and more so with high concurrency if that is how you intent to test.

[Andrew Matthews]

Indeed, I want to run with concurrency 15 for 5950x and 7 for 5800HX, in such cases specially keeping the IOps on the same storage 30 times in parallel in mind I just prefer 5 man.

It kinda allows me to maintain quality of games while being reasonably confident of running basically the same setup across all my machines new and old.

I am a big sucker for standardization

The outlook I have here is that I can utilize all these cores in parallel to push out 1CPU lists as quick as possible, then once we're all caught up with 1CPU I'll run 4/8/16 runs later.

[Ray]

Yes agreed.

It is CuteChess cli that you use ? I use CuteChess GUI myself, I like to look at the odd game here and there.

[Andrew Matthews]

Correct! I do use cutechess-cli, I kinda have it setup such I can automate runs back to back without having to intervene.

Though as you correctly pointed out, running cli means not seeing games, so if the setup is wrong I can waste a bit of time, but I think with the trial and error I've had to do with it and some engines, I think I've finally gotten it ironed out!

I usually just peak at the pgn being generated by cutechess (it only writes once per round though) to 1 make sure the run is going right but also 2 to see these engines duke it out (and try to make sense of the moves with my little humble brain )

[Ray]

Indeed, looking at the pgn afterwards is fine also.

[Gabor Szots]

I don't know the Ryzens, but an issue can be that the benchmark changes as the number of cores used changes. E.g. when I use 5 cores my benchmark is approximately 0.8 times the benchmark of the reference machine, using 6 cores (6 simultaneous tournaments) it slows down to about 0.85 etc. I change my time control accordingly.

[Andrew Matthews]

That does make sense, but in my experience as long as I am looking at 1 CPU runs and my concurrency less than my physical cores allows me to maintain the expected performance.

Having said that I will do a new time control scaling when I start a new category of testing (i.e. once I start 4CPU I'll do a unique scaling calculation for 4/8/16 afresh so we get apples to apples for 40/15.