Occasional random validation failures

In case 16962 [1] the question was raised why after a successful CI
validation you later get a failed validation.  I experience this
sometimes and guess others might interested in this too.

If multi-threads are involved, sometimes the failure indicates a real
problem due to a race condition that only occurs sometimes.  Here you
will see lots of tests passing and then a problem on a particular one.
In this case I find running the suspect test or class from playground
wrapped in a 1000 loop can expose the problem for debugging.

In the case of 16962 [1], looking at a report for success [2] you can
see "CIChangeCaseValidationRule" is followed by "Loading CASE 16962"
and then a long listing of tests run, and only at the end the
validation report.  However the report for failure [3] shows
CIChangeCaseValidationRule immediately followed by the validation
report.  So I wonder if that means the slice did not load - maybe due
to some network error or smalltalkhub error.

Now thats idle speculation, but maybe useful to consider if you
encounter such a scenario.

cheers -ben


2015-11-28 3:56 AM SUCCESS [2]
[...]
Starting validation of CASE 16962
http://smalltalkhub.com/mc/Pharo/Pharo50Inbox/main/SLICE-Issue-16962-Code-pane-should-not-show-all-spaces-selected-when-one-is-selected-DenisKudryashov.1
Notice: Running a CIAndRule
Notice: Running a CIChangeCaseValidationRule
Notice: Loading CASE 16962
http://smalltalkhub.com/mc/Pharo/Pharo50Inbox/main/SLICE-Issue-16962-Code-pane-should-not-show-all-spaces-selected-when-one-is-selected-DenisKudryashov.1
[...cut many,many lines of test cases...]
==========================================================
Validation Report: CASE 16962
http://smalltalkhub.com/mc/Pharo/Pharo50Inbox/main/SLICE-Issue-16962-Code-pane-should-not-show-all-spaces-selected-when-one-is-selected-DenisKudryashov.1
==========================================================
Validation Success: CASE 16962
http://smalltalkhub.com/mc/Pharo/Pharo50Inbox/main/SLICE-Issue-16962-Code-pane-should-not-show-all-spaces-selected-when-one-is-selected-DenisKudryashov.1
[...]



2015-11-26 10:29 PM FAILURE [3]
[...]
Starting validation of CASE 16962
http://smalltalkhub.com/mc/Pharo/Pharo50Inbox/main/SLICE-Issue-16962-Code-pane-should-not-show-all-spaces-selected-when-one-is-selected-DenisKudryashov.1
Notice: Running a CIAndRule
Notice: Running a CIChangeCaseValidationRule
==========================================================
Validation Report: CASE 16962
http://smalltalkhub.com/mc/Pharo/Pharo50Inbox/main/SLICE-Issue-16962-Code-pane-should-not-show-all-spaces-selected-when-one-is-selected-DenisKudryashov.1
==========================================================
Validation Errors: a CIChangeCaseValidationRule: Timeout (after 0:01:40:00)
[...]



[1] https://pharo.fogbugz.com/default.asp?16962
[2] https://ci.inria.fr/pharo/job/Pharo-5.0-Issue-Validator/20704//artifact/validationReport.html
[3] https://ci.inria.fr/pharo/job/Pharo-5.0-Issue-Validator/20587//artifact/validationReport.html