seq 1000 | while read i; do echo $i; echo run test suite; echo; done

All quality software should have an automated test suite: some way to ensure that at least the primary happy path works. Ideally, the test suite would verify much more than that, but at least that.

Unfortunately, it is the nature of software to be buggy, sometimes in weird ways. Automated test suites are software. A particularly unpleasant way for a test suite to fail is intermittently. A test will fail sometimes, but not always. It’s called a flaky test. This is particularly bad, because it tends to undermine trust in the test suite as a whole. If a test always fails, there’s clearly a bug in the test or the code under test, and it clearly needs to be fixed.

If a test fails once every now and then, it might be problem in the test, in the code under test, in the environment, or something else. who knows? The threshold to just disabling or deleting the test can be low.

A test can be flaky for any number of reasons. How do you know if you have flaky tests?

An easy way to get some confidence is to run the test suite many times. The more times you run the test in row, the less likely the test is flaky due to random chance. It might be flaky due to other reasons, of course, but running the test suite 1000 times is a good baseline.

Every time I do this to a new project with a test suite of significant size, there are problems. Which I then fix.

Can your test suite pass 1000 repetitions of the test suite in a row? Are you sure?