CircleCI have this feature called rerun failed tests. Using magic, only a portion of your flaky test suite is run again.
That magic is having your tool generate a list of tests,
filter it through circleci tests run,
and execute the tests that are still left,
writing the results to a JUnit style results file.
I was tasked with running jest tests through this, but the jest setup I had didn't look much like the example config:
1$ jest --ci --selectProjects svc
It took me quite a while to figure out that what I needed was the below snippet, since our jest setup had different projects that specified the test files to run (for the first invocation to find the right tests) but also global and project setup and teardown functions (for the second invocation to run tests correctly).
1$ jest \
2 --ci \
3 --selectProjects svc \
4 --listTests | \
5 circleci tests run \
6 --verbose \
7 --command "xargs jest --ci --selectProjects svc --runTestsByPath --"
Additionally, if you used circleci's native parallelism, but can't use circleci's test splitting because it never finds the right timings, then you need to override its default sharding behavior. This uses jest's sharding in place of circleci's sharding:
1$ jest \
2 --ci \
3 --selectProjects svc \
4 --shard "$(( CIRCLE_NODE_INDEX + 1))/${CIRCLE_NODE_TOTAL}" \
5 --listTests | \
6 circleci tests run \
7 --verbose \
8 --total 1 \
9 --index 1 \
10 --command "xargs jest --ci --selectProjects svc --runTestsByPath --"