Non deterministic Test Tracker - CDCgov/prime-simplereport Wiki

Background

SimpleReport engineers recently identified a pain point pertaining to CI/CD: non-deterministic or "flaky" tests.

Non-deterministic tests cannot be guaranteed to behave predictably, and may cause CI/CD failures even if the underlying code is healthy.

Below, please document any such automated tests. Unit tests, integration tests, and end-to-end tests are all relevant to this effort and may all be tracked on this page.

Tests

Backend

File Test label Description
QueueManagementTest.java queueOperations_standardUser_successDependsOnFacilityAccess() org.springframework.web.client.ResourceAccessException at QueueManagementTest.java:263 > Caused by: org.apache.http.ConnectionClosedException at QueueManagementTest.java:263 > {"time": "2022-06-24 18:33:13.403", "level": "INFO", "source": "o.s.s.c.ThreadPoolTaskScheduler:218", "message": "Shutting down ExecutorService 'taskScheduler'"}
TestEventExportIntegrationTest testEventSerialization() java.lang.AssertionError at TestEventExportIntegrationTest.java:65

Frontend

File Test label Description
PendingOrganizationsContainer.test.tsx organizations loaded › confirm/edit modal acts correctly › submitting an edit › saves information on change TestingLibraryElementError: Unable to find an element with the text: DC Space Camp. This could be because the text is broken up by multiple elements. In this case, you can provide a function for your text matcher to make your matcher more flexible.