Test reports - TanelOrumaa/Estonian-ID-card-mobile-authenticator-POC GitHub Wiki
| Test Case ID | Related requirements | Initial system state / input (data, action) | Expected system state/outputs | Actual Result | Fix? | Result after fix |
|---|---|---|---|---|---|---|
| TC #1 | UC #5 | Success scenario test: User with a valid Estonian ID card and saved CAN reads the ID card with the application. | The personal information from the ID card is displayed. | Expected result | No | Fixing not needed |
| --- | --- | Extension test: User with a valid Estonian ID card uses the wrong CAN to read the ID card info. | The user is notified that the CAN is wrong and the CAN is deleted from the application. | Expected result | No | Fixing not needed |
| --- | --- | Extension test: User with a valid Estonian ID cards removes it immediately when the card has been detected. | The user is notified that the ID card was removed too quickly. | Exected result | No | Fixing not needed |
| TC #2 | UC #4 | Success scenario test: The user saves the CAN of the ID card on the application. | The application notifies the user that the CAN has been saved and it is displayed to the user. | Expected result | No | Fixing not needed |
| TC #4 | UC #7 | Success scenario test: The user launches the application from the website. The user enters the CAN and PIN 1 of the card and puts the ID card against the smartphone. | The application closes itself at the end automatically | Expected result | No | Fixing not needed |
| TC #5 | UC #8 | Success scenario test: The user launches the application from the website. The user provides the required inputs and holds their ID card against the phone when requested. | After the app finishes reading the card and closes, the user is taken back to the website where the successful login attempt is confirmed. | Expected result | No | Fixing not needed |
| --- | --- | Extension test: The user tries to launch the application when it's not installed. | Nothing happens. | Expected result | No | Fixing not needed |
| --- | --- | Extension test: The user launches the application from the website. The user lets the 90 second period for placing the ID card against the phone expire. | The application closes and user is returned to the website | Application did not close, returned to the home screen instead. | No | Can not reproduce. |
| --- | --- | Extension test: The user performs all the same steps as in the main success scenario, except for removing the card from the phone as soon as it's detected. | Application notifies the user of the issue and returns to the website. | Expected result | No | Fixing not needed |
| --- | --- | Extension test: The user launches the application from the website. The application does not have the CAN saved, and when it requests it, the user enters an invalid one. | Application notifies the user of the issue and returns to the website. | Expected result | No | Fixing not needed |
| --- | --- | Extension test: The user launches the application from the website. The application does not have the PIN1 saved, and when it requests it, the user enters an invalid one. | Application notifies the user of the issue and returns to the website. | Expected result | No | Fixing not needed |
| TC #7 | UC #10 | Success scenario test: The user launches the application from the test app. The user provides the required inputs and holds their ID card against the phone when requested. | After the app finishes reading the card and closes, the user is taken back to the test app, where the successful login attempt is confirmed. | Expected result | No | Fixing not needed |
| --- | --- | Extension test: The user tries to launch the application when it's not installed. | Nothing happens. | Expected result | No | Fixing not needed |
| --- | --- | Extension test: The user launches the application from the website. The user lets the 90 second period for placing the ID card against the phone expire. | The application closes and user is returned to the test app | Expected result | No | Fixing not needed |
| --- | --- | Extension test: The user performs all the same steps as in the main success scenario, except for removing the card from the phone as soon as it's detected. | Application notifies the user of the issue and returns to the test app. | Expected result | No | Fixing not needed |
| --- | --- | Extension test: The user launches the application from the test app. The application does not have the CAN saved, and when it requests it, the user enters an invalid one. | Application notifies the user of the issue and returns to the test app. | Expected result | No | Fixing not needed |
| --- | --- | Extension test: The user launches the application from the test app. The application does not have the PIN1 saved, and when it requests it, the user enters an invalid one. | Application notifies the user of the issue and returns to the test app. | Expected result | No | Fixing not needed |
| TC #9 | UC #12 | Success scenario test: The user launches the application and goes to the settings screen. The user opts to add PIN1, enters a number between 4 and 12 digits long, and chooses to continue. | The application notifies the user that PIN1 was saved. | Expected result | No | Fixing not needed |
| --- | --- | Extension test: The user follows the same steps as in the success scenario, but enters a number that's either less than 4 or more than 12 digits long. | The user is notified of the incorrect length and is unable to save the PIN. | Expected result | No | Fixing not needed |
Non-functional requirements measurements
For testing purposes, the demo backend was set up according to the instructions in the README.md of the respective project using ngrok. The authentication application was launched through Android Studio. Measurements were taken manually using a stopwatch.
- USN9 - the application opens in under a second
- USN10 - the website receives the token in well under 15 seconds