Non functional requirements verification - Terminaator/chatbot GitHub Wiki

1. A average new user should be able to learn the application under 15 minutes. (Ease of Use)

We let some potential end users test our application. Here is their summary

  • There are still quite a lot of uncovered questions. Had some difficulty asking about things help command offers, but mostly got his answer. Said that it's fairly easy to use.
  • Also had a lot of unanswered questions at first. Tried to ask questions that we couldnt cover yet. Sometimes used synonyms, that should have been correct, but sentenceProcesser didnt know them. Said that Help command should be more visible. Overally was easy to use.
  • Also had unanswered questions at the beginning. After about 10 minutes knew what to ask for. Gave some recommendations: Improve help command, make to bot say what's missing from the users querry more often. for example user asks "hindamine", then the bot should say that the user should specify course.
  • In the beginning didn't understand how to ask questions. Help command didn't help much. After 11 minutes understood how to ask questions. Suggestions: Improve help command with some examples how to ask, make answers for unknown questions more interesting, remember asked questions longer, so user doesn't need to type course name everytime
  • At first, user had no idea what and how to ask. Example questions were missing. Help command didn't help that much. Test user just wanted to use keywords, like "tunniplaan". But after about 10 minutes, test user was able to ask different questions and get meaningful answers from bot.

In conclusion everyone had a bit of trouble using this bot at the beginning, but after about 10 minutes they knew how and what to ask for.

2. If the input data format changes, the developer will be able to make the required changes in < 20 person-hours (Extensibility)

It held true.

3. Authenticated user request about his/her personal information is accurate, authentic, and without corruption.

Should be true.

4. The application should have 0 CSS mistakes according to CSS Validator. CSS validator

5. The application should scale 100% with desktop, tablet and mobile screen sizes

On desktop and tablet it's currently fine. On mobile the app is a too long. Our app was back-end focused. The front end was only made so we could show our app better.

6. The bot should return an answer in at most 3 seconds. Unless there are more than 10 queries at the same time.

To test this I used a google chrome extension called RESTful Stress. I decided to test the bot with 20 with 10ms delays because if it can answer all these in 3s, it should easily do 10. For the question I chose "Mitu eap on tarkvaraprojekt?". The settings I used: Settings

This is the result: Stress test

The time it took (in ms) to answer is on the y-axis. As you can see the bot answered to 20 all questions in 2s.

7. Errors shouldn't crash the system.

Upon error, the bot says "Ma tõesti ei tea" and the user can ask again.

8. Project can be given over to next year students.

We made the project as easy to understand as possible. We also created guides for installing the application and adding new questions.

9. The application should support the latest web browser (Chrome v53.0+ and Firefox v48.0+) versions.

I used https://www.browserling.com/ to test this. Our application worked on both of these versions.

10. Project initial architectural choices (Python 3+, Framework Django, Library estnltk, Frontend Angularjs)

All the project's initial architectural choices remained the same.