A page that creates predictions for the optimal safe schedule given the clinic constraints.
Depends on
REST API
Angular
Interface
The UI interface for the prediction module is defined graphically as a form that when inputted the following constraint parameters:
Prediction range: int
confidence level
interval date ranges (intervals within prediction can be of different length)
Target service % for patient αi: int, for each triage category denoted as αi
target time window for patient αi:int for each triage category denoted as αi
(optionally) a file with the initial patient backlog
It returns the minimum number of required slots for all triage categories (i.e. urgent patient, semi-urgent patient, etc.) needed to see the expected number of patients in each category within their category's respective timeframe; the confidence interval around that number; and a breakdown of the expected slots per class;
Below are the major UI methods used to get the predictions:
function setPredictionResults(predictionResults: ): void
From the HTTP response, store the data in a "global" service to allow inter-component communication.
Returns
void
function getPredictionService(url: string, body: Object): Object
Makes an http request to the REST API to get the prediction given the url.
Parameters
url <string> - the request url
body <string> - the POST request body
Returns
Object - the http request response body
Secrets (Implementation Details)
The user input updates the state of the angular component (which is used to create the prediction module). Once all the fields are validated, for their correct type
The user inputs must be type validated using REGEX matching
Dynamic reloading is done via angular observables
Admin Module
Allows any technical or administrative staff to upload new data, and change any default behaviours.
Depends on
REST API
Angular
angular-file-uploader
Interface
The UI interface for the admin module is defined graphically in 2 parts:
i) Data uploading
More up-to-date data can be uploaded to keep the predictions relevant. Therefore the following can be uploaded in a dropbox:
Most recent patient waitlist
Past patient visitation patterns
ii) Configuration settings
Changing any default values can be done via a form that intake the following parameters:
The default prediction constraint values (mentioned above)
function updateCurrentWaitlist(): void
The file stored in the module state as a private variable is sent in the post body to the REST API
Returns
void
Secrets (Implementation Details)
Dynamic reloading is done via angular observables
Backend Modules
Rest API Spec Version v1
Some description
Depends on
Triage Controller
Flask
api.resources.predict
json
math
model.load_and_predict
scipy
Interface
The flask API interface can be defined via a REST schema
GET v1/predict
Given the following prediction constraint parameters, fetch the current wait list, get the patient arrival distribution, and then calculate the number of slots to allocate per triage class that maximizes the number of patients seen while satisfying the prediction/query parameters constraints. See the Secrets (Implementation Details) section for details.
Query Parameters
Parameter
Type
Description
intervals
list<string>
A list of dates (YYYY-MM-DD) on how to partition the prediction. This way some seasonality may be captured. I.e. (2020-10-01,2020-11-01)
Which translates to
(start-date - Oct 2020, Oct 2020 - Nov 2020, Nov-2020 - end-date).
Therefore make predictions for each of the three intervals. Required
Default is (start-date, end-date)
confidence
float
(1-α) of the overall prediction. Number [0,1]. Default is 0.95. Admin adjustable
num-sim-runs
int
The number of times to run the simulation. Default is 1000
waitlist.csv
csv file
The current patients that have yet to be seen
(triage_class, arrival_date). Default: empty wait list
clinic-id
int
The ID of a clinic. Required
Headers
Parameter
Type
Description
token
string<string>
The JSON web token retrieved from GET V1/auth
Response
Code
Description
200
Request is successful. Returns a JSON object in the form of Prediction
When the endpoint is invoked, load_and_predict_patient_total is invoked as a subroutine and returns the total number of patients predicted. From this, scipy linear optimization is used to find the minimum number of slots needed so that patients can be seen within the appointment deadlines. If no solution is found, a greedy algorithm for the number of slots is implemented, favouring patients by triage class priority. The slot distribution result is returned.
PUT v1/upload/past-appointments
Send the past appointment history by triage class data to base predictions off the most recent data history. Restricted to privileged users.
Query Parameters
Parameter
Type
Description
clinic-id
int
The ID of the clinic. Required.
Body Parameters
Parameter
Description
files
File containing the current waitlist. .txt, .docx, and .csv are accepted. Required.
Headers
Parameter
Type
Description
token
string<string>
The JSON web token retrieved from GET V1/auth
Response
Code
Description
200
Request succeeded
400
Bad request. Invalid file type, or no file was given.
401
Unauthorized
Secrets (Implementation Details)
Contents of the file are validated
GET v1/models
Get a list of all the prediction models for a particular clinic. Restricted to privileged users.
Query Parameters
Parameter
Type
Description
clinic-id
int
The ID of the clinic. Required.
Headers
Parameter
Type
Description
token
string<string>
The JSON web token retrieved from GET V1/auth
Response
Code
Description
200
Request succeeded
400
Bad request. Invalid file type, or no file was given.
401
Unauthorized
PATCH v1/models
Update a clinic’s prediction simulator to use a particular model. Restricted to privileged users.
Query Parameters
Parameter
Type
Description
clinic-id
int
The ID of the clinic. Required.
model-id
int
The ID of the pre-existing prediction model to currently use. Required.
Headers
Parameter
Type
Description
token
string<string>
The JSON web token retrieved from GET V1/auth
Response
Code
Description
200
Request succeeded
400
Bad request.
401
Unauthorized
Secrets (Implementation Details)
Prediction models have a creation date. Using this date, a cron job removes models that exceed a configurable retention time. This way, the list of models does not grow infinitely.
GET v1/classes
Display triage classes for a given clinic.
Query Parameters
Parameter
Type
Description
clinic-id
int
ID of the clinic. Required.
Body Parameters
Parameter
Description
Headers
Parameter
Type
Description
token
string<string>
The JSON web token retrieved from GET V1/auth
Response
Code
Description
200
Request is successful. Returns a JSON object containing list of triage classes.
400
Bad request. Could not update triage class.
401
Unauthorized
Secrets (Implementation Details)
DataBase module used to access data.
PUT v1/classes
Update a triage class for a given clinic.
Query Parameters
Parameter
Type
Description
triage-class
dict (keys: str, values: int / str)
Class information to update. Requires triage class severity (int), name (str), duration (int), and proportion (float). Required.
Body Parameters
Parameter
Description
Headers
Parameter
Type
Description
token
string<string>
The JSON web token retrieved from GET V1/auth
Response
Code
Description
200
Request succeeded
400
Bad request. Could not update triage class.
401
Unauthorized
Secrets (Implementation Details)
DataBase module used to access data.
Triage class changes are validated.
Database Module
A Postgres database that stored all the persistent data for Triage
Depends on
Postgres
Interface
The database interface consists of the tables and views.
Clinic
Name
Type
Description
id
int
Unique clinic identifier
name
string
Human friendly name for the clinic
Users
Name
Type
Description
clinic_id
int
Clinic the user is associated with
username
string
Unique username
password
string
Password hash
salt
string
Salt for protection against rainbow tables
admin
boolean
Whether the user is an admin
TriageClasses
Name
Type
Description
clinic_id
int
Clinic the triage class is associated with
severity
int
Used to order clinic triage classes. Lower is more severe
name
string
Human friendly name for the triage class
duration
TimeInterval
The target wait time duration
proportion
float
The target proportion of patients to see within the above duration
HistoricData
Name
Type
Description
id
int
Unique clinic identifier
clinic_id
int
Clinic the triage class is associated with
severity
int
Used to order clinic triage classes. Lower is more severe
date_received
date
The date the referral arrived at the hospital
date seen
date
The date the patient was seen
Models
Name
Type
Description
id
int
Unique model identifier
data
blob
The trained model weights
clinic_id
int
Clinic the model is associated with
accuracy
float
Measure of the successfulness of training. May be broken into more metrics in the future.
created
Timestamp
The time the model training finished
in_use
boolean
Whether the model is the active prediction model for the associated clinic
Schedules
Name
Type
Description
id
int
Unique schedule identifier
data
json
The schedule information
clinic_id
int
Clinic the schedule is associated with
Secrets (Implementation Details)
There is an index on User.username for faster user lookups
There is an index on Schedules.clinic_id to more efficiently find cached schedules
There is an index on Models.clinic_id to more efficiently get the model for a clinic
There is an index on TriageClasses.clinic_id to more efficiently get clinic triage classes
Triage Controller
The controller that provides the functionality exposed by the REST API
Depends on
Simulation Module
ML Training
Database
Interface
TriageController(): TriageController
Construct a new Triage Controller
Returns
A TriageController object
function predict(clinic_id, parameters): Schedule
Construct a Simulation
Parameters
clinic_id <int> - The clinic to run the prediction on
parameters <Dict> - The prediction parameters (all the triage class durations and proportions, and the number of times to run the simulation)
Returns
A Schedule object that contains the required number of slots for each triage class
function getDataManager(clinic_id): DataManager
Get a clinic’s DataManager
Parameters
clinic_id <int> - The clinic to get the DataManager of
Returns
A DataManager object with the following functions
getWaitlist
updateWaitlist
getPastAppointments(start, end)
addAppointments(appointments)
function getModelManager(clinic_id): ModelManager
Get a clinic’s ModelManager
Parameters
clinic_id <int> - The clinic to get the ModelManager of
Returns
A ModelManager object with the following functions
listModels
getModel
getModel(id)
trainNewModel(hyperparams)
Secrets (Implementation Details)
The controller schedules recurring training sessions on the updated data for each clinic
The predict function uses the database to cache prediction results
ML Training Module
A transient module that performs a training session
Depends on
Database
Interface
This module initialized with the following details
Clinic ID
Training hyperparameters
It then performs the training session and writes the model to the database module.
Secrets (Implementation Details)
Uses a GPU for training
Preprocesses the training data
Calls database to get training data
Simulation Module
Performs simulation using predicted data to produce a schedule
Current Design: Currently the data used for training/validation of the model contains the following values for each individual patient referral (See Prediction Module): Referral Date, Triage Classification, Date Seen (if available)
Possible Change: It is possible that the schema of this data may change in order to add values that can be used to ensure the prediction of accurate schedules.
Rationale: This data largely focuses on a singular patients referral without taking into account the state of the clinic at the given time. For example, it may be helpful to understand the wait queue at the time the patient was triaged in order to compare how the predicted schedules perform under varying circumstances compared to the hospital’s current solution. (Also see Data Constraints)
LC-DATA-2: Clinic Data Collection / Format / Delivery Changes
Current Design: Currently the patient referral data used for training/validation of the model is manually collected, anonymized, and stored in a CSV format. This data is then manually sent to the team to be used for improving the model’s accuracy. (See Datastore Module and Data Constraints)
Possible Change: It is possible that the schema of this data may change in order to add values that can be used to ensure the prediction of accurate schedules. It is also possible that the collection/delivery method may be altered to remove manual work done by hospital staff.
Rationale: It is likely that the project will be utilized by numerous clinics (and possibly hospitals). Manual data collection and delivery is simply not a scalable process. Given that patient referral data is already logged within ERB systems, it is possible that with the permission of the hospital, the application may be able to query data directly from a database (in non-CSV format), removing the need for members of a clinic to manually collect, anonymize, and deliver the data to the team. The goal is to ultimately automate the process found within the Prediction Module. (See Datastore Module and Data Constraints)
LC-DATA-3: Multiple Clinic Data
Current Design: Currently the patient referral data used for training/validation is solely provided by the nephrology clinic at the McMaster Children’s Hospital. (See Datastore Module and Data Constraints)
Possible Change: It is possible that the application will be utilized by multiple clinics and hospitals.
Rationale: The McMaster Children’s Hospital (and possibly other hospitals in the Hamilton area) are currently standardizing the triage process across all clinics. This includes developing consistent definitions for triage classifications across clinics as well as formalizing the data collection process and standards. As such, there has been interest in this application from McMaster Children’s Hospital specialists outside of the nephrology clinic. Ultimately this will require the Datastore Module to provide enough flexibility to handle acquiring, storing, and delivering data from multiple clinics to other modules of the project.
LC-DATA-4: Hospital / Region Wide Data
Current Design: Currently the patient referral data used for training / validation is solely provided by the nephrology clinic at the McMaster Children’s Hospital. (See Datastore Module and Data Constraints))
Possible Change: It is possible that utilizing hospital and region data may aid in the overall goal of providing shorter wait times to patients.
Rationale: The McMaster Children’s Hospital conducted an audit, the results of which suggest a series of data points that may have an effect on patient waiting times. Some of these proposed data points can only be found in a hospital or region-wide datasets. For example, it may be beneficial to understand for any given patient referral to a clinic whether there exists a number of alternative clinics that may be able to provide a shorter wait time. (See information about the Hospital Audit, as well as the need for more data in Data Constraints)
Behaviour
UI
LC-UI-1: User Experience
Current Design: Currently the design uses a series of tables and charts to display scheduling predictions to the user. (See Prediction Interface)
Possible Change: It is likely that the format of these tables and charts will change over the course of development.
Rationale: Ideally the application provides similar user experiences to that of other applications used by hospital staff. Given this goal, the team will work with various customers to develop a clear understanding of their technical capabilities and preferences. (See Stakeholders - Doctors and Hospital Administrators for more information about the customers)
LC-UI-2: Additional Functionality
Current Design: Currently the UI offers pages for predicting a schedule, altering settings/user profiles, and reading help documentation. (See all pages in External Interfaces - Software Components)
Possible Change: It is likely that with the addition of new customers (See LC-DATA-3), data formats/content / visualization formats (See LC-UI-1), that users will require new pages to interact with the various features of the project.
Rationale: Rationale: It’s not unlikely to believe that different customers may desire slightly different interface functionalities. Given that different clinics/hospitals may find importance in varying pieces of information, it's not unlikely that the project will need to provide various interfaces with flexibility that will evolve over time in order to fit the needs of the user. (Note that these changes may also come up due to Standards Compliance Constraints)
API
LC-API-1: New API Endpoints
Current Design: Currently the design offers a series of endpoints for managing the data used to predict scheduling as well as a single prediction endpoint. (See HTTP API Backend Module)
Possible Change: It is possible that given the future expectations of providing the product to new clinics and hospitals (See LC-DATA-3), along with changes to data (See LC-DATA-1 and LC-DATA-2), as well as the need for new UI functionality (See LC-UI-2), may result in a need for new API endpoints.
Rationale: It’s not unlikely to believe that different customers may find varying pieces of information useful to them. There may also be various methodologies and rulesets by which clinics prefer to schedule patients which may require varying approaches to the prediction of patient scheduling. In order to satisfy the needs of all customers, it is likely that we will require multiple similar endpoints, each tailored towards the goals of a given group of users. (Note that these changes may also come up due to Standards Compliance Constraints)
LC-API-2: API Endpoint Inputs / Outputs
Current Design: Currently the input and output structure and typing is defined as seen in this design documentation (See Rest API Spec Version 1).
Possible Change: It is possible that with possible data changes (See LC-DATA-1 and LC-DATA-2), new functionality requirements (See LC-UI-2), and new customers storing information in unique ways (See LC-DATA-3), that the application will be required to handle / return different inputs and outputs (both in structure and type).
Rationale: It is understandable that if the functionality changes, there is a high likelihood that new or different datapoints will be required from the user to predict schedules. Similarly, if the process of data collection / retrieval is automated (See LC-DATA-2), it is highly likely that varying clinics / hospitals will store information in unique ways and the project will require some form of adapter to remain usable in all customer use cases. (Note that these changes may also come up due to Standards Compliance Constraints)
Triage Controller / Simulation Module
LC-TCSM-1: API / Data Changes affecting Triage Controller / Simulation
Current Design: Currently the triage controller works with the simulation module (and the ML / Prediction module) to create a prediction given a set of parameters.
Possible Change: As described above, the parameters may change (See Likely API Changes) thus altering the behaviour of the triage controller and simulation module.
Rationale: Changes in input and output data would require the triage controller / simulation module to handle inputs and format outputs in new ways (See Rest API Spec Version 1). Further, changes in requirements that alter the contents of the inputs / outputs will require the triage controller and simulation module to operate under new assumptions and generate different data.
ML Module
LC-ML-1: ML Module Structure Changes
Current Design: Currently the ML module is a neural network operating under the assumptions that it is provided a very specific schema of data for training / validation / prediction. (See Prediction Module
Possible Change: It is possible that the data used for training and validation may change in order to improve the accuracy of predicted schedules. (See LC-DATA-1)
Rationale: Changes in this data would have to be reflected in the training / validation / prediction functionalities offered by the ML Module. (See more about the importance of having formalized data formats / content)
Hardware Specifics
LC-HS-1: Hosting Hardware Changes
Current Design: Currently the project is planned to be hosted on McMaster Children’s Hospital servers. (See Project Assumptions - Hosting)
Possible Change: It is possible that this project may expand to servicing customers from other hospitals (See LC-DATA-3) and will therefore be hosted on multiple different types of hardware.
Rationale: Given that the team is unaware of the hardware architecture that is utilized by possible future customers, it is possible that different hospitals may have different hardware architectures available. With this fact in mind, the technologies used by the project must be easily portable to a variety of different types of hardware (See Portability).
Security Requirements
LC-SR-1: Hosting Hardware Security Changes
Current Design: Currently the project is planned to be hosted on McMaster Children’s Hospital servers and will conform to their current security standards. (See Product Constraints - Standards Compliance Constraints)
Possible Change: It is possible that this project may expand to servicing customers from other hospitals (See LC-DATA-3) and will therefore be hosted on multiple networks at different locations with varying security standards. Similarly, it is possible that the McMaster Children’s Hospital IT staff will release new standards for hosting software on their networks.
Rationale: It is essential that the software conforms to the security standards of the systems it is being hosted on in order to continue running. Modifications to these standards will naturally result in a change to the security requirements of the project. (See Security Requirements)
Possible Change: It is possible that this project may expand to servicing customers from other hospitals (See LC-DATA-3) and will therefore need to conform to the data usage / storage specifications of these establishments. Further, the government of Canada may revise the guidelines it currently has in place over time.
Rationale: Given the complexity of legal challenges related to utilizing patient data, it is likely that different hospitals with varying types of patient data may have different rules that the project must follow in order to receive the necessary data to predict schedules. Further, it is not uncommon for the government of Canada to reevaluate and revise their legislation with regards to utilizing medical records in software projects. (See Security Requirements)
Unlikely Changes
Data
UC-DATA-1: Patient Data Dependencies
Current Design: Clinic data is required to train and verify the model as well as to predict future requirements. See Product Scope
Rationale: This data is required to make accurate and dynamic predictions and is the foundation of the project.
UC-DATA-2: Machine Learning Data Storage
Current Design: Machine learning models and results stored in the PostgreSQL database. See ID: DAT-5 from the data storing requirements.
Rationale: These results may be helpful for users in the future and should remain accessible. Even in the event that the model's predictions were to change, there should be no need to change the results storage method.
Rationale: This ensures the application is highly portable and can be deployed to various customers (clinics) as necessary. See: Portability
UC-UI-2: Frontend Technology (Angular)
Current Design: The frontend of the application is currently implemented using Angular. See Web Client Host
Rationale: It would require large sums of work to change this implementation choice and would likely offer little overall benefit.
API
UC-API-1: API Technology (Flask/Python)
Current Design: The API of the application is currently implemented using Flask, a Python library. See HTTP Server Host
Rationale: It would require large sums of work to change this implementation choice and would likely offer little overall benefit. Flask has direct libraries that allow for an extension to any foreseeable use case including authentication and routing.
UC-API-2: Data Input
Current Design: The API currently receives application input using both URL parameters and body parameters. See Backend API Module
Rationale: The dual input types allow for complex data-rich requests such as file transfers to be handled by the body of the request and less data-rich requests to be handled by URL input parameters.
UC-API-3: Data Serving
Current Design: The API currently serves data to the application frontend using JSON web objects. See Backend API Module
Rationale: JSON objects offer a data-rich, standardized format that can easily be extended based on use case.
Triage Controller / Simulation Module
UC-TCSM-1: Simulation Technology (Python)
Current Design: The triage controller and simulation module are currently implemented in Python. See Simulation Module
Rationale: It would require large sums of work to change this implementation choice and would likely offer little overall benefit.
ML Module
UC-ML-1: Machine Learning Model Technology (Keras/TensorFlow/Python)
Current Design: The machine learning module is currently implemented in Python using Keras for TensorFlow. See Predictive Model Training
Rationale: Keras is easy to work with and boasts easy to modify code to interact with the library. It is also an ideal choice when working with time series analysis.
Hardware Specifics
UC-HS-1: Customer Provided Hardware
Current Design: The application is expected to run on the customer's provided hardware and with no requirement of external resources such as cloud hosting. See IT Infrastructure and Service Costs
Rationale: This aids in security, decreases project costs, and limits host tailoring requirements.
Security Requirements
UI Authentication
UC-SRUI-1: Protected Pages
Current Design: Sensitive pages that contain user's information or user data will be authentication protected. See ID: SEC-3
Rationale: User's need their personal and uploaded data to remain confidential and protected.
API Authentication
UC-SRAPI-1: Protected Endpoints
Current Design: Sensitive endpoints that contain user information, user data or some data based off of user input will be authentication protected. See ID: SEC-4
Rationale: User's need their personal, uploaded, and prediction data to remain confidential and protected.
Data Usage Security
UC-DUS-1: Authenticated Access Only
Current Design: The application will restrict data usage to designated authenticated users. See Security
Rationale: Data is user-specific and should be protected from other users using the data for their own purposes.
Data Storage Security
UC-DSS-1: Anonymized Patient Data
Current Design: The data provided to the application is anonymized. See Assumptions, Data Storage
Rationale: This significantly reduces the burden of authentication as a breach is significantly less compromising. This will also aid with our ability to adhere to government guidelines and standards.
UC-DSS-2: Authenticated Access Only
Current Design: The application will restrict data access to designated authenticated users. See Security
Rationale: Data is user-specific and should be protected while it is stored. No user that does not have the authorization of the uploading user or is an administrator should be able to access this data.
Problems Related to Problem Domain
Data Processing
Definition: The process of collecting raw data, transforming this data into a usable form and then storing this data. This process occurs in six stages: 1. Data collection: the pulling or generating raw data from relevant sources. 2. Data preprocessing: Cleaning and organizing the data. 3. Data Input: cleaned data is entered into its destination system. 4. Processing: the inputted data is analyzed using statistical models or machine learning algorithms. 5. Data output: The results of the data processing are outputted in a usable form. 6. Data storage: the data is stored for future use.
Rationale: This is the central concept of our entire application. Every other concept listed is either one of the phases of or is related to one of the phases of data processing.
Design/Run Time Concept: Both. Currently, data collection does not appear in any module and is done manually so it only appears in our design and not in our code. Data preprocessing appears in the ML Training module. Data input appears in the ML Training and Simulation modules. Data output appears in the Prediction module. Data storage appears in the Database module.
Data Collection:
Definition: The pulling or generating of raw data (uncleaned and unedited data) from relevant sources.
Rationale: In order to generate accurate and reliable predictions, our application needs relevant up-to-date data. Collecting this data in a timely fashion so that it is still up-to-date is therefore a very important step.
Design/Run Time Concept: Design time. Current data collection is done manually where one of the doctors pulls data from the hospital systems and preprocesses and anonymizes it before sending the massaged data to us. So data collection does not appear in any module. However, this may change. See LC-DATA-2
Machine Learning
Definition: Machine learning is the use of computer systems that are able to learn and adapt without explicit human intervention by using algorithms and statistical models to analyze and draw inferences from patterns in data provided to the system.
Rationale: Machine learning is central to our application. If the system needs frequent human instructions to provide accurate modelling that both costs time and money and makes the system not much better than their current solution. Machine learning is also related to other concepts in our domain such as time series analysis.
Design/Run Time Concept: Run time. The data that we feed the model, which algorithms we use to train the model, how we use the model to generate predictions are all concepts that are present in both the ML Training module and the Simulation module
Time Series Analysis
Definition: Time series analysis is the examination of time series in order to be able to create a forecast for the future. Time series analysis looks at several aspects of time series such as seasonality (trends in the series that follow time periods) and autocorrelation (how similar events are as a function of the time difference between the events). Time series are a series of data points ordered by time. Typically for time series time is the independent variable.
Rationale: Time series analysis is the main type of analysis that our machine learning model will have to perform. This is because our goal is to be able to create a forecast based on historical data and because our early data indicates that there are seasonal trends in the number of patient arrivals.
Design/Run Time Concept: Run time. The aspects of time series we look at, our approach to modelling time series and the functions we use to model the time series, and how we generate our forecast based on the time series will all be present in our code in both the ML Training module and the simulation module. We’ll also be allowing the user (doctors and hospital administrators) to specify the amount of time they want to forecast for in the Prediction module.
Data Output
Definition: Data output is the translation of processed data into a usable form that is readable by the user, typically in the form of graphs, or images.
Rationale: Users being able to visualize the results of the data analysis is key to them being able to understand why they are using the system and why it gave them the predictions or results that it did. Good data output increases the trust our users will have in our system.
Design/Run Time Concept: Run time. How we display data and what we will display appear in the Prediction module.
Interface usability
Definition: The interface is the part of the application that a user would directly interact with. The usability of an interface measures a variety of factors such as learnability (how easy it is to learn to use the system), functionality (the range of functions the interface can support), retention (how easy is it to remember how to use a system if a user has not used the system in a while), error rates (how often the user makes an error and how easily are these errors caught and corrected), trustability (how much users trust the system) and subjective satisfaction (how pleasing is the interface to use).
Rationale: Designing a highly usable interface is a key aspect of our project. No matter how good our model or how accurate our predictions, if the doctors, and hospital administrators cannot use our interface that reduces the impact of all of our other work. Our interface needs to be easily learnable and give them good functionality. (in this case the ability to customize prediction parameters)This concept is highly related to data output as the main thing users will be viewing will be the data output.
Design/Run Time Concept: Design time. The user and how they will use the system is a very important factor in our design affecting not only which specific design goals we’ll target for the front-end but also the features and how they are implemented in the back-end. However, the user will never appear in our code.
Data Storage
Definition: Data storage is the process of storing data for future use. This includes both determining the schema for how the data is stored and the actual storing of data. Data is typically stored in some type of database but can also be stored in files or using other methods.
Rationale: Our application needs to store data in order for us to easily retrieve and use this data later. This allows us to retrain the model with recent data while also taking historical trends into account.
Design/Run Time Concept: Run time. The exact schema we use for storing the data needed for our system will explicitly show up in the Database module
Data Protection
Definition: Data protection is the process of safeguarding data from corruption (the data is changed or modified unintentionally), compromise (an unauthorized person/persons gain access to the data), or loss (the data becomes irretrievable).
Rationale: Because we are storing data and especially because we are working with patient data which even when anonymized can be highly sensitive, it is necessary to protect the data that we store to ensure that it is not compromised. Almost as important, because the system uses this data to make predictions, data loss or corruption make the entire system less accurate and reliable, so we need to minimize the possibility of this happening.
Design/Run Time Concept: Run time. How we plan to protect data such as data verification, user verification, and data encryption all appear in the Interface module and the Database module.
Data Privacy and Anonymization
Definition: Data privacy
Rationale: Because our application is working with patient data which is highly sensitive and heavily regulated by PIPEDA it is very important that all data we use and store is anonymized and that patient privacy is protected.
Design/Run Time Concept: Design time. Exactly what data we need and if we could get that data in an anonymized form is a very important to design considerations for us. The data we do request has all been anonymized and we ensured in our design not to request any non-anonymized data. Because currently our data collection is done manually, and all the data we receive is already anonymized there is no reference to anonymization in our modules. However, this may change. See LC-DATA-2
Rationale: it is ideal to have a graphical user interface to upload new training data, and a more recent wait list to the backend/database, and also a place to update default configurations
Commit: backend and UI: added optimization requirement to the backend. Added admin panel
Change: Specified optimization algorithm for the backend (See Backend API Module)
Rationale: the number of slots returned and their distribution should favour the urgent patients over the standard patients. Also it should try to maximize the number of patients seen under the number of slots available and the minimum % of patients seen for each triage class.
Commit:
backend and UI: added optimization requirement to the backend. Added admin panel
3.2.1.1: added optimization constraint to the backend API
Removed Requirements
Backend Modules
PUT v1/upload/waitlist
Send the most current clinic waitlist to base predictions off of. Restricted to privileged users.
Reason for Removal
Originally determined that the client could upload a waitlist but the client has chosen not to give us a waitlist and to base predictions off of only seen patients.
Query Parameters
Parameter
Type
Description
clinic-id
int
The ID of the clinic. Required.
Body Parameters
Parameter
Description
files
File containing the current waitlist. .txt, .docx, and .csv are accepted. Required.
Headers
Parameter
Description
user-pass
The hashed user’s password. Required
Response
Code
Description
200
Request succeeded
400
Bad request. Invalid file type, or no file was given.