Mobile(PWA) - COS-301/graduates GitHub Wiki

Documentation

NOTE: Assignment2 JavaScript Wiki Documentation can be found here:

Final_Wiki (3).pdf

1. Introduction

The purpose of the Mobile Feature is to ensure that the UP Computer Science Graduates portal fulfills the main requirements of a PWA (Progressive Web App), namely installability, offline functionality and mobile friendliness in terms of UI/UX. This is being done by the JavaScript Team who will be responsible for overseeing that all other features of the Graduates portal adhere to the PWA requirements, as well as assisting the various feature teams in implementing their features accordingly using the knowledge gained throughout our research into the matter.

2. Functional Requirements:

FR1. Once the user opens the graduates portal, they must be able to install the web app and thereafter access it on their device.

FR1.1. The web app must have a web manifest with the correct fields.

FR1.2. The web app must work on the localhost domain.

FR1.3. The web app must have an icon to represent it on the user's device.

FR1.4. The web app must have a registered service worker.

FR2. When the web app is opened the user must have an experience on par with a mobile app.

FR2.1. The web app must have a simple installation process

FR3. The web app must be responsive to a generic device (ie. smartphone).

FR3.1. The web app must have icons and images of appropriate size of the generic device.

FR4. The web app must work independent of internet connection (ie. work offline or at low network quality).

FR4.1. The user must be able to revisit pages they have once visited, even when internet connection is critical.

FR4.2. The web app must have a cached shell to load when offline.

FR5. All of the features of the web app must function as expected on a mobile app.

FR5.1 The web app must be mobile responsive.

2.1.1 Use Case Diagrams

3. Non-functional Requirements:

Quality Requirements:

NFR1. The web app must be linkable, the user must have the ability to link to the web app using a specific URL without the need for an app store or a complicated installation procedure.

NFR2. The web app must have a look that aligns with the user interface of the graduates portal.

Performance:

NFR3. The performance of the web app must be similar to that of a mobile app.

NFR3.1. The time it takes from when the web app loads to the user being able to interact with it must be under 3.8 seconds.

NFR3.2. The time at which the first text or image is painted must be under 1.8 seconds.

NFR3.3. The time in which content is visually displayed during page load must be under 3.4 seconds.

NFR3.4. The render time of the largest image or text block visible within the viewport, relative to when the page first started loading must be under 2.5 seconds.

NFR3.5. The total amount of time that the application’s pages are blocked from responding to user input should be under 0.2 seconds.

NFR3.6. The largest burst of layout shift of layout shift scores for every unexpected layout shift that occurs during the entire lifespan of a page should have a score of 0.1 or less.

Architectural Requirements:

Flexibility:

NFR4. The PWA must function on a generic mobile device (ie. smartphone).

NFR5. The PWA must work in all browsers (eg. Firefox and Google chrome).

NFR6. The PWA must be installable from a common variety of web browsers (ie. Firefox & Google Chrome).

NFR6.1. The PWA must be usable on a basic level (ie. version 1.0) on older browsers.

Maintainability:

NFR7. There must be constant communication between developers of the PWA and the project owners in order to identify and repair errors thus improving the quality of application and user experience.

NFR7.1. Clear and concise documentation on software requirements specification must be provided.

NFR8. The Angular framework is used to build the Graduates Portal and as the PWA uses a single codebase, in turn it must use the same
framework.

NFR8.1. Component-based architecture is more manageable and maintainable.

NFR8.2. Components must be decoupled to aid more efficient maintenance and updates of code.

NFR9. The PWA must contain a manifest file that must be kept up to date as it contains information about the website.

NFR9.1. The manifest file of the PWA can be modified to reflect updates.

NFR9.1.1. Updatable Web Manifest fields include:

-Theme colour

-Scope

-Display

-Icons

-Shortcuts

NFR9.2. The name or location of the PWA manifest file must not be tampered with, as this prevents the browser from updating the PWA.

Scalability:

NFR10. The server scalability is closely related to the scalability of the web app host server.

NFR11. The PWA caters for offline functionality once it has been installed, this in term increases the ability for users to use the PWA at the same time, as it is installed.

NFR12. PWA must work anywhere from a single codebase as it is installable.

NFR12.1. Single code base caters for wrapping up, which in turn provides high scalability.

Availability:

NFR13. The PWA must be available on any common web browser.

NFR14. The PWA must be available and easily accessed by any device, desktop or mobile.

NFR15. The PWA must be available at all times ( day or night, connected to the internet or not) once it is installed.

Reliability:

NFR16. The installation of the PWA is reliable on internet connection.

NFR17. The PWA must have the capability of working offline, (i.e. independent of internet connection).

NFR17.1. The service worker of the PWA must be responsible for caching content.

NFR17.2. The PWA must use the application shell and IndexedDB to enable the web app to function regardless of internet connection.

NFR18. The PWA must ensure a stable experience to the user regardless of the quality of internet connection.

Usability:

NFR19. The PWA must use UI components developed by the UI kit team.

NFR20. The PWA must use a single codebase as the graduates portal, which is built on the Angular Framework which uses component-based architecture.

NFR20.1. Components are self-sufficient and thus developers can reuse components throughout the the application (ie. the Graduates Portal, as a whole).

NFR21. The PWA must be mobile friendly.

Deployability:

NFR22. The PWA must be deployable to various platforms, such as:

NFR22.1. Web application (ie. web browser),

NFR22.2. Android

Architectural Constraints:

3.1.3.1. The PWA’s frontend must be implemented using the Angular framework.

3.1.3.2. The PWA’s backend must be implemented using the NestJS framework.

3.1.3.3. The PWA’s service worker capabilities must be extended using IndexedDB and app shell to make it work offline.

3.1.3.4. All testing of the PWA’s capabilities must be performed using Cypress and Jest.

3.1.3.5. Designs of the PWA must be done on (but limited to) Figma.

3.1.3.6. For retrieving data from an API into the PWA, GraphQL must be used.

-POST methods are used by GraphQL to send data to the API server.

-Service Workers that are set up by Angular only cache API with GET methods

-In order to solve this problem, another service worker needs to be implemented. This service worker must override the event listener of the Angular service worker.

3.1.3.7. GitHub and Visual Studio Code must be used to share source code and collaborate with other developers

4. Components/Services

Team roles have been specified for each member, however members may have contributed beyond the scope of their roles due to the nature of the feature.

4.1 AI (Andrey) Omeltchenko - u04534205 (Project Manager)

Communication and Coordination:
Multiple means of communication were established for various purposes for the duration of the first sprint, namely a WhatsApp group (for meeting plans and brief discussions) and a Discord server (to facilitate meetings and share preliminary resources and ideas).

Communication between PM and team members happened on a near daily basis in order to make sure there was an understanding of what is expected of the team as well as each team member.

Team members were tasked with participating in weekly meetings on Mondays and Fridays in order to make sure that everyone on the team was aware of the tasks that needed to be completed, as well as to discuss the progress made throughout the week. End of week meetings were meant as a weekly review session where each member presented their progress in terms of their role and responsibilities. This was done to ensure accountability and create a motivated environment so that team members felt that the team was slowly moving towards the end goal.

Minutes for each meeting were taken and can be found on GitHub, while initial minutes were posted on the BlackBoard Discussion Board.

Management:
In order for team members to collaborate, a Milanote board was set up, which provided a space to keep track of tasks and who would be responsible for those tasks. The board was also used to brainstorm ideas within the team. The GitHub Project boards were also used, although to a lesser extent as certain functionalities of Milanote were not replicable.

Milanote Board:
https://app.milanote.com/1NpFRF1D2ZkRbO?p=cSWgAYb060P

4.2 NA (Nasiphi) Mjobo - u18074074 (Business Analyst)

My role as the Business Analyst is to evaluate the client’s needs for the system and thereafter provide the first step to the solution. The first step in implementing the solution would be the identification of the respective requirements, these are namely: the functional requirements and nonfunctional requirements, as well as the acceptance criteria. Below are the contributions I’ve made with regard to my role.

Research:
I did research on what exactly PWAs are, they’re structure, how they work and how to implement the feature accordingly. (Please see references section)

Planning:
Prior to the use of gitHub project boards, our PM (Andrey Omeltchenko) created a Milanote board where we can start our planning, the below link conveys the planning I did. https://app.milanote.com/1Ntocm1Mu6Zr4l/business-analyst-board?p=e45FvES6CbR

Documentation:
After extensive research on what PWAs are and trying to understand how they work, I began to compile the requirements. Through the help of my teammates as well as research I was able to identify the functional requirements, non functional requirements and acceptance criteria of the mobile feature based on what the client wants. Thereafter I added this information on the Requirements wiki. See below:
https://github.com/COS-301/graduates/wiki/Requirements
After clarification and consent from Arne (Project Owner), I changed the format of the documentation into the SRS formatting standard.

Overleaf Software Requirement Specification documentation:
https://www.overleaf.com/read/mhxryzcsvxkx

Google Docs : Software Requirement Specification documentation: https://docs.google.com/document/d/121c7XtMvi7nSINLU-k-p6JLkunscnRgn_9_Nge9l0kc/edit?usp=sharing

I implemented the use case diagram depicted above.

Meeting Minutes:
I have also recorded minutes in the respective meetings:
The BA meeting:
https://github.com/orgs/COS-301/teams/business-analysts/discussions/9

JavaScript meeting(s):
https://github.com/orgs/COS-301/teams/javascript/discussions/1/comments/6 https://github.com/orgs/COS-301/teams/javascript/discussions/1/comments/15

4.3 TS (Tshego) Manthata - u17110310 (Designer)

PWA Designer responsibilities
My responsibility as the PWA Designer is to ensure mobile-friendliness/ responsiveness of the components designed throughout the whole application. As part of the designers' documentation, I have to ensure that every webpage has a corresponding mobile design adaptation. To guarantee that the components used throughout are mobile-friendly, I have collaborated extensively with the UI Kit Designer amongst other groups.

@COS-301/Designers Collaboration
During the course of the first splint I have collaborated with the @COS-301/Designers to create a colour palette that will match the theme of the University of Pretoria affiliated websites namely blue, white, red and gold. The colour scheme the @COS-301/Designers chose from were from the colour options I created. These colours were largely Inspired by the cs website as well as the up portal.

I outlined the skeleton in the wiki for the @COS-301/Designers. This includes grouping and ordering the wiki by features.
I collaborated with the UI Kit Designer and adapted essential components to an Android Large device.
I created mobile wireframe and mockups templates for the @COS-301/Designers to use in their mobile design mockups.

https://www.figma.com/file/Cf4kPiiGSrvITHUzbtzydC/Untitled?node-id=0%3A1

@COS-301/Designers Documentation

Milanote https://app.milanote.com/1NsnIt1MtLQ721/designer-board?p=e45FvES6CbR

Meeting Minutes For Javascript:
https://github.com/orgs/COS-301/teams/javascript/discussions/1/comments/2
https://github.com/orgs/COS-301/teams/javascript/discussions/1/comments/3
https://github.com/orgs/COS-301/teams/javascript/discussions/1/comments/5
https://github.com/orgs/COS-301/teams/javascript/discussions/1/comments/11

For @COS-301/Designers:
https://github.com/COS-301/graduates/discussions/6#discussioncomment-2407083

4.4 T (Tlholo) Koma - u20575085 (API Engineer)

My job as an API Engineer was to find a use of an API for our feature. Through research, I have found that no API has to be constructed for the feature because there has been no communication between two different computers or computer programs. The APIs that seem necessary for this feature (i.e. making resources fetched by requesting from APIs to display offline) are not to be constructed by the JavaScript team. This enables collaboration with the API Engineers from other teams. I have, however, kept up to date with the other API Engineers from other teams by attending the meetings.

I have worked together and am still working with my team members, K Haarhoff, K Bhoola and NL Ntuli, by participating in doing research about the offline requirement specified in the functional requirements. The part of the research was how to achieve such a requirement for the Graduates Portal. Our findings are expected to be implemented during the second sprint. Documentation of the different methods were compiled for future teams that wish to work on the Mobile (PWA) feature to review and, if necessary, change the method that my team has chosen and implemented. The documentation also includes the strategies to be used to accomplish the offline requirement.

https://github.com/COS-301/graduates/wiki/Mobile(PWA)#pwa-service-workers-and-offline-requirement

I’ve assisted the Designer, TS Manthata, with the wireframe designs of how the website should look on a mobile device. I was responsible for creating the wireframe for the login page.

Meeting Minutes:

https://github.com/orgs/COS-301/teams/javascript/discussions/1/comments/6

https://github.com/orgs/COS-301/teams/javascript/discussions/1/comments/14

4.5 K (Kyle) Haarhoff - u19033347 (Service Engineer)

As a service engineer my role was largely unnecessary in the context of a PWA, and as such part of my job was to attend the service engineer meetings to update my team on important discussions if they occurred. I conducted research about the functions and needs of a service worker and discussed concerns with the project owners in order to help elicit the requirements of the feature originating from the service worker. I further helped determine how we would use the service worker to fulfill the offline requirements of a pwa and what strategies would be employed to implement this.

Meeting Minutes:
https://github.com/orgs/COS-301/teams/javascript/discussions/1/comments/10

4.6 NL (Njabulo) Ntuli - u19062665 (DevOps)

As a DevOp, my role is to monitor PRs of the team. Fixing the pipeline, whenever there is a break. I did research on some of the DevOP tools. Which was suggested in the DevOps meeting that was held on the 12th of March 2022. Git-Lint is one of the tools I researched on. Which is a tool that analyzes Git commits. And ensure a clean history of commits. I collaborated with API, Data, Service Engineers for the offline functionality of the feature. Where caching method and strategy was decided. I modeled the Activity diagram of the functionality.

Meeting Minutes:
https://github.com/orgs/COS-301/teams/developer-operations/discussions/4
https://github.com/orgs/COS-301/teams/javascript/discussions/1/comments/13

4.7 K (Kajal) Bhoola - u20554240 (Data Engineer)

Collaborated with other data engineers:
I attended meetings with other data engineers which gave me a clear overview of the schema, general standards, and common conventions that all data engineers should adhere to when creating a database. After doing some research and receiving advice from other data engineers, I concluded that a database would not be needed for our feature.

Collaborated with other group members:

I am currently working on the offline functionality of the PWA, along with three other group members(Tlholo, Kyle and Njabulo). I have researched and created documentation and now have a clearer idea of how we plan on making the PWA installable and work offline.
I assisted the designer(Tshego) by creating wireframes for the student profile page.
I added my research to the Milanote board and made a few contributions to the project board on Github.

Resources:
Angular.io. 2022. Angular. [online] Available at: https://angular.io/guide/service-worker-config [Accessed 24 March 2022]. Developer.mozilla.org. 2022. Cache-Control - HTTP | MDN. [online] Available at: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control [Accessed 24 March 2022].

Meeting minutes:
https://github.com/orgs/COS-301/teams/javascript/discussions/1/comments/7
https://github.com/orgs/COS-301/teams/javascript/discussions/1/comments/8
https://github.com/orgs/COS-301/teams/javascript/discussions/1/comments/12

4.8 MJ (Matome) Makgopa - u18166793 (Tester)

Collaboration with Business Analyst:
I did research on the non-functional requirements of the system and helped the Business Analyst to identify the non-functional requirements of the progressive web app feature of the Graduates Portal.

References:
Below are the references to the documentation that I looked at to help me to better understand the non functional requirements of the system.

web.dev. 2022. Lighthouse performance scoring. [online]
Available at: https://web.dev/performance-scoring/ [Accessed 23 March 2022].

web.dev. 2022. What makes a good Progressive Web App?. [online]
Available at: https://web.dev/pwa-checklist/ [Accessed 23 March 2022].

Requirements Specification:
Below is the link to the requirements specification.
https://github.com/COS-301/graduates/wiki/Requirements

Formulating test plan: I worked with the team of testers to draw up a test plan that is going to be followed throughout the development process. Among the things that were discussed during our meetings were the protocols to be followed when logging tests for both automated and manual testing.We also discussed how we are going to go through the different phases of testing i.e. unit testing, integration testing, end to end testing and usability testing.

Meeting Minutes:
https://github.com/orgs/COS-301/teams/testers/discussions/3
https://github.com/orgs/COS-301/teams/testers/discussions/5

Test Plan:
https://github.com/COS-301/graduates/wiki/Testing

4.9 AI (Andrey) Omeltchenko - u04534205 (UI Engineer)

Research:

The necessary technologies required for the implementation were researched which included Angular, Angular Material, SCSS and Tailwind.

Collaboration:

A meeting was held with all the UI Engineers where it was discussed that for the first sprint the UI Engineers would need to wait for designs to be finalized by designers, and in the meanwhile research was to be done. I informed the UI Engineers that the implementation needed to account for mobile friendliness in regards to the platform being a PWA. Additionally, there was collaboration with the designer of the team where ideas were shared in order to find an optimal representation of the PWA when interacting with it on mobile devices.

PWA Mobile-friendliness

WireFrames:

UI Kit adaptations:

Wireframe templates we created for other designers to draw inspiration from:

Mockups:

The splash screen will be used when the PWA starts up, it will be the screen that loads up before the login page. When offline, a golden accent will appear underneath the header to indicate this.

Example pages of mobile designs in collaboration with @COS-301/Designers

PWA service workers and offline requirement

To convert a website into a PWA, 4 requirements are needed:

A service worker
A Web App Manifest (manifest.json)
HTTPS
Icons

SERVICE WORKER

Description

A service worker is the core of making a Progressive Web App. It is a programmable network proxy that lets you control how network requests from your pages are handled. It enables fast loading, offline access, application updates, push notifications, and many more. The service worker being used for the PWA development of the Graduates Portal will be implemented using Angular’s pre-existing service worker functionality.

HTTPS

The HTTPS protocol is required to register the service worker, because it can only run over HTTPS. Localhost is the only exception. HTTPS is also needed by the service worker for security reasons

Adding the service worker

The following command line will add the service worker to an existing Angular project ng add @angular/pwa --project *project-name* This will generate ngsw.json and ngsw-config.json

Components

The following components will be generated automatically with the service worker

ngsw-config.json: Contains all the information about what the service worker should cache and how the cache is updated. Generated with some default information but can be edited.
ngsw.json Generated from the ngsw-config.json and is downloaded by the end user. Renaming this file can be used to remove the service worker.

https://angular.io/guide/service-worker-config

Error Handling

It is important to note that for the service worker to be registered, the app must be stable, i.e., no constant polling must occur. Unrecoverable states may also need to be handled.

Updates

Updates to the application will be automatically downloaded and handled by the service worker, even if only a single file is modified. The service worker can check for any updates done on request during certain intervals, repeatedly. Forced update activation is handled by the service worker. If an update is available, the user is notified using the service worker notifications. If the resourceOutputPath or the assets path is modified, ngsw-config.json must be updated manually to reflect this. The method for updating the cache is specified in ngsw-config.json

Push notifications

The service worker can be given the responsibility for push notifications. The library SwPush is used from the @angular/service-worker package.

Bypassing the service worker

Using Ngsw-bypass as a request header will allow a request to bypass the service worker.

WEB APP MANIFEST

Provides information about a web application in a JSON text file, necessary for the web app to be downloaded and be presented to the user similarly to a native app (e.g., be installed on the home screen of a device, providing users with quicker access and a richer experience). It includes the app name, the icons the app should use, and the URL that should be opened when the app is launched.

OFFLINE REQUIREMENTS: STORAGE OPTIONS

Web storage

Web storage can be used to store small amounts of data to provide a better offline experience for your users. There are two types of Web Storage: localStorage and sessionStorage. Each type of Web Storage is maintained as a separate data store that's isolated to the domain that created it.

SessionStorage persists only for the duration of the browsing session. For example, while the browser is open, which includes refresh and restores.
LocalStorage persists until the data is removed by the code, the user, or the browser. For example, when there is limited storage available.

Session storage and local storage is not usually considered when implementing the offline functionality of a PWA since they are synchronous and usually block the main thread, storage capacity is between 5-10 MBs and it is not accessible through service workers.

Cache

Cache API is a system for storing and retrieving network requests and their corresponding responses. Can be used to provide access to managed cached resources. Cache API is used within the context of a Service Worker. The Service Worker checks to see whether the file already exists in any cache, by using the match() method:

If the request exists in the cache, that cached result is returned.
If the request doesn't exist in the cache, a new fetch for that resource is run, a copy of the response is cached for later, and the response is returned.
If the fetch fails because the network is unavailable, the offline page is returned from the cache.

This method is ideal for storing dynamic data. We can manipulate and check properties such as setting the amount of assets to cache, and old cache can be compared to new cache to decide on whether to update the cache. For this caching method to always work, the "Cache-Control Header" must be configured on the web server to set the max-age of the cache.

However, static data such as the company logo can be precached. Precaching refers to software that downloads data ahead of time in anticipation of its use. Cached data can be stored in the ngsw-conf.json. They will be added under the "assetsGroup" array in the JSON file. Using a service worker to provide precaching lets you intercept network requests and deliver responses directly from a local cache instead of retrieving them from the network.

The Angular “assetsGroup” also consists of:

installMode: determines how these resources are initially cached.

Prefetch: tells the Angular service worker to fetch every single listed resource while it's caching the current version of the application. This is done to decrease the time waiting for that resource and ensures that resources are available whenever they are requested, even when the browser is offline.
Lazy: only caches resources for which it is receives requests. This type of caching is more likely to be used for images of different resolutions, so the service worker only caches the correct assets for the screen and orientation.

updateMode: determines the caching behaviour, for resources already in the cache, when a new version of the application is discovered.

Prefetch: tells the service worker to download and cache the changed resources immediately.
Lazy: treats them as unrequested and waits until they're requested again before updating them. An updateMode of lazy is only valid if the installMode is also lazy.

Caching strategies:

Cache first, Network fallback: can be used when dealing with remote resources that are very unlikely to change, such as static images.
Network first, Cache fallback: Use this when data must be as fresh as possible, such as a real-time API response, but you still want to display something as a fallback when the network is unavailable
Stale while revalidate: It is a common strategy where having the most up-to-date resource is not vital to the application. It follows this approach:
1. Try to fetch from the network first.
2. If the network request does not complete after 0ms (that is, immediately), fall back to the cache (ignoring cache age).
3. Once the network request completes, update the cache for future requests.
4. If the resource does not exist in the cache, wait for the network request anyway.
Network only: This can be used when only fresh data can be displayed on your site.
Cache only: data is cached during the install event so that you can depend on the information being there. This can be useful if you have your own precaching step.

In terms of a PWA, stale while revalidate appears to be the most efficient solution because it allows you to respond to the request as quickly as possible with a cached response if available, falling back to the network request if it’s not cached. The network request is then used to update the cache.

Cache-Control: max-age=604800, stale-while-revalidate=86400

IndexedDB

This is an API for storing structured data. It is used for storing a significant amount of structured data on the client, or binary data, such as encrypted media objects. Similar to the Cache API, it's also asynchronous. This means you can use it in the main thread, or with Web Workers such as Service Workers. Cache API and IndexedDB are the most favourable for a PWA because they are accessible through service workers, and they are asynchronous.

Caching GraphQL APIs

Another method to make content available offline is by caching the API requests and responds. APIs in the Graduates Portal are built using GraphQL. The client side of the GraphQL needs to be cached as we need the requests and responds to be stored locally. This can be done using the @apollo/client and from it the InMemoryCache library. What it does is it:

generates a unique ID for every identifiable object included in the response
stores the object by ID
merges objects with same ID
fields that appear in only the existing object or only the incoming object are preserved

This method can be used alongside the service worker. In the ngsw-conf.json, the pages to be viewed offline can be added under the dataGroup array. In these pages to be viewed offline, if an API must be queried, the Stale While Revalidate cache method will be used. The cache will be fetched first before network is consulted. This will ensure that even offline, previously viewed dynamic resources will be visible.

Activity Diagram

RESOURCES

https://medium.com/samsung-internet-dev/pwa-series-service-workers-the-basics-about-offline-a6e8f1d92dfd
https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API
https://medium.com/animall-engineering/best-caching-strategies-progressive-web-app-pwa-c610d65b2009
https://angular.io/guide/service-worker-config
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
https://web.dev/add-manifest/
https://docs.microsoft.com/en-us/microsoft-edge/progressive-web-apps-chromium/how-to/offline
https://daily.dev/blog/caching-in-graphql