Meeting #1

Introductory meeting with clients

4 May 2022 [15:00 - 15:52]

Hosted/Recorded by: Ben Pietersen

Minutes Compiled by: Zoe Liebenberg

Attendees:

• Ben Pietersen
• Chiara Goncalves
• Dylan Pietersen
• Dylan Spies
• Steven Schormann
• Zoe Liebenberg

COS301 representative: Funmi Fagbola

EPI USE representatives: Francois Maass, Mvuyisi Scheepers

Discussion points:

• The first 10 minutes were spent discussing Funmi Fagbola to see what was expected of us from the COS301 team and what our clients will need to do to ensure that we obtain the maximal amount of practical knowledge about the industry during this project
  • We will need to work with both a lecturer mentor and an industry mentor
  • The industry mentor will be responsible for:
    • Attending demos and grading us
    • Joining demo 3 and providing oral feedback
    • Answering any questions from the team *Meeting with the team once every 2 weeks for about 30 minutes to an hour to discuss progress and future steps
• Jason Louw is the actual project lead, but he was on leave during the meeting. Francois Maass took his place
• Francois Maass and Mvuyisi Scheepers will be responsible for the industry mentor role
• The focus of the project will be on looking for matter such as:
  • Objects that could harm the animals (snares laid down by poachers)
  • Shifts in the ground
  • Sinkholes
  • Erosion
  • Drying up of water beds
  • Carcasses
  • Fence shifting
  • Roads washing away
  • Picking up on fundamental land changes
• EPI USE wants us to take aerial footage and map this together to create a flat plane plan of what the reserve looks like at any moment in time
  • There will be 4/5 flight paths to cover the entire reserve which will need to be stitched together
• The company makes use of both drones and propeller aeroplanes
  • The drones and planes will be flying at different heights and speeds
  • The drones fly about 50/100 metres above ground whereas the planes fly 15 000 feet (around 4 500 metres) above ground
• The footage received will be of video format
• Possible system requirements that the company will need:
  • Way to identify flights
  • Flight management
  • Group flights together (same map/stitching effort)
  • View map
  • Login
  • Dashboard (last num. flights done, num. images processed, which jobs(stitching) is running)
  • A progress screen of any existing jobs that are running
  • Catalogue to overlay different views
• If needed, it is possible for the pilot to alter the paths if the overlays are not strong enough for stitching
• Factors that can affect the drone flight:
  • Turbulence
  • Air pockets
  • Wind
  • Battery
• Reserves mentioned during the meeting to keep in mind
  • Madikwe
  • Rietvlei
    • Some issues with air space clearance because of the air force base next to the reserve
    • Flying the drones around 3 times a week
• Our team will need to make a single 2D image of the entire plane
  • A grid could be incorporated on the map
  • Steer clear of a fixed anchor point
• The most important aspect to take into account is that we will need to display everything visually
• In the project, there is an emphasis on resolving objects
• The maximum amount of time for the system to process small images to develop the large image must be a maximum of 1 hour
• We will need to submit a dashboard of the progress of the project on a weekly basis
• Ben will need to set up a recurring meeting ever second Friday. Preferably not between 10 and 12

Meeting #2

Meeting with lecturer mentor & client

20 May 2022 [12:45 - 13:34]

Hosted/Recorded by: Ben Pietersen

Minutes Compiled by: Zoe Liebenberg

Attendees:

• Ben Pietersen
• Chiara Goncalves
• Dylan Pietersen
• Steven Schormann
• Zoe Liebenberg

Discussion points:

• We first discussed with our lecturer mentor where we should be and pitched our potential use cases for demo 2 to him
• Use cases
  • 5 use cases with 2 being core requirements
  • The 2 must be used by the end user
  • Core requirements: image stitching, video into images and map grid
  • Other requirements: registration link (email), fleshing out UI (accounts page, image catalogue)
  • Will need to justify the use cases properly
  • Each use case needs to have a use case diagram
• We then spoke to our client
• Images: upload a file with the name of the image, longitude, latitude, altitude (drone images naturally come with this information but drone videos do not), date
• Application should take in images
• It will be nice to have a video and images are taken from the video (functionality already complete)
• Overlaying new images over old images
• Image stitching
  • OpenCV in Python, or
  • Tensorflow
    • Run on client side
    • Preferably not
  • If done on server side, the file uploads will be quite big so to increase time efficiency split the video into images locally and then upload the videos
  • Time is not a priority as of yet
  • Rules can be set up where the file is old and is already been processed it can be deleted over time
  • On server side: instead of rendering and image splitting, we can change the format to be .h27 video for compression
• File storage
  • Amazon S3 (needs credit card; using Ben's for now)
    • 30 GB free
    • 2 TB over the limit for a month is $5
• Split up videos client side and save images in S3 bucket to save costs
• Map building on server side
• Technologies to set up server
  • NodeJS
  • SAP (do not have access to SAP)
  • Python ad JS (easy to use)
  • Lambda functions in S3
    • Pay as you use
    • Upload images to bucket, set processes on the lambda functions that will set up a trigger when the bucket takes in an image and the trigger executes code to process the images
    • Can run Python, Java, NodeJS
  • Firebase might be slightly easier than S3
    • Firebase Cloud functions are server-less functions
  • Hosting on (application that cannot be remembered by the client)
• Lambda functions can only execute for a maximum of 30 minutes which will be an issue if the map building takes longer (check how long it takes to make the map)
• Can run the project locally for as long as possible
• Main priority is mapping images together. The client is not prioritizing technologies used. Hosting should be done at a later stage
• Images have coordinate metadata whereas videos do not
• Map collections: albums of maps that contain specific features (fires, rhinos, etc)
  • This is an extra feature
• User story:
  • User will upload the video
  • The application will split the video into images
  • The user can go back to the image catalogue and see all the images from their video in a folder
  • The user can select a folder, click run processing and then the images will build a map
• Will need to change the colours
• Possible outing with the entire team to fly the drone
• Current drone footage is not from a birds' eye view
• Currently have video splitting in Python. Will need to convert it to JavaScript
• Get our own drone footage so we can try do it ourselves

Meeting #3

Meeting with client

3 June 2022 [12:30 - 13:22]

Hosted/Recorded by: Ben Pietersen

Minutes Compiled by: Zoe Liebenberg

Attendees:

• Ben Pietersen
• Chiara Goncalves
• Dylan Pietersen
• Zoe Liebenberg

Discussion points:

• Using PostgreSQL for database
• Architectural design
  • Server will be hosted in a lambda function
  • Object recognition and stitching will be in a lambda function
  • NestJS server API backend (to do auth)
  • Angular frontend
  • With the API, we will make a call to the lambda function to get the results of that using Python to do the processing
  • Hosting of NestJS server: AWS
    • Cannot be hosted through a lambda function, will need to set up a bucket
    • Having Python on Flask and NestJS set up separately may become complicated. Makes more sense to have 1 server to do all the processing all built into lambda functions. We will then only need the lambda function set up
• To do the auth: currently using the database and JWT

Advice from our client for our architecture of the application:

• Database will be hosted on AWS
• The server-less functions (lambda functions) can be as many as we want
  • Auth specifically: url does not need to be authenticated to sign in
    • We will be sent a username and password, we will then need to check the database
    • In this way, we will not need NestJS to set up any of this. The logic will be set up in the server-less function
  • Another server-less function that does image stitching
  • etc
• Take current code and run it in server-less function
• Set up database connection in server-less function
• Login call from API
  • Trigger: how it connects
• Microservices approach
• Event-driven
• Angular service physically makes a POST request to the lambda function
• It is possible to call other lambda functions within a lambda function
• Can have 1 API gateway which directs to the lambda functions
• Will be difficult to migrate to just using lambda functions for the next demo
  • For now, do all code locally and test on NestJS then when we want to deploy, deploy the entire server
• NestJS is basically a framework on how a nodeJS server is set up but we do not have to use resolvers and gateways
• Keeps API separate from the backend but still running on the same server
• All the functions can be written in Python and this will deploy to AWS server which will then run on Lambda
• Do development locally and then do hosting
• Firebase
  • Has everything needed in terms of functionality
  • More simple than AWS
  • Authentication is already built
  • Able to do direct reads of the database from the Angular UI. Authentication doesn't have to go through a backend server
• Look at Epi Use website for colours
• Will need to decide on a technology and let Jason know
• No problem with running the application locally. Only problem is with the actual demo

Meeting #4

Meeting with lecturer mentor

6 June 2022 [10:00 - 10:13]

Hosted/Recorded by: Ben Pietersen

Minutes Compiled by: Zoe Liebenberg

Attendees:

• Ben Pietersen
• Chiara Goncalves
• Dylan Pietersen
• Dylan Spies
• Steven Schormann
• Zoe Liebenberg

Discussion points:

• Pitched the idea of going server-less
  • Not enough usage of the API as a standalone app to host it monthly (from the client's perspective)
  • We were not aware of this suggestion so it becomes more of a rewrite than a migration
• From lecturer's point of view:
  • As long as the architecture is still built well and we give good justification for why it is server-less
  • Time-frame: we will not have enough time to do it for the next demo
  • Do not prioritise this change. Look at user cases first and then we can look at migration. We do not have enough time for the big migration
  • Could be done for demo 3
  • Might be a bigger task to migrate later on. Build from here on out with an intention for migration
  • Do everything diagram-atically first
  • It is okay to demo locally. It must just be demoed from the master branch
• Still need to set up S3 bucket
• Have not had any regressions other than code coverage
• Code coverage should be around 40%. If it is around 50%-60%, we are doing well
• User manual example posted on ClickUp
  • Start with explanation of what the project is and does
  • Walk through of UI and what everything does
  • Adding FAQ at the bottom for additional use cases