Our system consists of four main components, each playing a critical role in its seamless operation. At its heart, we have the API, a robust backend service that runs on the highly reliable PM2 process manager, ensuring exceptional performance and high availability. The frontend, served using the efficient Nginx web server and reverse proxy, delivers a smooth and responsive user experience, effortlessly handling client requests and forwarding them to the API.

For data storage, we rely on MongoDB, a flexible NoSQL database that efficiently manages structured and unstructured data, allowing us to handle vast amounts of information effortlessly. To enable real-time communication and data exchange, our system employs the Mosquitto MQTT server, a lightweight message broker that facilitates fast and reliable messaging through the publish-subscribe pattern.

In line with our unwavering commitment to security, every component of our system employs SSL/TLS encryption protocols. This ensures that all data transfers, communications, and interactions are safeguarded against potential threats and unauthorized access, making our system a fortified fortress for your valuable data. With an emphasis on efficiency, scalability, and unparalleled security, our system stands ready to cater to a myriad of applications, offering you a truly innovative and dependable solution.

To get your development environment setup and ready to use, it is assumed that you have the following

• Git
• Yarn
• NodeJs
• Google App Password used for sending emails
• A MongoDB server connection string. You can get a free one using MongoDB atlas. Follow their documentation for setup
• An MQTT broker and credentials to that broker.
• That you have an OAuth consent screen setup using the Google Cloud Console
• That you have credentials setup on the Google Cloud Console with your domain and callback URLs setup

Clone the repository from GitHub

git clone https://github.com/COS301-SE-2023/Event-Participation-Trends

Change directory into the repository

cd Event-Participation-Trends

Create a .env file in the root of the project that contains the following fields.

Everything indicated with key=<value>, means that you have to replace <value> with your actual value

Everything indicated with key=<(value1|value2)>, means that you have to replace <(value1|value2)> with either value1 or with value2

ENVIRONMENT=<(production|development)>
PORT=<api_port>
MQTT_URL=wss://<mqtt_domain>:<mqtt_port>
MQTT_USERNAME=<mqtt_username>
MQTT_PASSWORD=<mqtt_password>
JWT_SECRET=<jwt_secret>
GOOGLE_CLIENT_ID=<google_client_id>
GOOGLE_SECRET=<google_secret>
GOOGLE_CALLBACK_URL=https://<domain>/api/auth/google/callback
FRONTEND_URL=/home
JWT_EXPIRE_TIME=1d
MONGO_ALTALS_CONNECTION_URL=<mongodb_connection_string>
MONGO_ATLAS_USERNAME=<mongodb_username>
MONGO_ATLAS_PASSWORD=<mongodb_password>
ADMIN_EMAIL=<gmail_account_email_from>
APP_KEY=<gmail_account_app_password>
MONGO_ALTALS_CONNECTION_URL_DEV=<mongodb_connection_string>
MONGO_ALTALS_CONNECTION_URL_TEST=<mongodb_connection_string>
TEST_USER_EMAIL_1=<gmail_addr1>
TEST_USER_EMAIL_2=<gmail_addr2>
TEST_USER_ROLE_1=manager
TEST_USER_ROLE_2=viewer

Description

After creating your .env file, we will be installing all the needed packages. Run the following in the root of the repository

yarn install

Done, you should now be able to start the project using

yarn start dev

Refer to the above Development setup for the environment variables, what they are used for and how to get them.

• nginx: 1.18.0 (Ubuntu)
• mosquitto: 2.0.11
• git: 2.34.1
• pm2: 5.3.0
• yarn: 1.22.19
• a MongoDB database, we are using a self hosted one

All of the above should be rather easy to install, there is a lot of resources available online to help with the setup of these packages

You will also need a domain, and have to figure out TLS/SSL setup for your domain. This will all depend on where you got your domain.

This will also not cover anything firewall related as the setup thereof can differ quite a lot.

/etc/nginx/sites-available/<some_file_name>

server {
        listen 443 ssl;
        listen [::]:443 ssl;
        root /var/www/html/ept;
        index index.html index.htm index.nginx-debian.html;
        server_name eventparticipationtrends.co.za;

        ssl_certificate /root/.ssl/fullchain.pem;
        ssl_certificate_key /root/.ssl/privkey.pem;

        location / {
                try_files $uri $uri/ /index.html;
        }
        location /api {
                proxy_pass http://localhost:3000/api;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
        }
}

Certificates for your domain should be placed in /root/.ssl/* according to this config. The paths in the config can be adapted to your environment

Use the folling command to create a symbolic link

sudo ln -s /etc/nginx/sites-available/<some_file_name> /etc/nginx/sites-enabled/

Enable nginx to run as a service on startup

sudo systemctl enable nginx

Start the nginx service using

sudo systemctl start nginx

or restart.

This file is placed in /etc/mosquitto/conf.d/default.conf

password_file /etc/mosquitto/passwd
allow_anonymous false
listener 8883
protocol websockets
certfile /etc/mosquitto/certs/fullchain.pem
keyfile /etc/mosquitto/certs/privkey.pem
tls_version tlsv1.2

You can also specify a cafile, this will depend on your setup and if your certificated were signed by an actual certificate authority.

Now place the needed certs in the /etc/mosquitto/certs/ directory. Run the following commands

sudo chmod 700 /etc/mosquitto/certs

sudo chmod 600 /etc/mosquitto/certs/*.pem

sudo chown -R mosquitto:mosquitto /etc/mosquitto/certs

This will ensure that the directory has the proper permissions and that certificates can only be read my Mosquitto

Enable mosquitto to run as a service on startup

sudo systemctl enable mosquitto

Start the mosquitto service using

sudo systemctl start mosquitto

or restart.

We are using pm2 for managing our API, here is the basic configuration. Please read their own documentation for details

module.exports = {
    "apps": [
      {
        "name": "api-production",
        "cwd": "/root/deploy/dist/apps/api",
        "script": "main.js",
        "autorestart": true,
        "restart_delay": 3000,
        "exec_mode": "fork",
        "instances": 1,
        "watch": true,
        "ignore_watch": ["node_modules", "logs"],
        "log_date_format": "YYYY-MM-DD HH:mm Z",
        "out_file": "/var/logs/api_ept/api.log",
        "error_file": "/var/logs/api_ept/error.log",
        "env": {
            ... all environment variables needed during production, as described in the Development environment setup above.
        }
      }
    ]
  };

Please refer to Espressif's own documentation on installing and setting up the ESP-IDF.

idf.py build

The .bin file in the build directory can be hosted on a server, in menuconfig, enter a link that points directly to the firmware file.

If OTA update is enabled using the menuconfig, the sensors will automagically update themselves and restart.

The menuconfig has a section with the name EventParticipationTrends where you will find all values needed by the firmware, as well as descriptions for those values.

• Disable OTA option, unless you have it set up
• Disable MESH option, unless you are modifying the firmware yourself

The rest of the options can be left as is.

Some other common settings that could be helpful to adjust (They might have different names depending on your hardware):

Also ensure that certificate bundles are attached, and that they have support for your certificate authority. Otherwise neither MQTT nor OTA updates will work.

If you are setting up OTA, ensure that partitions.csv is specified when using custom partitions, refer to Espressif's documentation regarding the details