Backend - OtagoPolytechnic/Cloudy-with-a-Chance-of-LoRa GitHub Wiki
Backend: Development Setup
We are looking at reworking this into nextjs so we can deploy our site and backend together.
We have decided that this would be a good approach to minimize on overhead from a serverside basis, we have looked into it and decided that since we aren't using anything that will require stringent security concerns such as a built-in user management system this would be a serviceable approach. I figure if we or a future team decides to put forward ideas that might require an account this could be solved with firebase and an id being linked to the account. Such as for the idea we had for store management data like widget designs and layout settings if we don't expand past that we could also use cookies to store layout data without a user management function.
NextJS
This change to nextjs will require some lateral thinking to allow for the adaptation of the express code to work with nextjs as well as some research into how the integration of this into the workflow will be done as we have almost exclusively used express in the past to build our backends. The same concept will apply with the express app where we will utilize a post request for receiving data and sending it off we will even be utilizing a function per data type that will allow us to send all the different payloads if received to the right tables in the database allowing for us to split up all the data and normalize it with avgs from a set time frame that will then be returned to the frontend allowing us to utilize. This is because the webhook is passing all payloads through and not specific ones so I will have to develop around this idea it will look something like the following
import {
DustData,
TemperatureData,
PressureData,
CO2Data,
GasData,
WindData,
RainData,
HumidityData,
} from '@/app/utils/receive-data-helper';
export const dynamic = 'force-dynamic';
export const POST = async (request) => {
try {
const authHeader = request.headers.get('authorization');
const data = await request.json();
let send = [];
/**
* Since the webhook will send all data we will the data
* into different tables based on there data type
*/
const decodedPayload = data.uplink_message?.decoded_payload;
if (!decodedPayload) {
return new Response(
JSON.stringify({ message: 'Decoded payload is required' }),
{
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Origin': '*',
'Access-Control-Allow-Methods': 'POST',
},
status: 400,
},
);
}
const device_id = data.end_device_ids.device_id;
const temperature = decodedPayload.temperature ?? null;
const pressure = decodedPayload.pressure ?? null;
const co2_level = decodedPayload.co2 ?? null;
const gas_level = decodedPayload.tvoc ?? null;
const dust = decodedPayload.dustDensity ?? null;
const wind_speed = decodedPayload.windSpeed ?? null;
const wind_direction = decodedPayload.windDir ?? null;
let rain_gauge = decodedPayload.rain ?? null;
const humidity = decodedPayload.humidity ?? null;
const sensorData = [
{ condition: dust, fetchData: DustData },
{ condition: temperature, fetchData: TemperatureData },
{ condition: pressure, fetchData: PressureData },
{ condition: co2_level, fetchData: CO2Data },
{ condition: gas_level, fetchData: GasData },
{ condition: wind_direction && wind_speed, fetchData: WindData },
{ condition: rain_gauge, fetchData: RainData },
{ condition: humidity, fetchData: HumidityData },
];
for (const { condition, fetchData } of sensorData) {
if (condition) {
const results = await fetchData(
device_id,
...(Array.isArray(condition) ? condition : [condition]),
);
send.push(results);
}
}
if (!authHeader) return new Response('Authentication Required');
const splitAuth = authHeader.split(' ')[1];
if (splitAuth !== process.env.PASSWORD)
return new Response('You are not authorized to post');
if (!device_id) {
return new Response(
JSON.stringify({ message: 'Device ID is required' }),
{
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Origin': '*', // Allow all origins
'Access-Control-Allow-Methods': 'POST', // Allow POST method
},
status: 400,
},
);
}
return new Response(JSON.stringify(send), {
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Origin': '*', // Allow all origins
'Access-Control-Allow-Methods': 'POST', // Allow POST method
},
status: 200,
});
} catch (err) {
return new Response(JSON.stringify({ message: err.message }), {
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Origin': '*', // Allow all origins
'Access-Control-Allow-Methods': 'POST', // Allow POST method
},
status: 500,
});
}
};
Backend (NEXT)
npm i- Create a .env.local file it will take an input of
DATABASE_URL="INPUT_POSTGRES_URL"- We now have a password field for authenticating Post requests and large Get Requests
PASSWORD="INPUT PASSWORD"
- We now have a password field for authenticating Post requests and large Get Requests
- You will need to expose the port to recievedata from ttn I have done this with NGROK 'ngrok https 3000'
- You will then need a webhook to forward this data you can follow what you see on mine My TTN webhook you will want to change the baseurl to match your url which will be gotten from ngrok
Depricated (Express)
Express Logic some of this will be outdated and updated when we implement nextjs
I am using a local postgres server currently we can change this in the future if we want to.
npm i- Create a .env file for the postgresql database details
DB_PASSWORD="InputPassword"
DB_USERNAME="InputUsername"
DB_NAME="InputDBName"
DB_HOST="InputHost"
DB_PORT="InputPort"
-
For development purposes i am using ngrok to expose my localhost for use in conjunction with the webhook to receive data from ttn.
ngrok https [API PORT] -
I have put together a very basic webhook while we are working with a local postgres server we will need our own webhooks the baseurl is what will change with your internet facing URL which I am getting from ngrok.
app.post('/receive-data', async (req, res) => {
try {
const device_id = req.body.end_device_ids.device_id;
const humidity = req.body.uplink_message.decoded_payload.humidity;
const temperature = req.body.uplink_message.decoded_payload.temperature;
if (!device_id || typeof temperature !== 'number' || typeof humidity !== 'number') {
return res.status(400).send('Invalid data');
}
await pool.query(
'INSERT INTO sensor_data (device_id, temperature, humidity) VALUES ($1, $2, $3)',
[device_id, temperature, humidity]
);
res.status(200).send('Data received');
} catch (err) {
console.error(err);
res.status(500).send('Server Error');
}
});
Our current method of receiving data is setup for use with our Temperature and Humidity Sensors
- Is from the req.body we can get humidity, device and temperature
- We then validate the data and if it fails we will return and invalid data msg.
- Then we will query the data into the database.
We will need to setup a datacleanup after a set time (Discussed likely data wont be a large amount so wont be an issue for a while But we are discussing ways so prune and maybe keep some historic data).
Webhook: The webhook takes in all the data from the weather station so the recieve_data block will need to be updated to check what data is given so it can decide where its sending the data.
AI: The approach we are looking at taking with the forecasting will be the main backend will send a post request too the cloud model backend which is listening to give it an image and then we can listen for its response and then store that data in the database in a forecast table and then it will be able to get it like normal for displaying in the frontend whenever we want