Introduction

• This documentation gives brief information about the WEBATOOL backend process and how the url get analyzed and returns the result.

Install Nodejs

https://nodejs.org/en/download

Environment Setup

• Create the subfolder named as backend/server in the WebAtool folder.
• open the subfolder in the vs code.
• create the folder according to the subdivision.

Install Packages

npm install sequelize
npm install express
npm install nodemon
npm install mysql2
npm install cors
npm install passport.js
npm install passport-local
npm install bycrypt
npm install passport-jwt

Create a Simple Express Server

• This is the simple express application.

import app from "./src/app";
const port = 8000;
app.get("/api", (req, res) => {
  res.send("App is running");
});
app.listen(port, () => {
  console.log(`Server is running on http://localhost:${port}`);
});

Folder Structure & File Structure

• Mainly four folder are created as per requirement they are:-

db/
  db.ts

scans/
  scansModel.ts
  scanServices.ts
  scanController.ts
  scanRoute.ts

scanDetails/
   scanDetailsModel.ts
   scanDetailServices.ts

utils/
   analyzerUrl.ts
   middleware.ts

auth/
   AuthController.ts
   AuthRoutes.ts
   AuthService.ts

Users/
   userModel.ts

app.ts
server.ts

What is Sequelize.js

Sequelize is promise based node js ORM (object-relational-Mapper) tool which helps to work with relational databases such as MySql,Postgresql etc.

follow the steps below for the connection of the sequelize with our database.

• We can configure the database connection by creating the instance of the Sequelize by providing connection parameters to the constructor, parameters like database name, username and password and option parameters like host and dialect. etc
• We can disable it by providing the logging option to false to the constructor of Sequelize while creating the instance.

import { Sequelize } from "sequelize";
// In new Sequelize parameter enter your database configuration data new Sequelize('database', 'username', 'password',{})
const sequelize = new Sequelize("rapdb", "appdev", "dev123", {
  host: "localhost",
  dialect: "mysql",
  logging: false,
});

• Use the authenticate() instance method and implement error handling using try/catch blocks.

async function connect() {
  try {
    await sequelize.authenticate();
    console.log("Connection has been established successfully.");
  } catch (error) {
    console.error("Unable to connect to the database:", error);
  }
}

• Now, We have to synchronize our database and the sequelize
• Create the synDb() first invoke connect() method to check whether the connection has been established.

const synDb = () => {
  connect();
  sequelize
    .sync()
    .then(() => {
      console.log("Scan table created successfully!");
    })
    .catch((error) => {
      console.error("Unable to create table : ", error);
    });
};

• to call execute function

synDb();

• Now export the sequelize model.

export default sequelize;

scans

scanModel.ts

• Now import the sequelize from db.js.
• create the Model from the imported file.

What is the Model ?

• A model is an abstraction that represents the table in your database. It tells several things about the entity it represents, such as the name of the table in the database and which column it has(and their data types)

• create row named url it will synchronize with database .

• Note:- while using the sequelize model to connect with database. After declaring and initializing the row.

• it creates the id automatically while inserting the data.

• We have discussed about the model before. So now create the our scan table Model named as ScanModel.

• declare the url attribute and declare it's type.

import { DataTypes } from "sequelize";
import sequelize from "../db/db";

// By using define()
const ScanModel = sequelize.define("Scans", {
  // Model attributes are defined here
  url: {
    type: DataTypes.STRING,
    allowNull: false,
  },
});

export default ScanModel;

scanServices.ts

• Import the created scanModel in the scanServices.js

import ScanModel from './scanModel'

const paginate = (query, { page, pageSize }) => {
  const offset = page * pageSize;
  const limit = pageSize;

  return {
    ...query,
    offset,
    limit,
  };
};

• create the class named ScanService then create the required methods.
• The methods are getById(),getAll(),remove(),update(),create().
• initiate the methods asynchronously.
• asynchronously because the sequelize methods such as findByPk etc are callback function.
• So this is why we have to use asynchronous function.
• getById() in this we have to find particular record from the database through api. so we use include:[].

class ScanService {
  // instance method used with creating instance of class
  async getById(id: Number) {
    return await ScanModel.findByPk(Number(id), {
      include: ["ScanDetails"],
    });
  }

• getAll() in this we get all the records. By using find().

  async getAll(inputs: any) {
    // console.log('inputs', inputs)
    const { page, pageSize, search, field, sort } = inputs;
    let query: any;
    if (search) {
      query = {
        where: { url: search },
        offset: page,
        limit: pageSize,
        order: [field, sort](/sandeep-g-more/webatool/wiki/field,-sort),
      }
    } else {
      query = {
        offset: page,
        limit: pageSize,
        order: [field, sort](/sandeep-g-more/webatool/wiki/field,-sort),
      };
    }

    console.log('page, pageSize', page, pageSize, search)
    return await ScanModel.findAndCountAll(query);
  }

• remove by using this method we delete the records from the database by using destroy().

async remove(id: Number) {
return await ScanModel.destroy({
where: {
id: id,
},
});
}

• update() by using this method we can update the records from the database by using update()

async update(id: Number, url: any) {
return await ScanModel.update(
{ url: url },
{
where: {
id: id,
},
}
);
}

• create() by using this method we create the records to the database.
• create the instance of the class by declaring new to the class and export it.

async create(url: any, trans: any) {
return await ScanModel.create({ url: url }, { transaction: trans });
}
}

export default new ScanService();

scanController.ts

js first we have to delete the record from the scanDetails table and then from the scan tables.

• getScansById in this handler we use try/catch method. it takes req and res as an parameter in which we take the id from the req.params now we check through the if condition whether it is the integer and whether it is non negative number.Now we use getById() service method then pass the id and get the value from the table.

• This are the handlers of the services of the rest api.

• now import the files which are given

• for the rest not consider about the analyzeUrl and scanDetailService we will get to know about this further.

import sequelize from "../db/db";
import analyzeUrl from "../utils/analyzeUrl";
import scanService from "./scanService";
import scanDetailService from "../scanDetails/scanDetailService";

• Now initiate the class named as ScanController.
• now initiate the methods such as getScans createScans removeScans getScansById
• initiate this methods asynchronously because we are calling the callback methods from scanServices.
• getScans() it takes req,res as parameter res means request get from the client either through the medium of body headers .

export class ScanController {
  //  private scanService = new ScanService();
  //  private scanDetailService = new ScanDetailService();

• res means response which we are sending the data to the client through api from the database.
• getScans() declare the const result get the data in it through the callback method.
• send the received data as response to the client.

  async getScans(req, res) {
    const result = await scanService.getAll(req.body);
    res.send(result);
  }

• createScans it takes the url from the body parameter.
• Write the function asynchronously in try/catch block by using the transaction because for rollback and committing the script.
• pass the url in the scanService.create(url, trans) save the data in the database.
• pass the url in the imported function analyzer. For analyzing the Accessibility of the website. analyzeUrl(url);.
• create the object named as source in which it includes the scanId and results.
• We have to save the url id and it's accessibility result in another table called as scanDetails.
• Pass this created object source in create scanDetails.service().
• Now send the res.statusCode and res in the json formate to the client which includes the object of the message,success,data.

  async createScans(req, res) {
    const { url } = req.body;
    try {
      await sequelize.transaction(async (trans) => {
        const scanRow = await scanService.create(url, trans) as any;
        const results = await analyzeUrl(url);
        const source = { scanId: scanRow?.id, results: results };
        // scanDetails
        await scanDetailService.create(source, trans);
        res.status(201);
        res.json({
          success: true,
          data: { scanId: scanRow.id },
          message: "Created successfully",
        });
      });
    } catch (error) {
      res.status(404).json({
        success: false,
        error: error.message,
      });
    }
  }

• If any error failure occurs catch that in the catch block throw the error.
• removeScan in this handler we use try/catch method. it takes req and res as an parameter in which we take the id from the req.params now we check through the if condition whether it is the integer and whether it is non negative number.
• After checking the integer type of the if we call remove() service method from the scanServices.

async removeScans(req, res) {
  const { id } = req.params;
  // Ensure a URL was provided.
  if (!Number.isInteger(Number(id))) {
    res.status(404);
    res.json({ success: false, error: "Id is not valid" });
    return;
  }
  try {
    // scandetails
    await scanDetailService.remove(id);
    await scanService.remove(id);
    res.sendStatus(204);
    // res.json({ success: true, message: "Deleted Successfully" });
  } catch (error) {
    res.status(404).json({
      success: false,
      error: error.message,
    });
  }
}

• getById this method is for getting the specific record from the database.
• Get the id from the req.params, check whether it is Integer or not.
• After passing the Integer test, Pass that id integer value to the getById(id) .
• if record is found it will return the value.If not it will throw an error.

// async function updateScans(req, res) {
//   const result = await update(req.params.id, req?.body?.url);
//   res.send(result);
// }
async getScansById(req, res) {
  const { id } = req.params;
  // Ensure a URL was provided.
  if (!Number.isInteger(Number(id))) {
    res.status(404);
    res.json({ success: false, error: "Id is not valid" });
    return;
  }
  try {
    const result = await scanService.getById(id);
    if (result === null) {
      res
        .status(404)
        .json({ success: true, error: "Scan records not found" });
      return;
    }
    res.status(200);
    res.json({
      success: true,
      data: result,
      message: "fetched Successfully",
    });
  } catch (error) {
    res.status(404).json({
      success: false,
      error: error.message,
    });
  }
}
}

ScanRoutes.ts

• In this we declare the basic *http methods of the Api such as POST GET DELETE PATCH.

• import scanController which is the handler for different http methods.

• import verifyToken model from the utils folder.

• call the static route method of the express.

• apply the http methods for the instance of static route method by passing the first parameter the path and second the scanController handler.

• Actually it takes three parameters. first the route,second middleware,third scanController.

• In the POST http method add the middleware which is initialize as the method of the scanRoute class. Initialize the middleware which was imported as verifyToken.

• In the POST http method add the middleware which is initialize as the method of the scanRoute class. The method is known as ValidUrl.

• In this check it just checks whether the entered url in is valid url or not.

• To check this validUrl we take the URL from the req.body initialize the try/catch method.

• Pass the url in NEW URL(req.body.url) it will validate whether it is URL

import express from "express";
import { verifyToken } from "../utils/middlewares";
import { ScanController } from "./scanController";

class ScanRouter {
scanController = new ScanController();
scanRoutes = express.Router();
constructor() {
this.scanRoutes.post("/scans", verifyToken, this.scanController.getScans);

    this.scanRoutes.post(
      "/scans/new",
      this.validUrl,
      this.scanController.createScans
    );

    this.scanRoutes.delete("/scans/:id", this.scanController.removeScans);

    // scanRoutes.put("/scans/:id", updateScans);

    this.scanRoutes.get("/scans/:id", this.scanController.getScansById);

}

validUrl(req: any, res: any, next: any) {
// Ensure a URL was provided.
const { url } = req.body;
console.log(req.body);
if (!url) {
res.status(404);
res.json({ success: false, error: "url is required" });
return;
}
// Ensure the URL is valid.
try {
new URL(url);
} catch (error) {
res.status(400);
res.json({ success: false, error: `Invalid URL` });
return;
}
next();
}
}
export default new ScanRouter().scanRoutes;

utils

analyzeURl.ts

• AnalyzeUrl feature we use as for the middleware in the http method.
• In this AnalyzeUrl we use AxeCore and AxePuppeteer.
• This library analyzes the Accessibility of the url.
• After analyzing it returns the result.

import puppeteer from "puppeteer";
import { AxePuppeteer } from "@axe-core/puppeteer";

const analyzeUrl = async (url) => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.setBypassCSP(true);
  await page.goto(url);
  const results = await new AxePuppeteer(page).analyze();
  await page.close();
  await browser.close();
  return results;
};

export default analyzeUrl;

middleware.ts

• verifyToken it takes request and response parameters.
• -Take the token from the req.headers.authorization
• Token will be received with Bearer formate.
• now remove the Bearer from the received token by using replace method to the string token.
• After receiving that token verify it with jwt.verify static method.
• Pass that token with second parameter as top_secrete.

import jwt from "jsonwebtoken";

export function verifyToken(req: any, res: any, next) {
  const token = req.headers?.authorization
    ? req.headers?.authorization.replace("Bearer ", "")
    : undefined;

  if (!token) {
    return res.status(401).send({ code: "401", error: "Unauthorized User" });
  }
  try {
    const decoded = jwt.verify(token, "top_secret");
    req.user = decoded;
  } catch (err) {
    return res.status(401).send({ code: "401", error: "Unauthorized User" });
  }
  return next();
}

scanDetails

• import scanModel from the scan features.
• This is the table which is created in the database which stores the scanId (which is the foreign key),isRescan and result of Accessibility of the url.
• Same as the scanModel create the scanDetail Model.
• Now use the Associations for this both models.
• check the Associations such as scanModel.hasMany(scanDetailModel)

scanDetailModel.ts

import { DataTypes } from "sequelize";
import sequelize from "../db/db";
import ScanModel from "../scans/scanModel";


const scanDetailModel = sequelize.define("ScanDetails", {
  // model attributes
  results: {
    type: DataTypes.JSON,
    allowNull: false,
  },
  isRescan: {
    type: DataTypes.BOOLEAN,
  },
});

// console.log(scanDetailModel === sequelize.models.results); // true
// creates the
ScanModel.hasMany(scanDetailModel);
scanDetailModel.belongsTo(ScanModel);

export default scanDetailModel;

scanDetailService.ts

• Same as the scanModelService initialize different methods such as update,getById,create and remove
• This is services are created as the methods of the class ScanDetailService.

import scanDetailModel from "./scanDetailModel";

• update is used for updating the Accessibility results.

class ScanDetailService {
  async update(id:Number, results:any) {
    return await scanDetailModel.update(
      { results: results },
      {
        where: { id: id },
      }
    );
  }

• remove is used for removing the record from the table.

async remove(scanId:any) {
return await scanDetailModel.destroy({
where: {
ScanId: scanId,
},
});
}

• create is used for creating the scanId and scanDetails.

async create(source:any, trans:any) {
// here we get the parameter value as
// the source which is passed from the scanController module
return await scanDetailModel.create(
{
ScanId: source.scanId,
results: source.results,
},
{ transaction: trans }
);
}

• getByScanId is used for getting the particular record from the scanDetails table.

async getByScanId(scanId:any) {
return await scanDetailModel.findAll({
where: {
ScanId: scanId,
},
});
}
}

• export this class and it's method by creating it's instance by using new keyword while exporting.

export default new ScanDetailService;

routes.js

• This is the main routes for all the subRoutes of the Models.
• import the scanRoutes and authRoutes.
• Then use static route method.
• By using the use as the middleware route the scanRoutes and authRoutes.
• use the '/' for the default route root route for the both routes.

import express from "express";

import scanRoutes from "./scans/scanRoute";
import authRoutes from "./auth/AuthRoute";

class Routes {
  routes = express.Router();
  constructor() {
    this.routes.use("/", scanRoutes);
    this.routes.use("/", authRoutes);
  }
}
export default new Routes().routes;

Auth

• This feature is used for authentication.
• for this create the authentication model called as UserModel.js

UserModel.js

• create the schema for storing the authentication.
• create Model attribute such as email,firstName,last Name,password,allowExtraEmails.
• Now,By using the prototype for the model.
• Initialize an asynchronous function named as comparePassword with password as an argument.'
• use bcrypt static method known as compare().

import sequelize from "../db/db";
import { DataTypes } from "sequelize";
import bcrypt from "bcrypt";

const UserModel = sequelize.define("User", {
  firstName: {
    type: DataTypes.STRING,
    allowNull: false,
  },
  lastName: {
    type: DataTypes.STRING,
    allowNull: false,
  },
  email: {
    type: DataTypes.STRING,
    allowNull: false,
  },
  password: {
    type: DataTypes.STRING,
    allowNull: false,
  },
  allowExtraEmails: {
    type: DataTypes.BOOLEAN,
    allowNull: true,
  },
});

• pass the first parameter as password model attribute. second as the argument which is passed to the function
• export the UserModel.

UserModel.prototype.comparePassword = async function (password) {
  return await bcrypt.compare(password, this.password);
};
export default UserModel;

AuthServices.js

• import the UserModel and bcrypt.
• Initialize the class named as AuthServices declare the asynchronous method known as signUp(). with source as parameter.
• destructure the source object.
• return the create method after implementing it on UserModel.
• while using the create method pass password with bcrypt implementation so that it get hashed while storing in the database.
• export the services by creating the instance of the class by using new keyword to the class.

import UserModel from "../users/UserModel";
import bcrypt from "bcrypt";

class AuthServices {
  async signup(source: any) {
    const { firstName, lastName, email, password, allowExtraEmails } = source;
    return await UserModel.create({
      firstName,
      lastName,
      email,
      password: bcrypt.hashSync(password, 10),
      allowExtraEmails,
    });
  }
}

export default new AuthServices();

AuthController.js

• import jwt
• import UserModel from user feature.
• import AuthServices
• Initialize AuthController class with two synchronous method such as signUp signIn.
• signUp take the data from the req.body pass it in to the try/catch AuthServices.signUp( req.body) method.

import jwt from "jsonwebtoken";
import UserModel from "../users/UserModel";
import AuthServices from "./AuthServices";

export class AuthController {
  async signup(req: any, res: any) {
    try {
      AuthServices.signup(req.body);
      return await res.send({
        success: true,
        data: null,
        message: "Created successfully",
      });
    } catch (error) {
      return res.send({ error });
    }
  }

• signIn take the email and password from the req.body.
• use findOne method on the UserModel by using the email to check whether the account credentials are present or not.
• If found fetch it and store in the variable called as user. If not send an error message.
• remember we have created in UserModel as prototype function known as comparePassword().
• implement that method pass the user entered password as second parameter.

  async signIn(req: any, res: any, next: any) {
    const { email, password } = req.body;
    try {
      const user: any = await UserModel.findOne({ where: { email: email } });
      if (!user) {
        return res.status(404).send({
          success: false,
          message: "User not found",
        });
      }
      const validate = await user.comparePassword(password, user.password);

• It will validate the password and returns if the entered password is correct or not.
• Now Generate the token by using jwt static method known as jwt.sign().
• In this it take firs parameter as data,keyword(such as top_secrete),next an object such expires in.
• Then return the response with the object message, data,success.

      if (!validate) {
        return res.status(404).send({
          success: false,
          message: "Username or password doest not match",
        });
      }
      const token = jwt.sign({ email, password }, "top_secret", {
        expiresIn: "5h",
      });
      return res.send({
        success: true,
        data: { token: token },
        message: "Sign in successfully",
      });
    } catch (error) {
      return res.send({ error });
    }
  }
}

AuthRoute.js

• import the express
• import AuthController
• create the instance of the static method of the express.Router() called as authRoutes.
• create the instance of the AuthController.
• Initialize the http method POST to the route instance for both signUp and signIn.

import express from "express";
import { AuthController } from "./AuthController";

class AuthRouter {
  authRoutes = express.Router();
  authController = new AuthController();
  constructor() {
    this.authRoutes.post("/user/signup", this.authController.signup);
    this.authRoutes.post("/user/signin", this.authController.signIn);
  }
}

export default new AuthRouter().authRoutes;

App.ts

• import the cors
• import the routes from the route feature.
• create the instance of the express call it is as app.
• In this we implement use middleware of the express.
• first we convert the file into the json by using express.json().
• express.json() parses incoming requests with JSON payloads and is based on body-parser.
• Apply middleware for cors() and routes as well

import express from "express";
import routes from "./routes";
import cors from 'cors';
class App {
  myLogger(req, res, next) {
    console.log("LOGGED");
    next();
  }
  app = express();
  constructor() {
    this.app.use(express.json());
    this.app.use(this.myLogger);
    this.app.use(cors());
    this.app.use("/api", routes);
  }
}

export default new App().app;

Server.ts

• It is the main file to the run the server.
• import the app.
• declare the variable PORT and initialize with any four digit number.
• Then use the listen method to the run the server on the declared port.

import app from "./src/app";

const port = 8000;

app.get("/api", (req, res) => {
  res.send("App is running");
});

app.listen(port, () => {
  console.log(`Server is running on http://localhost:${port}`);
});

tsConfig.json

• create the above file globally in the folder.
• This is the typescript setup.

{
    "compilerOptions": {
      "target": "es5",
      "module": "commonjs",
      // "strict": true,
      "esModuleInterop": true,
      "forceConsistentCasingInFileNames": true,
      "skipLibCheck": true,
      "outDir": "dist",
      "sourceMap": true
    }
  }