This section outlines the backend logging system in DataSeq, focusing on its differences from frontend logging and its importance in tracking system and job-level operations.

Backend logging shares the following characteristics with the frontend logger:

• Log Format:
  [YYYY-MM-DD HH:MM:SS] ACTION_TYPE: Description of action (Metadata)

• Output Modes:

  • Console
  • File (log.txt)
  • Both (configured via .env using LOG_MODE)

• Configuration:
  Logging behavior is defined in .env (e.g., LOG_MODE=both, LOG_FILE_PATH=logs/log.txt)

• Future Plans:

  • A runtime-configurable administrative panel for logging control was planned but has been backlogged.
  • Reason: The current implementation relies on static .env variables, which cannot be modified dynamically at runtime without restarting the server.
  • Impact: Enabling runtime log mode switching would require major project restructuring (e.g., introducing centralized configuration management or dynamic reload systems).
  • This feature has been deferred to a future major project revision.

Backend logs capture system-critical events, job execution states, and integration with external services:

• Job Execution

  • JOB_START: A backend job has been launched
  • JOB_COMPLETE: The job completed successfully
  • JOB_FAIL: The job terminated with an error

• System Errors

  • ERROR: Unhandled exceptions, file issues, or configuration errors

• Backend Operations

  • PREPROCESS, VALIDATE, CONFIG_PARSE: Backend-specific processing stages

• External Communication

  • HPC_SUBMIT, API_CALL: Interactions with external systems such as HPC clusters or APIs

Unlike the frontend, backend logs are written to persistent files and support automated rotation:

• Rotation Triggers:

  • File size exceeds a defined limit (e.g., 5MB)
  • Time-based rotation (e.g., daily)

• Rotated File Format:
  Standard timestamp-based naming (e.g., log-YYYY-MM-DD.txt)

• Common Rotation Tools:

  • logrotate (Linux-based systems)
  • winston-daily-rotate-file (Node.js)
  • RotatingFileHandler (Python)

Backend logging is fully implemented using Node.js with the winston library and .env configuration control.

const { createLogger, format, transports } = require('winston');
const dotenv = require('dotenv');

dotenv.config(); // Load .env vars

const LOG_MODE = process.env.LOG_MODE || 'both';

const loggerTransports = [];

// Console transport
if (LOG_MODE === 'console' || LOG_MODE === 'both') {
  loggerTransports.push(
    new transports.Console({
      format: format.combine(
        format.colorize(),
        format.printf(({ level, message, timestamp }) => {
          return `[${timestamp}] [${level}]: ${message}`;
        })
      )
    })
  );
}

// File transport
if (LOG_MODE === 'file' || LOG_MODE === 'both') {
  loggerTransports.push(
    new transports.File({
      filename: 'log.txt',
      format: format.combine(
        format.timestamp(),
        format.printf(({ level, message, timestamp }) => {
          return `[${timestamp}] [${level.toUpperCase()}]: ${message}`;
        })
      )
    })
  );
}

const logger = createLogger({
  level: 'info',
  format: format.timestamp(),
  transports: loggerTransports
});

module.exports = logger;

Here is an example usage of the backend logger in one of the controller.js files of the project's backend server code:

const logger = require('../utils/backEndLogger');

function login(req, res) {
    const user_email = req.body.user_email;
    const user_password = req.body.user_password;

    logger.info('/user/login request received');

    if (user_email && user_password) {
        logger.info(`Login attempt for: ${user_email}`);
        res.sendStatus(200);
    } else {
        logger.warn(`Bad login attempt. Email: ${user_email}`);
        res.status(400).json({ error: 'Null email or password' });
    }
}

function signup(req, res) {
    const user_email = req.body.user_email;
    const user_password = req.body.user_password;

    logger.info('/user/signup request received');

    if (user_email && user_password) {
        logger.info(`Signup attempt for: ${user_email}`);
        res.sendStatus(200);
    } else {
        logger.warn(`Bad signup attempt. Email: ${user_email}`);
        res.status(400).json({ error: 'Null email or password' });
    }
}

module.exports = { login, signup };

...

This section outlines the frontend logging system in DataSeq, which captures user interactions and UI-driven events for traceability, debugging, and auditing.

Frontend logging shares the following characteristics with the backend logger:

• Log Format:
  [YYYY-MM-DD HH:MM:SS] ACTION_TYPE: Description of action (Metadata)

• Output Modes:

  • Console
  • File (log.txt)
  • Both (controlled via .env using VITE_LOG_MODE)

• Configuration:
  Logging behavior is defined in .env (e.g., VITE_LOG_MODE=both)

• Future Plans:
  A runtime-configurable UI setting is planned for future enhancement

  Note: FEATURE HAS BEEN BACKLOGGED FOR NOW AS CURRENT IMPLIMENTATION UTILIZING A .ENV FILE MAKES IT IMPOSSIBLE TO CHANGE THE VARIABLES FROM THE FRON END REACT APP AS FAR AS MY RESEARCH HAS FOUND MAY BE CHANGED OR IMPLIMENTED DIFFERENTLY IN THE FUTURE.

Frontend logs are focused on user behavior and interface-level activity:

• User Interactions

  • CLICK, NAVIGATE, SUBMIT: Button presses, navigation changes, form submissions

• System Messages

  • ERROR, WARNING, NOTIFY: Feedback presented to the user

• Data Operations

  • IMPORT, EXPORT, QUERY: Initiated from the UI

Logs can be directed to the developer console, stored in memory for file-based export, or both:

• Console Output
  Logs appear in the browser’s developer console for immediate feedback

• File Output
  Logs are buffered in memory and can be downloaded on demand as log.txt

• Log Management Functions:

  • downloadLog(): Downloads the current log buffer
  • clearLog(): Clears the buffer (e.g., after saving)

Logging is implemented in src/utils/clientLogger.js using environment-based control.

Logging is implemented in JavaScript and uses a flexible utility architecture. Behavior is governed by the VITE_LOG_MODE environment variable.

Example (clientLogger.js):

// src/utils/clientLogger.js

let logBuffer = ""; // Used for file logging

const LOG_MODE = import.meta.env.VITE_LOG_MODE || 'console'; // 'console', 'file', or 'both'

function getTimestamp() {
  return new Date().toISOString().replace('T', ' ').substring(0, 19);
}

function formatLog(actionType, message) {
  return `[${getTimestamp()}] ${actionType.toUpperCase()}: ${message}`;
}

function log(actionType, message) {
  const formatted = formatLog(actionType, message);

  if (LOG_MODE === 'console' || LOG_MODE === 'both') {
    console.log(formatted);
  }

  if (LOG_MODE === 'file' || LOG_MODE === 'both') {
    logBuffer += formatted + '\n';
  }
}

// Optional: Trigger download of current log buffer as log.txt
function downloadLog() {
  const blob = new Blob([logBuffer], { type: 'text/plain' });
  const url = URL.createObjectURL(blob);

  const a = document.createElement('a');
  a.href = url;
  a.download = 'log.txt';
  a.click();

  URL.revokeObjectURL(url);
}

// Optional: Clear log buffer (e.g., after download or reset)
function clearLog() {
  logBuffer = "";
}

export default log;
export { downloadLog, clearLog };

Here is an example of the frontend logger being used to log what a user is uploading into the FASTQ upload form and if that file was successfully passed to the server or rejected for whatever reason.

function FastQForm({ setIsUploaded, setUploadID }) {
    const [selectedFile, setSelectedFile] = useState(null);

    const handleFileChange = (event) => {
      const file = event.target.files[0];
      setSelectedFile(file);
      log("SELECT_FILE", file ? `User selected file: ${file.name}` : "No file selected");
    };
  
    const handleUpload = async () => {
      if (!selectedFile) {
        alert("Please select a fastQ file for preprocessing");
        log("UPLOAD_FAIL", "Upload attempted with no file selected");
        return;
      }

      //Generates an id for associativity with config.yaml file
      const newID = uuidv4();
      const formData = new FormData();
      formData.append("uploadID", newID);
      formData.append('file', selectedFile);
      
  
      try {
        const response = await fetch('/api/upload', { 
          method: 'POST',
          body: formData,
        });
        // just invert with ! to test setIsUploaded functionality
        if (response.ok) {
          alert("FastQ file uploaded successfully");
          log("UPLOAD", `FastQ file uploaded: ${selectedFile.name}`);
          setUploadID(newID);
          setIsUploaded(true);
        } else {
          alert("FastQ file upload failed");
          log("UPLOAD_FAIL", `Server responded with status ${response.status}`);
        }
      } catch (error) {
        console.error("There was an error uploading the FastQ file:", error);
        alert("An error occurred during FastQ file upload");
        log("UPLOAD_ERROR", `Error during upload: ${error.message}`);
      }
    };

    return (
    <Card className="p-3 border">
        <h3>FastQ File Input</h3>
      <input className="fastQinputInput" type="file" onChange={handleFileChange} />
      <button className="mt-2" onClick={handleUpload} disabled={!selectedFile}>Upload File</button>
      {selectedFile && (
        <p>Selected file: {selectedFile.name}</p>
      )}
    </Card>
    )
}

This section outlines how both frontend and backend logs can be viewed or retrieved during development and deployment.

Frontend logs are available through two main methods:

• Web Developer Console (Web Terminal)

  • Logs are immediately output to the browser’s built-in developer console.
  • To view, press F12 or Ctrl+Shift+I to open Developer Tools, then navigate to the Console tab.

• Download Log File via UI

  • Users can click the "Download Log" button located on the header of the DataSeq application.
  • This triggers a download of the current log buffer into a text file (log.txt), containing all actions logged during the session.

Backend logs are accessible in two locations:

• Development Terminal Output

  • Logs are streamed live to the backend server’s terminal window (e.g., Visual Studio Code terminal, Node.js console, or cloud service terminal).

• Persistent Log File

  • A permanent record of backend logs is stored in a file (log.txt) in the server’s local filesystem.
  • This file accumulates logs across sessions and can be reviewed manually or archived as needed.

Notes:

• Console logs (both frontend and backend) are particularly useful during active development and debugging.
• File logs (log.txt) ensure long-term traceability for auditing, troubleshooting, and analysis after deployment.