NooBaa Directory Structure Explained

This guide should serve as a general directory structure reference in the NooBaa project.

In general, NooBaa contains a few deliverables each with its own directory structure. The top-level directories in the project are:

• frontend - Contains the NooBaa frontend component, see frontend
• gocode - Contains go code for the NooBaa operator and various CLI utilities, see gocode
• src - Contains the Backend (server), agents (storage and endpoints), deployment code and tests code, see src

frontend (frontend)

frontend

The frontend component provides the GUI for the NooBaa system. Most of the code resides in the app folder.

app

frontend/src/app Holds the management console web application code. The application code is mainly comprised of State Schema files (under ./schema), State Reducers (under ./reducers), UI Components (under ./components), action related side effect processes (under ./epics) and non state related services (under ./services). The folder is transpiled (using Babel.js) and bundled (using Browserify) into /frontend/dist/app.js during the application build process. A .js.map map file is also generated to help with debugging.

lib

Holds third party libraries used by the management console runtime code. These libraries are installed using bower and are bundled into frontend/dist/lib.js during the application build process. A .js.map map file is also generated to help with debugging.

assets

frontend/src/assets Holds all the raw assets which are used in the management console building process (images, icons and fonts), the results of processing these assets are copied to the frontend/dist/assets directory.

scaffolding

frontend/src/scaffolding Holds the implementation and templates for a scaffolding tool that ease the creation of new components in the management console application code base. The interactive tool can be launched using /frontend/scaffold.sh and is used to scaffold UI Components, UI Modals and State Reducers including injection of registration code.

gocode

• src/noobaa-operator - Contains NooBaa operator code

src

src

The src directory is broken down according to functionality, its main directories are

agent

src/agent

This directory contains the agent (daemon) code. The agent is a container which can run several functions - storage, s3 endpoint, and functions endpoint. This directory contains the container code, the storage and functions endpoint.

api

src/api

This directory contains the jsons which describe the NooBaa RPC API. They are broken according to the "server" or component which handles their logic. So bucket_api for example would be implemented by the bucket_server component and will handle bucket creation, policy changes on a bucket, bucket data retrieval, etc.

deploy

src/deploy

deploy is a directory which mainly contains scripts and files related to various deployments of NooBaa. So dockerfile, RPM build, virtual appliance build, mongo schema upgrades, etc. can be found in this directory.

endpoint

src/endpoint

endpoint contains the code the various endpoint NooBaa support. In this directory, you can find the code to the s3 endpoint, blob endpoint, and lambda functions endpoint.

hosted_agents

src/hosted_agents

hosted agents are the components which handle using API based resources (S3, GCP, Azure blob, etc.)

lambda_funcs

src/lambda_funcs

A few examples for existing lambda functions

native

src/native

This directory holds the native code implementation for a few components in the NooBaa system. In here the implementations for the NooBaa chunk encryption, inline dedup, agents n2n, third-party libraries, and various utilities can be found.

rpc

src/rpc

NooBaa's RPC layer implementation with its various transport options (TPC, WebSocket, n2n, etc.)

sdk

src/sdk

Serves as an API layer which runs on top of NooBaa's server, receives OPs from the various endpoints and prepare the requests for the internal NooBaa components.

server

src/server

This directory contains all the backend services of NooBaa. Content is broken down according to functionality

analytic_services

src/server/analytic_services

Activity logs, Bucket statistics, IO statistics, Prometheus integration, and system usage.

bg_services

src/server/bg_services

Background services of the system: blocks reclaimer, cluster heartbeat, db cleaner, server monitor, stats collector etc.

common_services

src/server/common_services

Auth server and Debug server.

func_services

src/server/func_services

Functions (lambda) server

mongo_services

src/server/mongo_services

mongo monitor

node_services

src/server/node_services

Handling the Agents (Daemon) components in the system: nodes monitor, nodes aggregator, nodes client etc.

notifications

src/server/notifications

Alerts server, Events server and notifications dispatcher.

object_services

src/server/object_services

NooBaa internal mapping services. Write/Read/Rebuild and the object server

system_services

src/server/system_services

Various components in charge of handling configuration entities in the system. For example, bucket server, tier server, pool server etc.

utils

src/server/utils

Various server specific utils.

test

src/test

NooBaa's tests are located in this directory.

tools

src/tools

Various tools (statistics, performance, analysis, etc.). See Developers Guide for more explanation on the NooBaa tests system.

upgrade

src/upgrade

Upgrade server and flow.

utils

src/utils

Various utils libraries used by the NooBaa components.