LAB 43 ‐ Step‐by‐Step Guide: Building a Scalable 3‐Tier Application with a Load Balancer - gautam-amitesh/OCI GitHub Wiki
Compartments
HUB
SPOKE
VCNs & Subnets
HUB
SPOKE
Security Lists
HUB
Public Subnet: Allow inbound SSH access (TCP port 22) from authorized sources. Allow outbound connectivity to the ITSM_subnet, all IAD services in the Oracle Services Network, and the internet for the installation of security patches, operating system updates, and software upgrades, and permit the corresponding return traffic.
SPOKE
WEB Subnet: Allow inbound HTTP connections from any source, and permit the corresponding return traffic. Allow outbound HTTP connections to the ITSM_subnet, and permit the corresponding return traffic.
APP/ITSM Subnet: Allow inbound HTTP connections from web_subnet, and permit the corresponding return traffic. Allow inbound ICMP connections from public_subnet of ITSM-hub_vcn, and permit the corresponding return traffic. Allow inbound SSH access (TCP port 22) from public_subnet of ITSM-hub_vcn, and permit the corresponding return traffic. Allow inbound HTTP connections from public_subnet of ITSM-hub_vcn, and permit the corresponding return traffic. Allow outbound connectivity to the public_subnet, web_subnet, db_subnet, Oracle Services Network, and the internet to enable the installation of update & upgrades, and permit the corresponding return traffic.
DB Subnet: Allow inbound TCP connections on port 1521 from ITSM_subnet, and permit the corresponding return traffic. Allow outbound TCP connections to all IAD services in Oracle Services Network, and permit the corresponding return traffic.
Gateways
DRG
HUB
SPOKE
Route Tables & Rules
HUB
SPOKE
WEB Subnet:
ITSM/APP Subnet:
DB Subnet:
Deploy Jump Host & Backend Server
So far, I have configured the networking components using the OCI Console. Next, I will deploy a Jump Host named bastion_server in the public_subnet and an APP Server named ITSM_server in the ITSM_subnet. On the application server, I will host an index.html page that represents the ITSM application.
Deploy bastion_server
Setup bastion_server
ssh -i /path-to-private-key opc@public-IP
sudo dnf install telnet
sudo su - root
mkdir ssh-keys
vi /root/ssh-keys/private_key.pem
:wq
chmod 400 ssh-keys
Deploy ITSM_server
Setup ITSM_server
ssh -i /root/ssh-keys/private_key.pem opc@private-IP
If you are unable to connect to internet to download package but your connection seems complete then navigate to your ITSM_subnet-RT and delete route to Service Gateway and try again, run following command.
sudo dnf install httpd
sudo firewall-cmd --list-all
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --reload
sudo firewall-cmd --list-all
ping ITSM_server-IP
telnet ITSM_server-IP 80
sudo su - root
cd /var/www/html
vi index.html
GitHub link to html file Html file Script for ITSM Tool
:wq
Deploy Load Balancer
To provide external access to the application, I will deploy a public load balancer (even if I have a single backend server) named ITSM_servers-LB in the web_subnet. The load balancer will receive incoming web traffic and forward it to the backend server in the ITSM_subnet. Users will access the ITSM application through the public IP address of the load balancer rather than directly accessing the backend server.
OCI Load Balancer acts as a buffer between users and backend servers. It receives incoming traffic and allows requests to reach backend servers only when the required ports are permitted by the configured security rules. Through continuous health checks, it monitors the health of backend servers, automatically removing unhealthy instances from the backend set and routing traffic only to healthy backends, helping maintain application availability without service disruption.
OCI Load Balancer distributes incoming traffic across backend servers using a configurable load-balancing policy such as Round Robin (deprecated), Weighted Round Robin (OCI default), Least Connections, or IP Hash. These algorithms help ensure that traffic is distributed efficiently across available backend servers.
In modern application architectures, instead of deploying a single large server, we deploy multiple identical instances and add them to a server pool. This approach is known as horizontal scaling (scaling out). To ensure that all active instances are utilized efficiently, incoming traffic is distributed among them using a load-balancing algorithm selected based on the application's requirements and traffic patterns. OCI Load Balancer supports the following load-balancing policies:
- Round Robin (deprecated): Round Robin is the simplest load balancing strategy, where incoming requests are distributed among healthy backend servers in a strict sequential order (Server A → Server B → Server C → repeat). This approach assumes that all servers have identical configurations (CPU/RAM) and that every request requires the same amount of processing time. In real-world scenarios, however, request processing times often vary. As a result, a server may already be busy handling existing requests when its turn comes again, causing the load balancer to continue routing additional requests to it, which can lead the server to choke.
- Weighted Round Robin (default): Weighted Round Robin addresses the unequal hardware limitation of basic Round Robin by assigning an integer weight to each backend node based on its compute shape. For example, if we have an older 4-core instance A (Weight = 1) and a modern 16-core compute instance B (Weight = 3), the load balancer distributes requests in a 3:1 ratio (Server A → Server B → Server B → Server B → repeat). This approach is ideal for transitional infrastructures where servers have different capacities, such as running legacy servers alongside newer high-spec nodes or combining smaller compute shapes with larger Bare Metal instances.
- Least Connections: Least Connections takes a more dynamic approach than following a fixed sequence. Instead of distributing requests in a rigid order, it evaluates the current traffic state and routes each new request to the server with the fewest active, open client connections. This helps prevent server overload when workloads are uneven or unpredictable. By continuously balancing traffic based on real-time connection counts, it ensures that no single server becomes overwhelmed with long-running requests while other servers remain underutilized.
- IP Hash: IP Hash takes the client's source IP address (and sometimes the destination IP/port), passes it through a mathematical hashing function, and maps the resulting hash value to a specific backend server. As long as the client's IP address remains unchanged, every request from that user is directed to the same backend instance, without the load balancer needing to inject tracking cookies or maintain state tables in memory. This makes it a simple way to achieve session persistence. However, IP Hash has an important limitation. If thousands of users are behind a single corporate NAT proxy or a university network, they all share the same public IP address. In that scenario, IP Hash maps all of those users to the same backend server, unintentionally creating a massive hotspot and resulting in an uneven distribution of traffic across the backend pool.
-
Navigation
Navigation menu > Networking > Load balancer (under Load Balancers) -
Click
Create load balancer -
Provide a Load Balancer name, select the Visibility type based on your access requirements, and choose the appropriate public IP configuration. Use
Publicvisibility when the load balancer must accept traffic from external users over the internet, orPrivatevisibility when access is limited to internal users, such as employees within the organization's network. If you select Public visibility, chooseEphemeralas the Public IP Address Type to have OCI automatically assign a temporary public IP address. -
Select
ITSM-spoke_vcnas the VCN andweb_subnetas the subnet. Because the load balancer is deployed in theweb_subnet(a public subnet), OCI assigns it a public IP address from the subnet's associated public IP space, enabling it to receive traffic from external users. -
Specify the
Load Balancing Policy, then select theCompartmentthat contains thebackend servers. ClickAdd Instancesand choose the compute instances that will serve as the Backend Servers for the load balancer. -
Specify the
Health Check Policyby selecting the appropriateProtocolandPort. The load balancer uses these settings to periodically probe backend servers and verify that they are healthy, responsive, and capable of serving client requests. -
Provide a
Backend Set Name, and in theSecurity Listsection, selectManually. This allows you to configure the required security rules yourself rather than having OCI automatically create or update them, giving you greater control over network access and traffic flow to the backend servers. -
Provide a
Listener Name, select the appropriateProtocolandPort, and then create the listener. The listener acts as the entry point for client traffic, accepting incoming connections on the specified protocol and port and forwarding them to the associated backend set based on the configured load balancing policy.
Application Testing
To access the ITSM Application, open any web browser, enter the Public IP address of the Load Balancer in the address bar, and press Enter. The load balancer will receive the request and route it to one of the healthy backend servers according to the configured load balancing policy.
If you have configured your browser to automatically upgrade all websites to HTTPS, as I have done, you may encounter an error similar to the one shown in the image below. This occurs because the load balancer is configured to accept HTTP traffic only and is not listening for HTTPS connections.
To fix this, go to the browser's address bar and click the URL containing the IP address and click once again. If the browser has automatically changed the protocol to HTTPS, edit the URL and replace https:// with http://, then press Enter. The browser will then connect to the load balancer using HTTP, which matches the current listener configuration.