This lab demonstrates how to recover access to an Oracle Linux instance when the SSH private key for the opc user is lost.

Instead of creating a new instance, we use an OCI Serial Console Connection to access the server, enter GRUB recovery mode, boot into a Bash shell, generate a new SSH key pair, update the authorized_keys file, and regain SSH access using a new private key.

This is a common troubleshooting scenario used by OCI administrators in production environments.

In this lab, we will:

• Create a Console Connection
• Connect using OCI Cloud Shell
• Reboot the instance
• Access the GRUB boot menu
• Boot into recovery mode using init=/bin/bash
• Access the Bash recovery shell
• Generate a new SSH key pair
• Replace the old SSH public key
• Reboot the instance
• Log in using the new private key

Navigate to:

OCI Console
→ Compute
→ Instances
→ Select Target Instance

Verify:

• Instance State = Running
• Console Connection Permission Available

Open:

Resources
→ Console Connections

Click:

Launch Cloud Shell Connection

Note: Use Cloud Shell Connection instead of Local Connection for this lab.

Cloud Shell opens automatically.

Press:

Enter
Enter
Enter

until console output appears.

Navigate to:

Instance
→ More Actions
→ Reboot

Wait for the reboot process to begin.

As soon as the reboot starts:

Press:

ESC

repeatedly until the GRUB menu appears.

Example:

Oracle Linux Server 9.8
Advanced options for Oracle Linux

Highlight the first boot entry.

Press:

e

to edit the boot configuration.

Scroll down until you find a line similar to:

linux ($root)/vmlinuz-...

Example:

linux ($root)/vmlinuz-0-rescue-xxxxxxxx

At the end of the Linux boot line add:

init=/bin/bash

Example:

linux ($root)/vmlinuz-0-rescue-xxxxxxxx init=/bin/bash

Boot using:

Ctrl + X

After booting, the system loads a Bash shell.

Expected output:

bash-5.1#

This confirms that the server has booted into recovery mode successfully.

Enable read-write access:

mount -o remount,rw /

Verify:

mount | grep " / "

Execute:

/usr/sbin/load_policy -i

Run:

cd /home/opc/.ssh

Verify the SSH key file:

ls -l

Example:

authorized_keys

Create a backup:

cp authorized_keys authorized_keys.bak

Rename the original file:

mv authorized_keys authorized_keys.old

Verify:

ls -l

Expected:

authorized_keys.bak
authorized_keys.old

Generate a new RSA key pair:

ssh-keygen -t rsa -b 4096

Press Enter for all prompts.

Example:

Enter file in which to save the key:
/root/.ssh/id_rsa

Enter passphrase:
Enter

Confirm passphrase:
Enter

Output:

Your identification has been saved in:

/root/.ssh/id_rsa

Your public key has been saved in:

/root/.ssh/id_rsa.pub

Copy the newly generated public key into the OPC user's authorized keys file:

cat /root/.ssh/id_rsa.pub > /home/opc/.ssh/authorized_keys

Verify:

cat /home/opc/.ssh/authorized_keys

Apply the correct permissions:

chmod 600 /home/opc/.ssh/authorized_keys

chown opc:opc /home/opc/.ssh/authorized_keys

Verify:

ls -l /home/opc/.ssh

Expected:

-rw------- authorized_keys

Reboot the server:

reboot -f

or

/usr/sbin/reboot -f

The instance exits recovery mode and starts normally.

Display the newly generated private key:

cat /root/.ssh/id_rsa

Copy the entire key and save it locally.

Example:

-----BEGIN OPENSSH PRIVATE KEY-----
...
-----END OPENSSH PRIVATE KEY-----

Save it as:

new-opc-key

Open:

PuTTYgen

Select:

Conversions
→ Import Key

Import:

new-opc-key

Leave Passphrase blank.

Click:

Save Private Key

Save as:

new-opc-key.ppk

Open PuTTY.

Configure:

Host Name = <Public IP>
User = opc
Private Key = new-opc-key.ppk

Connect to the server.

Successful login confirms that SSH access has been recovered.

Successfully recovered access to an Oracle Linux instance after losing the original SSH private key by using an OCI Serial Console Connection and GRUB recovery mode.

Answer:

I would create a Serial Console Connection, reboot the instance, access the GRUB boot menu, modify the boot entry by adding init=/bin/bash, boot into recovery mode, generate a new SSH key pair, replace the opc user's authorized_keys file, reboot the instance, and then log in using the newly generated private key.

Initially, after rebooting the instance and repeatedly pressing the Esc key, I was redirected to the GRUB command prompt (grub>) instead of the GRUB boot menu.

The grub> prompt does not allow editing the Linux boot entry required for recovery using init=/bin/bash.

The below screen shows the GRUB command prompt (grub>), which is not the menu required for SSH key recovery.

To access the correct GRUB boot menu, I rebooted the instance again. During the boot process, instead of continuously pressing Esc, I pressed the Up Arrow and Down Arrow keys repeatedly. This paused the automatic boot sequence and allowed me to capture the actual GRUB boot menu.

From this menu, I was able to select the Oracle Linux boot entry and press e to edit the boot configuration.

The below screen shows the actual GRUB boot menu used for troubleshooting and recovery. From this menu, the boot entry can be edited by pressing e and adding init=/bin/bash to boot into recovery mode.

Press e to edit the GRUB boot entry. Navigate to the Linux boot line, add init=/bin/bash at the end of the line, and press Ctrl + X to boot into recovery mode.

After pressing Ctrl + X, the instance booted using the modified GRUB configuration and entered recovery mode.

A Bash shell prompt appeared:

bash-5.1#

bash-5.1#

Execute the following commands to remount the root filesystem in read-write mode, load the required system policies, and access the SSH configuration directory for key recovery.

mount -o remount,rw /

mount | grep " / "

/usr/sbin/load_policy -i

cd /home/opc/.ssh
ls -l

cp authorized_keys authorized_keys.bak

ls -l

ssh-keygen -t rsa -b 4096

Display and copy the newly generated public key:

cat /root/.ssh/id_rsa.pub

cd /home/opc/.ssh
cp authorized_keys authorized_keys.bak
mv authorized_keys authorized_keys.old

ls -l

Replace the existing SSH public key with the newly generated public key.

cat /root/.ssh/id_rsa.pub > /home/opc/.ssh/authorized_keys

now verify

cat /home/opc/.ssh/authorized_keys

Verify that the contents of the authorized_keys file match the newly generated public key.

The output should be identical to the contents of /root/.ssh/id_rsa.pub, confirming that the new public key has been successfully updated.

chmod 600 /home/opc/.ssh/authorized_keys
chown opc:opc /home/opc/.ssh/authorized_keys

Interview Answer

chown opc:opc

ls -l /home/opc/.ssh

Run:

cat /root/.ssh/id_rsa

Open PuTTYgen and select:

Conversions → Import Key

Import the saved private key file and click Save Private Key.

Save the file as:

new-opc-key.ppk

now reboot the server

/usr/sbin/reboot -f

Server refused our key

or

Key mismatch

new-opc-key.ppk