Shared data based xCAT MN HA user case - xcat2/xcat-core GitHub Wiki

The shared data based xCAT MN HA user case

As a xCAT user, I have host1 and host2 with the same shared data directory, I want to configure xCAT management node HA rapidly.

Pre-requirements for User:

host1  host2
Shared data directory 
The nic that the virtual ip address attaches to
Virtual IP
Net mask

1. setup 2 xCAT MN nodes

There are 2 scenarios:

Scenario 1 xCAT is not installed on 2 xCAT MN nodes

host1 is installed as xCAT primary MN, host2 is installed as xCAT standby MN

  1. Install xCAT on xCAT standby MN host2 using VIP/its hostname, details:
    1. configure VIP and its hostname
    2. make symbolic link to share data directories 
      /install -> /HA-data/install
/etc/xcat ->/HA-data/etc/xcat
/root/.xcat -> /HA-data/root/.xcat
/var/lib/pgsql -> /HA-data/var/lib/pgsql
/tftpboot -> /HA-data/tftpboot
    3. install xCAT
  2. Switch DB to required type as needed
  3. Deactivate xCAT standby MN host2 ---> details refer to section 1 in "failover scenario 1"
  4. Install host1 as active xcat primary MN using VIP and its hostname, details steps:
    1. configure VIP and its hostname
    2. make symbolic link to share data directories 
      /install -> /HA-data/install
/etc/xcat ->/HA-data/etc/xcat
/root/.xcat -> /HA-data/root/.xcat
/var/lib/pgsql -> /HA-data/var/lib/pgsql
/tftpboot -> /HA-data/tftpboot
    3. install xCAT

Scenario 2 user uses host1 as xCAT MN, he wants to add another new node host2 as xCAT standby MN node

  1. The original xCAT MN host1 ip becomes virtual IP, user should add another new IP for host1,
  2. User should deactivate xCAT MN host1, deactivte virtual ip and its hostname
  3. User installs host2 as a xCAT standby MN using VIP and its hostname, then deactivate xCAT standby MN host2
  4. User activate host1 as xCAT primary MN
  5. Configure all related service stop from starting on reboot on both 2 nodes

2. Different failover scenarios:

failover scenario 1

when active xCAT MN host1 is broken and we can access to its OS

  1. I use deactivate-xcatmn to make host1 as non-active xcat MN node
    1. make sure all related services as followings are down, make sure all related services are configured stop from starting on reboot
      1. console service
      2. DHCP service
      3. named service
      4. xcatd
      5. database (mysql/postgresql/sqlite type)
    2. umount/un-link shared data directories on host1
    3. change hostname if needed
    4. remove virtual IP
  2. I use activate-xcatmn to configure host2 as active xcat MN node
    1. make sure virtual ip is not used (ping), or else, exit
    2. add virtual ip into its nic
    3. set hostname to virtual ip hostname
    4. check if current DB type is matched, if not, exit and clean up env
    5. make symbolic link to share data directories, for example: 
      /install -> /HA-data/install
/etc/xcat ->/HA-data/etc/xcat
/root/.xcat -> /HA-data/root/.xcat
/var/lib/pgsql -> /HA-data/var/lib/pgsql
/tftpboot -> /HA-data/tftpboot
    6. start/re-configure all related services as followings:
      1. database (mysql/postgresql/sqlite type)
      2. xcatd
      3. named service (makedns -n)
      4. DHCP service (makedhcp -n, makedhcp -a)
      5. Console Server
      6. ... ...

failover scenario 2

when active xCAT MN host1 is broken and we cannot access to its OS, restart this xCAT MN node host1, after it reboots:

  1. if we can access to its OS

    1. do the same with failover scenario 1

  2. if we cannot access to host1 OS, un-plugin its network cable.

    1. use activate-xcatmn to configure host2 as active xcat MN node, the same as section 2 in failover scenario 1
    2. Recommend to fix host1 OS or hardware.