Software - acislab/pragma-cloud-cluster GitHub Wiki
Rocks install
Wrote a USB drive with the kernel roll: http://www.rocksclusters.org/downloads/2017-12-01-download-rocks-7-0-manzanita.html
Followed installation instructions normally: http://central-7-0-x86-64.rocksclusters.org/roll-documentation/base/7.0/install-frontend-7.html
Add the following rolls at install time:
- area51
- base
- CentOS
- core
- ganglia
- kernel
- kvm
- python
- Updates-CentOS-7.4.1708
- zfs-linux
Fix HTTPD not starting
Apache won't start with:
A needed directory under /run does not exist. It looks like this may be fixed in an update but not sure about updating ROCKS from mainline CENTOS. Just created the directory:
mkdir /run/httpd
chown root:apache /run/httpd
chmod 0710 /run/httpd
service httpd start
Ref:
https://bugzilla.redhat.com/show_bug.cgi?id=1215667
https://ma.ttias.be/apache-2-4-ah01762-ah01760-failed-to-initialize-shm-shared-memory-segment/
ZFS
The ZFS roll no longer requires compilation so after installing all the ZFS tools are there. Make raid10-style zpool and fs for pragma boot directories on head node:
zpool create -f storage mirror ata-WDC_WD5000BPKT-00PK4T0_WD-WX11AA2R2477 ata-WDC_WD5000BPKT-00PK4T0_WD-WX51A92P7624
zpool add -f storage mirror ata-WDC_WD5000BPKT-75PK4T0_WD-WX81A23F9893 ata-WDC_WD5000BPKT-75PK4T0_WD-WX81A23R2662
zpool add -f storage mirror ata-WDC_WD5000BPKT-75PK4T0_WD-WX81A23R3759 ata-WDC_WD5000BPKT-75PK4T0_WD-WXB1E92LUR62
zpool status
zfs create -o compression=on storage/site_temp_directory
zfs create -o compression=on storage/site_repository_dir
zfs create -o compression=on storage/cziso_temp_directory
zfs list
If reinstalling these will already exist if you exported the zpool before wiping the install.
KVM roll
Use insert-ethers to PXE boot VM containers.
Main instructions are here: http://central-7-0-x86-64.rocksclusters.org/roll-documentation/base/7.0/install-compute-nodes.html
PRAGMA boot install
https://github.com/pragmagrid/pragma_boot
Also cziso as referred to in the above page.
Below is deprecated
The following notes were made during the ROCKS 6.x installation which was never completed:
Local CDROM drive was used because swapping CDs over remote emulation didn't work well. The local CDROM needed to have DMA disabled at the boot prompt of the first disk with build ide=nodma
.
Later, found the jumbo DVD image so CD swapping wasn't needed but still installed from local CDROM as it's a bit faster:
Add the following rolls at install time:
- ganglia
- area51
- kernel
- os
- base
- kvm
- python
Install for convenience yum --enablerepo=base install screen
Networking
After install, the KVM roll reconfigures the networking to be a physical device peth? with a bridge eth? attached which holds the IP configuration, a typical bridge VM setup. However, the regular devices are not correctly renamed to peth? at boot so networking fails.
Edit /etc/udev/rules.d/70-persistent-net.rules
to set the device named to peth? and reboot. Networking should now start up correctly eg:
# PCI device 0x14e4:0x1639 (bnx2)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="e4:1f:13:2e:18:c4", ATTR{type}=="1", KERNEL=="eth*", NAME="peth0"
ZFS roll
To install roll, insert CD and run:
mount /dev/cdrom /mnt/cdrom
rocks add roll
rocks enable roll zfs-linux
cd /export/rocks/install; rocks create distro
rocks run roll zfs-linux | sh
Follow installation instructions from the documentation to build the source and install the compiled roll:
https://pcloud-head/roll-documentation/zfs-linux/0.6.4.1/adding-the-roll.html
After following those instructions, make zpool and fs on head node:
zpool create -f storage mirror ata-WDC_WD5000BPKT-00PK4T0_WD-WX11AA2R2477 ata-WDC_WD5000BPKT-00PK4T0_WD-WX51A92P7624
zpool add -f storage mirror ata-WDC_WD5000BPKT-75PK4T0_WD-WX81A23F9893 ata-WDC_WD5000BPKT-75PK4T0_WD-WX81A23R2662
zpool add -f storage mirror ata-WDC_WD5000BPKT-75PK4T0_WD-WX81A23R3759 ata-WDC_WD5000BPKT-75PK4T0_WD-WXB1E92LUR62
zpool status
zfs create -o compression=on storage/site_temp_directory
zfs create -o compression=on storage/site_repository_dir
zfs create -o compression=on storage/cziso_temp_directory
zfs list
KVM roll
Use insert-ethers to PXE boot VM containers.
PRAGMA boot install
https://github.com/pragmagrid/pragma_boot
Also cziso as referred to in the above page.