Coherence Workspaces on VMs - padogrid/padogrid GitHub Wiki
◀️ Coherence perf_test App :link: Hadoop CLASSPATH ▶️ :link: Home ▶️
Coherence Workspaces on VMs
PadoGrid provides a simple way to manage Coherence clusters on laptop, PadoGrid pods, and VMs. This article describes how to create and run a Coherence cluster on multiple VMs and/or PMs (Physical Machines). The instructions provided apply to any VMs and PMs including AWS, Azure, GCP, physical machines, VirtualBox, vSphere, PadoGrid pods (VMs), etc.
With PadoGrid, you can have a complete control over Coherence running on VMs from your local machine. PadoGrid preserves the state of VM configurations and deployments on your local machine so that you can conveniently activate or deactivate the VM environment at any time. After you are done running VMs, you can tear them down and reinstate the very same environment later with a single command. This saves you time and reduces VM costs when you are working in a cloud environment, for example.
Installation Steps
There are four (4) main steps to installing PadoGrid on VMs. Once these steps are performed, you can then synchronize your local machine with the remote VMs by simply executing the vm_sync command, which enables the local control of VMs.
- Setup password-less SSH login
- Download required software
- Create VM workspace
- Synchronize VM Workspace
Password-less SSH Login
The first step is to check to make sure you are able to login to the VMs that you want to cluster without the password. For AWS EC2, for example, this is already done for you so you can skip this section.
If you need to manually setup password-less SSH login, then follow the instructions here.
:exclamation: Make sure you can ssh from any of the VMs including your local machine without the password.
Download Software
PadoGrid requires the following software products. If they are not installed on VMs, then download their tarball or zip distributions.
- Java
- Coherence
- Coherence distributions come in two forms: stand-alone and quick install. Download the stand-alone install.
For our demo, we'll assume they are downloaded in the following directory. Note that they must be tarball or zip distributions.
/home/dpark/Downloads
├── jdk1.8.0_212.tar.gz
└── fmw_14.1.1.0.0_coherence_Disk1_1of1.zip
Install Coherence
If you haven't installed Coherence previously on your local machine, e.g. your laptop, install it now by following the instructions provided in the following link.
Installing Oracle Coherence for Java
For our example, we will assume you have installed Coherence in the following directory:
~/Padogrid/products/Oracle/Middleware/Oracle_Home/coherence
Create Coherence tarball
In order to remotely install Coherence on VMs, we need to create a tarball. Using our example directory, let's create a tarball as follows.
tar -C ~/Padogrid/products/Oracle/Middleware/Oracle_Home --exclude doc --exclude examples -czf tmp/coherence.tar.gz coherence
The above command generates the ~/Downloads/coherence.tar.gz file without examples and documents.
Create VM Workspace
Create a VM workspace on your local machine.
create_workspace -vm -name ws-vm
The above command interactively prompts for software installation paths information. The -vm option enables VMs and -name option names the workspace. The following shows my laptop environment. The environment variables that begin with VM_ are specific to the remote VMs. All other environment variables without the prefix VM_ pertain to your laptop environment. For example, VM_PADOGRID_HOME is the PadoGrid installation path in the VM hosts you specified.
Please answer the prompts that appear below. If you are not able to complete
the prompts at this time then use the '-quiet' option to bypass the prompts.
You can complete the requested values later in the generated 'setenv.sh' file
You can abort this command at any time by entering 'Ctrl-C'.
Enter Java home path.
[/home/dpark/Padogrid/products/jdk1.8.0_212]:
Enter the local product home directory path. Choose one
from the defaults listed below or enter another.
/home/dpark/Padogrid/products/Oracle/Middleware/Oracle_Home/coherence
Enter workspace name.
[ws-vm]:
Enter default cluster name.
[mycoherence]:
Enable VM? Enter 'true' or 'false' [true]:
Enter VM JDK home path.
[/home/dpark/Padogrid/products/jdk1.8.0_212]:
Enter VM product home path.
[/home/dpark/Padogrid/products/Oracle/Middleware/Oracle_Home/coherence]:
/home/dpark/Padogrid/products/coherence
Enter VM padogrid home path.
[/home/dpark/Padogrid/products/padogrid_0.9.7]:
Enter VM workspaces path.
[/home/dpark/Padogrid/workspaces/myrwe]:
Enter VM host names or IP addresses separated by comma.
[]:
hubuntu1,hubuntu2,hubuntu3,hubuntu4
Enter VM user name.
[]: dpark
Enter VM SSH private key path. If you don't have the key file (.pem) then
leave it blank for now. You can place the file in the workspace directory or
set the path in the workspace 'setenv.sh' file later.
[]:
You have entered the following.
JAVA_HOME: /home/dpark/Padogrid/products/jdk1.8.0_212
PRODUCT_HOME: /home/dpark/Padogrid/products/Oracle/Middleware/Oracle_Home/coherence
PADOGRID_WORKSPACE: /home/dpark/Padogrid/workspaces/myrwe/ws-vm
Default Cluster: mycoherence
VM_ENABLED: true
VM_JAVA_HOME: /home/dpark/Padogrid/products/jdk1.8.0_212
VM_PRODUCT_HOME: /home/dpark/Padogrid/products/Oracle/Middleware/Oracle_Home/coherence
VM_PADOGRID_HOME: /home/dpark/Padogrid/products/padogrid_0.9.7
VM_PADOGRID_WORKSPACES_HOME: /home/dpark/Padogrid/workspaces/myrwe
VM_PADOGRID_WORKSPACE: /home/dpark/Padogrid/workspaces/myrwe/ws-vm
VM_HOSTS: hubuntu1,hubuntu2,hubuntu3,hubuntu4
VM_USER: dpark
VM_PRIVATE_KEY_FILE:
Enter 'c' to continue, 'r' to re-enter, 'q' to quit: c
----------------------------------------------------------------
WORKSPACE: /home/dpark/Padogrid/workspaces/myrwe/ws-vm
Cluster Created: mycoherence
CLUSTER_TYPE: coherence
Deployment: VM
CLUSTER_DIR: /home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mycoherence
CONFIG_DIR: /home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mycoherence/etc
CONFIG_FILE: /home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mycoherence/etc/cache-config.xml
LOG_DIR: /home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mycoherence/log
MEMBER_START_PORT: 9000
DEBUG_START_PORT: 9101
JMX_START_PORT: 12001
PROMETHEUS_START_PORT: 8091
The default config.xml file has been placed in the CONFIG_DIR directory.
Modify it or replace it with the appropriate file.
One (1) member has been configured (added) to run in this cluster as follows:
Member: mycoherence-member-padogrid-01
Working Dir: /home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mycoherence/run/mycoherence-member-padogrid-01
1. Make sure to replace or update cache-config.xml:
/home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mycoherence/etc/cache-config.xml
2. Place your application jar files in one of the 'plugins' directories:
/home/dpark/Padogrid/workspaces/myrwe/ws-vm/plugins/
/home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mycoherence/plugins/
3. Configure cluster properties in:
/home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mycoherence/etc/cluster.properties
4. You must configure password-less ssh login to all VMs. On AWS, this
is already configured but you will need to enter the correct private key
file path in the cluster.properties file.
5. To change VM host names, list the VM host names in the cluster.properties file
for members.
vm.hosts=host1,host2,...
6. Deploy the cluster configuration changes you made to all the VM hosts listed in the
previous step by running 'vm_copy' or 'vm_sync'.
vm_copy /home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mycoherence
7. To start the cluster:
start_cluster -cluster mycoherence
8. To verify the cluster state:
show_cluster -cluster mycoherence
9. To view log file:
show_log -cluster mycoherence
----------------------------------------------------------------
Workspace created:
/home/dpark/Padogrid/workspaces/myrwe/ws-vm
The workspace directory also contains the following files.
initenv.sh - Workspace initialization file. Source this file in to
enter the workspace.
setenv.sh - You can set your workspace-wide environment variables in
this file.
.addonenv.sh - A hidden file containing the workspace environment variables.
You can overwrite them in setenv.sh as needed.
IMPORTANT:
----------
To use this workspace, execute the 'switch_workspace' command as follows:
switch_workspace ws-vm
You have created a VM workspace. To add or remove VM hosts, set VM_HOSTS in
the workspace 'setenv.sh' file. Note that VM_HOSTS applies to the entire workspace
whereas the 'vm.hosts' property in the cluster 'etc/cluster.properties' file applies
only to the defining cluster. By default, 'vm.hosts' is initially set to VM_HOSTS.
/home/dpark/Padogrid/workspaces/myrwe/ws-vm/setenv.sh
SSH Private Key
If a private key file is required and you have not specified it in the previous example, then you can place it in the workspace directory. PadoGrid automatically picks up the first .pem file found in the workspace directory if the VM_PRIVATE_KEY_FILE environment variable is not set in the $PADOGRID_WORKSPACE/setenv.sh file. The following shows the contents of the ws-vm workspace directory we created.
# If a private key file (.pem) is required then place it in in the
# ws-vm workspace directory.
switch_workspace ws-vm
tree -L 1 .
.
├── apps
├── bundles
├── clusters
├── docker
├── initenv.sh
├── k8s
├── lib
├── plugins
├── pods
└── setenv.sh
Test VM Workspace
Before you sync the local workspace with VMs - which may initially take some time to complete - if you need to install software products, we should first test the environment to make sure SSH works properly. This is done by executing vm_test.
vm_test
vm_test outputs the following:
------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------
Workspace: ws-vm
Environment:
VM_ENABLED=true
VM_HOSTS=hubuntu1,hubuntu2,hubuntu3,hubuntu4
VM_USER=dpark
VM_PRIVATE_KEY_FILE=
VM_JAVA_HOME=/home/dpark/Padogrid/products/jdk1.8.0_212
VM_COHERENCE_HOME=/home/dpark/Padogrid/products/coherence
VM_PADOGRID_HOME=/home/dpark/Padogrid/products/padogrid_0.9.7
VM_PADOGRID_WORKSPACE=/home/dpark/Padogrid/workspaces/myrwe/ws-vm
------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------
Network Test:
hubuntu1 - OK
hubuntu2 - OK
hubuntu3 - OK
hubuntu4 - OK
------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------
WARNING:
Java is not installed on the following VMs. The workspace will not be operational
until you install Java on these VMs.
hubuntu1 hubuntu2 hubuntu3 hubuntu4
VM Java Home Path:
/home/dpark/Padogrid/products/jdk1.8.0_212
To install Java on the above VMs, download the correct version of Java and execute 'vm_install'.
Example:
vm_install -java jdk1.8.0_212.tar.gz
------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------
WARNING:
Coherence is not installed on the following VMs. The workspace will not be operational
until you install Coherence on these VMs.
hubuntu1 hubuntu2 hubuntu3 hubuntu4
VM Coherence Path:
/home/dpark/Padogrid/products/coherence
To install Coherence on the above VMs, download the correct version of Coherence and
execute 'vm_install'.
Example:
vm_install -product coherence.tar.gz
------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------
/home/dpark/Padogrid/products/padogrid_0.9.7
padogrid is not installed on the following VMs. It will be automatically installed
when you execute 'vm_sync'.
hubuntu1 hubuntu2 hubuntu3 hubuntu4
------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------
Summary:
One or more VM issues found. Please correct them before executing 'vm_sync'.
Workspace Issues:
None.
Network Issues:
None.
Software Issues:
Java missing from the following VMs. Install Java with 'vm_install -java'.
hubuntu1 hubuntu2 hubuntu3 hubuntu4
Coherence missing from the following VMs. Install Coherence with 'vm_install -product'.
hubuntu1 hubuntu2 hubuntu3 hubuntu4
------------------------------------------------------------------------------------------
vm_test scans all the VMs and reports any issues it encounters. For example, the abovevm_test report shows that SSH sessions are working fine but Java and Coherence are not installed on all of the VMs. To install Java and Coherence on the VMs, we execute the vm_install command as follows.
vm_install -java /home/dpark/Downloads/jdk-8u212-linux-x64.tar.gz \
-product /home/dpark/Downloads/coherence.tar.gz
vm_install outputs the following:
Java installation
Local Source: /home/dpark/Downloads/jdk-8u212-linux-x64.tar.gz
Remote Target: /home/dpark/Padogrid/products/jdk1.8.0_212
Installing hubuntu1...
Installing hubuntu1...
Installing hubuntu2...
Installing hubuntu3...
Installing hubuntu4...
Java installation complete.
Coherence installation
Local Source: /home/dpark/Downloads/coherence.tar.gz
Remote Target: /home/dpark/Padogrid/products/coherence
Installing hubuntu1...
Installing hubuntu1...
Installing hubuntu2...
Installing hubuntu3...
Installing hubuntu4...
Coherence installation complete.
Run 'vm_test' to check installation and configuration status.
If you run vm_test after installing Java and Coherence, you should see no issues in the VM workspace. You can now proceed to sync the VM workspace.
Synchronize VM Workspace
In the previous section, we have locally created and tested a VM workspace. Before we can use it on the remote VMs, we need to synchronize it with the VMs. To do so, we execute the vm_sync command, which synchronizes the workspace you just created and automatically installs PadoGrid on all the VMs if it is not already installed.
vm_sync
vm_sync outputs the following:
Deploying padogrid_0.9.7 to hubuntu1...
Deploying padogrid_0.9.7 to hubuntu2...
Deploying padogrid_0.9.7 to hubuntu3...
Deploying padogrid_0.9.7 to hubuntu4...
Workspace sync: ws-vm
Synchronizing hubuntu1...
Synchronizing hubuntu2...
Synchronizing hubuntu3...
Synchronizing hubuntu4...
Workspace sync complete.
Configure Coherence Cluster
In our example, we have created the default cluster named, mycoherence. Let's switch into the mycoherence cluster.
switch_cluster mycoherence
The cluster directory has the following files.
mycoherence/
├── bin_sh
│ └── setenv.sh
├── etc
│ ├── cache-config.xml
│ ├── cluster.properties
│ ├── hibernate.cfg-mysql.xml
│ ├── hibernate.cfg-postgresql.xml
│ ├── log4j2.properties
│ ├── pof-config.xml
│ ├── prometheus.yml
│ └── tangosol-coherence-override.xml
├── lib
├── log
├── plugins
└── run
etc/cluster.properties
In the mycoherence directory, you will find etc/cluster.properties which defines cluster-level properties. You can edit this file to change VM settings as needed. For example, you can change the list of VM hosts by setting the vm.hosts property or even disable VM altogher by setting the vm.enabled property to run the cluster on your local machine instead.
# Browse the cluster properties and change them as needed.
# Pay attention to the following two (2) properties:
# vm.privateKeyFile=
# vm.hosts=
vi etc/cluster.properties
etc/cache.xml
You can also update the cache-config.xml and tangosol-coherence-override.xml files at this time as needed.
vi etc/cache-config.xml etc/tangosol-coherence-override.xml
If you have other Coherence resource files then place them in the etc/ directory, which is included in CLASSPATH.
Sync Changes
Any changes you made can be easily deployed to all the VMs specified by the VM_HOSTS environment variable in the $PADOGRID_WORKSPACE/setenv.sh by running the vm_sync command, which copies the entire workspace directory to the VMs. Be aware that this can take some time to complete if you have many VMs and large binary files in the workspace.
After the VMs have been synchronized, if you make changes to only a few files, then instead of executing vm_sync again, you can execute vm_copy which copies only the specified file or directory to the VMs.
switch_cluster mycoherence
# To copy a single file
vm_copy etc/cluster.properties
# To copy the entire directory
vm_copy .
vm_sync vs vm_copy
The main difference between vm_sync and vm_copy is that vm_sync copies the entire workspace and vm_copy copies only the specified file or directory. Both commands allow you to apply the changes you made to all of the VMs. As you might have guessed, vm_sync will take longer to complete since it copies the entire workspace, but it is more convenient to use than vm_copy since you don't have to remember the files you modified. These commands will quickly become your favorite commands if you often reconfigure Coherence clusters or update application library files.
:exclamation: Note that vm_copy by default only copies the files that are in the workspace directory hierarchy. If you try to copy a file that is outside of the workspace directory then it will fail and output an error message. If you need to copy non-workspace files, specify the -mirror option, which copies the absolute file path to the same absolute file path in the remote VMs.
Start Cluster
You are now ready to start the mycoherence cluster.
start_cluster
Monitor Cluster
To monitor the cluster:
show_cluster
View Log
To view logs
# hubuntu1
show_log
# hubuntu2
show_log -num 2
Stop/Kill Cluster
To stop or kill the cluster:
stop_cluster
kill_cluster
Running from VMs
You can also run VM workspaces from any of the VMs. As with the local environment, you must first setup the PadoGrid environment on the VM in which you want to run workspaces by sourcing in the workspaces initenv.sh file.
# SSH into one of the VMs
ssh hubuntu1
# Source in intienv.sh
. ~/Padogrid/workspaces/myrwe/initenv.sh
# Optionally add the above line in .bashrc (or .bash_profile) so that PadoGrid is
# automatically initialized when you login next time. Make sure to include the '-quiet' option.
echo ". ~/Coherence/workspaces/myrwe/initenv.sh -quiet" >> ~/.bash_rc
# Once you have initenv.sh sourced in you can then execute any of the PadoGrid
# commands as before. The following starts the `mycoherence` cluster.
switch_workspace ws-vm
switch_cluster mycoherence
start_cluster
Teardown
If you want to remove the cluster from all the VMs, then you must first stop the cluster and execute the remove_cluster command. The workspace can be removed using the remove_workspace command.
# Stop cluster including members and locators
stop_cluster -all
# Remove cluster. Unlike other commands, this command requires the '-cluster' option.
remove_cluster -cluster mycoherence
# Simulate removing workspace from all VMs. Displays removal steps but does not
# actually remove the workspace.
remove_workspace -workspace ws-vm -simulate
# Remove workspace from all VMs. Runs in interactive mode.
remove_workspace -workspace ws-vm
◀️ Coherence perf_test App :link: Hadoop CLASSPATH ▶️ :link: Home ▶️