v3.14.9 Deployment Instructions - OpenWIS/openwis GitHub Wiki
The OpenWIS Installation Guide aims at providing the various steps to install and configure the OpenWIS components. This guide is provided for system administrator as most of the operations require root access.
The Installation Guide describes the installation and configuration process for each OpenWIS component in dedicated sections.
A full distribution of OpenWIS consists of the following artefacts. These are either available from Jenkins or can be built from source:
- openwis-securityservice.war: Security service endpoint.
- openwis-management-service.ear: Management services
- openwis-dataservice.ear: Data Services
- openwis-user-portal.war: Public user portal
- openwis-admin-portal.war: Administration portal
- openwis-portal-solr.war: Solr search index
- stagingPost.war: Staging post
- PopulateLDAP.zip: A tool used during installation to populate users and groups in OpenAM.
- openwis-dependencies.zip: An archive containing various dependencies used by OpenWIS.
OpenWIS requires the following dependencies.
- OpenAM
- OpenDJ
In addition to these, the following system and 3rd party dependencies may be required for specific components:
- OpenJDK 1.8
- Wildfly 15.0.1 Final
- Tomcat 9.0.33
- vsftpd
- httpd
From an installation point of view, deploying a GISC, a DCPC or a NC is equivalent. The only differences reside in configuration (described below).
Each deployment consists in deploying the following components:
- Security Service (OpenDJ, OpenAM and OpenWIS WebServices)
- Database (PostgreSQL)
- Data Service + Cache replication (GISC only)
- Management Service
- Portals and front-end
- Staging Post
OpenWIS installation has the following minimum structure:
It requires 3 tomcat instances. If all of Tomcat instances are at the same OS they must run under different user.
For maintenance or migration purpose, the whole system will need to be restarted. To avoid losing any data / requests, the following process may be performed:
- On Admin portal: stop all local services (in Backup / Availability of Local System) This will prevent any users to perform new requests and the system to ingest and process new subscriptions.
- Wait for all JMS queues to go down to 0 (see section 5.6.1) This will wait for current processing to complete.
- Stop servers (only if necessary) in this order:
- User Portal(s)
- Admin Portal
- Solr
- Security Service
- Data Service
- Postgres
- Perform the maintenance operations
- Start the servers in reverse order (only for servers that have been stopped):
- Postgres
- Data Service
- Security Service
- Solr
- Admin Portal
- User Portal(s)
All OpenWIS components can be installed on RedHat 5.5 or newer. The following system configuration is required:
- Time set in UTC and synchronized with NTP on an external time server
- English language (required by OpenAM installer)
- Firewall settings to allow communication between components
The first point is to ensure that the hostname is properly set.
Make sure you are logged in as root.
cd /etc/sysconfig
vi network
Look for the HOSTNAME line and replace it with the new hostname you want to use.
HOSTNAME=<HOSTNAME>
Next we will edit the /etc/hosts file and set the new hostname.
vi /etc/hosts
The changes to /etc/hosts and /etc/sysconfig/network are necessary to make your changes persistent (in the event of an unscheduled reboot). Now we use the hostname program to change the hostname that is currently set.
hostname <HOSTNAME>
And run it again without any parameters to see if the hostname has changed. hostname Finally we will restart the network to apply the changes we made to /etc/hosts and /etc/sysconfig/network.
service network restart
As root, create a user “openwis”, with the default home directory (/home/openwis).:
useradd openwis
passwd openwis
Changing password for user openwis…
The following Java version is required: OpenJDK 1.8.0
Java installation with yum:
As root:
yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel
If java is already installed, check that java-1.8.0-openjdk-devel.x86_64 is installed (jdk is required for maven compilation) and choose Java 1.8 version as root, using:
alternatives --config java
As openwis,
Set java environment variables. Add these lines in .bash_profile files:
JAVA_HOME=/etc/alternatives/jre_1.8.0_openjdk
export JAVA_HOME
Apply:
source .bash_profile
Verify:
echo $JAVA_HOME
and
java -version
- In this document java 1.8 release is : 1.8.0_242
If git is not installed, run as root:
yum install git-all
Note: OpenWIS versions may require older version of maven. See: Getting-Started#installing-apache-maven
As root:
yum install -y wget unzip
cd /opt/
wget https://downloads.apache.org/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.zip
unzip apache-maven-3.6.3-bin.zip
rm apache-maven-3.6.3-bin.zip
and as user:
cd ~
vim .bash_profile
Add:
export M2_HOME=/opt/apache-maven-3.6.3
export M2=$M2_HOME/bin
export PATH=$M2:$PATH
If needed:
export JAVA_OPTS="-Xms1g -Xmx2g -XX:MaxPermSize=2g"
Apply:
source .bash_profile
Verify:
mvn -version
Outputs:
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /opt/apache-maven-3.6.3
Java version: 1.8.0_242, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre
Default locale: fr_FR, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-957.21.3.el7.x86_64", arch: "amd64", family: "unix"
Maximum Number of Open File Descriptors
OpenWIS system makes intensive use of file access and network connections.
By default, RedHat limits the number of open file descriptor to 1024 by user, which may appear insufficient in case of burst of files or service to process.
Beyond this limit, errors occur in the system such as “Too many open files” or “cannot open connection” to the database, which may cause unexpected and unpredictable problems.
All the components are running as ‘openwis’ user or ‘postgres’ user (database only).
It’s then recommended to increase this limit on all the component hosts.
Test:
ulimit
If limit is below 8192 then configure as root the file /etc/security/limits.conf (system needs to be restarted):
openwis - nofile 8192
postgres - nofile 8192
To check this limit has been applied, log in as openwis or postgres user and check:
ulimit
Should show: 8192
Clone OpenWIS in ~/maven_projects directory:
As openwis,
mkdir maven_projects
cd maven_projects
git clone https://github.com/OpenWIS/openwis.git
cd openwis
git checkout openwis-3.14.10
mvn clean install -Pdependencies,openwis,admin
mvn clean install -Puser
The tests could be skipped with option:
-DskipTests=true
Copy and unzip openwis-dependencies/target/openwis-dependencies.zip to /home/openwis:
cp maven_projects/openwis/openwis-dependencies/target/openwis-dependencies.zip ~
unzip openwis-dependencies.zip
cd ~/openwis-dependencies/data-management-services/
unzip openwis-dataservice-config-files.zip
*Maven build likewise Windows, see Getting-Started Wiki
Note: For this installation the minimum required Tomcat instances is 3
- For Security service and utils
- For Admin portal
- For User portal
Each must be installed *under different user, under different Home and path. Following chapter details installation of 1 instance of Apache Tomcat.
You can download a copy of Tomat 9.0.36 from the project website (or one of the mirrors):
As openwis,
wget https://downloads.apache.org/tomcat/tomcat-9/v9.0.36/bin/apache-tomcat-9.0.36.zip
Once downloaded, perform the following commands in the openwis home directory:
As openwis,
unzip apache-tomcat-9.0.36.zip
chmod a+x apache-tomcat-9.0.36/bin/*.sh
If server is just installed as application: The provided scripts (@/openwis-dependencies/portals) can be used from user to start and stop Tomcat:
As openwis
cp ~/openwis-dependencies/portals/*.sh .
chmod a+x *.sh
Note start/stop scripts update may needed, $CATALINA_HOME must point to Tomcat home instance. Start script example:
#!/bin/sh
#
# Start OpenWIS Tomcat
#
# Settings
export CATALINA_OPTS="-Xmx512m -XX:MaxPermSize=256m"
export CATALINA_HOME=/home/openwis/apache-tomcat-9.0.36
export CATALINA_PID=$CATALINA_HOME/openwis-tomcat.pid
# Check if Tomcat did not crash (with PID file)
if [ -e $CATALINA_PID ]
then
ps -p `cat $CATALINA_PID` &> /dev/null
if [ $? = 0 ]
then
echo "Tomcat still running"
exit 0
else
echo "Tomcat crashed, cleaning remaining PID file"
rm $CATALINA_PID
fi
fi
# Start Tomcat
cd $CATALINA_HOME/bin
./startup.sh
- Memory adjustment
By default, Tomcat is configured to use a maximum of 512 Mbytes of memory for heap space.
If the host allows more memory allocation, it is recommended to increase this maximum value.
Edit start_openwis_tomcat.sh:
export CATALINA_OPTS="-Xmx512m -XX:MaxPermSize=256m"
The following parameters may be adjusted: -Xmx represents the maximum of memory usage (‘m’ means Mbytes)
-
Modify the script "tail_tomcat_log.sh" with the correct tomcat version:
tail -f ~/apache-tomcat-9.0.36/logs/catalina.out
-
Stop/start tomcat:
As openwis
~/start_openwis_tomcat.sh
~/stop_openwis_tomcat.sh
-
To verify Tomcat state, check catalina.out file:
less /home/openwis/apache-tomcat-9.0.36/logs/catalina.out
or:
~/tail_tomcat_log.sh
Tomcat application server may added as system service.
To add Tomcat as a service to start at boot, edit as root:
cd /etc/systemd/system/
vim tomcat.service
Create the 'tomcat.service' file with the following lines:
#!/bin/bash
[Unit]
Description=Apache Tomcat Web Application Server
After=syslog.target network.target
[Service]
Type=forking
User=openwis
Environment="JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk/"
Environment="CATALINA_HOME=/home/openwis/apache-tomcat-9.0.36"
Environment="CATALINA_PID=/home/openwis/apache-tomcat-9.0.36/openwis-tomcat.pid"
Environment="CATALINA_OPTS=-Xms512M -Xmx1024M -server -XX:+UseParallelGC"
ExecStart=/home/openwis/apache-tomcat-9.0.36/bin/startup.sh
ExecStop=/home/openwis/apache-tomcat-9.0.36/bin/shutdown.sh
[Install]
WantedBy=multi-user.target
*CATALINA_HOME, JAVA_HOME and process name may differ dependant on servers purpose.
then :
systemctl daemon-reload
- Tomcat stop/start as service
As root
systemctl start tomcat
systemctl stop tomcat
-
Enable the service to be automatically started at boot:
systemctl enable tomcat
If required to install more than one instance of Tomcat in the same machine
In any case multiple tomcats are added as services, user, processname and paths must not be the same.
Follow 2.7 set different $CATALINA_HOME (and $JAVA_HOME if needed)
then copy scripts and rename them accordingly.
Ex: start_openwis_portals.sh
Edit $CATALINA_HOME value to path of new Tomcat.
-
Locate server.xml in {Tomcat homr}\ conf \
-
Find following similar statements
<Connector port="8080" maxHttpHeaderSize="8192" maxThreads="150" minSpareThreads="25" maxSpareThreads="75" enableLookups="false" redirectPort="8443" acceptCount="100" connectionTimeout="20000" disableUploadTimeout="true"/>
-
About Tomcat’s server.xml file cites it’s runs on port 8080. Change every connector configuration port: Connector port=”8080″ to any other port number. For example:
-
Edit and save the server.xml file. Restart Tomcat. Done
If required to run a Tomcat which must run under different Java than default edit start/stop scripts to use the other Java home
As openwis user:
Download the version of WildFly 15.0.1 from the JBoss website:
cd ~
wget https://download.jboss.org/wildfly/15.0.1.Final/wildfly-15.0.1.Final.tar.gz
Unzip the file in the openwis home directory:
tar -xzf wildfly-15.0.1.Final.tar.gz
Edit ~/.bash_profile and add the following lines:
export JBOSS_HOME=/home/openwis/wildfly-15.0.1.Final
Save and execute
source ~/.bash_profile
Create link to JBOSS_HOME:
ln -s /home/openwis/wildfly-15.0.1.Final wildfly
JBoss is configured to use a maximum of 512 Mbytes of heap space by default. If the host allows more memory allocation, it is recommended to increase this maximum value. The following parameters may be adjusted:
- -Xmx represents the maximum of memory usage (‘m’ means Mbytes) Edit $JBOSS_HOME/bin/standalone.conf to adjust $JAVA_OPTS values:
JAVA_OPTS="-Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m
values example :
JAVA_OPTS="-Xms1024m -Xmx2048m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=768m"
*XX:MaxMetaspaceSize will be increase in case of "ERROR: java.lang.OutOfMemoryError: Metadata space"
The time-zone used by the JVM will need to be explicitly set to UTC. Also add at the end of the file the following line:
JAVA_OPTS=$JAVA_OPTS -Duser.timezone=UTC
The server logs will be written to $JBOSS_HOME/standalone/log/ . If you want to have the logs written to another location, you can replace the empty log directory with a symbolic link to desired log directory:
ln -s $JBOSS_HOME/standalone/log <target log dir>
As user copy WildFly stop/start scripts to user home :
cd ~
cp ~/openwis-dependencies/data-management-services/openwis-dataservice-config/*.sh ~
chmod a+x *.sh
The following configuration values for WildFly are configured as part of the start script and can be changed if necessary: If so, open start_openwis_jboss.sh in a text editor and change the appropriate configuration value.
• bindingAddress: The address WildFly will be bound to. Default = “0.0.0.0”
• multicastAddress: The multicast address used for communication between cluster members. All cluster members must have the same multicast address. Default = “239.255.100.100”
The provided scripts are used from openwis user to start and stop WildFly:
As openwis
./start_openwis_jboss.sh
./stop_openwis_jboss.sh
To verify Wildfly running, check server.log file :
tail -f $JBOSS_HOME/standalone/log/server.log
Wildfly application server may be added as systemd service.
- To add Wildfly as a service, there are 3 files to create/configure.
As root:
wildfly.conf
mkdir -p /etc/wildfly
vim /etc/wildfly/wildfly.config
Add:
# The configuration you want to run
WILDFLY_CONFIG=standalone.xml
# The mode you want to run
WILDFLY_MODE=standalone
# The address to bind to
WILDFLY_BIND=0.0.0.0
launch.sh
vim /home/openwis/wildfly-15.0.1.Final/bin/launch.sh
Add:
#!/bin/bash
echo "Start Wildfly"
echo "Configuration file: $2"
echo "Bind address: $3"
CONF_FILE="$2"
if [ "x$CONF_FILE" = "x" ]; then
CONF_FILE="standalone-full.xml"
fi
if [ "x$OPENWIS_HOME" = "x" ]; then
OPENWIS_HOME="/home/openwis"
fi
if [ "x$WILDFLY_HOME" = "x" ]; then
WILDFLY_HOME="$OPENWIS_HOME/wildfly"
fi
if [[ "$1" == "domain" ]]; then
$WILDFLY_HOME/bin/domain.sh -c $2 -b $3
else
$WILDFLY_HOME/bin/standalone.sh -c $CONF_FILE -b $BIND_ADDRESS
fi
Make the file executable:
chmod a+x /home/openwis/wildfly-15.0.1.Final/bin/launch.sh
wildfly.service
vim /etc/systemd/system/wildfly.services
Add:
[Unit]
Description=The WildFly Application Server
After=syslog.target network.target
[Service]
Environment=LAUNCH_JBOSS_IN_BACKGROUND=1
EnvironmentFile=-/etc/wildfly/wildfly.conf
User=openwis
LimitNOFILE=102642
PIDFile=/var/run/wildfly/wildfly.pid
ExecStart=/home/openwis/wildfly/bin/launch.sh $WILDFLY_MODE $WILDFLY_CONFIG $WILDFLY_BIND
StandardOutput=journal
[Install]
WantedBy=multi-user.target
Notify systemd for the new unit file:
sudo systemctl daemon-reload
Start the wildfly service:
systemctl start wildfly
Check the service status:
systemctl status wildfly
Output:
[root@linux]# systemctl status wildfly
● wildfly.service - The WildFly Application Server
Loaded: loaded (/etc/systemd/system/wildfly.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2020-06-11 20:01:18 CEST; 18min ago
Main PID: 4124 (launch.sh)
CGroup: /system.slice/wildfly.service
├─4124 /bin/bash /home/openwis/wildfly/bin/launch.sh standalone standalone-full.xml 0.0.0.0
├─4125 /bin/sh /home/openwis/wildfly/bin/standalone.sh -c standalone-full.xml -b 0.0.0.0
└─4182 java -D[Standalone] -server -Xms1024m -Xmx2048m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=768m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.a...
Stop the wildfly service:
systemctl stop wildfly
Check wildfly systemd log:
journalctl -u wildfly
Or, in order to see the last info:
journalctl -u wildfly --since "5 minutes ago"
Optional: redirect output of systemd service to a file
Replace in /etc/systemd/system/wildfly.service
, StandardOutput=journal
with the 3 following lines:
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=wildfly
Then:
sudo systemctl daemon-reload
Create the file /etc/rsyslog.d/wildfly.conf
:
vim /etc/rsyslog.d/wildfly.conf
Add:
if $programname == 'wildfly' then /var/log/wildfly/wildfly.log
& stop
Then:
systemctl restart rsyslog
And a copy of the wildfly log is located in /var/log/wildfly/ :
ll /var/log/wildfly/
total 224
-rw-r--r-- 1 root root 11190 Apr 9 10:50 console.log
-rw------- 1 root root 216544 Jun 11 20:26 wildfly.log
-
Enable the service to be automatically started at boot:
systemctl enable wildfly
Dataservice deployment is described in chapter 5
The Security Service deployment is composed of:
- OpenDJ (LDAP server)
- OpenAM
- OpenWIS Security Services (User & Group management) openJDK 7 is required.
Note: This has been developed and tested using openJDK7_0.65
At the following part there are lots of configurations to be done.
- Tomcat logs (OpenDJ, OpenAM, OpenWIS Security Services)
- JBoss logs (openwis-dataservice, openwis-management-service)
- OpenDJ logs located at:
/home/openwis/opendj/logs
- Openam logs located at:
/home/openwis/openam/log
or/home/openwis/openam/openam/debug
The following sections describe how to install OpenDJ 2.6.0.
Download OpenDJ-2.6.0.zip:
cd ~
wget https://...OpenDJ-2.6.0.zip
As openwis user, unzip this Archive in /home/openwis directory.
unzip OpenDJ-2.6.0.zip
Run OpenDJ setup:
cd opendj
for a GUI installation
./setup
for a command line installation
./setup --cli
The following steps depend on the way the installation is done: with graphical capabilities or without command line.
The splash screen is displayed.
The Welcome page is displayed, click Next
Accept the license, click Next.
Enter the hostname, the ldap listener port, the administration connector port and login/password for root User DN
Select Standalone server
Set Directory Base DN to dc=opensso,dc=java,dc=net
Leave default runtime settings, click Next.
Click on Finish
Installation is processing
OpenDJ 2.6.0 is now installed and ready for OpenAM installation.
Please take note that the installation script has some problems with:
- copy-pasting
- using backspace
- using space
User will be promoted to:
Please read the License Agreement above.
You must accept the terms of the agreement before continuing with the
installation.
Accept the license (Yes/No) [No]:
Enter: "Yes" to accept the license
What would you like to use as the initial root user DN for the Directory
Server? [cn=Directory Manager]:
Leave the default value [press Enter]
Please provide the password to use for the initial root user:
Please re-enter the password for confirmation:
Provide the fully-qualified directory server host name that will be used when
generating self-signed certificates for LDAP SSL/StartTLS, the administration
connector, and replication [srv-openwis-vm2.silogic.fr]:
Check fully qualified name of server
On which port would you like the Directory Server to accept connections from
LDAP clients? [1389]:
Leave the default value
On which port would you like the Administration Connector to accept
connections? [4444]:
Leave the default value
Do you want to create base DNs in the server? (yes / no) [yes]:
Leave the default value
Provide the base DN for the directory data:
Write: dc=opensso,dc=java,dc=net
Options for populating the database:
1) Only create the base entry
2) Leave the database empty
3) Import data from an LDIF file
4) Load automatically-generated sample data
Enter choice [1]: 1
Do you want to enable SSL? (yes / no) [no]:
Do you want to enable Start TLS? (yes / no) [no]:
Do you want to start the server when the configuration is completed? (yes /
no) [yes]:
Setup Summary
=============
LDAP Listener Port: 1389
Administration Connector Port: 4444
LDAP Secure Access: disabled
Root User DN: cn=Directory Manager
Directory Data: Create New Base DN dc=opensso,dc=java,dc=net.
Base DN Data: Only Create Base Entry (dc=opensso,dc=java,dc=net)
Start Server when the configuration is completed
What would you like to do?
1) Set up the server with the parameters above
2) Provide the setup parameters again
3) Print equivalent non-interactive command-line
4) Cancel and exit
Enter choice [1]: 1
Configuring Directory Server ..... Done.
Creating Base Entry dc=opensso,dc=java,dc=net ..... Done.
Starting Directory Server ....... Done.
See /tmp/opendj-setup-XXXXX.log for a detailed log of this operation.
To see basic server configuration status and configuration you can launch ~/opendj/bin/status
To start OpenDJ:
cd opendj/bin
./start-ds
or ~opendj/bin/start-ds
To stop OpenDJ:
cd opendj/bin
./stop-ds
or
~opendj/bin/stop-ds
Verify installation :
via GUI console:
~/opendj/bin/control-panel&
via command line:
~/opendj/bin/status
Warning: At status results Data sources Entries must not be 0.
--- Data Sources ---
Base DN: dc=opensso,dc=java,dc=net
Backend ID: userRoot
Entries: 0
Replication:
If so create new file named make-basedn.ldif:
Add:
dn: dc= opensso,dc=java,dc=net
objectClass: domain
objectClass: top
dc: opensso
If so a Data Source entry must be added manually, execute:
./ldapmodify -p 1389 -D "cn=Directory Manager" -w openwis -f make-basedn.ldif –a
OpenDJ status results at the end of OpenWIS installation
Install OpenDJ as service.
As root:
cp /home/openwis/openwis-dependencies/security/opendj /etc/init.d/
/sbin/chkconfig --add opendj
/sbin/chkconfig opendj on
Import SchemaOpenWIS.ldif
As user:
cd /home/openwis/opendj/bin
./ldapmodify -h localhost -p 4444 -D "cn=Directory Manager" -w <LDAP_PASSWORD> -a -X --useSSL -f /home/openwis/openwis-dependencies/security/SchemaOpenWIS.ldif
If replication is active the schema must be replicated on each instance (to have all the attributes of OPENWIS)
The following sections describe how to install release 12.0.0 OpenAM is deployed on Tomcat. Report to section 2.7 for details on Tomcat installation.
Configure Tomcat 7 as per usual for your environment:
- Start/Stop scripts to adapt
- Include UTF-8 configuration
- Include VM memory settings
- Tomcat failover consideration:
If failover is configured for component deployed in Tomcat 7 (such as OpenAM), the deployment of the component will be done on two Tomcat instances that run simultaneously.
The Web accessed will be done via a load balancer, such as Apache front-end with mod_proxy module (described in a dedicated section).
To enable session affinity configuration on the load balancer, the Tomcat instances needs to be configured:
vi /home/openwis/apache-tomcat-7.0.59/conf/server.xml
Set the jvmRoute attribute on Engine element:
<Engine name="Catalina" defaultHost="localhost" jvmRoute="node1">
Download the patched version of OpenAM from the middleware repository:
cd ~
wget https://repository-openwis-association.forge.cloudbees.com/private/binaries/openam-patched.war
Deploy it to the Tomcat container:
mkdir /home/openwis/apache-tomcat-9.0.36/webapps/openam
cp openam-patched.war /home/openwis/apache-tomcat-7.0.59/webapps/openam/
cd /home/openwis/apache-tomcat-7.0.59/webapps/openam
jar xvf openam-patched.war
Note: openam-patched.war is based on OpenAM 12.0.0. It includes a new JSP (spGetToken.jsp), used by OpenWIS portal to retrieve tokens.
Pre-requisite: Tomcat and OpenDJ are started.
Go on OpenAM Installation Page
http://<HOST_NAME>:<PORT>/openam/
- WARNING: Don’t use localhost or IP for OpenSSO URL. You have to use the fully qualified Hostname.
- WARNING: the System time between each entity must be synchronized with UTC time.
- WARNING: When configuring an installation with 2 OpenAM instances, each OpenAM instance must be installed with its own distinct hostname. The frontend will only be setup during “site configuration”.
Using HTTPS To setup OpenAM using HTTPS, configure first the Apache front-end, and perform the whole installation process via HTTPS instead of HTTP (IDP Discovery included).
Create a new custom configuration and accept the license.
Fill the password of the amAdmin user (OpenAM administrator)
Keep default values and press next
Leave default values.
Select OpenDJ as data store type -- WARNING: uncheck OpenDJ and check again OpenDJ (bug OpenAM)
Change only:
-
The port to 1389.
-
Root suffix to:
dc=opensso,dc=java,dc=net
-
Fill the password used during the OpenDJ installation.
Leave default values.
Set the default policy agent (not used in OpenWIS).
After installation is complete:
Connect the OpenAM console with User Name amAdmin
Password the one set during the previous step (step 1).
Login page timeout configuration OpenAM defines a timeout on the Login Page itself (before being logged in) and on timeout, the context is lost by OpenAM preventing the redirect back to the portal after a successful login. This is related to an OpenAM bug The bug should be fixed in next versions of OpenAM but the fix does not apply on the current OpenAM version.
As a workaround, the following configuration allows to increase the login page timeout to one week. On OpenAM console:
- Go to Configuration / Servers and Sites
- Edit server configuration and go to Session Tab
- Click on Inheritance Settings, uncheck Invalidate Session Max Time, then Save and go Back to Server profile
- Set Invalidate Session Max Time to 10081 and save
After installation go to :
cd ~/openam/openam/log/
and execute:
> amAuthentication.error
It appears this file is missing from current openAM version and must be added manually.
Start OpenDJ control panel to check OpenAM and OpenDJ are linked. The following steps depend on the way the installation is done either with graphical capabilities or without (command line).
cd ~/opendj/bin
./control-panel&
Click on Directory Data / Manage Entries
Check the tree structure is equivalent to this screen.
cd ~/opendj/bin
Replace “HOST-NAME” with your server’s hostname and execute:
./ldapsearch --hostname HOST-NAME --port 1389 --baseDN "dc=opensso,dc=java,dc=net" "(objectclass=*)" @groupOfUniqueNames
As user go
cd ~/opendj/bin
Copy from openam deployment find opendj_user_schema.ldif like:
cp ~/apache-tomcat-7.0.59/webapps/openam/WEB-INF/template/ldif/opendj/opendj_user_schema.ldif .
and after replacing HOSTNAME and PASSWORD execute:
./ldapmodify -h HOSTNAME -p 4444 -D "cn=Directory Manager" -w PASSWORD -a -X --useSSL -f ./opendj_user_schema.ldif
outputs:
Processing MODIFY request for cn=schema MODIFY
operation successful for DN cn=schema
As user install SSO Admin Tools as follows:
cd /home/openwis
mkdir ssoAdminTools
cd ssoAdminTools
Download the SSO Admin Tools from the middleware repository:
wget https://repository-openwis-association.forge.cloudbees.com/private/middleware/ssoAdminTools.zip
Unzip the tools and run the setup:
unzip ssoAdminTools.zip
./setup
After license acceptance, you will be prompted for:
Do you accept the license?
Write yes
Path to config files of OpenAM server [/home/openwis/openam]:
Enter
Debug Directory [/home/openwis/ssoAdminTools/debug]:
Enter
Log Directory [/home/openwis/ssoAdminTools/log]:
Enter
The scripts are properly setup under directory: /home/openwis/ssoAdminTools/openam
Debug directory is /home/openwis/ssoAdminTools/debug.
Log directory is /home/openwis/ssoAdminTools/log.
The version of this tools.zip is: OpenAM 12.0.0
The version of your server instance is: OpenAM 12.0.0 Build 11961 (2014-December-17 21:16)
cd /home/openwis/ssoAdminTools/openam/bin
Create a file containing the amAdmin user's password named passwd
echo "YOUR_amAdmin_PASSWORD" > passwd
Change the mode of passwd. It must be read-only by owner.
chmod 400 passwd
Reload iPlanetAMUserService: First Delete the service iPlanetAMUserService
./ssoadm delete-svc -u amadmin -f passwd -s iPlanetAMUserService
Service was deleted.
Then, add it back again
./ssoadm create-svc -u amAdmin -f passwd --xmlfile /home/openwis/openwis-dependencies/security/amUser.xml
outputs:
Service was added.
cd /home/openwis/ssoAdminTools/openam/bin
cp /home/openwis/openwis-dependencies/security/attrs.properties .
Edit attrs.properties and set the values:
- idrepo-ldapv3-config-ldap-server=YOUR_HOSTNAME:1389
- sun-idrepo-ldapv3-config-authpw to the LDAP password for cn=Directory Manager
Remove deprecated entry:
- sun-idrepo-ldapv3-ldapv3OpenDS
Next update the data store configuration Open OpenAM in a browser and find Data store name : Go to Access Control, select the / realm and then open Data Stores:
Current installation example <OpenDJ datastore name>
is 'Sun DS with AM schema'
execute likewise:
./ssoadm update-datastore -u amadmin -f passwd -e / -m 'Sun DS with AM schema' -D attrs.properties
Datastore profile was updated.
Check OpenAM / Data Store connection In OpenAM: Go to Access Control, select the / realm Go to Data Stores, select OpenDJ
The “LDAP User Attributes section” must contain the OpenWIS attributes (ex OpenWISNeedUserAccount)
The “Attribute Name for Group Membership” must contain isMemberOf
Select “Load schema when saved” and click on Save.
If the message “Profile was updated” appears, the connection is OK.
Otherwise, check the configuration on this page (LDAP host, port and password)
The following sections describe how to define a Circle of Trust via OpenAM, which will include:
- The hosted Identity Provider (IdP): the local IdP provided by this OpenAM
- An optional remote IdP, when IdP federation is required
- The IdP Discovery, allowing to discover an IdP from the portal
- The service providers, which are the OpenWIS portals
Connect to OpenAM
http://<HOST_NAME>:<PORT>/openam
Click on “Common Tasks” tab.
Select Create Hosted Identity Provider
Enter IDP Name: IDP1 Select “test” as Signing key Enter Circle of Trust name: e.g. “cot_openwis” Click On Configure
Click on Finish
Verify at:
Federation
Entity Providers
Download .war from the middleware repository and deploy idpdiscovery.war into Tomcat, as openwis user:
cd ~
wget https://repository-openwis-association.forge.cloudbees.com/private/middleware/idpdiscovery.war
cp idpdiscovery.war /home/openwis/apache-tomcat-7.0.59/webapps
Configuration Connect to configuring IDP Discovery Service:
http://<IDP_DISCOVERY_HOST_NAME>:<PORT>/idpdiscovery/Configurator.jsp
Set:
- /home/openwis/idpdiscovery as debug directory
- Persistent as cookie type Click on Configure.
Go to OpenAM
http://<IDP_HOST-NAME>:<IDP_PORT>/openam
Click on “Federation” tab. Edit Circle Of trust by clicking on its name ex. cot_openwis Add SAML2 reader and writer urls
like example:
http://openwisServer.com:8080/idpdiscovery/saml2writer
SAML2 Writer Service URL: http://
IDP_DISCOVERY_HOST_NAME
:PORT
/idpdiscovery/saml2writerSAML2 Reader Service URL: http://
IDP_DISCOVERY_HOST_NAME
:PORT
/idpdiscovery/saml2reader
Note: If an Apache frontend is used to access OpenAM / IdpDiscovery, set the external host name / port configured on the front-end for these services.
The WAR of OpenWIS Security Service and the configuration file are deployed in Tomcat that contains OpenAM. As openwis user:
cd ~
mkdir /home/openwis/apache-tomcat-7.0.59/webapps/openwis-securityservice
cd /home/openwis/apache-tomcat-7.0.59/webapps/openwis-securityservice
cp ~/openwis/openwis-securityservice/openwis-securityservice-war/target/openwis-securityservice.war .
jar xvf openwis-securityservice.war
Edit the configuration file located at:
nano ./WEB-INF/classes/openwis-securityservice.properties
The following parameters need to be set, replace all '@LABELS@' with your system's appropriate :
- ldap_host: set the hostname where OpenDJ has been installed
- ldap_port: keep 1389
- ldap_user: keep cn=Directory Manager
- ldap_password: set the LDAP admin password configured during OpenDJ installation
- @managementServiceServer@ : set ManagmentService server with port number(currently jboss. ex: http://openwis.server.com:8180/openwis-management-service-openwis-management-service-ejb/AlertService?wsdl )
- @openam.baseDir@ set openam home ex: /home/openwis/openam/openam/log/amAuthentication.error
The following optional parameters may also be set:
- register_users_threshold: number of registered users beyond which an alarm is raised
- global_groups: the default Global groups (comma separated list)
- log.timer.file: the OpenAM file to check for authentication failure
- openwis.management.alertservice.wsdl: the WSDL location of the alert service
The OpenWIS Security Web Services will accessible on URL:
http://<host>:<port>/openwis-securityservice/services/<SERVICE>?wsdl
For example:
http://your_host:8080/openwis-securityservice/services/UserManagementService?wsdl
When deploying a new Centre, the LDAP must be initialized. The provided OpenWIS distribution contains a tool called “PopulateLDAP”, which can be used to perform the LDAP initialization when defining a new Centre.
Zip file is located at .../openwis-securityservice/openwis-securityservice-utils/PopulateLDAP/target/ PopulateLDAP.zip
As openwis user install PopulateLDAP, for example on the IdP:
cd ~
mkdir PopulateLDAP
cd PopulateLDAP
cp ~/openwis/openwis-securityservice/openwis-securityservice-utils/PopulateLDAP/target/PopulateLDAP.zip .
unzip PopulateLDAP.zip
chmod a+x populateLDAP.sh
Edit script ./populateLDAP.sh to set your system's properites,
-
Update CLASSPATH to be consistent with your current version
openwis-securityservice-utils-populate-ldap-@[email protected]
you can find jar version information by executing:find ~ -name openwis-securityservice-utils-populate-ldap*
-
Replace @OPENWIS_SECURITY_HOST@:@PORT@ with your current system setting.
Run the script as user:
./populateLDAP.sh
- Enter 1, to create a new Centre
- Enter the Centre name (name of the deployment name: e.g. GiscMF) Enter the Administrator login, password, email, First name, Last name
./populateLDAP.sh
------ Populate LDAP ------ [1] Initialize Deployment [2] Populate LDAP [3] Reset LDAP Users [4] Reset LDAP Groups [5] Reset LDAP Users And Groups Choose one of the above values. (any other value to exit) 1 Initialize Deployment: Deployment Name: openwis Administrator Login: openwis Administrator Password: <admin password> Administrator Email: <Email> Administrator First name: <admin first name> Administrator Last name: <admin last name>
PostgreSQL 10 (10.0 is the tested and supported version) is required along with PostGIS 2.4 or 2.5 and the CIText extension. These are provided as RPM repositories from PostgreSQL. It is recommended to use these repositories to install the database to simplify the process of installing and enabling the extensions required by OpenWIS
If any help needed see : postgresonline.com
Select the appropriate repository for the database version you wish to use and the OS distribution you are installing the database on. The postgresql10 applicable is listed below:
PostgreSQL Version | Operation System | URL |
---|---|---|
10.0 | RHEL 7+ | https://download.postgresql.org/pub/repos/yum/10/redhat/rhel-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm |
URLs of other repositories can be found at: http://yum.postgresql.org/
Install the repository using rpm. Use the URL specific to your OS (e.g. the following is for RHEL7 x86_64):
As root:
rpm -ivh https://download.postgresql.org/pub/repos/yum/10/redhat/rhel-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
Install the PostgreSQL packages, including (the contrib package ?) the PostGIS package:
As root:
yum install -y epel-release
yum install -y postgresql10-server postgis24_10 postgis24_10-client postgis24_10-utils
... Complete!
Additional packages:
hdf5.x86_64 -> already installed and latest version
json-c.x86_64 -> already installed and latest version
yum install proj.x86_64
yum install gdal.x86_64
... Complete!
Optional, install "pspg", a usefull pager for psql:
yum install pspg
As root:
systemctl enable postgresql-10
The process of initializing the database depends on whether you are using a custom data directory. • If you are using a custom directory, run the initdb script as the “postgresql” user. Include the full path of the directory as the argument to the “-D” option:
As root:
su - postgres -c '/usr/pgsql-10/bin/initdb --data-checksums -D ~/10/data'
... Success.
Then after initdb, the data and log repertories are:
PGDATA=/var/lib/pgsql/10/data
PGLOG=/var/lib/pgsql/10/data/log
systemctl start postgresql-10
Verify that PostgreSQL is up and running:
systemctl status postgresql-10
[root@centos76 ~]# systemctl status postgresql-10
● postgresql-10.service - PostgreSQL 10 database server
Loaded: loaded (/usr/lib/systemd/system/postgresql-10.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2020-05-05 12:01:43 CEST; 2 days ago
Docs: https://www.postgresql.org/docs/10/static/
Process: 13311 ExecStartPre=/usr/pgsql-10/bin/postgresql-10-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
Main PID: 13317 (postmaster)
CGroup: /system.slice/postgresql-10.service
├─13317 /usr/pgsql-10/bin/postmaster -D /var/lib/pgsql/10/data/
├─13319 postgres: logger process
├─13321 postgres: checkpointer process
├─13322 postgres: writer process
├─13323 postgres: wal writer process
├─13324 postgres: autovacuum launcher process
├─13325 postgres: stats collector process
├─13326 postgres: bgworker: logical replication launcher
├─19055 postgres: openwis OpenWIS 127.0.0.1(36236) idle
├─19064 postgres: openwis OpenWIS 127.0.0.1(36242) idle
└─19067 postgres: openwis OpenWIS 127.0.0.1(36248) idle
May 05 12:01:42 centos76 systemd[1]: Starting PostgreSQL 10 database server...
May 05 12:01:43 centos76 postmaster[13317]: 2020-05-05 12:01:43.444 CEST [13317] LOG: listening on IPv4 address "0.0.0.0", port 5432
May 05 12:01:43 centos76 postmaster[13317]: 2020-05-05 12:01:43.445 CEST [13317] LOG: listening on IPv6 address "::", port 5432
May 05 12:01:43 centos76 postmaster[13317]: 2020-05-05 12:01:43.456 CEST [13317] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
May 05 12:01:43 centos76 postmaster[13317]: 2020-05-05 12:01:43.461 CEST [13317] LOG: listening on Unix socket "/tmp/.s.PGSQL.5432"
May 05 12:01:43 centos76 postmaster[13317]: 2020-05-05 12:01:43.688 CEST [13317] LOG: redirecting log output to logging collector process
May 05 12:01:43 centos76 postmaster[13317]: 2020-05-05 12:01:43.688 CEST [13317] HINT: Future log output will appear in directory "log".
May 05 12:01:43 centos76 systemd[1]: Started PostgreSQL 10 database server.
Note: ** For the rest of the install instructions, to log in as postres user**, login as root and then execute
su - postgres
As the “postgres” user
Create the "openwis" postgres user. The "openwis" user should not be a super user and should not need to create any separate databases or user roles. The "openwis" user, however, may need to create new tables or modify existing tables:
createuser -P openwis
Enter password for new role: <enter password>
Enter it again: <enter password again>
createdb -O openwis OpenWIS
$ psql -d OpenWIS -c 'create extension postgis;'
CREATE EXTENSION
$ psql -d OpenWIS -c 'create extension postgis_topology;'
CREATE EXTENSION
$ psql -d OpenWIS -c 'create extension citext;'
CREATE EXTENSION
cd /usr/pgsql-10/share/contrib/postgis-2.4/
psql -d OpenWIS -U postgres -f legacy.sql
CREATE FUNCTION
...
...
CREATE AGGREGATE
psql -d OpenWIS -U openwis
OpenWIS=> select postgis_full_version();
NOTICE: Topology support cannot be inspected. Is current user granted USAGE on schema "topology" ?
postgis_full_version
POSTGIS="2.4.8 r17696" PGSQL="100" GEOS="3.8.1-CAPI-1.13.3" PROJ="Rel. 7.0.0, March 1st, 2020" GDAL="GDAL 3.0.4, released 2020/01/28" LIBXML="2.9.1" L IBJSON="0.11" LIBPROTOBUF="1.0.2" RASTER (1 row)
OpenWIS=> select * from pg_available_extensions where name like '%postgis%';
postgis_sfcgal | 2.4.8 | | PostGIS SFCGAL functions postgis | 2.4.8 | 2.4.8 | PostGIS geometry, geography, and raster spatial types and functions postgis_tiger_geocoder | 2.4.8 | | PostGIS tiger geocoder and reverse geocoder postgis_topology | 2.4.8 | 2.4.8 | PostGIS topology spatial types and functions (4 rows)
OpenWIS=> select name, installed_version as ver, comment from pg_available_extensions where installed_version is not null;
citext | 1.4 | data type for case-insensitive character strings postgis | 2.4.8 | PostGIS geometry, geography, and raster spatial types and functions postgis_topology | 2.4.8 | PostGIS topology spatial types and functions plpgsql | 1.0 | PL/pgSQL procedural language (4 rows)
-
Quit OpenWIS DB:
OpenWIS=#\q
With large catalogues and spatial searches, very large SQL queries can be generated on the PostGIS spatial index table. This can cause PostgreSQL to exceed the default process stack size limit. You will know when this happens because a very long SQL query will be output to the SolR log file prefixed with a cryptic message something along the lines of “java.util.NoSuchElementException: Could not acquire feature:org.geotools.data.DataSourceException: Error Performing SQL query: SELECT .........”. It is therefore necessary to raise the stack size limits of the PostgreSQL database:
Get the permitted stack size of the postgresql user. The result will be return in kilobytes (1,024 bytes).
ulimit -s
10240 # KB = 10 MB
As root open “/var/lib/pgsql/10/data/postgresql.conf” in a text editor and search for the key “max_stack_depth”. Uncomment the value if it is not commented out, and raise this to about 80% of the value reported by ulimit:
max_stack_depth = 8MB
As root, restart PostgreSQL for the changes to take effect.
systemctl start postgresql-10
Verify as root :
systemctl status postgresql-10
If the error still appears these values need to be increased.
The OpenWIS distribution comes with two scripts: schema.dll and purge.sql. The schema.dll script is used to create the OpenWIS database tables used by the data services (the tables used by the user portals are created automatically on first start). Both of these scripts are located in “openwis-dependencies.zip”:
/home/openwis/openwis-dependencies/database/purge.sql
/home/openwis/openwis-dependencies/database/schema.ddl
As the “openwis” user:
- Openwis DB SCHEMA
The “schema.dll” file from “openwis-dependencies/database” needs to be loadded into the OpenWIS database:
psql -d OpenWIS -f /home/openwis/openwis-dependencies/database/schema.ddl
- Purge script
A purge script purge.sql is provided in openwis-dependencies/database. It allows to purge the blacklisting entries for users having no activity (request/subscription) since 30 days (default value, can be changed). This is needed in particular because the data service is not strictly related to a local user referential, and can manage requests/subscriptions of user belonging to remote Centre (via Remote Authentication). The user lifecycle (creation/deletion) is then not correlated to the Data Service. To execute the purge:
psql -d OpenWIS -f /home/openwis/openwis-dependencies/database/purge.sql
As postgres user, locate pg_hba.conf:
-
connect at postgres DB:
$ psql
psql (10.12)
Type "help" for help.
-
execute:
postgres=# show hba_file ;
hba_file ------------------------------------ /var/lib/pgsql/10/data/pg_hba.conf
(1 row)
-
Exit postgres DB console
postgres=#\q
-
Edit the pg_hba.conf to allow access to these components according to installation needs:
vim /var/lib/pgsql/10/data/pg_hba.conf
trust Allow the connection unconditionally. This method allows anyone that can connect to the PostgreSQL database server to login as any PostgreSQL user they like, without the need for a password.
reject Reject the connection unconditionally. This is useful for "filtering out" certain hosts from a group.
md5 Require the client to supply an MD5-encrypted password for authentication.
password Require the client to supply an unencrypted password for authentication. Since the password is sent in clear text over the network, this should not be used on untrusted networks.
gss Use GSSAPI to authenticate the user. This is only available for TCP/IP connections.
sspi Use SSPI to authenticate the user. This is only available on Windows.
krb5 Use Kerberos V5 to authenticate the user. This is only available for TCP/IP connections.
ident Obtain the operating system user name of the client (for TCP/IP connections by contacting the ident server on the client, for local connections by getting it from the operating system) and check if it matches the requested database user name.
ldap Authenticate using an LDAP server.
cert Authenticate using SSL client certificates.
pam Authenticate using the Pluggable Authentication Modules (PAM) service provided by the operating system.
Peer Obtain the client's operating system user name from the operating system and check if it matches the requested database user name. This is only available for local connections.
- For more info about pg_hba.conf file, see : https://www.postgresql.org/docs/10/auth-pg-hba-conf.html
-
Edit the postgresql.conf
vim /var/lib/pgsql/10/data/postgresql.conf
Set listen_addresses to the host IP address (or ‘*’) if the database needs to be accessible from outside this host.
-
restart postgresql
systemctl restart postgresql-10
The Data Service and Management Service run in Wildfly 15.0.1.
Note: Openwis application could be installed in /var/opt/openwis/ directory.
In this wiki, the middlewares and Openwis components are located in /home/openwis/
As openwis:
-
Create directory for dataservice properties files
mkdir /home/openwis/conf
-
Create directory for data
mkdir /home/openwis/harness mkdir /home/openwis/harness/incoming mkdir /home/openwis/harness/ingesting mkdir /home/openwis/harness/ingesting/fromReplication mkdir /home/openwis/harness/outgoing mkdir /home/openwis/harness/working mkdir /home/openwis/harness/working/fromReplication mkdir /home/openwis/cache mkdir /home/openwis/stagingPost mkdir /home/openwis/temp mkdir /home/openwis/replication mkdir /home/openwis/replication/sending mkdir /home/openwis/replication/sending/local mkdir /home/openwis/replication/sending/destinations mkdir /home/openwis/replication/receiving mkdir /home/openwis/status
These folders correspond to:
- Incoming: folder on which incoming GTS files are dropped and consumed by OpenWIS dispatch process
- Ingesting: folder containing files to ingest (after dispatch)
- Outgoing: folder on which OpenWIS feeds files to the GTS
- Working: internal working folder
- Cache: the cache root directory
- StagingPost : the staging post root directory
- Temp : the directory for temporarily stored files
- Replication: the directory for the replication folders, with the following sub-structure:
- sending / local: contains a buffer of 24 hours data received from GTS and that needs to be sent to replication destination
- sending / destinations /
<DESTINATION_NAME>
: contains links to sending/local data that are currently sent to the given destination - receiving /
<SOURCE_NAME>
: contains files received from the replication from the given source - Status: the directory containing status files (status of each services)
Depending on the deployment, all these directories, and in particular the Cache and Staging Post directories, may correspond to mounted folders, targeting network shared locations (NFS, Samba…).
The components and config files are available for deploying after the Maven compilation described at 2.4.3
-
Copy property files in /home/openwis/conf/
cp ~/openwis-dependencies/data-management-services/openwis-dataservice-config/config/localdatasourceservice.properties ~/conf/ cp ~/openwis-dependencies/data-management-services/openwis-dataservice-config/config/openwis-dataservice.properties ~/conf/
Make appropriate changes to the configuration files located in ~/conf. The following files can be configured:
localdatasourceservice.properties:
For a GISC deployment, leave the values commented
For a DCPC, configure the LocalDataSource.
- Identification of the local data source: o Key: logical name of the data source o Value: URL of the WSDL corresponding to the Local Data Source Web Service
- Polling mechanism (to detect availability of new products) o Key:
<logical name of the data source>
.polling o Value: true or false to enable/disable the polling of the data source
openwis-dataservice.properties:
- For both GISC and DCPC:
Adapt the file "openwis-dataservice.properties" to your deployment.
This configuration section contains many parameters that can be configured. Placeholders surrounded with “@” will need to be replaced.
Most of typical deployments will have to configure only the basic ones, shown in the table below.
Note: ejb JNDI names in "openwis-dataservice.properties"
Replace in each line containing "ejb" ejb:
with java:global/
as the following example:
extraction.timer.url=ejb:openwis-dataservice/openwis-dataservice-server-ejb/ExtractionTimerService!org.openwis.dataservice.common.timer.ExtractionTimerService
Becomes
extraction.timer.url=java:global/openwis-dataservice/openwis-dataservice-server-ejb/ExtractionTimerService!org.openwis.dataservice.common.timer.ExtractionTimerService
The following table describes the parameters
Data Service Directories Locations | description |
---|---|
dataService.baseLocation | Base location of the data services data directory (e.g. “/home/openwis”). |
cache.dir.stagingPost | The path to the staging post directory. (same as ‘staging.post.uri’ value) |
cache.dir.temp | The path to the temporary directory |
cache.dir.cache | The path to the cache root directory |
cache.dir.harness.incoming | The path to the incoming directory (folder in which GTS flow is dropped) |
cache.dir.harness.ingesting | The path to the folder containing files to ingest (after dispatch) |
cache.dir.harness.working | The path to the working directory (batch files being ingested are split in this folder, after ingesting) |
cache.dir.harness.outgoing | The path to the outgoing directory for feeding |
dataservice.service.status.folder | The path to the folder containing status files for each sub-services. |
cache.replication.config.folder | The path to the folder containing replication sub-folders (sending/receiving) |
FTP Replication Settings | |
cache.replication.config.fromReplication.folder | Name of sub-folder of ingesting and working folders where files are dropped when received from replication |
Cache Feed/Ingest Settings | |
cache.config.numberOfChecksumBytes | The number of bytes of a file on which the checksum will be calculated |
cache.config.location.sendingCentre | The 4-letter-location-code of the sending (the local) centre (used as a prefix of outgoing WMO FTP files). |
Dissemination Configuration | |
staging.post.uri | The path to the staging post directory. |
cache.dissemination.stagingPost.purgeTime | [minutes] The entries in the “StagingPostEntry” database table and the files on the staging post, referenced in those entries, which are older than this value will be deleted. |
cache.config.stagingPostMaximumSize | The maximum number of bytes in the staging post. |
cache.config.cacheMaximumSize | The maximum number of bytes in the cache. |
cache.dissemination.disseminationHarness.public.url | The URL of the public dissemination harness. |
cache.dissemination.disseminationHarness.rmdcn.url | The URL of the RMDCN dissemination harness. |
cache.dissemination.threshold.mail | The maximum number of bytes to send via email |
cache.dissemination.threshold.ftp | The maximum number of bytes to send via ftp |
mail.from Sender | of the blacklisting emailing |
mail.smtp.host SMTP | Host for the blacklisting emailing |
mail.smtp.port SMTP | Port for the blacklisting emailing |
blacklist.default.nb.warn Default | blacklisting warning threshold regarding the number of files |
blacklist.default.nb.blacklist | Default blacklisting threshold regarding the number of files |
blacklist.default.vol.warn | Default blacklisting warning threshold regarding the volume |
blacklist.default.vol.blacklist | Default blacklisting threshold for the volume |
Management WebService URLs | |
openwis.management.alertservice.wsdl | The WSDL URL of the alert service. (update Host/Port with the location of the Management service) |
openwis.management.controlservice.wsdl | The WSDL URL of the control service. (update Host/Port with the location of the Management service) |
openwis.management.controlservice.defaultFeedingFilterLocation | The full path and name (e.g. ‘defaultFeedingFilters.config’) of a file containing the default feeding filters. The content of this file must be one regular expression per line, each line thus defines a filter. |
openwis.management.disseminateddatastatistics.wsdl | The WSDL URL of the disseminated data statistics. (update Host/Port with the location of the Management service) |
openwis.management.exchangeddatastatistics.wsdl | The WSDL URL of the exchanged data statistics. (update Host/Port with the location of the Management service) |
openwis.management.replicateddatastatistics.wsdl | The WSDL URL of the replicated data statistics. (update Host/Port with the location of the Management service) |
openwis.management.ingesteddatastatistics.wsdl | The WSDL URL of the ingested data statistics. (update Host/Port with the location of the Management service) |
Other Settings | |
cache.cacheManager.housekeepingTimer.expirationWindow | [days] The number of days the cache content will be valid before it gets removed |
cache.cacheManager.purgingTimer.expirationWindow | [ms] The number of milliseconds a file will be kept in temporary directory. |
Table 1: Data Service configuration properties
Note: Advanced parameters should not to be modified on a typical deployment.
-
Download JDBC Driver for PosgreSQL 10 and java 1.8
wget https://jdbc.postgresql.org/download/postgresql-42.2.6.jar
-
JBOSS_HOME - Check that JBOSS_HOME is properly configured:
$ echo $JBOSS_HOME
/home/openwis/wildfly-15.0.1.Final
It's convenient to install the Openwis Data&Management service using jboss-cli.
Assuming that openwis is builded as described in 2.6 and that JBOSS_HOME is correctly configured, execute a script with the following commands:
Note:
Wildfly must be started
As openwis user:
#!/bin/bash
#
echo "---Deploy Openwis Data and Management Services via CLI Tools---"
echo ""
echo "-- JBOSS_HOME setting check";
if [ "x$JBOSS_HOME" = "x" ]; then
echo "JBOSS_HOME undefined -> exit !";
exit;
else
echo "JBOSS_HOME ok : $JBOSS_HOME";
fi
echo "--Set the HTTP port to 8180"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="/socket-binding-group="standard-sockets"/socket-binding="http":write-attribute(name="port",value=8180)"
sleep 5
echo "--Set the Management Inet Listen Address"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="/interface=management:write-attribute(name=inet-address,value=127.0.0.1)"
echo "--Set the Public Inet Listen Address"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="/interface=public:write-attribute(name=inet-address,value=127.0.0.1)"
echo "--Set the WSDL Inet Listen Address"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="/subsystem=webservices:write-attribute(name="wsdl-host",value=127.0.0.1)"
echo "--Set the WSDL port to 8180"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="/subsystem=webservices:write-attribute(name="wsdl-port",value=8180)"
sleep 5
echo "--Configure Deployment Scanner scan-interval"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="/subsystem=deployment-scanner/scanner=default:write-attribute(name=scan-interval,value=500)"
echo "--Configure Deployment Scanner auto-deploy true"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="/subsystem=deployment-scanner/scanner=default:write-attribute(name=auto-deploy-exploded,value="true")"
echo "--Reload WildFly..."
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 --command=":reload"
sleep 5
echo "--Setup logging"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command='/subsystem=logging/periodic-rotating-file-handler="CollectionHandler":add(level=INFO, file={"relative-to"=>"jboss.server.log.dir", "path"=>"collection.log"}, formatter="%d %-5p [%c] %m%n", append=true, autoflush=true, suffix="yyyy-MM-dd")'
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command='/subsystem=logging/periodic-rotating-file-handler="RequestHandler":add(level=INFO, file={"relative-to"=>"jboss.server.log.dir", "path"=>"request.log"}, formatter="%d %-5p [%c] %m%n", append=true, autoflush=true, suffix="yyyy-MM-dd")'
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command='/subsystem=logging/periodic-rotating-file-handler="AlertsHandler":add(level=INFO, file={"relative-to"=>"jboss.server.log.dir", "path"=>"alerts.log"}, formatter="%d %-5p [%c] %m%n", append=true, autoflush=true, suffix="yyyy-MM-dd")'
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command='/subsystem=logging/logger=org.openwis.dataservice.util.WMOFTP:add(use-parent-handlers=true,handlers=["CollectionHandler"])'
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command='/subsystem=logging/logger=org.openwis.dataservice.gts.collection:add(use-parent-handlers=true,handlers=["CollectionHandler"])'
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command='/subsystem=logging/logger=org.openwis.dataservice.dissemination:add(use-parent-handlers=true,handlers=["RequestHandler"])'
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command='/subsystem=logging/logger=org.openwis.datasource:add(use-parent-handlers=true,handlers=["RequestHandler"])'
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command='/subsystem=logging/logger=org.openwis.management.service:add(use-parent-handlers=true,handlers=["AlertsHandler"])'
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command='/subsystem=logging/periodic-rotating-file-handler=FILE:write-attribute(name="level",value="INFO")'
sleep 5
echo "--Reload WildFly..."
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 --command=':reload'
sleep 5
echo "--Deploy the postgresql driver"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 --command='deploy postgresql-42.2.6.jar'
echo "--Setup the OpenDS data source"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command=' data-source add --name=OpenwisDS --jndi-name="java:/OpenwisDS" --connection-url="jdbc:postgresql://localhost:5432/OpenWIS?stringtype=unspecified" \
--user-name="openwis" --password="$openwis" --driver-name="postgresql-42.2.6.jar" --driver-class="org.postgresql.Driver" \
--min-pool-size=10 --max-pool-size=40 --idle-timeout-minutes=15 --blocking-timeout-wait-millis=15000 --background-validation-millis=50000'
sleep 10
echo "--Setup the JMS queues"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="jms-queue add --queue-address=CollectionQueue --entries=[java:/jms/queue/CollectionQueue]"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="jms-queue add --queue-address=IncomingDataQueue --entries=[java:/jms/queue/IncomingDataQueue]"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="jms-queue add --queue-address=RequestQueue --entries=[java:/jms/queue/RequestQueue]"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="jms-queue add --queue-address=DisseminationQueue --entries=[java:/jms/queue/DisseminationQueue]"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="jms-queue add --queue-address=PackedFeedingQueue --entries=[java:/jms/queue/PackedFeedingQueue]"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="jms-queue add --queue-address=UnpackedFeedingQueue --entries=[java:/jms/queue/UnpackedFeedingQueue]"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="jms-queue add --queue-address=StatisticsQueue --entries=[java:/jms/queue/StatisticsQueue]"
sleep 10
echo "--Reload WildFly..."
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 --command=":reload"
sleep 10
echo "--Deploying Management Service ear file"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="deploy /home/openwis/maven_projects/openwis/openwis-management/openwis-management-service/openwis-management-service-ear/target/openwis-management-service.ear"
sleep 10
echo "--Deploying Data Service ear file"
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="deploy /home/openwis/maven_projects/openwis/openwis-dataservice/openwis-dataservice-server/openwis-dataservice-server-ear/target/openwis-dataservice.ear"
sleep 10
echo "--checking deployments"
$JBOSS_HOME/bin/jboss-cli.sh --controller="localhost:9990" -c --commands=ls\ deployment
sleep 5
echo ""
echo "*** Installation COMPLETE ***"
Note: If needed, Solr can be deployed in wildfly with the following command:
#Deploying SolR war
$JBOSS_HOME/bin/jboss-cli.sh --connect --controller=127.0.0.1:9990 \
--command="deploy /home/openwis/maven_projects/openwis/openwis-metadataportal/openwis-portal-solr/target/openwis-portal-solr.war"
-
Verify that Wildfly is running port 8180:
$ netstat -an | grep LISTEN|grep -v LISTENING tcp 0 0 127.0.0.1:9990 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:3528 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:8180 0.0.0.0:* LISTEN
-
Verify Wildfly working:
tail -f $JBOSS_HOME/standalone/log/server.log
...
2020-06-08 16:06:15,884 INFO [org.openwis.dataservice.cache.CacheManagerImpl] (EJB default - 8) +++ purging temporary directory
2020-06-08 16:06:15,885 INFO [org.openwis.dataservice.cache.CacheManagerImpl] (EJB default - 7) +++ Start housekeeping of cache since Sun Jun 07 16:06:15 CEST 2020
2020-06-08 16:06:15,885 INFO [org.openwis.dataservice.cache.CacheManagerImpl] (EJB default - 7) +++ expiration +++ deleting mapped metadata
2020-06-08 16:06:15,886 INFO [org.openwis.dataservice.cache.CacheManagerImpl] (EJB default - 7) +++ expiration +++ deleting cached files
2020-06-08 16:06:15,887 INFO [org.openwis.dataservice.cache.CacheManagerImpl] (EJB default - 7) +++ expiration +++ deleted : 0
2020-06-08 16:06:15,888 INFO [org.openwis.management.service.IngestedDataStatisticsImpl] (EJB default - 7) No ingested data found at Mon Jun 08 02:00:00 CEST 2020
2020-06-08 16:06:15,888 INFO [org.openwis.dataservice.cache.CacheManagerImpl] (EJB default - 7) +++ Finished housekeeping of cache since Sun Jun 07 16:06:15 CEST 2020
...
As openwis user: Create the Solr home folder and the Solr data folder:
mkdir /home/openwis/solr
mkdir /home/openwis/solr/data
The SolR war is deployed as followed:
mkdir <TOMCAT_HOME>/webapps/openwis-portal-solr
cd <TOMCAT_HOME>/webapps/openwis-portal-solr
copy openwis-portal-solr.war at TOMCAT_HOME/webapps/openwis-portal-solr ex:
cp ~/maven_projects/openwis/openwis-metadataportal/openwis-portal-solr/target/openwis-portal-solr.war .
unzip openwis-portal-solr.war
The configuration is done in WEB-INF/classes/openwis.properties of the SolR webapp:
###
### OpenWIS SolR Configuration
###
### SolR data folder
openwis.solr.data=/home/openwis/solr/data
### Spatial folder (only used in case PostGIS is not available)
openwis.solr.spatial=/home/openwis/solr/spatial
### Database configuration
jdbc.driver=org.postgresql.Driver
jdbc.url=jdbc:postgresql://127.0.0.1:5432/OpenWIS?stringtype=unspecified
jdbc.user=openwis
jdbc.password=DB_PASSWORD
Verify that postgresql-XXX-jdbc4.jar” version is the same as installed postgres at:
<TOMCAT_HOME>/webapps/openwis-portal-solr/WEB-INF/lib/
In any other case replace it with the consistent one.
2 possibilities exist for spatial indexing:
- Shape file: When using shape files (and not PostGIS), local files in a dedicated directory will be used to store the spatial information. Create the SolR spatial directory:
mkdir /home/openwis/solr/spatial
Adjust the related spatial folder in openwis.properties described above (if needed).
Difference with Iteration 2: the shared folder between SolR and Portals are not needed anymore
- PostGIS: when using PostGIS, the spatial information are stored in the common OpenWIS database. Adjust the database configuration in openwis.properties described above.
Note: When starting up, SolR will first try to connect the database and test if a valid PostGIS installation exists, otherwise it automatically switch to shape file indexing.
Finally restart Tomcat service tomcat restart (or use stop start tomcat scripts provided)
The Staging Post is a file system accessible via a Web interface. This can be configured using the provided StagingPost web application deployed on a Tomcat server or alternatively deployed on JBoss. As mentioned in data service section, the Staging Post file system is configured to be in /home/openwis/stagingPost.
As openwis: Deployment of the Staging Post web app:
mkdir /home/openwis/stagingPost (if needed)
cd /home/openwis/stagingPost
cp ~/openwis/openwis-stagingpost/target/stagingPost.war .
jar xvf stagingPost.war
In case the Staging Post is deployed elsewhere, adjust the Staging Post globalXsltFile locations in WEB-INF/web.xml.
To deploy Staging Post on Tomcat: Edit the server.xml file of Tomcat:
vi ~/apache-tomcat-9.0.36/conf/server.xml
Add the following element at the end of the Host element:
<Context docBase="/home/openwis/stagingPost" path="/stagingPost" reloadable="false"/>
Note: Admin Portal, User portal and OpenAM cannot be in the same Tomcat instance neither run under the same OS user.
Deployment of the admin portal:
As a new user (ex openwisAdmin):
mkdir apache-tomcat-9.0.36/webapps/openwis-admin-portal
cd apache-tomcat-9.0.36/webapps/openwis-admin-portal
Find openwis-admin-portal.war or openwis-admin-portal-admin.war (in later versions of openwis) and copy it at apache-tomcat-9.0.36/webapps/openwis-admin-portal.
jar xvf openwis-admin-portal-admin.war
Likewise deployment of the user portal:
Also as new user (ex openwisUser):
mkdir apache-tomcat-9.0.36/webapps/openwis-user-portal
cd apache-tomcat-9.0.36/webapps/openwis-user-portal
jar xvf openwis-user-portal-user.war
The configuration is based on the same file/parameters on user and admin portals.
⚠️ This configuration must be done for both deployments: User Portal and also for Admin Portal
Edit as your defined: portal admin for Admin Portal And as portal user for User Portal
-
config.xml: database connection (Geonetwork)
vi WEB-INF/config.xml
Edit at section postgresql, set the user, password and URL used to connect the PostgreSQL instance.
-
openwis-metadataportal.properties: OpenWIS specific configuation
vi WEB-INF/classes/openwis-metadataportal.properties
Property | Description |
---|---|
openwis.metadataportal.dataservice.*.wsdl | WSDL location of the Data Services |
openwis.metadataportal.securityservice.*.wsdl | WSDL location of the Security Web Services Update with SecurityService host/port |
openwis.metadataportal.harness.subselectionparameters.wsdl | WSDL location of the SubselectionParameters Harness Web Service |
openwis.metadataportal.harness.mssfss.wsdl | WSDL location of the MSS/FSS Harness Web Service |
openwis.management.*.wsdl | WSDL location of the Security Web Services,Update with ManagementService host/port |
openwis.metadataportal.mssfss.support | Whether the MSS/FSS is supported by the current deployment |
openwis.metadataportal.url.staging.post | Base URL of the Staging Post |
openwis.metadataportal.cache.enable | Whether the current deployment has a Cache facility |
openwis.metadataportal.solr.url | SolR location: http://server:port/openwis-portal-solr |
openwis.metadataportal.date.format | Date format (test only) |
openwis.metadataportal.datetime.format | Date time format (test only) |
openwis.metadataportal.deploy.name | Deployment name as defined with Populate LDAP described in section 3.5(e.g. GiscMeteoFrance) |
openwis.metadataportal.datapolicy.default.name | Default data policy that will be applied to newly created groups |
openwis.metadataportal.datapolicy.default.operations | Default data policy operation that will be applied to newly created groups |
openwis.metadataportal.sso | OpenSSO URL (used only on Administration portal) |
openwis.metadataportal.oai.maxRecords | Max records processed by OAIPMH in one page |
openwis.metadataportal.acceptedFileExtensions | Accepted list of file extensions (deduced from the metadata and used during file unpacking) |
openwis.metadataportal.monitoring.userportal.url | URL of user portal used to test availability (used on admin portal) |
openwis.metadataportal.monitoring.synchro.warn.limit | Percentage beyond which the availability of synchronization process will be marked in error |
openwis.metadataportal.monitoring.harvesting.warn.limit | Percentage beyond which the availability of harvesting process will be marked in error |
openwis.metadataportal.session.securityservice.tooManyActiveUsers | Number of connected users beyond which an alarm is raised |
openwis.metadataportal.securityservice.tooManyActiveAnonymousUsers | Number of anonymous users beyond which an alarm is raised |
openwis.metadataportal.extract.xpath | XPath of the GTS Category / Data Policy in ISO19139 schema |
openwis.metadataportal.extract.gtsCategoryAdditionalRegexp | Regular expression to interpret a GTS Category found in metadata as WMOAdditional |
openwis.metadataportal.extract.gtsCategoryEssentialRegexp | Regular expression to interpret a GTS Category found in metadata as WMOEssential |
openwis.metadataportal.extract.gtsPriorityRegexp | Regular expression to interpret a GTS Priority found in metadata |
openwis.metadataportal.extract.urnPatternForIgnoredFncPattern | The pattern applied on URN to determine if FNC Pattern found in metadata should be ignored when inserting the metadata.Default ignored FNC Pattern when URN is TTAAiiCCCC based) |
openwis.metadataportal.catalogsize.alarm.period | Catalog |
openwis.metadataportal.catalogsize.alarm.limit | Catalog size limit beyond which raise an alarm |
openwis.metadataportal.lang.list | Available languages: list of <lang_value/lang_label> |
openwis.metadataportal.report.file.path | Directory where harvesting reports are stored on the admin portal server |
- openwis-deployments.properties: Multi-deployments configuration
vi WEB-INF/classes/openwis-deployments.properties
Property | Description |
---|---|
openwis.cots | List of deployment names that can be used to consult the remote requests/subscription.These deployments must be in the same circle of trust. |
openwis.backups | List of backup deployment names |
openwis.deployment.url.DEPLOYMENT_NAME | The User portal URL of the backup deployment (for each backup deployment, including the current one) |
openwis.deployment.url.DEPLOYMENT_NAME.admin | The administrator email of the backup deployment (for each backup deployment, including the current one) that will be notified in case of availability error detection |
openwis.backup.warn.rate | The rate of available function (in %) bellow which we consider that |
Each OpenWIS portal (User or Admin) will be a Service Provider (SP), and will need to be configured before being registered in the Circle of Trust defined previously. This phase requires the OpenWIS portal to be deployed.
GenerateSPConfFiles is a tool provided to ease the configuration process of a Service Provider. The tool can be downloaded from the middleware repository: https://repository-openwis-association.forge.cloudbees.com/private/binaries/GenerateSPConfFiles.zip
Unzip GenerateSPConfFiles tool:
cd <PORTAL USER or PORTAL ADMIN HOME>
mkdir GenerateSPConfFiles
cd GenerateSPConfFiles
wget https://repository-openwis-association.forge.cloudbees.com/private/binaries/GenerateSPConfFiles.zip
unzip GenerateSPConfFiles.zip
Edit configuration.properties in conf directory:
#Home Directory (Windows : c:/Documents and Settings/<USER-NAME>, Other /home/<USER-NAME>)
directory=<PORTAL USER or PORTAL ADMIN HOME>
#Circle Of Trust Name
cot-name =cot_openwis as defined in step 3.3.1
# Service Provider Name
sp-name =GiscA # logical name of the portal: GiscMF, GiscMFAdmin; name to identify the Service Provider in OpenAM
# Service Provider URL (example: http://<HOST-NAME>:<PORT>/<SP-NAME>)
sp-url =http://<external-sp-hostname>/openwis-user-portal (for user portal)
# IDP Discovery URL (example: http://<HOST-NAME>:<PORT>/idpdiscovery)
idp-discovery=http://<external-idpdiscovery-hostname>/idpdiscovery
# IDPs and SPs of the circle of trust. (comma separated)
trusted-providers =IDP1 # at least the IdP, as defined in step 3.3.1
#All IDPS names
idps-names =IDP1 # at least the IdP, as defined in step 3.3.1
#For each IDP name, create a variable <IDP-NAME>.url=http://<HOST-NAME>:<PORT>/openam
#<IDP-NAME>.url=
IDP1.url=http://<external-idp-hostname>/openam at least the IdP, as defined in step **3.3.1**
Run GenerateSPConfFiles
cd <PORTAL USER or PORTAL ADMIN HOME>/GenerateSPConfFiles/apache-ant-1.8.1/bin
chmod a+x *
cd <PORTAL USER or PORTAL ADMIN HOME>/GenerateSPConfFiles
chmod a+x Generate-SP-Conf-Files.sh apache-ant-1.8.1/bin/ant
./Generate-SP-Conf-Files.sh
The tool generates files in /fedlet, which allows the portals (service provider) to manage SAML2 communications with the other entities:
Generate-SP-Conf-Files.sh
more than once . You must manually delete all files contained in fedlet folder. Script wont replace them, thus re-execution wont have any effect.
⚠️ If any Fedlet error occurs, see Fedlet log files located at
/fedlet/debug
After Fedlet has been created properly
As openwis user Create storepass and keypass files:
echo A_PASSWOD > storepass.cleartext
echo A_PASSWORD > keypass.cleartext
using ssoAdminTools generate hidden keypass and storepas files:
~/ssoAdminTools/openam/bin/ampassword -e keypass.cleartext > .keypass
~/ssoAdminTools/openam/bin/ampassword -e storepass.cleartext > .storepass
And as root copy them to new created folder /home/USER-NAME/fedlet and change theirs ownership like example:
chown USER-NAME:USER-NAME filename
Note: each Portal needs its own Fedlet. If both portals are on the same machine must be placed under different user and also different Tomcat server. Openwis Portals are currently set to reach Fedlet folder from user.home/fedlet folder.
Restart Portal server and then verify via a browser:
For admin Portal:
<Admin-PORTAL-HOST-NAME>:<PORT>/openwis-admin-portal/saml2/jsp/exportmetadata.jsp
For user Portal:
<User-PORTAL-HOST-NAME>:<PORT>/openwis-user-portal/saml2/jsp/exportmetadata.jsp
In order to add Portals at COT they must be registered as remote Service Provider on OpenAM Connect to OpenAM. Click on “Common Tasks” tab.
Select “Register Remote Service Provider”.
http://<SP_HOST_NAME>:<PORT>/<SP_NAME>/saml2/jsp/exportmetadata.jsp
This URL is the URL of the User or Administration portal to register. For example: http://openwis-oai.akka.eu/openwis-user-portal/saml2/jsp/exportmetadata.jsp The portal must be running to complete this step.
Add this circle of trust to the existing previously creating named cot
Click on Configure.
For each Identity Provider in the Federation tab, select “Assertion Processing” tab.
Add one by one the following lines as attributes to the Attribute Mapper (the attribute names are case sensitive and there must be no spaces ) and save:
OpenWISAddress=OpenWISAddress
OpenWISAddressCity=OpenWISAddressCity
OpenWISAddressZip=OpenWISAddressZip
OpenWISAddressState=OpenWISAddressState
OpenWISAddressCountry=OpenWISAddressCountry
OpenWISClassOfService=OpenWISClassOfService
OpenWISBackUps=OpenWISBackUps
OpenWISProfile=OpenWISProfile
isMemberOf=isMemberOf
OpenWISNeedUserAccount=OpenWISNeedUserAccount
cn=cn
sn=sn
givenname=givenname
mail=mail
After installation is complete portals must be accesible via :
http://<USER_PORTAL_SERVER>:<PORT>/openwis-user-portal
http://<ADMIN_PORTAL_SERVER>:<PORT>/openwis-admin-portal
At login prompt through OpenAM user must enter the credentials of user(s) created via populateLDAP (see 3.5)
If any error occurs, information may be found at:
- Tomcat logs
- OpenWIS portals log files located at:
/home/<Admin or User>/logs
- Tomcat logs (OpenDJ, OpenAM, OpenWIS Security Services)
- JBoss logs (openwis-dataservice, openwis-management-service)
An Apache server is used as a front-end to access:
- the user and admin portals
- OpenAM
- OpenWIS Security Services (load balancing/failover)
- Data Service (load balancing/failover)
- the Staging Post
Apache is already installed by default in RedHat 5.5 +.
The configuration of Apache is done via the configuration file: /etc/httpd/conf/httpd.conf.
See details of configuration at next paragraphs
The relevant section is:
Start Apache Web Server:
service httpd start
service httpd status
Notes:
Enable SELinux working with Apache Proxy As root: setsebool
httpd_can_network_connect on
if getsebool: SELinux is disabled Edit :/etc/sysconfig/selinux
and set SELINUX=disabled -> SELINUX=permissive and reboot OS
To test the value
getsebool httpd_can_network_connect
httpd_can_network_connect --> on
Warning:
- If failover and site are not configured : the host name of the proxied OpenAM needs to be exactly the same as used during OpenAM installation (hostname and not IP address) or the whole OpenAM installation needs to be done via the front-end host name.
- In failover and site configuration: the frontend hostname will be the frontend of the configured site.
For a simple front-end configuration:
ProxyPreserveHost On
# Proxy for OpenAM
ProxyPass /opensso http://<HOST_OPENAM>:8080/openam
ProxyPassReverse /opensso http://<HOST_OPENAM>:8080/openam
For a failover configuration with load balancing between 2 OpenAM instances:
ProxyRequests Off
# Load balancer for OpenAM
<Proxy balancer://cluster-opensso>
Order deny,allow
Allow from all
BalancerMember http://<HOST_OPENAM_1>:8080/opensso route=node1
BalancerMember http://<HOST_OPENAM_2>:8080/opensso route=node2
ProxySet lbmethod=byrequests
ProxySet stickysession= APLBCOOKIE
</Proxy>
Header add Set-Cookie "APLBCOOKIE=APACHE.%{BALANCER_WORKER_ROUTE}e;path=/;" env=BALANCER_ROUTE_CHANGED ProxyPass /opensso balancer://cluster-opensso ProxyPassReverse /opensso http://<HOST_OPENAM_1>:8080/openam ProxyPassReverse /opensso http://<HOST_OPENAM_2>:8080/openam
For a simple front-end configuration:
# Proxy for IDP Discovery
ProxyPass /idpdiscovery http://<HOST_IDP_DISCOVERY>:8080/idpdiscovery
ProxyPassReverse /idpdiscovery http://<HOST_IDP_DISCOVERY>:8080/idpdiscovery
For a failover configuration with load balancing between 2 IdpDiscovery instances:
# IDP Discovery
<Proxy balancer://cluster-idpdiscovery>
Order deny,allow
Allow from all
BalancerMember http://<HOST_IDPDISCOVERY_1>:8080/idpdiscovery route=node1
BalancerMember http://<HOST_IDPDISCOVERY_2>:8080/idpdiscovery route=node2
</Proxy>
ProxyPass /idpdiscovery balancer://cluster-idpdiscovery lbmethod=byrequests
ProxyPassReverse /idpdiscovery http://<HOST_IDPDISCOVERY_1>:8080/idpdiscovery
ProxyPassReverse /idpdiscovery http://<HOST_IDPDISCOVERY_2>:8080/idpdiscovery
When configuring failover for OpenWIS Security Web Services, the Apache frontend is used as load balancer between the 2 Tomcat instances (used for OpenAM). The Apache configuration is as followed:
# Security service Loadbalancer
<Proxy balancer://cluster-securityservice>
Order deny,allow
Allow from all
BalancerMember http://<HOST_OPENAM_1>:8080/openwis-securityservice route=node1
BalancerMember http://<HOST_OPENAM_2>:8080/openwis-securityservice route=node2
</Proxy>
ProxyPass /openwis-securityservice balancer://cluster-securityservice lbmethod=byrequests
ProxyPassReverse /openwis-securityservice http://<HOST_OPENAM_1>:8080/openwis-securityservice
ProxyPassReverse /openwis-securityservice http://<HOST_OPENAM_2>:8080/openwis-securityservice
ProxyPreserveHost On
# Proxy for User portal
ProxyPass /openwis-user-portal http://<HOST_USER_PORTAL>:8080/openwis-user-portal
ProxyPassReverse /openwis-user-portal http://<HOST_USER_PORTAL>:8080/openwis-user-portal
# Proxy for Admin portal (if admin portal is available on this zone)
ProxyPass /openwis-admin-portal http://<HOST_ADMIN_PORTAL>:8080/openwis-admin-portal
ProxyPassReverse /openwis-admin-portal http://<HOST_ADMIN_PORTAL>:8080/openwis-admin-portal
Replace HOST_USER_PORTAL and HOST_ADMIN_PORTAL by the virtual address used for the Active/Passive configuration.
Redirect / It’s generally useful to be able to access the application without having to precise the name of the web application and just give the hostname; for example: using http://wispi.meteo.fr instead of http://wispi.meteo.fr/openwis-user-portal to access the user portal. This can be configured with the following Redirect directive: Redirect / http://<HOST_USER_PORTAL:PORT>/openwis-user-portal
Or similarly for Admin portal:
Redirect / http://<HOST_ADMIN_PORTAL:PORT>/openwis-admin-portal
ProxyPass /stagingPost http://<HOST_STAGINGPOST_WEBAPP>:8080/stagingPost
ProxyPassReverse /stagingPost http://<HOST_STAGINGPOST_WEBAPP>:8080/stagingPost
[OpenWIS Association] (http://openwis.github.io/openwis-documentation/)
[OpenWIS repository] (https://github.com/OpenWIS)
[IG-OpenWIS-1-v02 9-MigrationOpenAM-v3] (https://github.com/OpenWIS/openwis/blob/master/docs/IG-OpenWIS-1-v02%209-MigrationOpenAM-v3.doc)