Configuring and Running an OpenMigrate PUMA - tsgrp/OpenMigrate GitHub Wiki

High Level Description of OpenMigrate PUMA

An OpenMigrate PUMA is an instance of OpenMigrate that runs based on a scheduled job (can be every 5 minutes, every day, etc.). Each time the job is run it executes a migration. There are two different types of runs, the initial run and delta runs. The initial run is only run once and will migrate all documents that fit the query criteria. On the initial run each document that is migrated over is put into a table in a SQL database. A Delta run is run on based on the frequency of the job. During a delta run OpenMigrate will only migrate documents that fit the query criteria and that have a modified date that is later than the latest modified date in the SQL table (this prevents migrating the same document every time the job runs).

Configuring your Solr Instance

  1. Solr can be deployed OOB using the Solr binaries downloaded from http://lucene.apache.org/solr/

  2. Extract the zip somewhere on your machine (Ex: C:\Apache\solr-5.3.1)

  3. Copy the 'portal' config folder from http://svn.tsgrp.com/repos/hpi/trunk/demo-setup/solr/solr-core and paste it to your SOLR_HOME (by default will be wherever you extracted your zip + \server\solr.

  4. You should now have a directory structure that looks like this:

/                           # Root folder where solr was extracted Ex C:\Apache\solr-5.3.1
β”œβ”€β”€ bin                     # location of all of the startup scripts should be run from
... other folders
β”œβ”€β”€ server                  #    
β”œβ”€β”€β”€β”€ solr                  # This is SOLR_HOME by default
β”œβ”€β”€β”€β”€β”€β”€ portal              # This is the core that we extracted from TSG's svn
β”œβ”€β”€β”€β”€β”€β”€β”€β”€ conf              # Folder contains the core's configurations that define the schema, etc.
β”œβ”€β”€β”€β”€β”€β”€β”€β”€ core.properties   # portal core definition

  1. Open a command prompt to /solr-5.x.x/bin and run solr start. This will start solr server on port 8983.

  2. Run solr stop -all if you need to stop the instance for any reason.

In the solr installation:

  1. Add the solr-cell-4.9.0.jar to your {solr home}/collection1/lib folder
  2. Add solr-dataimporthandler-4.4.0.jar and solr-dataimporthandler-extras-4.4.0.jar to the lib folder
  3. Make sure your app-ctx.xml TargetSolrListener bean has the correct document type attributes listed
  4. Make sure your schema.xml in solr home/collection1/conf has the same attributes you added to the app-ctx.xml
  5. Restart the tomcat and run setenv-COMPUTER_NAME and om.bat from the OM project folder

Installing MySQL and Setting up Database

  1. Download MySQL Client Installer. (http://dev.mysql.com/downloads/mysql/ MySQL Community Server 5.x.x (GA) Release).

  2. Run the Installer.

    1. Make sure to note the password you select.

Database and Table Setup

  1. Navigate to: C:\Program Files\MySQL\MySQL Server 5.6\bin

  2. To create the database execute the following lines:

    1. mysql -u root –p

    2. CREATE DATABASE ;

    3. exit

  3. To create the todo table execute the following command:

    1. mysql.exe -h localhost -u root -p < {pathToCreateQueueScript} (where {pathToCreateQueueScript} is the path to the create_queue_table.sql script on the local machine. Usually located in the project folder of the project being used.)

  4. NOTE: for all the above replace with the desired name for the database.

OpenMigrate Setup

  1. Extract OpenMigrate.zip to C:\OpenMigrate

    1. This will be your OM_HOME

    2. (subject to change)

  2. Open OM_HOME/config/ctx-placeholder-{env}.properties and update the following lines with the correct values:

source.opencontent..username
source.docbase.password
sql.driverClassName=com.mysql.jdbc.Driver
sql.url
sql.username
sql.password
solrURL         #{solr-home-directory}/solr/portal
solr.base.path

Initial Run The initial run for a puma is only run once.

  1. Navigate to the OM_HOME\code\projects\clients\projects<clientProject>\config

  2. Open the file app-ctx.xml.

  3. Locate the QueuePopulator bean and set the property deltaRun to have a value of β€œfalse”

  4. Navigate to OM_HOME\code\projects\clients\projects<clientProject>

  5. Execute the following line in a command prompt: om.bat {env}

Delta Runs Delta runs are run after the initial run. Delta runs use the puma table when populating the queue to make sure that it only migrates over documents that have been added/modified since the last migration.

  1. Navigate the the app-ctx.xml file, locate the QueuePopulator bean and change the property of deltaRun to have a value of β€œtrue”

  2. Navigate to OM_HOME\code\projects\clients\projects<clientProject>

  3. Execute the following line in a command prompt: om.bat {env}

APPENDIX A (Using SquirrelSQL) Setting up Puma Table with the SQuirrel SQL Client Download SQuirrel SQL Client (http://squirrel-sql.sourceforge.net/#installation version 3.6) Follow installation instructions to setup the client. Launch squirrel-sql by running the squirrel-sql.bat.

Setting up the MySQL-Connector Driver Download the mysql-connector-java-5.1.26.jar (it can be version 5.1.26 or up). Close the SQuirrel SQL Client. Copy the mysql-connector-java-5.1.26.jar and put it into the lib folder in the squirrel-sql-3.6 directory. File path: C:\Program Files\squirrel-sql-3.6\lib Reopen the SQuirrel SQL Client. Confirm that the MySQL Driver has a blue check next to it when you view all drivers.

Setting up the MySQL Alias Navigate to the Aliases Tab and click the plus sign. Supply the following information: Name: (your choice) Driver: MySQL Driver URL: jdbc:mysql://:/ : name of server (eg. localhost) : usually the default port for mySql is 3306 : name of the database created in above section User Name: root Password: the chosen MySQL password from the above section. Click ok. Double click on the newly created alias and try to connect to test the connection.

Creating Puma Table Open the SQuirrel SQL Client if not already open and connect to the Alias created in the above section. Select the SQL tab if it’s not already selected. Locate the file create_queue_table.sql Copy the text and paste it into the text area in the SQuirrel SQL Client. NOTE: make sure the table name is puma_todo. Run the SQL. To test that the table was created run the following SQL: SELECT * FROM puma_todo An empty table should be the result.

OpenMigrate    Β© 2020 Technology Services Group

⚠️ **GitHub.com Fallback** ⚠️