ARCHIVED

New build step document link is here

Build Steps to follow to build up to 4.0.43

To setup new environment refer here

*Stop node before any build

Upgrade revision 3.x.xx - upgrade to - 4.0.11

eXist build
Run createIndex service for all communities (without community parameter)
Run following insert on mysql

            INSERT INTO `roles` (`roleId`, `roleName`, `description`) VALUES (3, 'Guest', 'Guest');

run following npm commands, standing inside "mobile1.8" folder

            npm config set strict-ssl false
            npm install --save gulp-order
            npm install --save gulp-uglify-es
            npm install --save ajv

Copy following documents from support server to target prod server's kwantu_configs bucket

            nodeResult_chart1_template
            nodeResult_chart2_template
            nodeResult_chart3_template
            nodeResult_chart4_template

{
  "_id": "system_settings_to_kwantu.biz",
  "_rev": "9-66041c3c869f9f7f124f820029daeafe",
  "user_ctx": {
    "name": "Administrator",
    "roles": [
      "_admin",
      "_reader",
      "_writer"
    ]
  },
  "source": {
    "headers": {
      "Authorization": "Basic <replace this>"
    },
    "url": "https://kwantu.support:6984/kwantu_configs"
  },
  "target": {
    "headers": {
      "Authorization": "Basic <replace this>"
    },
    "url": "https://kwantu.biz:6984/kwantu_configs"
  },
  "create_target": false,
  "continuous": false,
  "owner": "Administrator",
  "doc_ids": [
    "nodeResult_chart1_template",
    "nodeResult_chart2_template",
    "nodeResult_chart3_template",
    "nodeResult_chart4_template",
    "System_Settings"
  ]
}

Node build.
Migration

For recent and favourites files migration

Make sure you have updated primary set id for all profile, subprofile and new instance configurations
Do config build for community
Run following script

https://<servername>/exist/rest/db/kwantu-resource/migration/recentFilesUpdate.xq?communityId=<COMMUNITY_ID>

If this service returns a message with an error count on completion, that means some of the configs are still missing...... click on details to check the process id etc.. fix that and run script again...

Upgrade revision 4.0.11 - upgrade to - 4.0.13

npm install string --save
Copy following documents from support server to target prod server's kwantu_configs bucket

            nodeResult_chart1_template
            nodeResult_chart2_template
            nodeResult_chart3_template
            nodeResult_chart4_template
            System_Settings

Node build

Upgrade revision 4.0.13 - upgrade to - 4.0.14

eXist build
Run DDL on MySql

CREATE TABLE `password_requests` (
  `userId` varchar(50) COLLATE utf8_unicode_ci NOT NULL,
  `email` varchar(100) COLLATE utf8_unicode_ci NOT NULL,
  `uuid` varchar(50) COLLATE utf8_unicode_ci DEFAULT NULL,
  `createDate` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `isUsed` tinyint(4) DEFAULT '0',
  KEY `userId` (`userId`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

Node build

Build 4.0.19

Migration for enabling grids

Migration of recent file adding primaryId
Verify primary indicators for subProcessUUID in model.pending section

Others

Processes file - variable file and deleted processes migration - Mandatory for all communities

Important: This migration also includes variable file migration so this is mandatory for all such communities which are using EPWPRS project.

This migration needs to be ran using ETL processor... to run ETL insert following entry for each community in the database. Do run this for all active and inactive communities

INSERT INTO `etl_control` (`etl_type`, `communityId`, `pending_approved`) VALUES ('processesIterator', '<COMMUNITY_ID>', 'type01');

Push create index Xquery service
Run create index for f2074118-1a17-4e22-923c-ce22ea047afa
Replicate kwantu_geo database
Make sure you have c++ version 11+ otherwise install it as below

yum install centos-release-scl
yum upgrade --skip-broken
yum install -y devtoolset-7
scl enable devtoolset-7 bash
export CC=/opt/rh/devtoolset-7/root/usr/bin/gcc
export CXX=/opt/rh/devtoolset-7/root/usr/bin/g++

mkdir ~/glibc_install
cd ~/glibc_install
wget http://ftp.gnu.org/gnu/glibc/glibc-2.14.tar.gz
tar zxvf glibc-2.14.tar.gz
cd glibc-2.14
mkdir build
cd build
../configure --prefix=/opt/glibc-2.14
make -j4
sudo make install
export LD_LIBRARY_PATH="/opt/glibc-2.14/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"

cd /opt/betterdata-mobile/mobile1.8/
npm install pouchdb --save
SQL

ALTER TABLE user_
ADD COLUMN  `lastVisited` datetime DEFAULT '2000-01-01 00:00:00',
ADD COLUMN  `lastDevice` varchar(20) COLLATE utf8_unicode_ci DEFAULT NULL,
ADD COLUMN  `mobile` varchar(20) COLLATE utf8_unicode_ci DEFAULT NULL;

UPDATE user_ SET lastVisited = '2000-01-01 00:00:00';

Build 4.0.20

npm install @hapi/joi
npm install basic-auth

Build 4.0.24

Update MySQL Schema
eXist code build
below commands

npm install -g n
n lts
cd /opt/betterdata-mobile/mobile1.8/
git fetch --all
git reset --hard origin/Production01
npm install --only-prod

Note: There might be an error when installing above. The typical error code is the following. It seems to be specific to RHEL/Centos7

make: Entering directory `/opt/betterdata-mobile/mobile1.8/node_modules/sharp/build'
  CC(target) Release/obj.target/nothing/../node-addon-api/nothing.o
  AR(target) Release/obj.target/../node-addon-api/nothing.a
  COPY Release/nothing.a
  TOUCH Release/obj.target/libvips-cpp.stamp
  CXX(target) Release/obj.target/sharp/src/common.o
make: g++: Command not found
make: *** [Release/obj.target/sharp/src/common.o] Error 127
make: Leaving directory `/opt/betterdata-mobile/mobile1.8/node_modules/sharp/build'
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/root/.nvm/versions/node/v9.11.2/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:258:23)
gyp ERR! stack     at ChildProcess.emit (events.js:180:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:12)
gyp ERR! System Linux 3.10.0-957.12.1.el7.x86_64
gyp ERR! command "/root/.nvm/versions/node/v9.11.2/bin/node" "/root/.nvm/versions/node/v9.11.2/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /opt/betterdata-mobile/mobile1.8/node_modules/sharp
gyp ERR! node -v v9.11.2
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok

To address this, make sure that the node version is 12.8 and that the following support tools have been installed: (https://github.com/kwantu/platformconfiguration/wiki/Installation-Strapi#install-strapi)

Install with nvm https://itnext.io/nvm-the-easiest-way-to-switch-node-js-environments-on-your-machine-in-a-flash-17babb7d5f1b

in root user

wget https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh
chmod 755 install.sh
./install.sh
nvm install 12.8

yum group install "Development Tools"
yum install gcc gcc-c++ kernel-devel make

Another error was then reported with installation of libvips. (https://github.com/libvips/libvips/issues/1184) (https://tufora.com/tutorials/linux/general/install-vips-vips-tools-and-vips-devel-libvips-on-centos-7)

# Install the Remi repository configuration package
yum install http://rpms.remirepo.net/enterprise/remi-release-7.rpm

# Install the yum-utils package (for the yum-config-manager command)
yum install yum-utils

# Command to enable the repository
yum-config-manager --enable remi

# Install libvips (+ development files and command-line tools)
yum install vips vips-devel vips-tools

cd /opt/betterdata-mobile/mobile1.8/
npm install sharp

In some cases the installation fails because it cannot find python

gyp ERR! find Python
gyp ERR! find Python checking Python explicitly set from command line or npm configuration

# then specify the location of python explicitly
npm --python=/usr/bin/python install sharp
---

4. Update Jasper Jars
```code
cd /usr/local/exist/lib
rm -f user/jasperreports*.jar
rm -f user/poi-3*.jar
rm -f user/poi-ooxml-*.jar
rm -f user/commons-beanutils*.jar
rm -f user/com.fasterxml.jackson*.jar
rm -f user/gs.jar

Copy jars 16+2 from DEV server from following location /Script/build/exist-jars/lib
Install fonts

yum install curl cabextract xorg-x11-font-utils fontconfig
yum install https://downloads.sourceforge.net/project/mscorefonts2/rpms/msttcore-fonts-installer-2.6-1.noarch.rpm

To verify if your fonts are installed properly run following commands

sudo fc-cache -fv
fc-list | grep "Arial"

If you do not find any result in grep then install above packages using "reinstall" command again.

Restart exist server.
Node build

Build 4.0.26

This is critical release so we need to stop node before build and make couchdb inaccessible from global domain using ngnix to stop any replication from outside

Execute following SQL


CREATE TABLE `changes_feed_control` (
  `db_name` varchar(200) COLLATE utf8_unicode_ci NOT NULL DEFAULT '',
  `bookmark` varchar(1000) COLLATE utf8_unicode_ci DEFAULT NULL,
  `status` varchar(20) COLLATE utf8_unicode_ci DEFAULT 'idle',
  `mode` varchar(20) COLLATE utf8_unicode_ci DEFAULT NULL,
  `lastRun` datetime DEFAULT NULL,
  `lastUpdated` datetime DEFAULT NULL,
  `result` varchar(5000) COLLATE utf8_unicode_ci DEFAULT NULL,
  `processedTo` bigint(20) DEFAULT '0',
  `serverSince` bigint(20) DEFAULT '0',
  `serverBookmark` varchar(1000) COLLATE utf8_unicode_ci DEFAULT NULL,
  `is_active` tinyint(4) NOT NULL DEFAULT '1',
  `type` varchar(20) COLLATE utf8_unicode_ci NOT NULL DEFAULT '',
  PRIMARY KEY (`db_name`),
  KEY `lastRun` (`lastRun`),
  KEY `status` (`status`),
  KEY `type` (`type`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;


INSERT INTO `changes_feed_control` (`db_name`, `bookmark`, `status`, `mode`, `lastRun`, `lastUpdated`,  `is_active`, `type`) 
VALUES ('all_dbs', '0', 'Updated', 'setup', now(), now(), 1, 'all_dbs');

INSERT INTO `roles` (`roleId`, `roleName`, `description`)
VALUES
	(4, 'UserManager', 'UserManager');

 INSERT INTO `etl_types` (`etl_type`, `elt_type_function`, `is_active`, `notes`)
VALUES
	('elasticSubprocess', 'elasticSubprocess', 'T', 'New type for spo with all sdo');

INSERT INTO `etl_types` ( `etl_type`, `elt_type_function`, `is_active`, `notes`)
VALUES   (    'processesIterator',    'processesIterator',    'T',    'Service to call update processes accross communities'  );

After eXist build run createIndex service (IMPORTANT: RUN IT FOR ALL COMMUNITIES - INCLUDING INACTIVE COMMUNITIES ).
Allow indices to settle for 15 mins.
Processes file - variable file migration - Mandatory for all communities which are using EPWPRS application


This migration needs to be ran using ETL processor... to run ETL insert following entry for each community in the database. 

INSERT INTO `etl_control` (`etl_type`, `communityId`, `pending_approved`) VALUES ('processesIterator', '<COMMUNITY_ID>', 'type01');

Run following

cd /opt/betterdata-mobile/mobile1.8/
bower uninstall page --save --allow-root
bower install page --save --allow-root

Node build.
To activate ETL where needed

we need to do some inserts

1.

 INSERT INTO `etl_types` (`etl_type`, `elt_type_function`, `is_active`, `notes`)
VALUES
	('elasticSubprocess', 'elasticSubprocess', 'T', 'New type for spo with all sdo');


2. Insert needed app codes in etl_appcodes and etl_types_appid_mapping

3. Copy required SDO config files to ETL folder.

4. Insert control entry for community in  etl_control

example :   

INSERT INTO `etl_control` (`etl_type`, `communityId`, `pending_approved`)
VALUES
	('elasticSubprocess', '644a37d3-cf43-4b62-94b2-48c25923de4d', 'approved');

Set up the community short codes for elasticSearch. Please run the queries in this order for EPWP only. For other contexts run the first only.

update community set `elasticCode` = mid(communityid,1,locate("-",communityid,1)-1);
update community set `elasticCode` = MID(communityid,1, LOCATE("-",communityid,7)-1) WHERE `elasticCode` IN ('ec','nat','fs','wc','nc','nw','mp','lp','gp','kn');
update community set `elasticCode` = 'epwprs' where communityid = '962dd628-36b5-4a41-b959-a6e8827b6a7e';

Make sure that the appcodes for each app is in the etl_appcodes AND etl_types_appid_mapping tables

CONSOLIDATED STEPS FOR EPWP UPGRADE

Stop node
Stop exist
Change nginx to stop global access of couchdb from outside
Take backup of CouchDB
Take backup of MySQL
Do config build for required communities community
Sync MYSQL schema from DEV server
Install elastic and Kibana
Copy jars 16+2 from DEV server from following location /Script/build/exist-jars/lib
Install fonts

        yum install curl cabextract xorg-x11-font-utils fontconfig
        yum install https://downloads.sourceforge.net/project/mscorefonts2/rpms/msttcore-fonts-installer-2.6-1.noarch.rpm

        To verify if your fonts are installed properly run following commands

        sudo fc-cache -fv
        fc-list | grep "Arial"

        If you do not find any result in grep then install above packages using "reinstall" command again.

Do eXist build and merge mod server
Run following MYSQL commands

    INSERT INTO `changes_feed_control` (`db_name`, `bookmark`, `status`, `mode`, `lastRun`, `lastUpdated`,  `is_active`, `type`) 
    VALUES ('all_dbs', '0', 'Updated', 'setup', now(), now(), 1, 'all_dbs');
    INSERT INTO `roles` (`roleId`, `roleName`, `description`) VALUES (3, 'Guest', 'Guest');
    INSERT INTO `roles` (`roleId`, `roleName`, `description`) VALUES (4, 'UserManager', 'UserManager');
    UPDATE user_ SET lastVisited = '2000-01-01 00:00:00';
    update community set `elasticCode` = mid(communityid,1,locate("-",communityid,1)-1);
    update community set `elasticCode` = MID(communityid,1, LOCATE("-",communityid,7)-1) WHERE `elasticCode` IN ('ec','nat','fs','wc','nc','nw','mp','lp','gp','kn');
    update community set `elasticCode` = 'epwprs' where communityid = '962dd628-36b5-4a41-b959-a6e8827b6a7e';

Replicate kwantu_configs for folling docs

    "System_Settings",
    "nodeResult_chart1_template",
    "nodeResult_chart2_template",
    "nodeResult_chart3_template",
    "nodeResult_chart4_template",

Replicate kwantu_geo database
Make sure you have c++ version 11+ otherwise install it as below

                    yum install centos-release-scl
                    yum upgrade --skip-broken
                    yum install -y devtoolset-7
                    scl enable devtoolset-7 bash
                    export CC=/opt/rh/devtoolset-7/root/usr/bin/gcc
                    export CXX=/opt/rh/devtoolset-7/root/usr/bin/g++

Install glibc

                    mkdir ~/glibc_install
                    cd ~/glibc_install
                    wget http://ftp.gnu.org/gnu/glibc/glibc-2.14.tar.gz
                    tar zxvf glibc-2.14.tar.gz
                    cd glibc-2.14
                    mkdir build
                    cd build
                    ../configure --prefix=/opt/glibc-2.14
                    make -j4
                    sudo make install
                    export LD_LIBRARY_PATH="/opt/glibc-2.14/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"

Run follwing on terminal

            cd /opt/betterdata-mobile/mobile1.8/
            git fetch --all
            git reset --hard origin/Production02
            npm config set strict-ssl false
            npm install --only-prod
            bower uninstall page --save --allow-root
            bower install page --save --allow-root

Install latest node with

            nvm install --lts
            nvm use --lts
              
              OR

            npm install -g n
            n lts

Run createIndex service for all active and non-active communities
Leave server for 15 mins to settle indices
Run following for all communities

    https://<servername>/exist/rest/db/kwantu-resource/migration/recentFilesUpdate.xq?communityId=<COMMUNITY_ID>

# this query will prepare the curl script
SELECT CONCAT('curl --noproxy "*" "http://localhost:8080/exist/rest/db/kwantu-resource/migration/recentFilesUpdate.xq?communityId=',communityid,'"  >> tocurl.out') 
FROM mobile1_8.community;

Processes file - variable file and deleted processes migration - Mandatory for all communities

Important: This migration also includes variable file migration so this is mandatory for all such communities which are using EPWPRS project.

This migration needs to be ran using ETL processor... to run ETL insert following entry for each community in the database. Do run this for all active and inactive communities

# INSERT INTO `etl_control` (`etl_type`, `communityId`, `pending_approved`) VALUES ('processesIterator', '<COMMUNITY_ID>', 'type01');
INSERT INTO `etl_control` (`etl_type`, `communityId`, `pending_approved`) 
SELECT 'processesIterator', communityId, 'type01' FROM community;

IMPORTANT: DO "NOT" make couchDB accessible to outside

restart exist
start node
wait till the time changes_feed_control all_dbs entry is in setup mode.
Once changes_feed_control moves to execute mode then only
make CouchDB accessible to outside world

4.0.33 Build Steps

Prepare databases for kwantu_configs and taxonomies on staging server. This includes several steps so please discuss with Satinder.
eXist build
MySQL sync
Replicate new databases for config and taxonomies from staging server to the target server
Run Query update user_ set buildAccess = 0;
Run /db/kwantu-resource/_tools/userpreferencesMigration.xq (It will take around 15 mins, so monitor its completion from monex as browser or rest client will timeout after 10m as per setting in nginx but it will keep running behind the scene till completion.)
Update nginx for http2: In your nginx config for the domain

edit listen 443; to listen 443 ssl http2;

Now restart nginx
Run following

cd /opt/betterdata-mobile/mobile1.8/ git fetch --all git reset --hard origin/Production01 npm install --only=prod sh /opt/buildNode.sh

back

Build steps for the latest build

Build for 4.0.37

Note: The builds for exist are now taken from staging.kwantu.net and no longer dev.kwantu.net

stop node
exist build
check for MySQL schema change
Run createIndex.xq service for all communities (i.e. without any parameter)
Node build
replicate System_Settings document in kwantu_configs bucket
Start node

Build for 4.0.39

Drop constraint on mysql user_.mobile
Replicate System_Settings in kwantu_configs
Exist build
Node build
create kwantu_media database
Make sure ngnix file size is 200m - for video upload capability where required
Taxonomy replication

Specific for DESTEA

Make sure DESTEA Incentive users are changed to mobile as username

Specific for EPWP to deal with biometrics

EPWPMonthlyAttendanceBulkUpload (v1)
EPWPMonthlyEmploymentPerLocality (v2)
EPWPLocality (v2)
Taxonomy replication
Config build

Build for 4.0.40

MySQL schema sync
eXist build
Replication System_Settings, nodeResult_chart4_template
Community config builds as needed
Node build

Build for 4.0.43

MySQL schema sync
eXist build
Node build
Community config builds: for all the updated forms
Run the following processes for each community

5.1 Run reindex for the community
5.2 Update the historical forms for EPWPMonthlyAttendanceBulkUpload version 1
5.3 Update the historical forms for EPWPMonthlyEmploymentPerLocality version 2
5.4 Update the historical forms for EPWPLocality version 2
5.5 Update the historical forms for EPWPMonthlyProgressSummary version 1
5.6 Update the historical forms for EPWPDailyAttendance version 1
5.7 Update the historical forms for EPWPDailyAttendanceReview version 1

Upload the latest version of the EPWPRS Biometrics App to the google play store, and the huawai store
Make sure that the SMS engine works

ARCHIVED

New build step document link is here

Build Steps to 4.0.43 - kwantu/platformconfiguration GitHub Wiki

ARCHIVED

Build Steps to follow to build up to 4.0.43

*Stop node before any build

Upgrade revision 3.x.xx - upgrade to - 4.0.11

Upgrade revision 4.0.11 - upgrade to - 4.0.13

Upgrade revision 4.0.13 - upgrade to - 4.0.14

Build 4.0.19

Migration for enabling grids

Others

Build 4.0.20

Build 4.0.24

Build 4.0.26

This is critical release so we need to stop node before build and make couchdb inaccessible from global domain using ngnix to stop any replication from outside

CONSOLIDATED STEPS FOR EPWP UPGRADE

IMPORTANT: DO "NOT" make couchDB accessible to outside

4.0.33

Build Steps

Build steps for the latest build

Build for 4.0.37

Build for 4.0.39

Specific for DESTEA

Specific for EPWP to deal with biometrics

Build for 4.0.40

Build for 4.0.43

ARCHIVED

⚠️ GitHub.com Fallback ⚠️

Build Steps to 4.0.43 - kwantu/platformconfiguration GitHub Wiki

ARCHIVED

Build Steps to follow to build up to 4.0.43

*Stop node before any build

Upgrade revision 3.x.xx - upgrade to - 4.0.11

Upgrade revision 4.0.11 - upgrade to - 4.0.13

Upgrade revision 4.0.13 - upgrade to - 4.0.14

Build 4.0.19

Migration for enabling grids

Others

Build 4.0.20

Build 4.0.24

Build 4.0.26

This is critical release so we need to stop node before build and make couchdb inaccessible from global domain using ngnix to stop any replication from outside

CONSOLIDATED STEPS FOR EPWP UPGRADE

IMPORTANT: DO "NOT" make couchDB accessible to outside

4.0.33

Build Steps

Build steps for the latest build

Build for 4.0.37

Build for 4.0.39

Specific for DESTEA

Specific for EPWP to deal with biometrics

Build for 4.0.40

Build for 4.0.43

ARCHIVED

⚠️ **GitHub.com Fallback** ⚠️

⚠️ GitHub.com Fallback ⚠️