Logistics and Issues for the Timeline

This page is about the nature of the software and the required logistics that we need to take into account.

Caveat

Before any decision is made about following these Logistics, the "test of a test" version site will be available soon as soon as the test-of-the-upgrade scripts finish.

Make no decisions yet. Once you see what the tent-pole is for doing the UX chances you can decide.

Frankly there is no real decision to make. The database itself is the key to all of the configuration. New DB means new configuration. Any "configuration work" done in the test-of-test upgrade is going to be lost when the real upgrade happens. See "Rationale" below for reasons.

CAUTION: The current "test of a test" site is where all the muscle memory is going to be achieved. LEARN how and document what you want to do on the new upgrade on the test-of-a-test site. You'll need those notes when it's time to do it all over again on the real upgrade. The DB cannot persist those changes. The database (mysql DB) will be lost and reset once the upgrade begins. There is no way to 'preserve' the changes to the UX on the test-of-test site on the upgrade site.

Some of the necessary steps involve maneuvering within the process using some skills documented here.

Anyway, here goes:

Simple Terms

There are three servers in the mix:

1. The legacy HW that is currently running now.
2. The test HW
3. The TBD production HW

The bottom line is that the legacy site will need to be offline for as long as it takes to do these steps in general terms:

1. Export and copy data (Legacy to Test)
2. Run the PHP per the Upgrade (On Test)
3. Configure UX for the new site (including enablement and configuration of the new Blog/Article/etc features) (On Test)
4. Copy the kit onto the Production HW. (On TBD Production)

All of that -- schedule at least 10-12 days of "tasks" back to back tending the shell to coax everything along.

Ring Zero

Ring-zero means the stuff that is at the center. Two things are at the center:

1. The DNS changes
2. The New Production HW is rented and warmed up (the steps to configure the new HW are a 1-2 day task depending on the delay from the vendor to spin-up a new provision).

Assume unless otherwise noted that the version of the new software License uses is the -TESTINSTALL instance. We only "burn" the production license on the production TBD HW.

Upgrade of Software

When does the upgrade actually start?

The upgrade starts when the current site is disabled (we put in a static HTML page that things are being upgraded and will be back in X days)

That shuts off all access to the site and allows the upgrade to begin.

Step 1 - Extract Database.

The first step is to make a DB extract (via mysql) to generate a extract.sql that is used to load into the new database server. The extraction happens on the Legacy HW and is copied to the Test HW. This process can take up to a day to complete.

The steps for that are documented and I will copy then here. TBD.

Step 2 - Extract files.

This involves rsync between the Test HW and the Legacy HW. Luckily an initial grab has been made so this should not take more than a day for picking up any residual files.

The files have already been extracted with rsync so the next attempt to extract files will only copy those deltas (gallery or any uploads)

Step 3 - Apply Legacy database

This is where the new site begins.

On the Test HW:

Hereafter the term Document Root refers to the directory on the server that corresponds to the root directory of any content of the site. The legacy site used a Document Root of ~user/public_html On the new server it will be /var/www/something That directory is the Document Root and corresponds to the "root" directory in some URL like http://example.com/

The Legacy DB is loaded into the new server MySQL Server. A username and password is created with access to that database on the new MySQL server. Instructions for that are already documented.

Once the DB is loaded, then we move to the files

Step 4 - Copy Legacy Files

This involves the Legacy HW and the Test HW

The files are copied into the new Document Root. This process can take several hours.

Step 5 - Updated Software

On the Test HW:

The updated software is copied ON TOP of the new Document Root. It will overwrite the files that are necessary for the upgrade.

Step 6 - Configuration File

On the Test HW:

The conf_global.php file in the Document Root is edited with:

1. The new database name (if it changed) which it might. But that's not relevant.
2. The new username that the site uses to access the DB.
3. The new password for that DB user.
4. The URL path of the TEST site. (this site will use the -TESTINSTALL suffix on the license so this is a Test Site not the Production Site)

Step 7 - The Upgrade is initiated

On the Test HW:

The admin visits http://domain/admin/upgrade

The next is a long process of:

1. UTF-8 encoding conversions. Recommend using the CLI version of this, it's a tad faster and safer than the Web based invocation. I've written this up in another page.
2. Database conversions during the upgrade. Even after the UTF-8 conversion, then the Updated software has to make referential changes to the database. This takes a long time (1-2 days)

This step (step 7) takes several days to complete. The processing itself is literally a multi-day "wait for paint to dry" operation.

After the Upgrade

On the Test HW:

So once the upgrade is finished and the new software is up and running the site should have the legacy data, legacy files and upgraded software.

Status:

1. The legacy site is still OFFLINE
2. The new site (TESTINSTALL instance) is up and running on http://beta.example.com/

Step 8 - New Features Configured.

On The Test HW:

It is only here at this phase that new features can be "enabled" and "configured"

ANY UX changes including the enablement of Blogs, Articles, etc... are all configured on this TESTINSTALL version of the site.

REMEMBER: The original legacy site is still off-line

The users still have to wait until the UX (enablement, config and UX styling) is done on the TESTINSTALL instance while the legacy site is still OFFLINE We're not ready yet for any production or beta-access to the site. The new DB cannot be changed by users yet.

Step 9 - Prepare for the Production Use

This step involves moving files between the Test HW and the TBD Production HW.

1. The now clean and updated DB is extracted to a dumpfile in Test Server (takes 3-6 hours)
2. The now clean files set is copied to a new directory on TBD Production Server (this takes about 10-16 hours)
3. The DB dump is re-imported under a different database (production_ prefix) on TBD Production Server (takes 4-6 hours)
4. The fresh copy of the files (conf_global.php) is edited to use the new DB name on TBD Production Server.

The beta site is turned off (simply pulling the files away into a directory out of reach)

Step 10 - Turn on the Production Site

On TBD Production HW:

Load the http://example.com/admin/install and point it to:

1. The new production_ database
2. The official URL prefix / domain

Step 11 - The site is upgraded and in Production

This is the point where the new upgraded site is now public, reachable and in use.

Rationale

If you read through this carefully you may have noticed an important issue. Let me be clear.

ALL UX look and feel, all configuration, all setup of ANY feature of the site is stored in the database.

If the database is refreshed, that implies:

1. 2-3 days of UTF-8 conversion
2. 1-2 days of import scripts
3. Redo of all UX and Menu configuration of the entire site that uses Blog/Articles.

We cannot "save" the configuration of how the UX works (Blog/Article) in menu or permissions etc.. and then re-import the DB after that hoping to preserve the UX setup. It all gets blown away.

Which means -- we either are down for an extended period of time while not only the upgrade is performed (waiting for the paint to dry aka PHP scripts), but also waiting for the manual Admin work of UX changes for all of the features and their configuration.

We cannot have it both ways -- update the UX on a test server and then just selectively import those changes into the production site. The database differences are complex and untangling that is not in scope of the Invision upgrade software.

The only reasonable compromise I can make is this: If the desire is to have the "site" remain up while the actual upgrade is worked on with the -TESTINSTALL instance on the new TBD hardware, the legacy (existing) site does not allow for changes -- no posts, no gallery submissions. The site is effectively in a read-only mode while the upgrade is done on the TBD new hardware. This gives people an outlet to browse, but obviously we cannot let the database or files in the filesystem be modified during the upgrade. If we don't want a "read-only" version of the legacy (old) site to remain active during the upgrade, we're left (again) with the notice if being offline for as long as it takes to:

1. Do the upgrade
2. Configure the New/Updated UX
3. Re-align and configure the new Application features (Blog, Article)
4. Any and all UX configurations

The bottom line is that changing the new upgraded site in anyway affects the database of the new upgraded site and re-integration of changes to one part of the database but leaving other parts of the database unchanged is not a path provided by the upgrade process.