Upgrade Migration Scripts - odoo-ps/pshk-process GitHub Wiki

Overview

In general, the purpose of upgrade/migration scripts is to adjust models/fields to a new environment, whether it is from Saas to SH, or upgrading the Odoo version.

Since Odoo Model and Fields are fundamentally database tables and columns, upgrade/migration scripts are implicitly a database schema alteration

Basic

One of the most basic tasks of migration that is often used by TC is data migration.

FCs often create studio fields that will at some point be needed for the development of a module. If the studio field already has a lot of data in it on the production database, instead of just creating a new field, we should also migrate the data from the studio field to our module field.

For these simple data migrations, we are going to create a server action with the following format: migration_1

The script usually follows this format:

env.cr.execute("""			   
update model set new_field = x_studio_new_field;
""")

which will migrate the data of the studio field to the new field (row by row).

On top of data migration, to ensure consistency, we also di view migration. This is because often the studio fields will be put on view where the user can edit. If we don't change the fields on the views that involve studio fields, the data input will still be done on the studio field.

The script for view migration often follow this format:

env.cr.execute("""
update ir_ui_view set arch_db = replace(arch_db, 'x_studio_field', 'new_field') where arch_db like '%x_studio_field%';
""")

which will replace all occurrences of the studio field to the new field.

HOWEVER, PLEASE NOTE THAT: using the above script can be dangerous depending on the field name. Before you have any view changes, you have to first check the field names.

For example:

The client has created 2 studio fields: x_studio_data and x_studio_data_lines
Let's say you want to migrate x_studio_data to a new field: data_content and x_studio_data_lines to data_lines
When you run the script for view change for x_studio_data to data_content, it will replace across all views, which means x_studio_data_lines will be converted to => data_content_lines
Later on, there will be an error saying the field name data_content_lines doesn't exist

Please make sure that you understand everything before you change anything as these scripts are directly impacting the database (bypassing the framework, so checking/validation is done on the database only).

Upgrade/Migration

Script Writing

As mentioned in the beginning, upgrade/migration is implicitly a database operation. The format of the migration file begins like such:

from odoo import SUPERUSER_ID, api
from odoo.upgrade import util


def migrate(cr, version):
    env = api.Environment(cr, SUPERUSER_ID, {})
    cr.execute("SELECT latest_version FROM ir_module_module WHERE name='base'")
    util.ENVIRON["__base_version"] = util.parse_version(cr.fetchone()[0])

Some useful methods:

remove_module(cr, module): removing module, usually put on post-migrate
rename_xmlid(cr, old_xmlid, new_xmlid): Renaming xmlid of specific record, used in conjunction with renaming studio fields
rename_field(cr, model, old_field_name, new_field_name): renaming the field name, usually when transitioning from Saas to Odoo SH, where custom fields are started with x_, and removing x_ on SH.
rename_model(cr, old_model_name, new_model_name: Renaming module name, usually when moving studio model to an existing model. When moving, might need to also change the state from custom to base.

For any other operations, we can usually make do with database manipulation:

env.cr.execute(
"""

""")

Local Upgrade

Upgrade scripts require python file, hence will only affect SH projects/repositories.

Requirements:

Clone upgrade repository

Migration scripts are module/app-based, which means the whole migration directory is created under the module.

Module tends to be structured this way the migration is :

module_name/
|__ models/  # just some models
|__ views/  # just some views
|__ __init__.py
|__ __manifest__.py
|__ migrations/
   |__ version_number/  # i.e. 14.0.1.0.0
      |__ migration_file.py  # will be explained later

There can be multiple migration_file and each file can either be a pre or post migrate.

premigrate files are: written as pre-~migrate.py, they are executed before the module installation
postmigrate files are written as post-~migrate.py, and they are executed after the module installation

Concept

When running upgrade/migration, it will only be triggered when odoo notices there's an increase in the version (defined on the manifest) of the module. Therefore when running a migration script for a module, you would need to have the module 'fake-installed'/'downgraded' first.

The version_number you have to include in the directory dictates which particular migration will be run depending on the version. If in my manifest I put version: 14.0.1.0.0, that means, if the module is upgraded to 14.0.1.0.0, then it will look for directory: 14.0.1.0.0 under migrations/ directory of the module and run the valid script files.

Fake-install

You can run the following SQL statement on the DB:

INSERT INTO ir_module_module (name, state, latest_version) VALUES ('module_name', 'to upgrade', '0.1.0.1');

Which will make sure the module is recognized

Fake-upgrade

Run the following sql statement on the db:

udpate ir_module_module set latest_version='0.1.0.1' where name='module_name';

Running upgrade (locally)

Once all scripts and everything is set up, run the following odoo-bin command:

odoo-bin --addons-path=..... -d db --upgrade-path=""/path/to/upgrade/migrations" -u module_name

To test whether the upgrade is actually executed, you can put a debugger on the line under migrate() method,

migration_2

Or you would notice that the log will show migration is running migration_3

SH Upgrade

Technically, what you have done when you do testing on local upgrading, you should be able to do the same on Odoo SH during the upgrade. The difference in Odoo SH is that the upgrade directory is not found on the SH server. In order to do that, we will need a tool to temporarily put the upgrade directory on the server and run the upgrade.

And that tool is the odev.
Additional to clone: custom-util

You can do the upgrade in 2 ways, manually or by auto/

For the manual one (which is to wait until you drag and drop from dev to staging branch):
odev upgrade-wait --upgrade-repo-path /home/odoo/Github/upgrade --psbe-upgrade-repo-path /home/odoo/Github/custom-util repository_name test_branch_name

For the upgrade-manual which is to actually like running some command directly on OdooSH shell:

odev upgrade-manual --upgrade-repo-path /home/odoo/Github/upgrade --psbe-upgrade-repo-path /home/odoo/Github/custom-util repository_name branch_name -i module_name

Migration in Production:

Use the following command to merge your commit to the production branch and run the util in the migration scripts (be careful no way back!!):

odev upgrade-merge --upgrade-repo-path /home/odoo/Github/upgrade --psbe-upgrade-repo-path /home/odoo/Github/custom-util repository_name branch_name pull_request_number rebase [-i module_name]

Util

There are two util libraries providing the migration helper function.

util Github: https://github.com/odoo/upgrade

Example Usages: rename fields, rename XML ids, remove modules, etc.

from odoo.upgrade import util
util.remove_view(cr, xml_id=view)

custom util Github: https://github.com/odoo-ps/custom-util (previously was in https://github.com/odoo-ps/psbe-custom-upgrade)

Example Usages: rename models and rename fields when you need to change the state of fields to 'base'. Useful when migrating models.

from odoo.upgrade import custom_util
custom_util.custom_rename_model(cr, "x_references", "sale.references")
    field_rename_pair = [
        ("x_name", "name"),
    ]

    for field in field_rename_pair:
        custom_util.custom_rename_field(cr, "sale.references", field[0], field[1])

Localhost testing: (fake install the module with migration script first for a new module)

./odoo/odoo-bin --addons-path=odoo/addons,enterprise,design-themes --upgrade-path=[upgrade util path on your local host if imported],[custom util path on your local host if imported] -d database -u all --stop-after-init

Migrating a SAAS module

Remember to add this when you are migrating the SAAS module (when the module is already inside the SH DB), else the module LOC is still counted as SAAS!! cr.execute("UPDATE ir_module_module SET imported=FALSE WHERE name = 'module_name'")