Upgrade Process - odoo-ps/psbe-process GitHub Wiki

Upgrade Process (Technical)

Find the code of the project
Make modules installable in an EMPTY DATABASE
Make modules installable in the UPGRADED DATABASE
Tests
Custom Upgrade Issues
Dress Rehearsal
Going to production
Pull requests
Good to know / Problems you may encounter

Find the code of the project

SAAS
- The code should either be on odoo-ps/psbe-custom or odoo/ps-custom
- Look for a branch with the version and name of the customer (there might be more than one branch for the same customer)
- The code for the modules might also be in more than one branch (there could be one branch per module)
- If the code is in more than one branch, keep the latest code of every particular module. ADVISE: Merge them into one unique branch for that customer in that version.
- For upgrades from SAAS to SAAS: If the branch is on /odoo/ps-custom move it to /odoo-ps/psbe-custom. Example on how to do it here.
- For upgrades from SAAS to SH: Follow this steps to move the branch into a new repo and create the new SH project.
SH
- The code should be on a repo in https://github.com/odoo-ps
- You can find it on the Odoo SH project page.
- For projects moved from SAAS to SH without migrating their modules, some/all of the code could still be on odoo-ps/psbe-custom or odoo/ps-custom

Recommendations

For SH projects, clean the repo branches as much as possible.

Remove already merged dev branches.

Make staging branches ISO prod when nocessary.

Make modules installable in an EMPTY DATABASE

Make a local template of your db with all required modules installed. (Will help you save time during numerous drop/create).
- find all standard modules required by your custom project (e.g. via manifestoo)
- create odoo db in the desired version with required std modules installed (e.g. via odev init <new_db_template> <version> <comma_separated_list_of_std_module_requirements>; odev docs)
- by e.g.: createdb <new_db_to_test_against> -T <new_db_template>
Create a new branch to work in (Could be one branch per module to upgrade)
- by e.g.: <version>-<upgrade>-<trigram> or <version>-<upgrade>-<module>-<trigram>. Like: 15.0-upgrade-aup
Create a new branch to merge the code once it's ready (For SH projects, this should be the expected branch to be on the staging environment)
- by e.g.: <version>-<staging> on SH or <version>-<customer> on SAAS customers. Like: 15.0-staging (SH) or 15.0-customer-name (SAAS)
In the following image, you can find an example of good practises on how to work in the repo on the different stages of the Upgrade Process:
Add pre-commit to the project for the intended version and Run it on all the files.
Check the warnings and fix them, check the code modified by pre-commit and commit it. You can create a PR for the code to be reviewed.
For __manifest__ tips, check the following point.
Check __manifest__ file:.
- Upgrade the version, the new convention for manifest version is: ODOO_MAJOR.ODOO_MINOR.MODULE_MAJOR.MODULE_MINOR.BUGFIX.
  For example: 15.0.1.0.0 where 15.0 describe the full odoo version and 1.0.0 describe the module version.
- Sometimes dependencies can be cleaned.
- Replace description with summary.
- If not present add:
  - "author": "Odoo PS"
  - "license": "OEEL-1"
  - "website": "https://www.odoo.com"
Case moving from Saas to Sh:
1. Convert fields and models from XML to PY.
  - Create a model.py file for every new model or standard model that has fields added.
  - Rename x_ models by removing the x_ prefix, and replacing _ with . (except when model without x_ already exists).
  - Rename fields by removing the x_ prefix. (Except when field without x_ already exists).
  - If the models/fields are not in English, you can translate them (it's recommendable to leave a comment next to the model/field with its previous name, so you can find it easily in case there are references to it). This is NOT mandatory, it's just improvment for the developers, not for the customer. If it's likely to create more work than advantages, avoid it, or leave it for the end. (You might need to fix views, reports, templates, translations, actions, ...).
2. Everything that uses models/fields should be adapted to use the new name. (E.g.: domains, views, filters, ...).
3. Convert Server Action from XML to PY with comment (E.g.: #SA756 for server action 756):
  - Move the Server Action to the corresponding model.py file (create a new file if necessary).
  - Replace call to actions for model method.
  - Pay attention, some Server Actions could have been created to update every model record after a development and might not be required anymore.
  - Upgrade Server Actions not declared in the modules if referenced in views/code.
Check views:
- If views are declared in a single file, divide the file into separate files (one file for every model that it contains).
- For inherited views, check if its references to standard still exist: inherit_id, xpaths, buttons, fields, etc.
- Don't forget to rename x_ models/fields [Case Saas to Sh].
Check reports and templates:
- If reports/templates are declared in a single file, divide the file into separate files (one file for every model that it contains).
- For inherited reports/templates, check if its references to standard still exist: inherit_id, xpaths, buttons, fields, etc.
- Don't forget to rename x_ models/fields [Case Saas to Sh].
- Check for bootstrap classes that changed in newer versions, you can find the most relevant in the Bootstrap cheatsheet wiki.
- Reformat reports/templates into <template> tags if they're declared in <record model="ir.ui.view">.
- If reports/templates are in the views folder, move them to a reports folder.
Clean code as much as possible:
- Clean commented code (either upgrade it or delete it).
- Examples on what to check on the Code review guidelines (files, models, fields, tests).
Upgrade every reference to __export__ module xmlids, into the custom module in a data.xml file.
Replace references of ids to xmlids. Upgrade every reference without xmlid or whose xmlid belongs to __export__ module, into the custom module in a data.xml file.
Install your module in an empty db:
- If everything is working at first time, try again. Are you sure you changed your Odoo version?
- Not enough? Install with all non-Python files commented.
- Crash? You can comment and add a tag for later review (E.g.: TODO-UPG, FIXME-UPG).
- Don't forget to fix everything you postpone.
Regularly reinstall from an empty db if you encountered crashes. Some bugs may not show up again with just an update of the modules.
Test your module:
- Test every custom view (E.g.: tree, form, calendar, ...): Check that fields and buttons appear where they have to (and if they have invisible, readonly or other conditions).
- Test every custom report, template: Check fields, bootstrap classes, styles, ...
- Test that you're able to create/read/edit/delete every custom model/field.
- Test computed fields.
- Test Server Actions (E.g.: on create, edit, buttons, ...).
- Tests crons, security (groups, records rules, access rights) and every other custom development present.
- Test usual flows according to the feature's list (both developer and functional).
Make sure your branch is green on Sh.
- Ideally, fix standard tests if needed.
- As last resource, you can Monkey patch standard tests.
Write tests so that future devs and upgrades will be easier to check. (About 1-2 days).
At this stage, your branch should be GREEN and your code CLEAN. Everything is WORKING on an EMPTY DB.

Recommendations

If the modules are big, create one branch to work on each of them.

Tackle one module at a time (Upgrade code, test, make it green on SH, ...).

Create a PR for each module when installable in an empty db, so it's easier for the reviewer.

Make all modules installable in an empty db before moving to the upgraded db.

Make modules installable in the UPGRADED DATABASE

Request a new upgraded database:
- SAAS:
  - Download the database from db/_odoo/support/dump.zip
  - Request a new upgraded database on the Upgrade platform
  - Once the upgrade is finished, download the upgraded database from the upgrade platform.
- SH:
  - If you have a staging branch for the upgraded version, you can use the SH Upgrade Feature to request a new upgraded database.
  - Once the upgrade is finished, download the upgraded database from the upgrade platform.
Check the upgraded database you got:
- Are there new apps installed? (Pay attention that extra apps means extra fees. If not necessary, contact Standard Upgrade Team for case-by-case solution)

Make a template of the upgraded db. (Will help you save time during numerous drop/create).

On a terminal run:

createdb <name_templae_db>;
psql <name_templae_db> < <path_to_upgraded_db_dumpl.sql>;

Neutralize the local database with the Neutralization scripts.

You can also run some cleaning scripts so it's easier to login on the database:

UPDATE res_users SET login = 'superadmin' WHERE id = 1;
UPDATE res_users SET login = 'admin', active = True WHERE id = 2;
UPDATE res_users SET password = 'admin';
DELETE FROM fetchmail_server;

Update the data (More info in the Update the data - Migration scripts wiki):
- Rename records xmlid from __export__ module.
- Rename x_ models/fields [Case Saas to Sh].
- Activate custom views that could have been archived during the upgrade.
Case moving from Saas to Sh:
- Set the custom modules as NOT imported.
  On SAAS, custom modules are imported. When we transfer the project to SH, those modules have to be set as not imported so the CLOC (Tool used to count the custom lines of code on a database) takes them into account to invoice the customer for their developments.
  To do so add a migration script to set the modules as not imported. You can use the set_not_imported_modules function on Custom-util to do so.
Try to upgrade the custom modules on the upgraded database and enjoy multiple fixes.
- Crash? Fix issues: Use migrations scripts to fix data, remove deprecated records, ...
- Deactivate all custom views.
Test your modules and keep enjoying multiple fixes:
- Re-test views, reports, actions, flows.
- If required, compare the behavior with the original db.
Check if there are studio_customizations (active and inactive) (E.g.: in views, reports, fields, ...):
- You can always wait for the customer's feedback when testing to fix them. But if there are just a few and it is an easy fix, it is recommended to do it earlier in the process.
- Check if the active ones are working fine.
- If there are inactive studio customizations, try to activate and fix them (only if active in the original db).
At this stage, everything is WORKING on the UPGRADED DB.
Prepare a test database for the reviewer/customer:
- SH:
  - The use of the SH Upgrade Feature will provide a testing database when the commit with last fixes is pushed.
  - If the project has a reviewer assigned, let the reviewer know the test database is ready so they can test it and ensure there are no blocking issues before delivering the database to the customer.
    The reviewer can also create Custom Upgrade Tickets with the issues they find the database for you to fix.
    The reviewer is responsible to let the customer know the database is ready to be tested.
  - If there's no reviewer, let the customer know their database is ready to test. Check step 6 on the Upgrade Process: Functional wiki.

Recommendations

Keep the same strategy when making modules installable on an empty db:

Use one branch per module.

Tackle one module at a time (migrations scripts, test, ...).

Create a PR for module.

After everything is merged you can create new branches for fixes, tags check (E.g.: FIXMEMIG to fix, CHANGEDMIG), feedback from customer, ...

Tests

For client testing purpose, ask for an upgraded db with up-to-date data.
Make as many tests as possible in collaboration with the functional and the customer.
Don't forget the mail.template, translations, daily flows, website, ...
At this stage, everything is FULLY TESTED.

Custom Upgrade Issues

When the reviewer or the customer find a problem on the upgraded database, they will create a ticket. This ticket is created in PSBE Custom Upgrades Issues project and the upgrade task is set as its parent.
There should be one ticket per issue found in the database.

For each Custom Upgrades ticket you are assigned to:

Create a new branch from the upgrade staging branch.
Suggested name for the branch: <version>-<task-id>-<trigram> or <version>-<task-id>-<small-issue-detail>. By e.g.: 15.0-123456-aup or 15.0-123456-account-report
Once the issue is fixed: commit, push and create a PR to the upgrade staging branch.
Remember the Commit guidelines: [UPG][TASK_ID] module_involved: brief description
Once the PR is approved, squash and merge the commit/s.
Remove the development branch merged.
Test the fix on the upgraded database.
Let the reviewer/functional know the fix is ready to be tested. Send the new database link if necessary.

Remember to move the ticket to the corresponding stage when needed (Functional - Technical - Cust. Feedback - Done)

Dress rehearsal

A few days before going to production (2-3 days) request a new upgraded db to make sure everything is still working properly. Send it to the customer to test too.

N.B. if on SH, be sure to exit and re-enter the upgrade mode on the staging branch, wait for the upgrade to complete on the platform, then trigger a new build (push an empty commit or redeliver the last webhook for the branch). Only this way you're sure you're testing the upgrade on the latest version of production data (well, the latest backup really).
Make a list of all steps for the day of the final upgrade - Read the wikis for going to prod in case you have any question about it.
Test the whole upgrade process and track the time to communicate to the client the down time period.
Check the Lines of Code of the upgraded database:
- Connect to the Shell of the staging branch and run odoo-bin cloc.
- Compare the LOC with the current production database, module by module and the total.
- If there's a difference >= 100 LOC (more or less), analyse where it comes from (pre-commit, clean code, migrated fields, views, ...).
- Inform the sales about the potential upsell (or downsell) coming.
  Provide as much information as possible (ideally, more detailed the more difference there is, mainly for upsells).
Now, you're READY TO UPGRADE.

Going to Production

Let the customer know the upgrade process is about to begin. Remind them they should not use the database.
For SH projects, the production branch MUST be named main (Special exception for branches called master).
If the the production branch's name is not main or master, it must be renamed the day of the upgrade to ensure minimun downtime for the customer.
Follow the steps on the Rename production branch wiki.
Upgrade the Production database:
- SH: Use the Upgrade Feature.
- SAAS: More info in the Saas to Saas upgrade wiki.
- SAAS to SH: More info in the Saas to SH upgrade wiki.
Connect to the upgraded database and check that everything looks okay and it's in the right version.
Connect to the upgraded database support page (_odoo/support) and make a "Ping!" to update the version of the database on Odoo.
If the project has a reviewer assigned, let the reviewer know the upgrade is ready so they can take a look and inform the customer.
If there's no reviewer, let the customer know their database has been succesfully upgraded and they can use it again.
If there's no reviewer, set the upgrade to "Done". All the subtasks should be closed (Done or Cancelled).
Otherwise, this step should be done by the reviewer.
Let the maintenance team know the upgrade is finished. Send a message to the Maintenance channel. Don't forget to tag the team.
- by e.g.: Hello @Team: Maintenance, [CUSTOMER] is now upgraded to [VERSION NUMBER] 🥳 Their maintenance tickets can be assigned to me
Ring the bell 🔔 and enjoy the good work done! 🎉
Stay available for urgent fixes: Proactively check for new tickets/issues on the Help and PSBE - Maintenance projects with filter on the subscription during 2 weeks post production.

After the upgrade, set the upgraded staging branch ISO prod. You can use:

git checkout main
git pull
git checkout <staging-branch>
git reset --hard origin/main
git push --force

Pull Requests

All the code pushed to the production branch of the customer should have been reviewed at some point.

For us, the best moment to do so is before the customer begins the tests on that code.
That means, that the code has to be reviewed when attempting to merge into staging.

For that we create Pull Requests (PR) on GitHub.

To create a Pull Request, once the code is ready and on a Development branch:

On the repo, select the option Pull requests.
Click "New pull request".
Select the base branch (branch you'd like to merge your changes into) and the branch that contains your commits. GitHub will show the difference between them (both commits and files).
If everything is okay and there are not conflicts, click on "Create pull request".
Write a title for the PR and fill the template in the description with:
- Summary of the changes
- Link to the task
- Checkboxes
- All the information you consider useful for the reviewer to know before checking the code.
Assign the PR to PSBE-upgrade-reviewer
A reviewer / team of reviewers will check the code and approve it or suggest changes / improvements on it.
If necessary, fix the code according to the reviewer suggestions.
Once the PR is approved, you will be able to merge the changes.
Take into account that there are different kind of merges:
- Create merge commit
- Squash and merge (recommended)
- Rebase and merge
Once the development branch is merged, delete it from the repository.

Recommendations

Don't wait to have all the modules tested and ready to merge in staging to create the Pull Request.

Smaller PRs keep the reviewers happier. And take less time for you to get feedback.

You can create partial PRs (one per module, only python files, only views, only templates, ...).

You can create an intermediate branch not in the staging environment, merge the changes there in small batches with PRs and then merge that branch into staging when everything is ready. If all the code was already reviewed, you won't need a new PR for the staging branch.

Good to know / Problems you may encounter

Case upgrade from v12 to a newer version:
- Records from account.invoice have been moved to account.move.
- Fields with Many2one to account.invoice are now pointing out to account.move.
  (Handled by Standard Upgrades Team for integrity constraint respect).
%refd for record reference is interpreted even when commented. -> Temporarily remove the %.
Big database:
- is it because of Binary fields that are stored in the database instead of filestore? (see project Chou)
ids in code:
- Always use external id. If the data has to be moved from one table to another, the ids will change, not the external id.
- Create the record you need so that you can use it even on an empty db.
- Create the external id associated with the record on the customer's db before the upgrade request.
- Or recreate the record and remove the old one by script when the data is imported in db.
Green branch:
- Making your branch green will help you save time during the upgrade!
- Keep in mind, errors are not shown all at once. If you don't fix the first, you won't know about the second one.