Migrate package activity - ckan/ckan GitHub Wiki

migrate_package_activity.py does a database migration that is separate from the ones done by 'paster db upgrade'. migrate_package_activity.py should ideally be run before upgrading to CKAN 2.9 or higher, but can be done at any point. It is designed to be safely done while CKAN is running.

What it does

migrate_package_activity.py works out what the dataset looked like after every change, and stores these snapshots in the Activity Stream. These detailed views can be viewed in CKAN 2.9 or higher by admins and optionally the public too. They are shown in the Activity Stream pages:

So you can view what it looked like in CKAN at that time, and the diff of each change. The API also includes the dataset, as if you'd done package_show at the time.

Why do it while CKAN is running?

This migration can take a long time - e.g. a few minutes for a thousand datasets up to several hours for millions of datasets. CKAN can be running during the migration without any problem, and modern practice is to absolutely minimize downtime.

How to upgrade and migrate

If you're upgrading from CKAN 2.8 or earlier

Here's the migrate procedure (assuming CKAN's installed in the default way):

cd /usr/lib/ckan/default/src/ckan/
wget https://raw.githubusercontent.com/ckan/ckan/3484_revision_ui_removal2/ckan/migration/migrate_package_activity.py
wget https://raw.githubusercontent.com/ckan/ckan/3484_revision_ui_removal2/ckan/migration/revision_legacy_code.py
python migrate_package_activity.py -c /etc/ckan/production.ini

Now you can upgrade CKAN as normal to 2.9 and beyond.

If you're running CKAN 2.9 or beyond

When you upgrade to CKAN 2.9 or beyond it will remind you during the paster db upgrade step to do this extra migration.

Here's the migrate procedure (assuming CKAN's installed in the default way):

cd /usr/lib/ckan/default/src/ckan/
wget https://raw.githubusercontent.com/ckan/ckan/master/ckan/migration/migrate_package_activity.py
wget https://raw.githubusercontent.com/ckan/ckan/master/ckan/migration/revision_legacy_code.py
python migrate_package_activity.py -c /etc/ckan/production.ini

Questions

For help, ask on stackoverflow and tag it: #ckan