6. Stand alone migrations - nuvoleweb/integration GitHub Wiki

The Integration Consumer module is built leveraging the popular Migrate module. Still all Migrate-related components are part of an independent Integration Migrate module, on which the Integration Consumer module depends upon.

In order to user stand-alone migration components we need to enable the Integration Migrate module:

$ drush en integration_migrate
The following extensions will be enabled: integration_migrate, registry_autoload, migrate
Do you really want to continue? (y/n): y

The Integration Migrate module provides two classes that can be used to migrate a set of data exchange documents without relying on any specific module configuration:

AbstractMigration class

The Drupal\integration_migrate\AbstractMigration class extends Migrate's Migration class and must be extended by all custom migration classes that wish to import a set of data exchange documents:

use Drupal\integration_migrate\AbstractMigration;

/**
 * Custom article migration class.
 */
class ArticleMigration extends AbstractMigration {
	...
}

MigrateItemJSON class

The Drupal\integration_migrate\MigrateItemJSON class extends Migrate's MigrateItemJSON class and must be used as Migrate item class when setting migration source in all custom migration classes that wish to import a set of data exchange documents:

use Drupal\integration_migrate\MigrateItemJSON;
use Drupal\integration_migrate\AbstractMigration;

/**
 * Custom article migration class.
 */
class ArticleMigration extends AbstractMigration {

  /**
   * Constructor.
   */
  public function __construct($arguments) {
    parent::__construct($arguments);

    ...

    $list_class = new \MigrateListJSON('/path/to/documents/list.json');
    $item_class = new MigrateItemJSON('/path/to/documents/:id.json');    
    $this->setSource(new MigrateSourceList($list_class, $item_class));
  }

}

This will ensure support for multilingual content and standard JSON data exchange format.

A stand alone migration example

Let's walk through a step-by-step guide to build a stand alone migration using the classes introduced above.

Say we have our data stored in our local file-system with the following structure:

$ tree /path/to/documents
/path/to/documents/
└── articles
    ├── document-1.json
    ├── document-2.json
    ├── document-3.json
    └── list.json

Each of the document-*.json contains a separate document having the same numeric id as specified in the file name, so document-1.json would look like:

{
  "_id": "1",
  "type": "news",
  "default_language": "en",
  "languages": ["fr", "en"],
  "fields": {
    "title": {
      "en": ["English title"],
      "fr": ["French title"]
    },
    "abstract": {
      "en": ["English abstract"],
      "fr": ["French abstract"]
    }
  }
}

While list.json just contains an array of document IDs in JSON format, so it will look like:

[
  "1",
  "2",
  "3"
]

Given the setup above say we want to import the JSON documents as Drupal articles using the default "article" content type.

Step 1: Create custom migration class

Our custom class lives in ``custom_module/articles_migration.inc``` and it would look like the following:

use Drupal\integration_migrate\MigrateItemJSON;
use Drupal\integration_migrate\AbstractMigration;

/**
 * Custom article migration class.
 */
class ArticleMigration extends AbstractMigration {

  /**
   * Constructor.
   */
  public function __construct($arguments) {
    parent::__construct($arguments);

    // Set source id and destination entity type.
    $this->setMap(new MigrateSQLMap(
      $this->getMachineName(),
      [
      '_id' => [
        'type' => 'varchar',
        'length' => 255,
        'not null' => TRUE,
      ],
      MigrateDestinationNode::getKeySchema()
    ));

    // Import data as "article" content type.
    $this->setDestination(new MigrateDestinationNode('article'));

    // Entity translation requires that both title fields are mapped.
    $this->addFieldMapping('title', 'title');
    $this->addFieldMapping('title_field', 'title');
    $this->addFieldMapping('body', 'abstract');

    $list_class = new \MigrateListJSON('/path/to/documents/list.json');
    $item_class = new MigrateItemJSON('/path/to/documents/document-:id.json');    
    $this->setSource(new MigrateSourceList($list_class, $item_class));
  }

}

Step 2: Expose migration class to Migrate

In order for Migrate to be aware of our custom migration class we would first need to declare it in our custom_module/custom_module.info file as follows:

name = Custom module
description = Custom module importing articles
core = 7.x
...
files[] = articles_migration.inc

And then implement hook_migrate_api() hook.

/**
 * Implements hook_migrate_api().
 */
function custom_module_migrate_api() {
  return [
    'api' => 2,
    'groups' => [
      'CustomModule' => [
        'title' => t('Custom Module'),
      ],
    ],
    'migrations' => [
      'ArticleMigration' => [
        'class_name' => 'ArticleMigration',
        'group_name' => 'CustomModule',
      ],
    ],
  ];
}

Step 3: Run import

In order to run the import we would need to first register our statically defined migration by visiting admin/content/migrate/configure and clicking on "Register statically defined classes". We can then visit the Migrate dashboard at admin/content/migrate and run our migration.