Drupal to Drupal 8 via Migrate API

Drupal 8 MigrationMost of us already familiar with the Migrate module in previous versions of Drupal and I personally have been using it for several years to perform content migrations from different CMS's into Drupal. Migrate module is now part of Drupal 8 core which supposed to make it much easier to use. Unfortunatley this is not 100% true and in order to perform content migration you have to install several additional contrib modules. As of today there is no way to run migrations through interface as it was possible in Drupal 6 or 7 (and personally I don't think you really need the UI).

In order to be able to run migrations in command line you will need to install the following modules:

Migration used to be a piece of configuration, like any other module configuration prior Drupal 8.1, and as of Drupal 8.1 all migrations are plugins.

Migration file contain the source plugin, the processing pipeline, and the destination plugin. The source plugin is responsible for reading rows from some source. The processing pipeline defines how each field in each row will be transformed into a value that is appropriate for the destination. Then the destination plugin takes the processed row and saves it somewhere in Drupal (it could be a user or any other entity).

Drupal to Drupal 8 migration is fairly simple task, in most cases it is one-to-one. Drupal 8 already contains migration templates. Migrate Drupal (migrate_drupal) module is the module that lets you to migrate from older versions of Drupal to Drupal 8 and this might not work well if you're using contributed modules that don't have their Drupal 8 versions.

However migration template help a lot when you partially migrating content from your previous versions of Drupal. For instance you need to migrate only certain taxonomy vocabularies, terms, users or any other data.

Example templates could be found in core modules: node, user, taxonomy, file and etc. Migration templates are located in src/Plugin/migrate/source/d6/ or src/Plugin/migrate/source/d7/.

In this blog post I will show simple examples of YAML templates and Migrate API classes.

I usually use the following structure for my migration modules:

- my_migration/
- my_migration.info.yml
- my_migration.module
  - config/
    - install/
      - migrate.migration.my_migration_blog.yml
      - migrate.migration.my_migration_tags.yml
      - migrate.migration.my_migration_image.yml
  - src/
    - Plugin/migrate/source/
      - Node_Blog.php
      - Term_Tags.php
      - File_Image.php

Node

This is a simple example of how to migrate Blogs from previous version of Drupal to Drupal 8.

migrate.migration.my_migration_blog.yml

langcode: en
status: true
migration_group: Drupal 7 to Drupal 8
dependencies:
  config:
    - migrate.migration.my_migration_blog
  module:
    - my_migration_blog
    - node
id: my_migration_blog
migration_tags:
  - 'Drupal 7'
label: 'Blogs'
source:
  plugin: my_migration_blog
  node_type: blog
process:
  type: type
  title: title
  langcode:
    plugin: default_value
    source: language
    default_value: und
  status: status
  created: created
  changed: changed
  path: alias
  body: 
    plugin: iterator
    source: body
    process:
      value: value
      format: 
        plugin: default_value
        default_value: html_pure
  field_image:
    plugin: migration
    migration: my_migration_image
    source: field_image_fid
  field_tags:
    plugin: migration
    migration: my_migration_tags
    source: field_tags_tid
destination:
  plugin: entity:node
  node_type: blog
migration_dependencies: 
  required:
    - my_migration_tags
    - my_migration_image

Node_Blog.php

/**
 * @file
 * Contains \Drupal\my_migration\Plugin\migrate\source\Node_Blogs.
 */

namespace Drupal\my_migration\Plugin\migrate\source;

use Drupal\migrate\Row;
use Drupal\node\Plugin\migrate\source\d7\Node;

/**
 * Drupal 7 node source from database.
 *
 * @MigrateSource(
 *   id = "my_migration_blog",
 *   source_provider = "node"
 * )
 */
class Node_Blogs extends Node {

  /**
   * {@inheritdoc}
   */
  public function prepareRow(Row $row) {
    // Destination conent type.
    if (isset($this->migration->get('destination')['node_type'])) {
      $row->setSourceProperty('type', $this->migration->get('destination')['node_type']);
    }
    // Get Field API field values.
    $nid = $row->getSourceProperty('nid');
    $vid = $row->getSourceProperty('vid');
    foreach (array_keys($this->getFields('node', $row->getSourceProperty('type'))) as $field) {
      $row->setSourceProperty($field, $this->getFieldValues('node', $field, $nid, $vid));
    }
    // Migrate URL alias.
    $alias = db_select('url_alias', 'ua')
      ->fields('ua', ['alias'])
      ->condition('ua.source', 'node/' . $nid)
      ->execute()
      ->fetchField();
    if (!empty($alias)) {
      $row->setSourceProperty('alias', '/' . $alias);
    }
    // Blog image.
    $image_fid = [];
    foreach ($row->getSourceProperty('field_image') as $item) {
      $image_fid[] = $item['fid'];
    }
    // Taxonomy tags.
    $field_tags_tid = [];
    foreach ($row->getSourceProperty('field_tags') as $item) {
      $field_tags_tid[] = $item['tid'];
    }
    $row->setSourceProperty('field_image_fid', $image_fid);
    $row->setSourceProperty('field_tags_tid', $field_tags_tid);
    return parent::prepareRow($row);
  }

  /**
   * {@inheritdoc}
   */
  public function fields() {
    $fields = parent::fields();
    $fields += [
      'field_image_fid' => $this->t('Image'),
      'field_tags_tid'  => $this->t('Tags Terms'),
    ];
    return $fields;
  }

}

Taxonomy

Migrate Tags terms. Please note this migration script won't migrate terms hierarchy.

migrate.migration.my_migration_tags.yml

id: my_migration_tags
status: true
langcode: en
migration_group: Taxonomies
dependencies:
  module:
    - taxonomy
label: Migrate Tags taxonomy terms
source:
  plugin: my_migration_tags
destination:
  plugin: entity:taxonomy_term
process:
  vid:
    plugin: default_value
    default_value: tags
  name: name
  description: description
  weight: weight

Term_Tags.php

/**
 * @file
 * Contains \Drupal\my_migration\Plugin\migrate\source\Term_Tags.
 */

namespace Drupal\my_migration\Plugin\migrate\source;

use Drupal\migrate\Plugin\migrate\source\SqlBase;

/**
 * Taxonomy: Tags.
 *
 * @MigrateSource(
 *   id = "my_migration_tags"
 * )
 */
class Term_Tags extends SqlBase {

  /**
   * {@inheritdoc}
   */
  public function query() {
    $query = $this->select('taxonomy_term_data', 'td');
    $query->join('taxonomy_index', 'ti', 'ti.tid = td.tid');
    $query->join('taxonomy_vocabulary', 'tv', 'tv.vid = td.vid');
    $query->join('node', 'n', 'n.nid = ti.nid');
    $query->fields('td', ['tid', 'name', 'description', 'weight'])
      ->distinct()
      ->condition('n.type', 'blog')
      ->condition('tv.machine_name', 'tags');
    return $query;
  }

  /**
   * {@inheritdoc}
   */
  public function fields() {
    return [
      'name'        => $this->t('Category name'),
      'description' => $this->t('Description'),
      'weight'      => $this->t('Weight'),
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function getIds() {
    return [
      'tid' => [
        'type'  => 'integer',
        'alias' => 'td',
      ],
    ];
  }

}

Files

Example of how to migrate images.

migrate.migration.my_migration_image.yml

id: my_migration_image
status: true
migration_group: Files
dependencies:
  module:
    - file
label: Migrate Blog Images
process:
  filename: filename
  uri: uri
  filemime: filemime
  status: status
  created: timestamp
  changed: timestamp
  uid: uid
  alt: alt
source:
  plugin: my_migration_image
destination:
  plugin: entity:file
  source_path_property: filepath
  urlencode: true
  source_base_path: /path/to/source/files/

File_Image.php

/**
 * @file
 * Contains \Drupal\my_migration\Plugin\migrate\source\File_Image.
 */

namespace Drupal\my_migration\Plugin\migrate\source;

use Drupal\file\Plugin\migrate\source\d7\File;
use Drupal\Core\Database\Query\Condition;
use Drupal\migrate\Row;

/**
 * File: Blog images.
 *
 * @MigrateSource(
 *   id = "my_migration_image"
 * )
 */
class File_Image extends File {

  /**
   * {@inheritdoc}
   */
  public function query() {
    $query = $this->select('file_managed', 'fm');
    $query->join('field_data_field_image', 'fi', 'fi.field_image_fid = fm.fid');
    $query->join('node', 'n', 'n.nid = ti.nid');
    $query->fields('fm', ['fid', 'uid', 'filename', 'uri', 'filemime', 'status', 'timestamp'])
      ->distinct()
      ->condition('n.type', 'blog')
      ->orderBy('n.changed', 'DESC');

    // Filter by scheme(s), if configured.
    if (isset($this->configuration['scheme'])) {
      $schemes = array();
      // Accept either a single scheme, or a list.
      foreach ((array) $this->configuration['scheme'] as $scheme) {
        $schemes[] = rtrim($scheme) . '://';
      }
      $schemes = array_map([$this->getDatabase(), 'escapeLike'], $schemes);

      // uri LIKE 'public://%' OR uri LIKE 'private://%'
      $conditions = new Condition('OR');
      foreach ($schemes as $scheme) {
        $conditions->condition('uri', $scheme . '%', 'LIKE');
      }
      $query->condition($conditions);
    }
    return $query;
  }

}

Finally once you have migration module ready you run the following Drush commands:

drush ms to see the list of all migrations and drush mi --all - to run all migrations. You could also run migrations individually: drush mi [migration_name]

Full list of Drush migrate commands:

  • migrate-status - Lists migrations and their status.
  • migrate-import - Performs import operations.
  • migrate-rollback - Performs rollback operations.
  • migrate-stop - Cleanly stops a running operation.
  • migrate-reset-status - Sets a migration status to Idle if it's gotten stuck.
  • migrate-messages - Lists any messages associated with a migration import.

More links about migration in Drupal 8: