Drupal
A custom drush command to import nodes from RSS/XML to Drupal 8
Sometimes you need to copy nodes from one Drupal website to another where the content types do not have exactly the same structure or where Drupal versions do not match.
When I need to import nodes into a Drupal 7 page, I usally install the Views Data Export module on the old page and create a view that displays the desired content as XML Feed. Then I use the Feeds module on the new page, to import that XML Feed to new nodes. This usually works like a charm even when the new content type has more or different fields, than the one on the old page.
Today, when I needed to copy loads of nodes from a Drupal 6 page to a new Drupal 8 project, I realized that the Drupal 8 Version of my beloved Feeds module has not been fully ported to Drupal 8 yet. Even worse, it broke the site and couldn't be uninstalled using drush or from the GUI. That was also the moment, when I started properly reading the project page and found the author's note: "It's not ready yet. We're hoping it will be ready soon. If you decide to use it, don't be mad if we break it later. "
After playing around with a couple of other modules like Migrate or Backup and Migrate but still failing to import the nodes in my Drupal 8 site, I decided to write a dirty custom drush command to import nodes from a RSS/XML Feed into my Drupal 8 site.
My data source is a RSS/XML Feed, like that:
<nodes>
<node>
<Datum>18.03.2010 - 20:00</Datum>
<Titel>Polen, Opole</Titel>
<Body><p>Some text we gonna read is here.</p></Body>
</node>
<node>
<Datum>19.03.2010 - 20:00</Datum>
<Titel>Radzionkow-Katowice</Titel>
<Body><p>Bozos live here.</p></Body>
</node>
<node>
...
The custom drush command performs three steps:
- Read the XML File
- Format the read data (like dates or file attachments)
- Create a node for each <node> Object in our RSS/XML Source.
The code that reads XML and creates new nodes looks someting like that:
//$feed_source is the full url of our xml source file http://url/to/source.xml
$content = file_get_contents($feed_source);
$x = new SimpleXmlElement($content);
foreach($x->node as $entry)
{
//the $entry object holds all fields from our <node> tag in the XML source.
//In our example this would be $entry->Datum, $entry->Titel and $entry->Body
drush_print('creating new node: ' . $entry->Titel);
//prepare new node for creation
$newNode = array
(
'nid' => NULL,
'type' => 'your_custom_nodetype', //machine name found at /admin/structure/types
'title' => 'Your node Title',
'uid' => 0, //the id of the user that owns this node. 0 == Guest
'revision' => 0,
'status' => 1, //published or not
'promote' => 0,
//'created' => 'optional: properly formatted date string for custom creation date'
'langcode' => 'de', //holds the language code
'field_veranstaltungsort' =>
[
value => 'place for this event',
],
'field_datum' =>
[
//Make sure you use the right formatted date string when.
//inserting data in a date field. See below for some more about this.
value => $strDateFormatted,
],
);
$node = Node::create($newNode);
$node->save();
}
If your new node type contains a date field, you need to make sure, that you convert the date read from the XML source to the right date format for your new node. The right type depends on the field settings of your content type. If it's a date-only field, the right format would be Y-m-d which translates to something like 2016-12-24. If your content type on the other hand uses a datefield with date and time, your string format would be Y-m-d\TH:i:s, which translates to something like 2016-12-24T23:30:00.
Let's go back to our import function and see what this would look like in PHP:
$content = file_get_contents($feed_source);
$x = new SimpleXmlElement($content);
foreach($x->node as $entry)
{
//the $entry object holds all fields from our <node> tag in the XML source.
//In our example this would be $entry->Datum, $entry->Titel and $entry->Body
drush_print('creating new node: ' . $entry->Titel);
$strDatum = $entry->Datum; //date string from XML source ie. 07.08.2010 - 20:00
//create datetime object from supplied date string
$objVeranstaltungsDatum = DateTime::createFromFormat("d.m.Y - H:i", $strDatum);
//incorrect timezone. subtract 1 hour. I couldn't manage to include the timezone in our datestring, so we do it this way :(
$objVeranstaltungsDatum->sub(DateInterval::createFromDateString('60 minutes'));
//get proper format for storing to node
$timestamp = $objVeranstaltungsDatum->getTimestamp();
$strDateFormatted = \Drupal::service('date.formatter')->format($timestamp, 'custom', 'Y-m-d\TH:i:s', drupal_get_user_timezone());
The $strDateFormatted variable should hold the date string in proper format now. As mentioned in the code, I couldn't find a way to attach the time zone in the date string. Maybe this has to do with timezone settings of that field.
To run this code from bash using drush, we have to create a little custom module that wraps around our code. Create a new directory for your module in your installation's modules directory. Ie. /modules/custom/lofi_feeds_import/. Add two files there, lofi_feeds_import.info.yml and lofi_feeds_import.drush.inc.
lofi_feeds_import.info.yml
name: lofi Feeds Import
type: module
description: Import nodes from external feeds
core: 8.x
package: lowfidelity
lofi_feeds_import.drush.inc
<?php
/**
* @file Contains the code to generate the custom drush commands.
*
* lowfidelity heavy industries 2016
*
* import nodes from rss feed
* make sure to adjust the field names and node creation
* function below.
*
* based on these tutorials
* http://valuebound.com/resources/blog/how-to-write-custom-drush-commands-...
* https://code.tutsplus.com/articles/how-to-read-an-rss-feed-with-php-scre...
*/
use Drupal\node\Entity\Node;
//use Drupal\Core\Datetime\DateFormatterInterface;
/**
* Implements hook_drush_command().
*/
function lofi_feeds_import_drush_command()
{
$items = array();
//import appointments from drupal 6 page. views export module
//was used to create a xml view. this is the name of our custom command.
//to run from command line we'd use drush lofi-import-termine http://url.of/our/source.xml
$items['lofi-import-termine'] =
[
'description' => 'import termine nodes from old page',
'arguments' =>
[
'feed_source' => 'The url to the feed',
],
'drupal dependencies' => ['lofi_feeds_import'],
];
return $items;
}
/**
* Call back function drush_custom_drush_command_say_hello()
* The call back function name in the following format
* drush_{module_name}_{item_id_for_command}()
*/
function drush_lofi_feeds_import_lofi_import_termine($feed_source = '')
{
if(strlen($feed_source) <= 0)
{
drush_print('Please specify URL to a RSS/XML Feed' . $feed_source);
return;
}
drush_print('Reading from feed source ' . $feed_source);
$content = file_get_contents($feed_source);
$x = new SimpleXmlElement($content);
$numNodesCreated = 0;
foreach($x->node as $entry)
{
drush_print('creating new node: ' . $entry->Titel);
//creates a new node based on the infos from the entry object
_lofi_feeds_import_create_termin_node($entry);
//nodes counter
$numNodesCreated++;
}
drush_print('------------------');
drush_print('finished importing from feed. created ' . $numNodesCreated . ' nodes');
}
function _lofi_feeds_import_create_termin_node($objEntry)
{
//static settings for new nodes
$nodeType = 'termin';
$userId = 0;
$revisionId = 0;
$bPromoted = 0; //or 1
$langCode = 'de';
//node content read from rss feed
$strNodeTitle = $objEntry->Titel;
$strOrt = $objEntry->Titel; //veranstaltungsort
$strDatum = $objEntry->Datum; //datum ie. 07.08.2010 - 20:00
//create datetime object from supplied date string
$objVeranstaltungsDatum = DateTime::createFromFormat("d.m.Y - H:i", $strDatum);
//incorrect timezone. subtract 1 hour
$objVeranstaltungsDatum->sub(DateInterval::createFromDateString('60 minutes'));
//get proper format for storing to node
$timestamp = $objVeranstaltungsDatum->getTimestamp();
$strDateFormatted = \Drupal::service('date.formatter')->format($timestamp, 'custom', 'Y-m-d\TH:i:s', drupal_get_user_timezone());
//prepare new node for creation
$newNode = array
(
'nid' => NULL,
'type' => $nodeType,
'title' => $strNodeTitle,
'uid' => $userId,
'revision' => $revisionId,
'status' => 1,
'promote' => $bPromoted,
'langcode' => $langCode,
'field_veranstaltungsort' =>
[
value => $strOrt,
],
'field_datum' =>
[
value => $strDateFormatted,
],
);
$node = Node::create($newNode);
$node->save();
}
To finally run our custom command, we'll have to enable the module from either GUI or using drush en lofi_feeds_import. Then run our command using drush lofi-import-termine http://path.to/our/source.xml
<3
Larry E. Lektrik