Our client, Greenlandic Broadcasting Corporation (KNR), is the official TV and Radio Broadcasting agency of Greenland. Their website www.knr.gl was originally in TYPO3 (retained at: http://typo3.knr.codespry.com/).
From 2004 to 2009 we had been working on TYPO3 extensively which is what won us the project. However, during the post-award analysis and design phase, we recognised that the project would be better of done in Drupal instead of TYPO3. We had had an experience building www.indiaenvironmentportal.org.in, an environment news portal, in Drupal and the experience gained from this development led us to the belief that news and online mediapublishing sites should be built in Drupal. We had the benefit of having seen both the systems from close quarters.
We'll start with a disclaimer that we really liked working in TYPO3. It works very well for some applications, such as for intranets. Our reason for choosing Drupal over TYPO3 for this website was because of the following rationale.
We must acknowlege anothr key reason. A lot of our developers, after the indiaenvironmentportal.org.in experience started loving working with Drupal more than in TYPO3 because of the "control" they experienced - such as the ability to write Themes using PHP Template, instead of depending upon Typoscript, which was the native TYPO3 scripting tool for theming TYPO3 websites. Typoscript adds advantages for non-technical people, but developers and themers often dislike it.
We faced many challenges during the design and ideation of the project, which were related to issues of management (at the client and our end) of the project and attrition challenges at Srijan. Both us and the client were concerned about the future of the project. However, both the client and Srijan stayed committed to the relationship. Soon, the client appointed a full-time Project Manager for the project, and Srijan re-committed itself to the project.
As the OpenPublish website describes, it is:
"OpenPublish has been designed to meet the needs of any publisher – whether large newspaper, TV news site, niche information publication or something in between. It is a flexible solution easily tailored to fit any organization’s needs."
It is based on Pressflow, a performance tuned implementation of Drupal, has Memcached and Varnish implemented by default, and an Apache Solr integration built in.
It took the same team another 5-6 weeks from research completion to get the website live at www.knr.gl, including migration from the TYPO3 website to OpenPublish.
Our starting point was the Drupal migrate module and the case study written for migration of the Economist Magazine to Drupal.
We studied the data that in TYPO3 that needed to be migrated. Here's the metric of content we identified to migrate, and eventually migrated.
Do note that Gallery images were migrated in a different manner, and it is for this reason that the above screenshot shows 0 in the "migrated column". For migrating the gallery images we used simple PHP scripts, which also took care of "incremental migrations".
There were several challenges mapping the TYPO3 database structure onto Drupal. This challenge was magnified due to a poorly implemented TYPO3 setup on the KNR website.
The TYPO3 implementation done for KNR was a BIG mess. Here are some examples:
To handle the above situations an intermediate database schema had to be prepared. This would a clean migration of content between TYPO3 and Drupal, according to their own structures.
Since the KNR webite (TYPO3) was in production, post UAT, the content would have to be continually migrated; the new Drupal website would have to start serving with the live real-time content. For this an incremental migration process had to be followed for News stories (including images) and for the Photo Gallery and any new photographer user registrations.
Here's a sample of the code we used for such incremental migrations for News stories:
$data['name'] = str_replace($data['path'],'',$
data['name']);
$node->type = $type;
$node->title = $data['caption'];
$node->uid = $uid;
$node ->status = $status;
$node->created = $data['crdate'];
node_save($node);
$nid = $node->nid;
$node->nid = '';
$fileInsert = "insert into files (uid,filemime,filename,filepath,status,timestamp) values
('" . $uid ."', '" . $fileMime ."', '" .$data['caption'] . "', '" . $filePath . $sourcePath . $data['name']
."', '" . $status . "', " . $data['crdate'] .")";
db_query($fileInsert);
$fid = mysql_insert_id ();
$galleryInsert = "insert into node_galleries (gid , nid, fid) values (" . $gid .", " . $nid . "," . $fid . ")";
db_query($galleryInsert);
$updateStatus = "update " .$typo3Db . "migrate_image_status set data =" . $data['uid'] . " where name like 'image_migrate'";
db_query($updateStatus);
The TYPO3 site was multilingual - English, Danish and Greenlandic. The TYPO3 DB had Latin1 charset tables with UTF8 data stored (Are you sure about this? How do you know?) which needed to be converted to UTF8 for a Drupal database.
Our initial approach was to change the DB and table charset to UTF8, which would convert Latin1 data to UTF8 with commands like:
But this was actually not working. After searching through Google we came upon this post - http://bit.ly/1RAqTO - which gave us the breakthrough. The solution was to convert the fields to BLOB, and then the BLOB field to UTF8. A more detailed case of this is available at: http://www.srijan.in/blog/converting-latin1-charset-tables-utf8-data-set
We've written a TYPO3 migrate module during the process of this KNR website migration, as well as for another client East West Center (coming up soon).
This migration has been an exciting project for us, and for the Drupal community as well, as this is probably the first migration from TYPO3 to Drupal, and certainly the first release of a generic TYPO3 migration module.
Updated Content Migrations
We had not utilized a feature ofmigrate_module, which allows for migrating updated content records - such as News, Articles stories - which have already been migrated, and were updated post this migration. We had instead compiled all such updated records based on a Date indicator, and migrated them separately by first doing a roll-back of these stories, and then re-migrating them using the migrate_module itself.
In our next migration we would like to use the update feature of the migrate_module.