How We Got the Project
We had won an international bid to re-design and re-build the website in TYPO3. From 2004 to 2009 we had been working on TYPO3 extensively which is what won us the project. However, during the post-award analysis and design phase, we
Key Reasons for choosing Drupal over TYPO3
We'll start with a disclaimer that we really liked working in TYPO3. It works very well for some applications, such as for intranets. Our reason for choosing Drupal over TYPO3 for this website was because of the following rationale.
-
Concepts of "Taxonomy" is central to Drupal. These could be a "tag library" which is used for tagging all stories written in a website. TYPO3 had no concept of such a Taxonomy; not to date.
-
Availability of Apache Solr integration module. This module came much later in
TYPO3, and is just now about reaching maturity, and that too in a near-commercial model by the agency which wrote it. The Taxonomy concepts are easily hooked onto Apache Solr for filtering based on meta-information, such as Tag library, Authors, Publications, etc. This can be seen in action beautifully at www.indiaenvironmentportal.org.in -
The TYPO3 Backend from for a News publishing website with hundreds and thousands of
articles, added a level of usability complexity. -
Meta information such as "Authors" are simple entries in tt_news (a core TYPO3 extension - actually the heart of TYPO3), are simple Label entries; in Drupal, however, an Author can be part of a Taxonomy and complete User Profiles can be made for these authors - by default
-
Article/News creation by way of simple forms. Easy bulk publish/unpublish of news
-
Complex news-news and news-article relations easily managed using in-
bulit relationships in Drupal
We must
Management Challenges
We faced many challenges during the design and ideation of the project, which were related to issues of management (at the client and our end) of the project and attrition challenges at Srijan. Both us and the client were concerned about the future of the project. However, both the client and Srijan stayed committed to the relationship. Soon, the client appointed a full-time Project Manager for the project, and Srijan re-committed itself to the project.
Enter OpenPublish
What is OpenPublish
As the OpenPublish website describes, it is:
"OpenPublish has been designed to meet the needs of any publisher – whether large newspaper, TV news site, niche information publication or something in between. It is a flexible solution easily tailored to fit any organization’s needs."
It is based on Pressflow, a performance-tuned implementation of Drupal, has Memcached and Varnish implemented by default, and an Apache Solr integration built in. See the complete feature here.
Research complete; time to roll
It took the same team another 5-6 weeks from research completion to get the website live at www.knr.gl, including migration from the TYPO3 website to OpenPublish.
Migration from TYPO3 to Drupal
Our starting point was the Drupal migrate module and the case study written for migration of The
Analysis of the data to be migrated
We studied the data that in TYPO3 that needed to be migrated. Here's the metric of content we identified to migrate, and eventually migrated.
Do note that Gallery images were migrated in a different manner, and it is for this
Challenges with TYPO3 DB migration
There were several challenges mapping the TYPO3 database structure onto Drupal.
Poor TYPO3 implementation
The TYPO3 implementation done for KNR was a BIG mess. Here are some examples:
- The site allowed photographers to register and upload their photographs in a TYPO3 extension called smooth_gallery. However, instead of one instance of
the gallery to manage all photographers and their photos in albums, a separate TYPO3 page was created for each photographer with their respective name,an an smooth_gallery created and embedded into the page. smooth_galleryinstance offurther created a folder with the photographer name in which all images were finally stored. - There were 1800 pages, one for each of the 1800 registered photographers of the KNR website
Differences between the TYPO3 DB structure and Drupal DB structure
-
An "Author" (internal users at KNR) of a tt_news news story entry is a simple
label entry . Therefore, while the same author may have entered several news items,the name of the same are stored multiple times simple as a field entry. The email entry of the same author could be different. However, while migrating this to Drupal, we had to ensureintegrity of data in terms of the author profiles being made for internal as well as for external users - Photographers who registered on the site to upload their photos. -
These photos were residing independently in
folders, and had to be made available to the News editors for use in the News Stories in the website. Therefore a Digital Assets repository had to be implemented. -
In Drupal, however, an Author can be part of a Taxonomy and complete User Profiles can be made for these authors. Also, the photographs and photo-galleries they made, had to be associated with their profiles
Intermediate Database design
To handle the above situations an intermediate database schema had to be prepared. This would a clean migration of content between TYPO3 and Drupal, according to their own structures.
Incremental Migration
Since the KNR
Here's a sample of the code we used for such incremental migrations for News stories:
$data['name'] = str_replace($data['path'],'',$
$node->type = $type;
$node->title = $data['caption'];
$node->uid = $uid;
$node ->status = $status;
$node->created = $data['crdate'];
node_save($node);
$nid = $node->nid;
$node->nid = '';
$fileInsert = "insert into files (uid,filemime,filename,filepath,status,timestamp) values
('" . $uid ."', '" . $fileMime ."', '" .$data['caption'] . "', '" . $filePath . $sourcePath . $data['name']
."', '" . $status . "', " . $data['crdate'] .")";
db_query($fileInsert);
$fid = mysql_insert_id ();
$galleryInsert = "insert into node_galleries (gid , nid, fid) values (" . $gid .", " . $nid . "," . $fid . ")";
db_query($galleryInsert);
$updateStatus = "update " .$typo3Db . "migrate_image_status set data =" . $data['uid'] . " where name like 'image_migrate'";
db_query($updateStatus);
Converting Latin1 charset tables with UTF8 data set
The TYPO3 site was multilingual - English, Danish and Greenlandic. The TYPO3 DB had Latin1 charset tables with UTF8 data stored (Are you sure about this? How do you know?) which needed to be converted to UTF8 for a Drupal database.
Our initial approach was to change the DB and table charset to UTF8, which would convert Latin1 data to UTF8 with commands like:
- ALTER TABLE {tablename} MODIFY {table column} CHAR(20) CHARACTER SET utf8
- ALTER TABLE {tablename} DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci
But this was actually not working. After searching through Google we came upon this post - http://bit.ly/1RAqTO - which gave us the breakthrough. The solution was to convert the fields to
The TYPO3 migrate module
We've written a TYPO3 migrate module during the process of this KNR website migration, as well as for another client
Business Benefits
-
Srijan worked like a true partner to KNR, enabling them to move the platforms that would serve their needs better, even incurring costs to ensure that KRN received the Drupal implementation that would work better.
-
KNR saw a smooth migration of a huge number of photographs and news stories, even with the TYPO3 website being live
till the OpenPublish version was ready. -
Srijan enabled an end-to-end solution for KNR by bringing in design partners to handle concepts and wireframes.
Learnings for the future
Updated Content Migrations
We had not utilized a feature of migrate_module, which allows for migrating updated content records - such as News, Articles stories - which have already been migrated, and were updated post this migration. We had instead compiled all such updated records based on a Date indicator, and migrated them separately by first doing a roll-back of these stories, and then re-migrating them using the migrate_module itself.
In our next migration, we would like to use the update feature of the migrate_module.
If you wish to get in touch with us, drop us a line below.