Chuck Connell of CHC-3 Consulting, Inc. discusses migration from Lotus Notes Domino to EMC Documentum.
I have been working with Lotus Notes/Domino for about twenty years. Rumors of the product’s death have been swirling around the technosphere for the past ten. Variations of the whispers include: “IBM is dropping Notes”, “There will be no new version after the next one”, “All IBM resources are now going into Websphere”, “The NSF storage format will be dropped in favor of a standard DBMS”, etc.
The recent releases of Notes and Domino 8.5, along with Sametime 8.5 (messaging engine built on Domino), should put these fears to rest, at least for a while. Notes and Domino are among the most platform-agnostic products anywhere, running on Windows, Linux, AIX, Solaris, Macintosh, iSeries mid-range, and zSeries mainframe, with an installed base of 100 million seats (assuming you believe IBM’s count). It would take a lot to kill off this product line.
In spite of the fortitude of Notes/Domino, some organizations are still making rational decisions to move off that platform. The reasons include standardizing on one office software vendor, merging formats with other parts of the company, FUD (fear/uncertainty/doubt) spread by Microsoft, and available internal skill-sets to administer and customize the products.
For document-management applications, which are common for Notes, one of the dominant competing products is EMC’s Documentum. I am currently helping just such a migration project, and have learned some things which may prove useful to other people facing the same task.
First, a few general points;
• The goal of migration is a one-time one-way data transfer, with high fidelity for data and formatting. We don’t care about techniques for two-way data sharing between Notes and Documentum, such as environments where both platforms will co-exist.
• The overall data model for Documentum is a document object (such as a Word or PDF file) with some associated “meta-data”. Meta-data includes items that describe the document, such as creation time, author, approval status, etc.
• Lotus Notes has a native format, so you can compose a significant piece of writing directly in a Notes document, usually as a rich-text field. Documentum does not contain a similar native format. So, in general, you do not compose documents directly in Documentum; instead you store them as separate document objects.
• Migration is easy if you only have a small number of Notes documents. In that case, you can do the work by hand with a few minutes of copy-and-paste between the product UIs. But I am assuming that you have hundreds or thousands of documents. So you want methods that process an entire Notes view at once, or at least all the documents selected in a view.
Now let’s look at the pieces of a Notes database you will migrate to Documentum.
Standard data fields – These are Notes fields such as plain text, dates, and numbers. Documentum supports these data types, natively, so they can easily be exported from Notes and imported to Documentum. You can do the export directly from the Notes client, with the command File / Export / Type = Tabular Text. Or you can write some simple LotusScript/Java code to create a similar text file. On the Documentum side, you can import these text files with several of the tools listed below. This kind of text file also can quickly become an Excel spreadsheet, simply by opening the file into Excel, which performs an automatic conversion. This is useful because some of the tools for loading data into Documentum are driven by a spreadsheet.
Rich-text fields – As mentioned above, a Notes document can contain lengthy information in one or more rich-text fields. In many cases, such a Notes document is the entire content that you care about. Documentum does not support rich-text fields however. In general, the “body” of a document in Documentum is a document object of some standard type, such as PDF or Word or PowerPoint. The solution, in my experience, is to convert Notes rich-text fields to PDF, if you will treat the document as read-only in Documentum. If you plan to modify the document within Documentum, convert the Notes rich-text to Word.
There are several Notes-to-PDF converters. I tested DominoPDF. This product is a LotusScript library, so you embed its calls within some relatively simple LotusScript code that you write. I tried it and found excellent fidelity between the on-screen rendering of a Notes document (within the Notes client) and the generated PDF file.
This covers the case where you want the Notes document to become a read-only file within Documentum. But what if you want to edit it? PDF files, technically, are editable, but I found those editing tools to be awkward. For most users, editable files should be in Microsoft Word, because that is the editing software they are comfortable with. So we want a way to convert Notes rich-text to Word. Surprisingly, I found that the best solution was to convert Notes to PDF, using the method above, then convert the PDF files to Word, using an external tool. While the two-step method seems kludgy, I found good results during my tests.
One final word about rich-text fields. When you use these methods for converting Notes to PDF (or Word), you get a rendering of the entire Notes document, not just a particular rich-text field. This may be just what you want, but you should be aware of it at the start. If you really want just one rich-text field converted to PDF/Word, you are able to do that by changing the Notes form used, so that only one field is displayed, and then running your PDF output agent.
Attachments – Notes attachments violate the Documentum model of controlled access to each document. The reason is that a Notes attachment can be detached, modified, and then re-attached, with no record of the events. Also, Notes attachments do not have any version history, check-in/check-out feature, or permission list naming who is allowed to edit them.
So for a Notes document (and its attachments) to become proper citizens of Documentum, the attachments must be stripped off and “raised up” to have equal status to the main document. Then, each attachment can have its own permission list, version history, meta-data fields, etc. The easiest way to do this within Notes is via LotusScript or Java. As each attachment is removed, it is given a unique filename (in case more than one attachment has the same name) and the main document is marked with the filenames of the attachments removed (so the main document still points to them).
Document links – Doc links are an especially handy feature of Notes. Documentum uses a browser client, so it is possible to embed http URLs in a Documentum application (or document object) which map to the same location as the original Notes doc link. Unfortunately, I am not aware of any method to automatically do this mapping during data migration. But it is certainly possible to construct the Documentum URLs with some custom programming that uses the Notes and Documentum APIs.
Work-flow – Some Notes applications can contain complex logic for work-flow. An example is a purchasing application that sends purchase requests to the appropriate person for signature, depending on the dollar value of the item. These applications can also contain email reminders timers, and multiple levels of approval. The programming behind these applications can be fairly complicated. Documentum contains its own workflow feature called “lifecycle”, which can mimic Notes workflow processing. I am not aware of any automatic conversion tool, however, which translates Notes workflow logic to a matching Documentum lifecycle. So re-creating the workflow/lifecycle application may be a manual process after migration.
After considering which parts of a Notes database to migrate, the next task is to choose a tool that will do the actual loading into Documentum. There are a fair number of options, both from EMC and third party vendors, including:
• Built-in import command – This built-in feature (within Documentum Webtop) can import a single file or many files at once, so long as you don’t want to automatically specify different meta-data about each file. If you have exported each Notes document to a separate file, and don’t need variable meta-data, the built-in import command might be all that you need.
• Documentum Foundation Classes (DFC) – This is the built-in Java API for Documentum. If you are comfortable writing Java code, you can include both DFC and Notes API calls in a single program, to perform just about any data migration you want.
• Documentum Discovery Manager with Notes/Domino adaptor – The Discovery Manager is a large product capable of recognizing and importing many data sources, only one of which is Notes. Since this is an official EMC solution however, it is worth considering for your project.
• McLaren DocLoader – This third-party product is driven by an Excel spreadsheet, which lists the meta-data and file object for each document to be imported.
• Blue Fish DIXI – A third-party tool for Documentum import/export that exposes an XML interface to Documentum storage.
• OpenMigrate – A free migration tool from Technology Services Group. This software is a general migration platform, covering more than Documentum and Notes, so it may require more configuration that some of the other tools listed here.
By mixing and matching these resources, you can tailor a Notes-to-Documentum migration strategy to your needs.
Resources and References
Documentum Foundation Classes: https://community.emc.com/community/edn/documentum
Documentum Discovery Manager: http://www.emc.com/products/detail/software/discovery-manager.htm
McLaren DocLoader: http://www.mclarensoftware.com/products/docloader.aspx
Blue Fish DIXI: http://www.bluefishgroup.com/content-migrations/dixi.php
TSG OpenMigrate: http://tsgrp.com/Open_Source/OpenMigrate/open-migrate.jsp
About the Author
Chuck Connell is president of CHC-3 Consulting, which helps organizations use, and sometimes move away from, Lotus products.