Thinking Migration

A migration project could be as simple or complex as the system, data, and logic you are migrating.

It is always useful to know the main points you need to consider while thinking about how to approach your migration project, below are the main points that came across my mind from my previous experience:

Data Mapping:

-         Constructing the data mapping sheet, where you will map each field that will be migrated from the source system to the target system, for example, the sheet will contain that field name “First name” that is Mandatory=No and that has Max. length = 200 will be mapped to the new field name “Customer First name” that is Mandatory=Yes and that has max. length=250.

-         You may find like the above example that in the old system the first name wasn’t mandatory but the business has decided that going forward the customer first name will be a mandatory field, in such cases you need to agree on how those fields will be handled either by migrating a static value  like automatically populating the field with ‘XYZ’ or if it is a unique field then XYZ concatenated with the customer ID ‘XYZ-123’, you may also choose to migrate it as null  and implement a process when the user is dealing with the customer record for the first time on the new system that they have to fill it manually.

-         While migrating records that has a financial nature you need to ask if there is any regulation you need to adhere to, for example in our project we had a regulation that the company couldn’t have two invoices generated with the same serial number, accordingly the new  invoice serial number must start after the old invoice serial number. For example, if the old system invoice serial is ‘566’ then the new system invoice serial will be ‘567’. Another way of doing it is to add a prefix to the new invoices serial, you can’t add this prefix to the old invoices serial because if you did and generate this invoice from the new system it will have different serial than the one you already gave to your customer which will be against regulation in some countries.

-         Not only financial records that come with migration complexity, I remember that in the new system we introduced a new field which is a customer status, the customer status is automatically driven from the statuses of the customer’s orders. And that is a logic you need to build as well in your migration activities.

-         Some business requires to distinguish the old system records from the new system records, this is usually done by adding a prefix to the record's serial, take care when doing that you are not violating any regulation.

Data Cleansing:

-         Data cleansing activity is crucial, in the data cleansing you need to help your business by giving them a template that will be used in the migration. This template will illustrate that in the new system the customer first name is mandatory, customer E-mail must have a specific format XX@yy.xx 

-         The business then can use marketing campaigns to help them cleaning their data by generating a targeted marketing list where the first name is null and start approaching those customers to collect the needed information.

-         The earlier the data cleansing activity starts the better results you will get.

Migrating all the records against migrating partial amount of the records:

-         With systems that have been running for quite a few years you will need to think if you need to migrate all the records that were created from the old system to the new system or maybe migrate only the records with creation date in the last 5 years and leave the old system available in a read-only mode for older records.

-         Always think about the regulation.

-         Leaving the old system in a read-only mode is always a wise choice.

Rehearsals:

-         Like we test the system functionalities we need to test the migration logic. Testing the migration logic is done through migration rehearsals.

-         You agree with your business on the number of rehearsals needed, you agree about the set of records that will be in each rehearsal and you agree on the way of validating and reporting the rehearsal results.

-         According to the rehearsal results you may need to refine your migration logic again.

Cutover:

-         You need to agree on the cutover date, if data is entered after the cutover date in the old system it will not be migrated.

-         If your customer runs a business that has a fulfillment process behind it then you need to agree as well on how you will deal with the inflight orders, inflight orders for us where orders that were created before the cutover, but its fulfillment didn’t finish. Accordingly, the order doesn’t have a final status. If I remember correctly in our case, the choice was to complete fulfillment of those orders in the old system and then add them manually in the new system once the fulfillment is completed.

The above were the points I’ve learned from migrating in a big bang project approach where you close the old system and turn on the new system.

Some companies choose a less invasive approach like the parallel run where you run the old and new system in parallel till the company’s users have confidence in dealing with the new system, I am sure that migrating the data in such approach will have its own complexity.

Inspired by the conversation I had with my dear college Drew and from the knowledge I gained from working with the amazing minds Menrit Samir and Dina Waheeb.