Incremental load using Change tracking feature of CRM 2015 Update 1 with SSIS Integration toolkit – Kingswaysoft

Note:  This article assumes experience on SSIS Integration Toolkit for Dynamics CRM and understanding of CRM Online.

Change tracking is a new feature available in CRM update 1 that can help in keeping the data synchronized in source and target systems by detecting the data changes since the data was initially created or last synchronized.

Change tracking is used for incremental load

STEPS:

1. Before we begin to implement change tracking, we need to enable it on the entity for which we need to track changes

   Customizations → Entity → Enable change tracking

• We need to maintain a table in Sql so that the output tokens can be saved for future purpose which will help to maintain a version for the records in the form of tokens

  [Note: The tokens can be maintained in a file also]

  Let’s start by creating the SSIS package using SSIS integration toolkit for Dynamics CRM-Kingswaysoft

  Flow of package will be:

• Drag and drop the Execute Sql task which will be used to fetch the tokens from Sql

  [Note: In case of initial migration, the token will be NULL]
  
  Since initially the table will be empty we need to handle the NULL scenario

  1. Create a variable for maintaining the count of rows for the table(we need to store the result set in this variable created)

• If the count=0 (1^st time migration), it goes to a sequence container passing the input token as NULL
• If the count !=0 , it goes to a sequence container passing the input token as the last inserted token in the table
• Let’s consider we migrating the data for the 1^st time

  The sequence container will contain a data flow task (having the source and target CRM components) and Execute Sql task (storing the new token to Sql)

  Data Flow task:

  1. Drag the CRM source component
  2. Set up the connection, select source type as ‘Entity changes’, mention the entity and define the input and output token variables

  [Note: In this scenario the Input token will be NULL as it is 1^st time migration]

  Now we drag two CRM destination components

  1. CRM Primary Input (Tracking the Updates and Inserts)
  2. CRM Secondary Input (Tracking the Deletes)
  1. For CRM Primary Input destination component, set up the connection, action as ‘UPSERT’, mention the destination entity and upsert matching criteria

• For CRM Secondary Input destination component, set up the connection, action as ‘DELETE’ and mention the destination entity

Execute Sql task:

1. We need to store the output tokens in Sql, using Execute Sql task

• In parameter mapping, we mention the input and output tokens variables that we get from the Data flow task

1. Let’s consider the scenario where we already have the data set up

The sequence container will contain a Execute Sql task (for fetching the last output token from Sql), data flow task (having the source and target CRM components) and Execute Sql task (storing the new token to Sql)

Execute Sql task:

1. We need to fetch the output token from Sql

• The result set can be stored in the Input token variable which we pass to the CRM source component

The steps for data flow task and the execute Sql task are the same as mentioned earlier

• The final package will look like this:
  The package can be scheduled on a daily basis thus keeping the data in sync in the source and target systems.