Data Services Best Practices Migration Strategies
Data Services supports two methods to move code in the development cycle: export/import and using central repository. Export/import method can be used for a small project, while central repository is better suited for a medium to large project team.
1. Migration Using Export/Import
Export/Import is the basic method Data Services supports to move code between local repositories. Export Editor is used to exclude, or include objects in the export process. The target can be either another local repository with the same version number, or a flat file.
When you import a file, the objects with the same name in the destination repository will be overwritten. It is recommended that you keep a backup copy before importing.
2. Migration Using a Central Repository
In a typical development lifecycle, applications move among several different phases including development, testing, and production. In a migration, application code is copied and new configuration profiles are applied. A single central repository can support the application during these phases with job labeling and projects.
The following diagram shows the central repository as a source control tool during the development phase, and a migration tool to move applications among different phases in the lifecycle.
Workflow when using a central repository for migration
Labeling should not be confused with object versioning. The version information of an object is managed by the central repository as an integer number, and automatically increased whenever an updated version is checked in. On the other hand, labels are descriptive text user enters into the central repository during the development phase usually to indicate a milestone is reached, such as a release is ready for testing. The same label can be defined on different versions of a group of objects (such as objects belong to a project) so that they can be retrieved as a single one. The following example taken from the Data Services technical manual can illustrates the difference between versioning and labeling.
For example, suppose a job is added to the central repository where user 1 works on a work flow in the job, and user 2 works on a data flow in the same job. At the end of the week, after user 1 checks in two versions of the work flow and user 2 checks in four versions of the data flow into the central repository, the job is labeled “End of week 1 status.” This label contains version 1 of the job, version 2 of the work flow, and version 4 of the data flow. Later, user 1 and user 2 continue to change their respective work flow and data flow.
At some later point, if you want to get the job with the version of the data flow with this label, getting the job by its label accomplishes this, whereas checking out the job and its dependents does not.
The label “End of week 1 status” serves the purpose of collecting the versions of the work flow and data flow that were checked in at the end of the week. Without this label, you would have to get a particular version of each object in order to reassemble the collection of objects labeled “End of week 1 status.”
It is a good practice to follow a naming convention for labels. One recommended way is to define each label in the following format:
<project name>_rel_<number>_<date in ‘yyyymmdd’>
For example, for project “dw project 1”, you can label the first baseline of the project as “dw project 1_rel_1.1.1_20150928”. Here a three digit release number is used to indicate major, minor, and maintenance release numbers. You can use your own numbering system to keep track each release.
For more detailed discussions of labeling, refer to Chapter 8, “Working in a multi-user environment”, in the “Data Services Advanced Development and Migration Guide”.
Moving a Local Repository to a New Database Server
The following steps are recommended to move a local repository to a new database server:
- Create a local repository on the new database server with Repository Manager;
- Export the old repository to a file and import it into the new repository;
- Use the Server Manager to delete the old repository entry, and then add the new one.