Copy Data Management and Jenkins Continuous Integration

Catalogic 10/04/2016 0 Comments

Jenkins is a continuous integration solution used by many development organizations to automate software building, testing, and deployment. Copy Data Management (CDM) is a term used to describe software solutions that manage the proliferation of storage, virtualization and application snapshots within an operations context. Solutions in this category make it easy to re-use production copies of systems-of-record for development and testing purposes.

DevOps fosters a culture that bridges operations and development disciplines toward streamlining value creation for the business and end user.  This post will describe how Catalogic’s ECX CDM solution integrates with Jenkins, providing a powerful solution that produces copies of production environments updated with development’s latest code modifications within minutes.  

DevOps Use Case

As a QA member, I would like to have a copy of a live production system with the newest set of components from development to perform testing. Ideally, the creation of this updated production copy should be initiated by a QA or development member in a self-service/on-demand manner.  The entire process should take no more than a few minutes. Here is the activity that is expected to be carried out:

A self-service portal will facilitate the selection of a specific production copy version that will be leveraged for testing the latest changes from development. The latest changes from development may be organized as labeled source control repository branches. The developer or QA engineer will select a particular source code version from a repository along with a version of a production environment and then perform an operation to spin up the copy.  A Jenkins project will  be executed that will perform the following activity:

  • Pull the specific source code from the source code repository (git or SVN)
  • Transfers the source to a build machine, which runs a Jenkins slave agent
  • Run the build to produce the binaries and executables
  • Deploy the copy of a production environment
  • Update the production environment with the binaries and executables
  • Report back to the developer or QA engineer with job status

Jenkins-CDM Plugin

Integration between ECX-CDM and Jenkins makes the above use case possible. This integration takes the form of a Jenkins plugin that participates in the life cycle of a Jenkins Continuous Integration job which calls into ECX-CDM via its REST APIs.  The following is a high-level illustration of the plugin and integration:



The diagram depicts the Jenkins workflow leveraging the plugin to spin up a copy of a production environment so that it can push updates to it. The plugin’s GUI facilitates the following:


  • Allows one to define, through the Jenkins user interface, login credentials to ECX
  • Select a particular ECX policy to run that will produce a copy of a production environment

In Conclusion

The following diagram depicts the workflows managed by ECX-CDM and Jenkins leveraging the Jenkins ECX-CDM plugin.


For more details on how Catalogic can help you with DevOps, visit our DevOps solution page. Catalogic can also help if you are using more traditional Dev/Test methods

Jay Hamilton is Senior Manager of Software Research and Development at Catalogic. He has blogged previously on “Delivering Continuous Value to Customers via Agile Software Development and The Marketplace.

Let us show you around