Data Assurance

Data Assurance

Introduction

Maintainer: chet_metcalf@bmc.com

Data Assurance is a new data quality service for Control-M and Helix Control-M. It provides an integrated solution to evaluate data quality at key checkpoints in your data pipelines. Provided as a SaaS add-on for both Control-M and Helix Control-M, Data Assurance builds on Control-M’s extensive orchestration capabilities, used by enterprises worldwide to develop, run, and monitor critical data pipelines.

Use Case Overview

The Data Assurance workflow demonstrates the Trusted Data Quality use case where an inventory file, employee file, and personnel file are received from various sources. In order to ensure data quality, the files need to be checked before the workflow can continue. Including data quality checks within the pipeline will keep bad data from being processed in downstream workflows.

Use Case Technical Explanation

The ZZZ-Data-Assurance workflow can be run on demand.

If one of the Data Assurance jobs fail, the ending job ZZZ-Data-Assurance-on-HR-files will not run.

Here are short descriptions of the function of each of the jobs included in the workflow:

• ZZZ-Retrieve-Inventory-File – Dummy job to simulate a file transfer of the inventory file.
• ZZZ-List-Inventory-file – An OS job type that lists the contents of the Inventory file.
• ZZZ-Verify-Inventory-file – A dummy job that is a placeholder for Data Assurance on the Inventory file. The job is currently being tested, so has been set to dummy.
• ZZZ-Transfer-Personnel-file – A dummy job to simulate receiving the Personel file.
• ZZZ-HR-Process – An OS job to list the contents of the employee file.
• ZZZ-List-personnel-file – An OS job to list the contents of the personnel file.
• ZZZ-Verify-Employee-file - Data Assurance job where a quality check is performed to validate that the records count in the file is between 2,500 and 6,000.
• ZZZ-Verify-Personnel-file – Data Assurance job where a quality checks are performed to validate that the record count is between 25 and 500, and the Salary is less than $255,000.
• ZZZ-Data-Assurance-on-HR-Files - SLA Job for the process

Demo Environment Information

Environment	Status	Folder
Helix Production	Coming soon!
Helix Pre-Production	Coming soon!
VSE CTM PROD	Available	zzz-data-assurance
VSE CTM QA	Coming soon!