Data Quality Diagnostic and Assessment (DQD)

The Universal Data Challenge

According to a survey conducted in 2005 by industry consulting firm PriceWaterhouseCoopers, 66% of executives within the Global 2000 say they cannot rely on the quality of their corporate data. Inaccurate financial reporting, uncollected receivables, overpayments, poor product specifications, and excess inventory are only a few of the problems created by inaccurate or "dirty data". They all affect the bottom line.

To achieve and strengthen the maximum benefit from supplier relationships, certain information must be shared, including product information. Without shared access to centralized data, supply chain partners and companies will be unable to realize the promised value from technology investments, or to realize more effective cross-functional decision-making abilities, analysis, and optimization capabilities.

Inadequate Visibility

Organizations that lack global visibility of purchased goods are bound to have fragmented results. These results often stem from disparate sources of product information…or no information at all. These types of issues impact design, procurement, manufacturing, and distribution as well as sales and marketing. Symptoms of this type of disparate data include:

Different part numbering
Different commodity coding schemes
A variety of supplier coding information systems
Multiple units of measures

Without the ability to aggregate, append, cleanse, and normalize this data into a common language and enable adaptation to a variety of use models, it is difficult to collaborate data across an organization. This problem is compounded by the dynamic that exists when working with multiple manufacturing partners, and as organizations expand through mergers, adding additional disparate systems to the mix.

Roadblocks Within The Enterprise

Resolving multi-million dollar data issues is a sensitive proposition for most corporations. Roadblocks may exist for a host of reasons:

Data services expertise is outside the organization's core competencies
The company lacks the inability to anticipate costs and benefits related to cleansing and maintaining massive global enterprise content
They may not have the experience, tools, techniques and resources required to cleanse, augment and maintain data and meet global content challenges
Their technical resources are not sophisticated enough to provide data in formats for every technology use (online, print, procurement, EDI, marketplace, etc.)
Existing technical data is incomplete or non-standardized
The requirements for functional data are complex and difficult to interpret
The organization lacks corporate data governing standards to guide content certification and validation processes
The value of customer access to accurate, up-to-date product information is not clearly defined

Where to Begin?

ByteManagers has developed a fast and easy method for diagnosing data quality problems across all aspects of the enterprise: Data Quality Diagnostic and Assessment (DQD). Engaging to conduct a DQD is fast, non-intrusive, reliable, and provides an accurate picture of the overall health of your corporate data.

Framing the Data Solution: The Data Quality Diagnostic (DQD)

Purpose

Determine readiness of existing available data for solution implementation.
Determine condition and completeness of existing available data.
Determine potential content gaps and identify appropriate actions for cleanup.
Develop proposal for data cleanup.

Prerequisites For The DQD

Solution design should be known or established.
Customer will be prepared to provide complete overview of existing data sources.
Access to data sources is vital for successful assessment.
Data flow diagrams will be reviewed or established.
Existing data status and other customer-specific issues must be thoroughly reviewed.
Current customer data management and data flow systems must also be reviewed.

Resources Required For The DQD

ByteManagers Data Assessment Team
ByteManagers Data Cleansing Analysts
ByteManagers Project Team
Customer identification of existing data sources
Customer knowledge of data available within individual applications and sources
Customer capability for building quick data extracts/dumps from source systems for data analysis
Availability of internal customer business and subject matter experts for review of processed data

Major Activities of The DQD

Discovery Phase

Engage the Data Assessment Team and kick off the assessment
Review customer data use objectives and data status
Review data required for customer solution
Aggregate and map existing data from sources to required meta tables
Implement BMI data analysis process
Discuss and document customer-specific data use requirements as they relate to the stated project objectives
Extract customer data for data analysis

Analysis Phase

Receive and prepare data for analysis and assessment
Consolidate and process data
Perform data analysis
Identify data gaps and key issues affecting successful project execution
Clarification of data evaluation and observations with the customer

Results Phase

Documentation and presentation of the data analysis results
Preparation of Data Assessment Report, including
- Identification of gaps represented within sampled source data
- Identification of inconsistencies between disparate sources of data (referential integrity)
- Itemization of data issues into logical order for assessing, prioritizing, and determining how to address the data issues (staging, cleansing, appending, normalizing, etc.)
- Identification of data issues that have impact on important processes upstream/downstream (requisitioning, invoicing, plant floor maintenance, procurement, warehousing/inventory management, fulfillment, etc)
Presentation of Data Assessment Summary, including:
- Data Condition:
  - Uniqueness (duplicates)
  - Data validity (proper type of data)
  - Referential Integrity (between data sets)
  - Determine if additional processing is required
  - Accuracy where possible (limited)
- Data Completeness:
  - Suitability for successful use
  - Determine data items not available from sources
  - Identify corresponding importance of data completeness (critical, important, minor, etc.)
  - Assess impact on solution and relation to objectives

Deliverables from The DQD could Include:

EXAMPLE
Phase 1: Analysis of the General Health of Your Existing Data Sources (ERP, CRM, SCM, Legacy)
Phase 2: Data Assessment for Condition and Completeness of Data
Phase 3: Data Clean-Up Process Definition Across Systems
Phase 4: Clean-Up Execution and Proposed SOW's
Phase 5: Ongoing Data Maintenance and Process Definition