Toward Improving Enterprise Data Quality: A Perspective

By Ed Johnson (c)  2002-2010

Every enterprise – from the smallest one-person operation to the most complex end-to-end process – continually amalgamates “things” into a holistic system of functional capabilities.  Things amalgamated include people, material, methods, and machines; places, processes, actions, and events; ideas, qualities, and ways of thinking and behaving.

Enterprise functional capabilities get essential meaning and purpose from the relationships between the things amalgamated rather than from the individual things themselves.  It is always through some manner of enterprise “data” architecture (EDA) that things amalgamated are first conceptualized, related, and then eventually made concrete and operational.  Thus enterprise quality derives from amalgamation quality, which derives from EDA quality.  The situation cannot possibly be otherwise.

The proof is quite simple: one cannot say or write anything meaningful about an enterprise without making reference to at least two related things relevant to the enterprise.  For example, "We make bicycles."  This statement mentions "We" and "bicycle" related by "make."  To say "We" or "bicycle" alone makes no sense; neither does to say "We bicycle" without mentioning the relationship, "make."

Thus EDA is both fundamental and foundational to the enterprise.  It reflects inherent interdependencies the enterprise concerns itself with more so than just discrete things alone.  Moreover, the statement "We make bicycles" constitutes an instance of an EDA fragment.

People throughout the enterprise always will make sense of the enterprise by reference to some manner of EDA.  In the absence of a commonly knowable EDA, people will rely solely upon their own internal conceptualization or “mental model” of EDA.  Why?  Because this is what people do; it is human nature to conceptualize.  People are not machines nor can they be entirely reduced to behave like machines.  Here, too, the situation cannot possibly be otherwise.

People tend to accept as information those models they can know the meaning of and that answer their questions, rightly or wrongly, and by which to reject other models as mere noise.  Such noise tends to bring about “information overload,” which tends to lead to value-reducing activity.

This means high-leverage data quality improvement opportunities lie not within implementing EDA, since we now see that no enterprise exists without EDA.  Instead, high-leverage data quality improvement opportunities lie within continually exposing and improving the quality of a commonly knowable EDA.  Such opportunities missed mean high positive business value from data for leading, managing, operating, and sustaining the enterprise will remain hopes and wishes.

A commonly knowable EDA makes it possible to continually improve the quality of data accuracy, consistency, completeness, timeliness, and accessibility in their structure, values, and presentations.  A commonly knowable EDA also makes it possible to continually improve and align people’s internal mental models of the enterprise.

Quality data serve to answer questions, both known and unanticipated, much more readily, more often, and more flexibly with greater agility and value-adding effectiveness than non-quality, value-inhibiting data. Quality data also provide for people to form clearer conceptions and testable theories.

Software tools, however sophisticated, cannot improve EDA for the simple reason software tools cannot know the semantics (meaning) of the amalgamation of things relevant to the enterprise; only people can.  Software tools can, however, facilitate the rendering and subsequent presentation of a commonly knowable EDA.

The enterprise whose lifeblood is data of outstanding quality attains it continually through principled and rigorously defined EDA, the prime guiding principle being a single EDA underlies all enterprise processes.  Again, the situation cannot possibly be otherwise.  Thus the key question simply is, how commonly known and knowable is the EDA?

Much as the human skeletal system provides the foundation for an extremely broad yet highly economical range of human activities, a principled and rigorously defined EDA provides the foundation for broad ranging yet highly economical enterprise processes. The EDA that accurately states and otherwise properly defines and communicates the fundamental and foundational structure of the enterprise engenders enterprise agility, adaptability, flexibility, and dependability in the face of rapidly changing demands from both inside and outside the enterprise.

But when EDA goes unknown and unseen, disparate and redundant, verbose and non-communicative, or too specific and too lacking of abstraction, because EDA wrongly is believed to be strictly an information technology artifact, data of poor quality invariably arise to inhibit enterprise functional capabilities, as when redundant bones and broken bones inhibit human behavior.

Data of high quality arise when

• EDA is relied upon explicitly rather than tacitly
• EDA comes forward through semantic modeling methods
• EDA comes forward from enterprise business semantics (Figure 1) rather than from disparate information technology designs (Figure 2)

Disparate information technology designs usually originate with heavy reliance upon software tool functionality and KISS theory, where KISS theory too often amounts to justification for rejecting the quality way.  In spite of the complex real world, software tool functionality and KISS theory often become standards that aim to assure quality but invariably assure conformity.  Conformity differs remarkably from quality, just as quality assurance (QA) differs remarkably from continual quality improvement (CQI).

An examination of the terms in Table 1 reveals a concept the term quality improvement conveys that the term quality assurance does not.  That concept is change for the betterment of the customer.  All other change amounts to non-quality change, or simply change for the sake of change, no matter how well intentioned.  Hence, quality assurance very often amounts to, in effect, a prescription for maintaining status quo, which is a vicious constraint upon the continual quality improvement way of thinking.

More specifically, quality assurance tends to inhibit quality improvement for the simple reason quality assurance aims to inspect results against standards, specifications, and requirements in order to independently judge goodness and badness of results.  Bad results must, of course, be dealt with.  But when bad results are dealt with such that they always go to rework or scrap and little, if any, attention goes to improving the processes producing the bad results, invariably high entropy will win out.

Compensating actions by management often include putting up posters exhorting everyone to “Do Your Best!” followed by accountability systems and systems of reward and punishment.  Exhortations and systems of accountability and reward and punishment make improving quality more difficult.  Scant positive value is known to come from attempting to change for the better by merely managing and “QA-ing” results.

Consider the seemingly innocuous Year 2000 “bug,” for example.  In many cases, quality assurance contributed to the creation of this problem by standardizing on misrepresenting the calendar year as two digits.  While the intention to deliver efficient computer usage was laudable, the problem arose from application of KISS theory that clearly misrepresents the real world.

Table 1 - Quality Related Terms

EDA Quality Categories

• Data Structures – business model and technology models
• Data Values – facts reflecting business semantics and realities
• Data Presentations – data usefulness as information

EDA Quality Dimensions

• Accuracy - conformity of data characteristics to the real world
• Consistency - logical coherence among data characteristics
• Completeness - having all necessary and normal data characteristics
• Timeliness - temporal considerations given to the Completeness dimension
• Accessibility - ease with which data can be gotten to and understood

As a practical matter, to improve the quality of the enterprise’s data will require working each EDA quality category against the EDA quality dimensions.  Stated differently, it is essential to improve data accuracy, consistency, completeness, timelessness, and accessibility in their structures, values, and presentations.

Note that the EDA quality categories are dependent, top to bottom, from one to the next, in this way:

• Data Structures that lack accuracy, consistency, completeness, timeliness, and accessibility guarantee…
• Data Values that lack accuracy, consistency, completeness, timeliness, and accessibility, which guarantee…
• Data Presentations that lack accuracy, consistency, completeness, timeliness, and accessibility

The EDA quality dimensions, however, are highly interdependent.  It is not possible to materially improve the quality of enterprise data by improving, for example, the accuracy of data values alone.  It is also necessary to improve their consistency, completeness, timeliness, and accessibility.

EDA from disparate information technology designs exemplifies the paradigm that promotes low quality data (Figure 2).  This paradigm embodies data management practices known to proliferate application-centric data structures, data values, and data presentations.  Data replicated between software applications through data flows and data interfaces contribute to exponential growths in data redundancy with the illusion of providing data sharing and data integration.

However, in reality, limited data sharing and integration arise because each application invariably defines and imparts its own context and semantics to the data it handles, further distancing those data from their fundamental, inherently integrated enterprise semantics.  Moreover, data that enter the enterprise from the outside, whether through manual or automated data flows and data interfaces, can also contribute to the problem.  When data from the outside enter the enterprise very much as-is, they tend to make the enterprise vulnerable to outside factors over which the enterprise has little or no control.

Such data management practices predate relational methods and the general availability of relational database management systems.  The introduction of relational methods into the commercial sector established data flows and data interfaces as root causes of uncontrolled data redundancy and of data lacking structural quality.  In spite of that, heavy investments in application-centric data management practices remain popular, consequently negating effective usage of relational methods and perhaps evidencing dated data management methods.

Figure 1 – Positive Business Value from Enterprise Data Architecture

EDA provides for knowing data as fully integrated and sharable enterprise data (Figure 1). A high-value EDA culture embraces time-tested semantic modeling methods, relational methods, and honored quality principles.

Figure 2 – Negative Business Value from Disparate Data Designs

Additionally, a high-value EDA culture realizes that relational database management systems are also accommodating to disparate data management practices sufficiently enough to abandon the following wrong theories:

• Usage of relational database management systems automatically give rise to integrated and sharable quality data
• Gainful returns on investments necessarily result from the usage of relational database management systems and related software tools and computing platforms

Relational methods reflect a basis for architectural rigor.  Because they do, relational methods offer the opportunity to supplant the inherently problematic nature of application-centric data as well as paradigms fundamentally oblivious to integrated enterprise data (e.g., object oriented paradigms).  Moreover, relational methods apply significantly to defining business views of data as well as technology views of data.  Data quality improves as the technology views of data approach business views of data by way of commonly knowable EDA.

Data quality improvement aims for transformation from the paradigms of application-centric data and data flows and data interfaces between applications to truly integrated and sharable data reflecting the fundamental structure of the enterprise.  Use of established frameworks, time-tested methods, and honored quality principles engender the aim.

Examples of established frameworks, time-tested methods, and honored quality principles include:

• Zachman Framework for System Architecture
• ANSI/X3/SPARC Three-Schema Architecture
• Integrated Definition for Information Modeling, Extended (IDEF1X)
• Integrated Definition for Function Modeling (IDEF0)
• Continual Quality Improvement, mainly in the style of W. Edwards Deming’s Plan-Do-Study-Act (PDSA) cycles, which Deming himself called Shewart Cycles in honor of Walter A. Shewart of Bell Telephone Laboratories fame

From a strategic viewpoint, commonly knowable EDA must evolve to a fully integrated entity through a transformation process. This process requires dedicated resources.  The persons involved must be or become engaged with requisite frameworks, methods, and quality principles.  They must be unavailable to “fire fighting” and “heroics” in response to crises of the moment.  Ideally, they will be Systems Thinkers, capable to engage synthetic thinking and analytic thinking, as necessary.  Pursued this way, enterprise data of higher quality will continually replace application-centric data of lower quality with minimal interruptions to continuing enterprise operations.

Thus, this strategy calls for pursuing truly integrated, truly sharable, commonly knowable EDA in the face of continuing enterprise operations.  Moreover, this strategy posits that today’s enterprise data of low quality, mainly a consequence of application-centric data management practices, warrants sufficient reason to cease risking the long-term survival of the enterprise.

But then, by Dr. Deming’s wisdom, “Survival is not a requirement.”

***

Ed Johnson is president and principal consultant at Quality Information Solutions, Inc., a consultancy committed to human social and cultural systems receiving quality information from information systems.  Ed also is former president of Atlanta Area Deming Study Group.

Copyright © 2002-2010 Quality Information Solutions, Inc.  All rights reserved.  This material may be copied, provided the source is cited.  Zachman Framework for System Architecture is a trademark of John A. Zachman, Zachman International, Zachman Institute.  Three-Schema Architecture, IDEF1X, IDEF0, and Continual Quality Improvement are nonproprietary and reside in the public domain.