The Top 10 Mistakes in Data Architecture - part 1

The IT Architecture Journal is meant to help both practitioners of IT Architecture as well as those who might hire architects. It is vital that both of these stakeholder group understand not only the core principles behind the practice of Architecture but also where and when Architecture can go wrong. Today, we're going to talk about some of the more common problems or mistakes that arise in the practice of Data Architecture. While there are no concrete rules or really even standard practices in relation to some of the following considerations; we have attempted to clarify where standard practice is already present or perhaps just emerging.

Architecture can help solve puzzles, but should never produce its own

Mistake 1 - Lack of Context
A great deal of Data Architecture being practiced today is totally generic in nature. This is not just a question as to whether the effort may be high-level or not, even higher-level analysis and design can be targeted or contextual in nature. Part of the problem here is the desire by many organizations to standardize best practice based upon industry-defined expectations. This tends to happen most often in relation to the DMBOK (Data Management Book of Knowledge).

Now, there's nothing wrong with either the DMBOK or the desire to standardize approaches to data management or architecture based upon industry lessons learned. What is problematic however is taking much of that information at face value without the realization that the information was deliberately made generic in order to reach the widest possible audience. As with any standard practice or technical standard, it is only as good as its applicability to any specific scenario.

So the question here is what value the purely generic information has; the answer is, as in other cases; it can be an excellent starting point but it should never be the destination. For example, XML or JSON represent different approaches for forming data messages, yet knowledge of the core JSON or XML standards is only valuable in the context of how it might be applied to your own data (which standard to choose, and how it will be used, structured and integrated into your solution).

Far too many Data Architecture projects present the generic information without the follow-on "contextualization." This is a sure path to project failure.

Mistake 2 - A Disconnected Approach
Where does Data Architecture start and where does it end? This is more than an interesting philosophical question. If one stops to think about it for moment, it becomes clear that there is data lurking nearly everywhere in the enterprise - yet much of it - perhaps even the majority of it is not officially managed as part of the "Data Architecture." This can include the following types of data:

• Email and content (including Social Media)
• Metadata
• System logs (for all / any type of systems) 
• Portfolio Management or EA tools
• Smaller DBs or tools not officially managed by the IT department (often business users)
• Mobile and Web App related data 
• Applications (commercial software packages) that have their own databases
• Device data (and / or embedded databases) 

Obviously, there will be situations as a Data Architect where you'll be asked to focus on only one particular 'slice' of the overall data environment for an organization. However, this mistake is more directed to the larger question as to how the enterprise views data architecture and data management. If an organization views portions of its data assets to be outside the enterprise processes associated with data management - then there is a problem. There is also a problem when the management of data is further divided across semi-autonomous groups within the organization. Both of these are very common practices today.

What occurs on these situations is the creation of data silos or islands. The natural consequences of that includes:

• Duplicative data across the enterprise 
• An inability to deploy truly effective analytics
• Significantly higher costs for IT

The easiest way to tear down these walls and connect the data islands is to make the commitment up front that the Data Architecture will include all data meaningful to the organization so that design and management can be aligned with enterprise goals and expectations.

Mistake 3 - Lack of Actionable Governance
Note that we didn't just say 'Governance.' Everyone in the data business loves to talk about Governance, but what does that really amount to? Governance tends to be both a workflow process as well as an associated toolset. If either the tool or the workflow process has limitations or is not present, the Governance effort generally fails. When we say "Actionable," what we mean is very specific; it involves the following elements:

1. Integration of Data Governance with all pertinent system management processes.
2. Integration of Data Governance with security standards and processes.
3. Integration of Data Governance and Portfolio Management process.
4. Integration of Data Governance and Data Architecture.
5. Adherence to clearly defined metrics, tracked on a continual basis. (this can include ALM metrics, integrity metrics as well as performance metrics).
6. The Governance process has clearly defined roles with authority and decision gates built in. 
7. The Governance is closely aligned to both tactical and strategic organizational goals.

This set of elements goes well beyond what many groups consider Governance. Often times this happens because they aren't willing to make the investment in order to support these activities. Ultimately any perceived savings are in fact highly illusory given the well-defined cost impacts of environments suffering from a out of control data sprawl (which always seems to come as a surprise but really shouldn't - recreating the same solution 4 times costs 4 times more, right?).

Mistake 4 - Data Architecture isn't Taken Seriously
As you might imagine, this is a problem not limited to Data Architecture. So, what do we mean when we say Architecture isn't taken seriously? This tends to manifest itself in the following ways:

• No investment at all in architecture
• Limited or one-off investments (periodic reviews but no dedicated architecture practice)
• No investment in architecture tools
• Lack of integration between architecture and other key processes, such as Portfolio Management
• No architecture methodology employed

Architecture is like anything else - you get out of it what you put in it. If the resources and commitment to an effort is minimal, it is likely that the value derived will be less than may have been expected (unless it was understood to be a token effort from the start). In many ways, the Architecture lifecycle is similar to an ALM (application lifecycle management) approach. Architecture is design writ large - typically taking into account all potential aspects of a ecosystem of individual solutions. In this sense, it is very difficult to translate Architecture into Agile methodology except for time-boxing delivery expectations. 

But architecture without any methodology is simply ineffective. 

Mistake 5 - Product Only Focus
This one deserves a bit of caveat. If your project is narrowly focused on deployment of one particular product / technology, then it is obvious that the product architecture will be the primary focus of the effort. However, this problem refers to the larger enterprise. An important value that an IT Architecture effort brings to the table is the ability to abstract capability from the specific technologies that will ultimately manifest that capability. What is this important?

1. Because in a large environment, there will likely be dozens of technologies involved.
2. Because technologies and products change. 
3. Because the Architect is not generally an agent or advocate for any particular technology, but rather an unbiased 'change agent.' This role is also referred to as the "Honest Broker."

So, the mistake that occurs most often in relation to this issue is the idea that one product or technology will provide some sort of "Silver Bullet" solution that will remarkably transform the enterprise. While this does on occasion happen, it does not happen very often. Generally the scope for any Data Architect will be wider than one product or product suite (or even several), and most importantly the true scope is the problem space associated with the organization's goals rather than mere management of specific commercial software packages.

We will cover the next five mistakes in an upcoming post...

copyright 2014,  Stephen Lahanas

#ITarchitectureJournal
#StephenLahanas
#Semantech-Inc