The Top 10 Mistakes in Data Architecture - part 1

The IT Architecture Journal is meant to help both practitioners of IT Architecture as well as those who might hire architects. It is vital that both of these stakeholder group understand not only the core principles behind the practice of Architecture but also where and when Architecture can go wrong. Today, we're going to talk about some of the more common problems or mistakes that arise in the practice of Data Architecture. While there are no concrete rules or really even standard practices in relation to some of the following considerations; we have attempted to clarify where standard practice is already present or perhaps just emerging.

Architecture can help solve puzzles, but should never produce its own

Mistake 1 - Lack of Context
A great deal of Data Architecture being practiced today is totally generic in nature. This is not just a question as to whether the effort may be high-level or not, even higher-level analysis and design can be targeted or contextual in nature. Part of the problem here is the desire by many organizations to standardize best practice based upon industry-defined expectations. This tends to happen most often in relation to the DMBOK (Data Management Book of Knowledge).

Now, there's nothing wrong with either the DMBOK or the desire to standardize approaches to data management or architecture based upon industry lessons learned. What is problematic however is taking much of that information at face value without the realization that the information was deliberately made generic in order to reach the widest possible audience. As with any standard practice or technical standard, it is only as good as its applicability to any specific scenario.

So the question here is what value the purely generic information has; the answer is, as in other cases; it can be an excellent starting point but it should never be the destination. For example, XML or JSON represent different approaches for forming data messages, yet knowledge of the core JSON or XML standards is only valuable in the context of how it might be applied to your own data (which standard to choose, and how it will be used, structured and integrated into your solution).

Far too many Data Architecture projects present the generic information without the follow-on "contextualization." This is a sure path to project failure.

Mistake 2 - A Disconnected Approach
Where does Data Architecture start and where does it end? This is more than an interesting philosophical question. If one stops to think about it for moment, it becomes clear that there is data lurking nearly everywhere in the enterprise - yet much of it - perhaps even the majority of it is not officially managed as part of the "Data Architecture." This can include the following types of data:

• Email and content (including Social Media)
• Metadata
• System logs (for all / any type of systems) 
• Portfolio Management or EA tools
• Smaller DBs or tools not officially managed by the IT department (often business users)
• Mobile and Web App related data 
• Applications (commercial software packages) that have their own databases
• Device data (and / or embedded databases) 

Obviously, there will be situations as a Data Architect where you'll be asked to focus on only one particular 'slice' of the overall data environment for an organization. However, this mistake is more directed to the larger question as to how the enterprise views data architecture and data management. If an organization views portions of its data assets to be outside the enterprise processes associated with data management - then there is a problem. There is also a problem when the management of data is further divided across semi-autonomous groups within the organization. Both of these are very common practices today.

What occurs on these situations is the creation of data silos or islands. The natural consequences of that includes:

• Duplicative data across the enterprise 
• An inability to deploy truly effective analytics
• Significantly higher costs for IT

The easiest way to tear down these walls and connect the data islands is to make the commitment up front that the Data Architecture will include all data meaningful to the organization so that design and management can be aligned with enterprise goals and expectations.

Mistake 3 - Lack of Actionable Governance
Note that we didn't just say 'Governance.' Everyone in the data business loves to talk about Governance, but what does that really amount to? Governance tends to be both a workflow process as well as an associated toolset. If either the tool or the workflow process has limitations or is not present, the Governance effort generally fails. When we say "Actionable," what we mean is very specific; it involves the following elements:

1. Integration of Data Governance with all pertinent system management processes.
2. Integration of Data Governance with security standards and processes.
3. Integration of Data Governance and Portfolio Management process.
4. Integration of Data Governance and Data Architecture.
5. Adherence to clearly defined metrics, tracked on a continual basis. (this can include ALM metrics, integrity metrics as well as performance metrics).
6. The Governance process has clearly defined roles with authority and decision gates built in. 
7. The Governance is closely aligned to both tactical and strategic organizational goals.

This set of elements goes well beyond what many groups consider Governance. Often times this happens because they aren't willing to make the investment in order to support these activities. Ultimately any perceived savings are in fact highly illusory given the well-defined cost impacts of environments suffering from a out of control data sprawl (which always seems to come as a surprise but really shouldn't - recreating the same solution 4 times costs 4 times more, right?).

Mistake 4 - Data Architecture isn't Taken Seriously
As you might imagine, this is a problem not limited to Data Architecture. So, what do we mean when we say Architecture isn't taken seriously? This tends to manifest itself in the following ways:

• No investment at all in architecture
• Limited or one-off investments (periodic reviews but no dedicated architecture practice)
• No investment in architecture tools
• Lack of integration between architecture and other key processes, such as Portfolio Management
• No architecture methodology employed

Architecture is like anything else - you get out of it what you put in it. If the resources and commitment to an effort is minimal, it is likely that the value derived will be less than may have been expected (unless it was understood to be a token effort from the start). In many ways, the Architecture lifecycle is similar to an ALM (application lifecycle management) approach. Architecture is design writ large - typically taking into account all potential aspects of a ecosystem of individual solutions. In this sense, it is very difficult to translate Architecture into Agile methodology except for time-boxing delivery expectations. 

But architecture without any methodology is simply ineffective. 

Mistake 5 - Product Only Focus
This one deserves a bit of caveat. If your project is narrowly focused on deployment of one particular product / technology, then it is obvious that the product architecture will be the primary focus of the effort. However, this problem refers to the larger enterprise. An important value that an IT Architecture effort brings to the table is the ability to abstract capability from the specific technologies that will ultimately manifest that capability. What is this important?

1. Because in a large environment, there will likely be dozens of technologies involved.
2. Because technologies and products change. 
3. Because the Architect is not generally an agent or advocate for any particular technology, but rather an unbiased 'change agent.' This role is also referred to as the "Honest Broker."

So, the mistake that occurs most often in relation to this issue is the idea that one product or technology will provide some sort of "Silver Bullet" solution that will remarkably transform the enterprise. While this does on occasion happen, it does not happen very often. Generally the scope for any Data Architect will be wider than one product or product suite (or even several), and most importantly the true scope is the problem space associated with the organization's goals rather than mere management of specific commercial software packages.

We will cover the next five mistakes in an upcoming post...

copyright 2014,  Stephen Lahanas

#ITarchitectureJournal
#StephenLahanas
#Semantech-Inc

Read More

Security Architecture, Defined

There are some people who don't recognize Security Architecture as its own niche within IT Architecture, but given the number of roles / positions now using the title "Security Architect," perhaps the naysayers have missed out on something. This post will examine what Security Architecture consists of and how it fits within the larger context of IT Architecture.

Back in the old days, security was a lot easier. We didn't have internet connectivity everywhere, few if any viruses, and only a handful of hackers who were mostly in it for the fun. Things have sure changed. What used to be referred as "Information Assurance" or IA has gradually morphed into "Cyber Security." While IA was primarily focused on management of information assets (which included copious amounts of paper locked in vaults), Cyber Security has tentacles that reaches all the up and down every solution stack and into every nook and cranny of the newly dubbed "Internet of Things." (this year's hype king).

Security Architecture is related both the practice of Cyber Security as well as the practice of IT Architecture. We've tentatively classified it as a subset of Solution Architecture, but just like some other areas like SOA, Security Architecture proves to be a bit ubiquitous in both principle and practice. Originally, when those of us worked on Security Architecture projects, we tended to fund them primarily concerned with Data Center design (or consolidation) or more specifically focused on tightening up perimeter security within a Data Center. Now, Security Architecture has become much more expansive, including aspects of UI coding all the way to management of mobile devices (to encrypt data at rest and prevent identity theft for example). So lets move on to some definitions.

Security Architecture, Defined

Security Architecture represents the ability to represent and resolve any IT-related security issues using IT Architecture techniques. These security issues can be internal or external and can either be technology agnostic or technology specific. The solutions developed as a result of Security Architecture analysis and design are often decomposed into "Security Controls," which focused on narrowly defined portions of the overall security landscape. 

Security is still based on core IA principles but those have been expanded
 to include a much wider arena of potential action

In our initial definition, we alluded to something called Security Controls; this is worthy of further examination...

Security Controls, Defined

Security Controls are the individual measures necessary to mitigate specific security threats or concerns. These controls can be part of larger information security standards (NIST, FISMA Cyber Framework, etc) or they can be defined within specific organizations (or a combination of both could be used). The controls consist of guidelines and policies (for example, coding practices), specific configurations for systems, and test cases for security evaluation.

Management of Security Controls isn't necessarily a Security Architecture function, however definition of Security Controls often is (coming after some type of audit that may include review of the existing architecture).

Which brings us back to the larger question of what Security Architecture is and what exactly a Security Architect does. As we alluded to earlier, once upon a time it was common to associate Security Architecture with data center technology (like Intrusion Detection Systems for example) and Information Assurance with control over information or data. Today, Security Architecture spans the entire stack from infrastructure (Cloud or Traditional) through Data, Application up through to UX. It also extends out to other enterprises and to mobile platforms and social media (or other external Cloud based services). Security Architecture is always only as strong as its weakest link - and with the ever-expanding scope of operations the extent of the oversight necessary to mitigate those weak links makes the job of the Security Architect more daunting than ever before.

Security Architecture must facilitate both prevention of threats as well as active oversight of operational integrity. While the Security Architect does not sit in a NOC (Network Operations Center) he or she does often determine how operational analysts will conduct their jobs. This field within IT Architecture is an excellent example why cross-domain expertise is needed for many Architecture roles.

We will be examining a number of specific Security Architecture case studies in future posts here on the IT Architecture Journal.

copyright 2014,  Stephen Lahanas

#ITarchitectureJournal
#StephenLahanas
#Semantech-Inc

Read More

How to Create Product Maps

In some of our articles here you've probably noticed various types of diagrams thrown in as examples. All of these come from real-world project related exercises. When an IT Architect produces a diagram we generally refer to it as an "Artifact." While this may sound like we're talking about relics, it is simply an odd naming convention of our industry. An Artifact can be any sort of deliverable; a Word doc, a spreadsheet or perhaps even a file associated with a specific EA software environment. The common thread across any of these deliverable formats is the notion that the Artifact must be more or focused on one topic (sometimes this isn't the case, but most times it is).

One particularly interesting type of Artifact is the "Product Map." So what exactly is a Product Map? Well, there are two distinct types of Product Map and there are only tangentially related.

Type 1 - A Map showing the relationship between products, comparison of products or elements of a complex product suite.
Type 2 - A Map showing product components and their evolution (past and future roadmap).

So, you might be asking yourself, why is such a thing necessary and what does this have to do with IT Architecture?

Why We Need Product Maps
Believe it or not, the companies that produce these products don't always do a good job at depicting the composition of their products or the relationships between products. And, if we extend our view to a large heterogeneous enterprise environment, there is little incentive for a product company to spend too much time thinking about how their product might relate to their competitors or other products which don't seem to be related to the function their solution performs.

There are a variety of situations where product maps might be needed; they include:

• Representation of "As-Is" architecture. Product maps would be drill-down artifacts that illustrate more of a physical view.
• Product Evaluation and Source Selection efforts.
• Transition Planning exercise (in other words, being able to illustrate migration of one set of tools to another).
• Enterprise Integration Design. Getting different product sets to work with one another or even send messages to other can be challenging. 

The product map differs from other types of architecture artifact in that it depicts some aspect of a larger solution from a specific product perspective or context. The way we might highlight an identity management solution in an agnostic logical view could look far different than a product specific IAM stack (see the diagram below).


IT Architecture & Product Integration 
So how does the IT Architect facilitate Product Integration? Well, at its core this represents a problem solving exercise; in fact one of the more complex ones that the majority of enterprise tend to face on a routine basis. One might think that since so many similar organizations face this type of challenge that there would be a wider set of industry standard approaches to this - but this isn't the case. While many enterprises share the same tools, the exact configuration and inventory will almost certainly vary widely. And as most of you know, even moving from one version of a product to the next can be major hassle in itself.

The IT Architect takes this problem and turns into a design exercise. This begins by documenting all of the pertinent aspects surrounding the product:

• Key features
• Standards support
• Administrative and management considerations
• Customization support (languages, configuration)
• Inter-operability mechanisms (database connectivity, APIs, etc.)
• Version control and version specific variations of the above
• Overlap of features or deprecation of features (either between different versions or options of one product or competing products).

The Architect then defines all of the related design decisions based on a larger set of expectations for how the product might be used. Some products of course merely provide a platform for even more custom development (for example an DBMS like Oracle) - with the larger Oracle stack then there are opportunities for both custom and packaged analytics. The determination as to whether to go one way or the based upon what products are already in the inventory or which ones ought to be included near-term and long-term is a classic example of contextual product mapping.

An example of a product map of a suite of related security products

IT Architecture & Product Design 

Can IT Architecture support product design or development? In a word yes. This represents Type 2 Product Mapping. For example, what if you had an excellent idea for a new mobile app and wanted to manage it in some start-up scenario? There are of course various business considerations for a typical start up, but just as important is management of the product itself and that should start with its architecture.

The architecture will drive many key decisions related to the product's introduction and evolution including:

• Determination of what technologies should be harnessed to actualize the product.
• Development of the product roadmap (including conditional variations based upon contingencies or opportunities).
• Determination of key systems design options and solutions.
• Governance of the product (and all its related infrastructure and code) across its lifecycle.
• Determination of product synergies in the market (the larger ecosystem of related solutions).

We will look at Architecture and Product Design in greater depth in upcoming posts here on the IT Architecture Journal.

Copyright 2014,  Stephen Lahanas

#ITarchitectureJournal
#StephenLahanas
#Semantech-Inc

Read More

Aligning Lifecycle Methodologies - Part 1

In a previous post, I contended that requirements are still an important part of most enterprise environments, even those that might be using Agile lifecycle methodologies. However, I didn't specify exactly how those requirements should be captured — or even what constitutes a requirement. This is an important question when determining how to align multiple lifecycle approaches within one organization (the need for which is much more common than most people realize)

This question encompasses two major lifecycle approaches; Agile and Waterfall methodology. Many organizations feel that User Stories (Agile) are sufficient to build both code and project structures around them. Is it out of the question to consider that Use Cases might be used within an Agile approach?

Before answering that, let’s take a look at User Stories, Use Cases and Requirements in context again:

• A User Story can be thought of as a high level or conceptual scenario, or perhaps just a problem. An example might something like this: A user needs to be able to save a report into multiple file formats for downloading, including both .pdf and .xls. Even though we’ve described two different formats, it still represents the same scenario or problem.
• A Use Case, on the other hand, can be considered a functional requirement. The difference between a functional requirement and technical one is the level of detail the document includes about how specific actions are carried out. For example, a use case would capture the process flow, or steps involved in generating the report and the expected outcomes or alternatives. This helps to build test cases later on. A use case is slightly more structured and detailed and is generally written by someone IT or in combination with the IT group. That way, some of the technical expectations are captured. In other words, the use case describes not just behavior, but how that behavior is achieved.
• A Requirement in this context refers to a technical requirement. This is entirely specific to the elements needed to develop and/or deploy the capabilities described in the previous two documents. It allows us to formalize performance constraints, data elements and other pertinent technical details. This is especially important in environments where EA, service or data governance paradigms are in place. If written properly, requirements can then serve as system documentation.

Mapping across lifecycle approaches begins with requirements (however they're captured)

While many organizations practicing Agile bypass requirements, they may in fact still capture the same information after the fact if they actually document their deployed solutions (after the fact). When you’re considering whether or how to employ user stories and use cases, keep these points in mind:

• They are not the same thing.
• The user story ought to come first.
• The use case ought to be derived from the user story.
• Any requirements managed from this process should be embedded within, or otherwise traceable to, a specific use case and user story.
• User stories could be considered either scenarios, high-level processes or problems.
• Every organization approaches this process somewhat differently. For example, in a less Agile environment, a number of other architecture artifacts could be added to the illustration above:
• At the user story level, many people like to capture a data-flow diagram or a context diagram.
• At the use case level, many folks like to add a UML sequence diagram and conceptual data model.
• At the requirement level, many UML diagrams are often added including activity models, state models, component diagrams or deployment diagrams.
• If a formal EA framework is being applied, those UML or ERD diagrams are often replaced or folded into EA Framework views/artifacts.

Getting back to the original question — should use cases be used with Agile lifecycle management? — my recommendation is yes. Here’s why and how.

Why to Do It

1. It ensures some level of systems engineering is included in the Agile design process.
2. It helps to define the tasks associated with each user story and doesn’t add that much extra effort - it also doesn’t require a use case diagram (which in most cases doesn’t convey that much meaning anyway).
3. It ensures that both IT and business stakeholders capture their expectations in formats tailored to their roles and needs.

How to Do It

• Define your use case template and what it must convey.
• Determine whether user stories require more than one use case. It happens occasionally, although in most situations user stories should be in a 1-1 ratio.
• Ensure that someone from IT helps with, or provides much of the detail, in the use case.
• Ensure that use case development is aligned with both the quality and testing paradigms. This is especially important if no requirements are captured.

The majority of Agile development environments are fairly flexible in regards to storing of related attachments, so the biggest impact to employing use cases along with user stories is in the design and development process. This is the first step towards aligning lifecycle methodologies; we will examine this further in several upcoming posts.

Copyright 2014,  Stephen Lahanas

#ITarchitectureJournal
#StephenLahanas
#Semantech-Inc

Read More

Understanding Architecture Accessibility

The topic of Architecture Accessibility may sound a bit abstract upon first glance, but it is in fact one of the most important considerations as to whether an architecture effort will add value or not to an enterprise (or any particular solution lifecycle). It is also directly related (or perhaps a part of) the most important consideration for any architecture - its usability.

Let's step back for a moment and paint the background a bit.

IT Architecture has progressed in fits and starts over the past 20 years or so partially because of the widely varied results or outcomes associated with its practice under various scenarios. This is to be expected when considering that this is a fairly new aspect of Information Technology - it appeared rapidly and has been evolving ever since.

But why has IT Architecture sometimes gotten a bad reputation? The common causes for this include the following:

• A lack of clear expectations as to what the Architecture activity would really provide.
• A lack of connection between different types of Architecture efforts - for example Solution Architecture being managed separately from Enterprise Architecture or Data Modeling or application design not being considered as architecture.
• A lack of integration with governance and portfolio planning and management processes. 
• A lack of detail and / or specific implementation-level guidance.
• A lack of integration between architecture activities and ALMs / SDLCs (Software Development Lifecycles) 

You may have recognized a common thread in most of these problems - that organizations which produce Architecture in a vacuum seldom succeed. A variation of that theme is directly related to our core thesis on architecture accessibly.

Making Architecture accessible requires clear organization and a componentized, targeted approach

What is Architecture Accessibility and Why is it Important?
Accessibility in this context refers both to how people can reach the Architecture but also to how it is structured. So, here is a fundamental consideration - if an Architecture is produced yet few within the larger organization have access to it - how likely is it that the Architecture will achieve its originally intended purpose? Taken a step further, if we examine the structure issue; how likely is it that an Architecture will be exploited efficiently if it is difficult to find the pertinent information within it that any given stakeholder might need?

As Architectures grow in complexity, there tends to be a larger and larger potential audience associated with it. For nearly everyone in that audience, only a portion of the Architecture will be relevant to them. Thus the idea of a monolithic architecture artifact (say, a giant MS Word doc) becomes highly problematic. Use of a formal Enterprise Architecture repository can become equally problematic if added licensing is required for anyone who needs to view the Architecture (which in my view is a highly counter-intuitive strategy on the part of companies which produce such software).

What is at risk if Architecture becomes inaccessible is "Shelfware Syndrome." Architecture like software can provide no value if isn't applied to something - it has to be used. And this is how we begin to make the connection between Architecture Accessibility and Usability.

Architecture Usability, Defined
"Architecture Usability" refers to the ability for the Architecture to communicate the right level of information to specific stakeholder groups. Architecture must generally support multiple stakeholder groups simultaneously. What proves to be 'usable' in one specific organization may be different than in another, but generally it will involve a mix of descriptive (design explanation) and prescriptive (implementation guidance) content. Usable Architecture then is technical design content which is structured such that it can be easily accessed by the right audience at varying levels of detail.

Architecture is a continuum - it provides a mechanism whereby any number of projects and systems can be tracked and managed in context with one another. It represents a sort of meta-integration mechanism if used properly and the best architectures are those which are fully integrated within the organizations they serve. Good Architecture is Democratic as well in that the wider the shared designs are disseminated, the more likely it will be used effectively to guide the enterprise towards a unified set of goals (and some tangible end-state payoffs).

Copyright 2014,  Stephen Lahanas

#ITarchitectureJournal
#StephenLahanas
#Semantech-Inc

Read More

Introducing Design Patterns

Design Patterns are an important part of software engineering today, but what many folks don't realize is how ubiquitous they are becoming within the full spectrum of IT activities. Part of the reason this is the case is due to the flexible nature of Patterns.  Mastering Design Patterns is a critical part now of most IT Architects' roles and from a larger perspective is also becoming increasingly import for most enterprises.

Before discussing how to manage Design Patterns, we first need to define what they are:

Design Pattern - This represents a description and (usually) a visualization of some specific aspect of a larger design solution. Patterns can employ standard or non-standard notation and may or may not be used as Enterprise Architecture (EA) artifacts. The most important characteristic of a Pattern is that it encapsulates at least one significant design principle. Patterns are usually combined to produce comprehensive solutions.  Patterns can also be used as code "templates/stubs" to facilitate rapid development of applications and services. 

Beyond this basic definition, Patterns can mean many things to many different types of IT specialists. For example, Cyber Security experts can manage Patterns as mechanisms for modeling Threat behaviors. Network Engineers can view combinations of Data Center layouts and hosting configurations as Patterns. Data Architects could, if they wished, view prototypical Conceptual Models as Patterns and in fact a Canonical Model could be considered a specific type of Data Pattern applied across multiple enterprises. This sounds a little strange perhaps because many people associate the concept of Patterns with Object Oriented Programming and the original Gang of Four book of OOD patterns. Yet, there is nothing that explicitly states anywhere that Patterns have to be limited to application design (Object-Oriented or otherwise).

The flexibility of the Design pattern is simultaneously both its greatest asset and drawback.  It's an asset because it allows architects, developers and engineers the flexibility to define re-usable solutions for common design challenges. It's a drawback because it tends to promote decentralized management (or perhaps no management at all) of design artifacts and may also lead to more situations where solution elements are designed out of context of one another.  So how do we manage something that is flexible, effective yet often disruptive? Here are five suggestions for how to master Design Patterns:

1. Create your own Design Pattern Taxonomy - Keep in mind that there is no standard approach from industry as to how an organization (or individual) ought to manage patterns (unless of course you wish to blend it with your EA approach and integrate them within one of several EA Frameworks). So what's being recommended here is to create a taxonomy of the types of Design Patterns you think you may need. This might include application patterns, data patterns, UI patterns etc. - whatever makes sense.

2. Come with a Standard approach for Pattern Visualization - Many people like to use UML for this but quite frankly there's some design problems that UML isn't so great in representing. The benefit of using UML is that it should make alignment with your ALM easier (in most cases).

3. Come up with Standard Template for Pattern Description - You'll need to ask yourself here just how much information is required. While this shouldn't become a huge specification you will need to provide enough information so folks not involved in the original design won't have to come back and ask what you really meant.

4. Standardize your own Design Terminology - As we all know there's many ways to describe the same thing. Building a glossary that can support all of the Design Patterns you may create will come in handy for you and your enterprise. (again for those who are already utilizing an EA approach you can probably borrow from that or at least make sure that any Patterns not managed as part of the EA share common terminology).

5. Align your Design Patterns with your ALM / SDLC - This is perhaps the most important of the five recommendations. The majority of enterprises are already working with some sort of Application Lifecycle Management methodology. Often times ad hoc Design Patterns don't get "baked in" to the ALM. This can lead to much confusion and generally makes the overall design approach somewhat less efficient than it might otherwise be.

Design Patterns are meant to accomplish three critical things:

• Increase the speed of the design process.
• Ensure that lessons learned are adopted.
• Reduce overall costs by finding ways for certain capabilities to be reused. 

Actively managing Design Patterns helps to ensure that all three goals can be achieved.

Copyright 2014,  Stephen Lahanas

#ITarchitectureJournal
#StephenLahanas
#Semantech-Inc

Read More

Exploiting SharePoint as an Architecture Repository

Enterprise Architecture (EA) as a practice has gained quite a bit of traction over the past decade. Where EA hasn’t taken root yet, Solution Architecture is often practiced. One of the biggest challenges to the adoption of any type of IT Architecture practice or department is the ability to integrate it into the larger solutions lifecycle of any given organization. This is typically a challenge for the following reasons:

1. The culture of the organization isn’t sure about the value proposition for architecture.
2. The working model between architects and the rest of the solutions team isn’t always clear at the beginning.
3. The architecture practice / department lack the tools to do its work or properly communicate back to the rest of the organization.  

Of these three elements of the problem, we’re going to focus on the last one. Usually, there is always some level of resistance to doing anything new within an IT group and of course it takes time to find the right working models for how to implement new processes or capabilities. In the case of IT Architecture, having a tool available at or near the start-up of  a practice makes a huge difference though in helping to resolve the first two issues and ultimately will lead to greater overall success of the endeavor.

SharePoint as a repository is basic - but it does meet several core requirements

Architects tend to use a variety of tools to do their jobs; for example a typical EA may utilize the following software to support a typical data architecture project:

• Microsoft Visio – for free-form design / modeling
• CA ERwin – for ERD or dimensional modeling
• A UML tool – for developing Use Cases, Sequence Diagrams etc.
• Specific tools to support individual software packages (BI, Hadoop)
• Ontology Modeling or mind Mapping
• Network Design tools

All of these tools produce their own architecture “artifacts” or files. There is a also a class of software out that specifically tries to unify all of this information by providing both design interface and architecture repository. The best known of these tools includes IBM System Architect, TrouxMetaverse Mega and Sparx EA. There are quite a few advantages to using this type of software but surprisingly few architecture practices actually deploy them and often when they do try to deploy them they don’t “stick.” In other words, some aspect of the deployment fails or adoption is restricted for other reasons – for example due to a lack of licensing to make the architecture accessible to all of the stakeholders who actually need to be involved. So what happens when an organization deploys an IT architecture practice without any dedicated tool support for it?

In many cases, when an architecture group isn’t given clear guidance on what tools to use; the ability to define and develop clear processes becomes highly problematic. One of the first things an architecture practice or department ought to be doing is defining exactly what patterns or designs are standard for that organization. This set of decisions then tends to manifest itself into design templates in whatever tools happen to be available. Without any standard tools then it almost impossible to accomplish this vital step. Without standard views of the design patterns, then the ability to consistently capture design and apply it in a uniform fashion across projects suffers. The lack of design tool also tends to imply not having a repository as well. Without a repository, architecture artifacts then get managed as content by whatever means may be available in the enterprise:

• Shared folders
• CMS or portal
• QA or other backlog or PLM tools

Of these available options, the use of an existing Content Management / Portal capability represents the best opportunity for organizing an Architecture practice when a dedicated architecture  repository tool is unavailable. Shared folders have access issues and a serious lack of functionality and many QA/PLM also tools share the same cost issues associated with architecture repositories. While any CMS / Portal will work for what’s being described here; I’ve zeroed in on SharePoint due to the sheer volume of the enterprise organizations now using it. Those organizations that don’t already have

SharePoint deployed internally now also have access to SharePoint in the Cloud on Office 365 at fairly reasonable rates (although it takes a while to figure out how to order and configure the options focused on SharePoint only).

SharePoint isn’t always the easiest tool to work with, but it does support some of the key capabilities needed to help establish, unify and organize an IT Architecture practice; those include:

1. The ability to save and version control any type of content.
2. The ability to support complex publishing (e.g. through a basic wiki). 
3. Some application capability (in this case, through the use of List web parts in SharePoint).

While many or perhaps most installations of SharePoint don’t really support Collaboration very well, the SharePoint online solution is now being bundled with Yammer which opens this open up as the 4th major capability. Once you have your architecture “platform” in place, you can go back and align your designs process approach with the “public face” of the practice. A typical SharePoint Architecture site might include the following elements (see the diagram below):

• A Design Pattern Wiki – This is how stakeholders can access your architecture.
• A Business Glossary – This can include the ontology and core business terminology.
• Data Dictionary – This is where architecture meets data; very few organizations have comprehensive data dictionaries, even fewer make them available publicly.
• Architecture Governance Portal – Architecture requires both technical and business input – that Governance can be managed using simple web part Lists for tracking, approval etc.
• Technology Catalog – While SharePoint is not recommended as a comprehensive IT asset management solution, it can be used to capture all reference architectures and information about deployed technology (using a wiki and / or lists). 

In an upcoming post, we will talk more about how architecture patterns are managed using this type of approach or traditional EA repositories.

Copyright 2014,  Stephen Lahanas

#ITarchitectureJournal
#StephenLahanas
#Semantech-Inc

Read More

The Key Qualities of Effective Enterprise Roadmaps

One of the most important and challenging tasks that IT architects are often asked to participate in or even lead is the development of enterprise roadmaps. Roadmaps differ from schedules or integrated master schedules in that they are meant to represent the realization of capability goals rather than just merely tracking project tasks, milestones or intended delivery dates. Often times Enterprise Roadmaps do however provide the starting point or reference framework for all schedules related to the enterprise goals highlighted.

There is no standard format for a Roadmap because it is more than anything a Strategy, or a starting point for everything else that's to follow. More than that, Roadmaps are quite often used as "What If" tools as well - providing multiple potential paths to achieve organizational goals. There are quite a few software tools that provide various types of Road-mapping features. The types of tools most likely to have these features are Enterprise Architecture suites, but it is important to keep in mind that Roadmaps can be built from scratch if necessary.  An important question you might be asking yourself is why are Architects asked to contribute or lead this type of strategic activity; here's the answer:

1. Because Architects are tasked to manage the solution from a holistic perspective and thus must be involved to some extent in tempering the expectations of the organization in relation to what is or isn't technically feasible. It's generally customary to allow the person who must execute a plan to be involved in developing it.

2. Because most Architects are trained to be able to take complex situations and provide visual reference artifacts that help illustrate both the problem and the solution in an effective manner. Sometimes this is done using standard modeling notation, like UML, other times it's dictated by the feature sets of the software used to create Roadmaps. Sometimes though, it is completely unique - coming from the mind of the Architect.  The common denominator is the ability to take highly technical information and make it relevant and understandable to just about anyone without over-simplifying it to the point where its meaningless. This, as one might imagine, is a bit tricky.

3. Because Architects are trained to look for dependencies, constraints and risks at both a holistic and a detailed level. If this examination occurs up front during the Road-mapping process, the initiative has a much higher likelihood of success.

Once embarked on Road-mapping exercise, the most important qualities of a successful roadmap to strive for include:

1. The ability to illustrate capabilities in context (both with one another as well as existing constraints and future expectations of the organization in question).

2. The ability to illustrate sequential implementation while highlighting inter-dependencies. This of course doesn't have to be as detailed as an Integrated Master Schedule but it does need to be able to illustrate the highlights - those areas which will have the most impact or the greatest risk.

3. The ability to illustrate the core business challenges or problem sets being tackled.

4. The ability to properly define and prioritize chunks of capability.

5. The ability to visualize the roadmap in such a way to emphasize the linkage between core goals and outcomes.

6. The ability to serve as a communication tool or facilitation mechanism. (as the roadmap is often used to generate internal support or pursue funding)

7. A sense of realism and an understanding of the business impacts of what's being recommended. Architects are often the go-to resource when developing IT cost estimates and business case justifications because of this ability.

Building a Roadmap can be a daunting task, but putting the right people in charge is a good way to start.

Copyright 2014,  Stephen Lahanas

#ITarchitectureJournal
#StephenLahanas
#Semantech-Inc

Read More

Why Requirements Still Matter

IT Architects deal requirements in many contexts; there are requirements associated with the management of architecture itself, the requirements of the enterprise which drive an entire portfolio and the more specific detailed requirements associated with solution design. Today, we're going to examine the requirements in the context of Solution Architecture - a domain where Agile is sometimes interpreted as a "requirements-free zone."

These days, requirements management is often misunderstood or under-appreciated in IT. Many people have used migration to Agile methodologies as an excuse to discard requirements or rethink how they’re used to support most projects.

In the overall lifecycle of lifecycles, requirements becomes critical in establishing traceability

One of the key tenets of Agile application development is not to over-think technical requirements in advance. The premise is that it’s usually impossible to accurately predict the details, which would invariably lead to costly rework after the fact—as opposed to an Agile approach of incremental refinement.

So who’s right? Do Agile advocates make a convincing case that a too-precise definition of requirements is counterproductive, or does the more traditional view of requirements management make more sense? Before we answer that question, let’s review several pertinent factors:

• Not all IT projects involve coding (for example, software development).
• Even Agile methodology includes some level of requirements definition.
• Many, if not most, organizations track “functional” level business requirements, a task usually managed by business analysts.
• The majority of people who collect and manage requirements don’t follow a specific methodology, since it’s often considered less important than the follow-on development or implementation activities. What’s more, most people don’t realize that requirements management is a collaborative activity.

Maybe both camps are wrong, or perhaps both approaches suffer from a similar flaw. Traditional requirements management and Agile requirements represent two ends of a spectrum—and each assumes the other end is ineffective.

Common Ground

As in most things, the truth lies somewhere in between. Not all requirements are the same, so they shouldn’t all be managed the same way. Either camp could go wrong in this area. Also, the main reason that requirements management has come under criticism isn’t that defining solutions up front is intrinsically wrong, it’s that defining solutions out of context is.

Followers of both traditional Waterfall and Agile techniques can fail to define solutions in context. The difference is that with Agile, the flawed results are more likely to be noticed early. Context can be lost when a team develops requirements without:

• End-user input
• Stakeholder input
• Use cases and test cases
• Mechanisms for validation and incremental development
• Integration of business and technical requirements processes.

It’s worth pointing out that the controversy over whether requirements management is a worthwhile practice hasn’t led to widespread insight into how it can be corrected. Instead, it has led to sporadic use of requirements or a complete disregard for them. Meanwhile, even with Agile, the success rates of IT projects haven’t improved all that much.

The Best of Both Worlds

Requirements management is the most logical place to integrate business and technical processes and teams. Instead of moving from one either/or philosophy to another, we should take the best elements of each camp and create a better approach.

Far from abandoning requirements as a central part of every project, it’s time to start advocating their use in all projects. Here’s why: Requirements are the foundation of all good design and architecture, all portfolio and project management, and all project or solution collaboration. If compiled properly, they capture vision and thinking from a diverse spectrum of team members and document the solution as it’s progressing.

As we said, not all projects need to apply requirements the same way. Conditional requirement approaches might be applied to these project categories:

• Pure Development: Little or no code reuse. This might involve a more Agile approach, including more experimentation and more test-driven or scenario-driven requirements.
• Commodity Development: Significant use of existing libraries or software packages. This might focus more on integration and performance requirements. For example, an existing knowledge base may support more detailed technical requirements.
• User Interface-Driven Solutions: Much more focused on event-driven expectations and extensive user stories.

Requirements don’t have to predict all aspects of a solution, and they can be corrected or updated as the solution emerges. They can easily become evolutionary in nature. The bottom line is, don’t skip them. Compiling them now can save you a lot of trouble later.

Copyright 2014,  Stephen Lahanas

#ITarchitectureJournal
#StephenLahanas
#Semantech-Inc

Read More

The Practice of Architecture Assessments

One of the most common tasks that an IT Architect will be asked to do across the course of their career is the Architecture Assessment. It is more common to conduct these if one is working as a consultant rather than an architect on an internal team, but even in those instances it is likely that an IT Architect will have to perform at least a few of these.

In the context of a larger solution lifecycle, the Architecture Assessment tends to occur most often in the initial phases (Discover, Analyse etc.), however assessments can occur at any point within the lifecycle as part of a problem-solving exercise or can occur entirely outside of the scope of any given solutions lifecycle. Assessments are disproportionately (in relation to time / dollars invested) important activities for a number of reasons:

1. It represents a dedicated effort to validate or discount core assumptions
2. It provides a mechanism for honest feedback (w/o fear of retribution)
3. It allows for rapid problem-solving and design exercises in environments where that might not otherwise occur.

Assessments are not a clear part of any particular EA Framework methodology (they may be implied in some sense), however that doesn't mean that they should be conducted in a haphazard manner. Assessments should be planned, estimated and designed in order to provide clear outcomes (be they recommendations, design deliverables or both).

Problem Space analysis is one of the techniques used with assessments - it can apply
both to the business and technical aspects of a project

An Architecture Assessment is also different than other traditional Architecture activities in that the expectation is generally that third-party personnel are more likely to perform them. The reasons for this include the following:

1. Assessments are one of the key tools involved in IT oversight activities (sometimes referred to as Independent Validation and Verification or IV&V).
2. It is more likely that an accurate assessment can be obtained by architects / investigators without a vested interest in the project. 
3. The skillset of the person doing the assessment is critical - it needs to be an architect and not merely a technical or product expert. This is the only way to ensure that all options / alternatives are properly considered / assessed. 

So what exactly is an Architecture Assessment? A typical assessment tends to include the following categories of activity:

• Information Gathering  
• Design & Project Review
• Design & Project Recommendations

An assessment also typically includes one or more types of specific analysis as well:

• Analysis of Alternatives
• Root Cause Analysis
• What if Analysis
• Feasibility Analysis
• Trade off Analysis
• Problem Space Definition & Resolution 

One way to capture alternatives is through Decision Trees.

Perhaps the most important function that an Architecture Assessment can provide is a mechanism to challenge assumptions and combat complacency. One of the main reasons that IT projects fail today is because there is generally no incentive or expectation to raise issues or problems. Rather than being viewed as a healthy activity - identification of the problems is feared in itself and thus ensures even more pain later on when the issue finally surfaces (which they always do).

When compared to the cost of the rest of a typical IT project (and any potential loss or cost overruns associated with problems that aren't identified or managed in a timely fashion), a relatively brief assessment exercise is generally less than 1% of total cost yet could make the difference between success or failure...

Copyright 2014,  Stephen Lahanas

#ITarchitectureJournal
#StephenLahanas
#Semantech-Inc

Read More

How to Set up an Enterprise Architecture Practice

What if someone walked into your office and told you; "you're now in charge of Enterprise Architecture," what would you do? Where would you start? Here are some suggestions...

First, you'd need to be cognizant of the mission implied with such a request. There are in fact two missions that might be involved depending on what type of organization you support. Those missions are:

1. Internal Architecture Management - This mission is focused entirely on supporting internal processes, systems and infrastructure. While these solutions may themselves facilitate core client-facing business capabilities, the architecture group does not directly interact with those clients. This type of EA group or department would of course interact with all other internal organizations and with partners.
2. External Architecture (EA) Consulting - EA Consulting is an entirely customer-facing mission (even though many of the customers you'll have will their own customers). This mission is the one more traditionally associated with the concept of a "Practice." The primary difference between a practice and an internal department is one of flexibility versus absolute subject matter expertise. An EA practice by nature must be able to support any type of organization and thus must be structured for rapid assessment and problem-solving whereas the EA department is more focused on ensuring project success and corporate continuity.

There are some occasions when an EA group may take on both missions, but that's relatively rare (reserved for some of the larger IT consulting companies). Once you understand the primary mission of the EA group you've been tasked to create, then there are five basic steps you'll need to follow in order to get it up and running:

• Step 1 - You'll need to do a thorough assessment of your organization (and its target market). This encompasses quite a lot actually, including - all existing systems, all currently used lifecycle approaches and tools as well as all business processes, goals and the corporate culture/s.

• Step 2 - You'll have to decide what EA means to you and your organization. Rule number 1 about EA - it means different things to different people and no two organizations implement it the same way. In defining what EA means what we're really referring to here is the ability to map EA frameworks, tools or processes to the business culture and environment and then align all of that with corporate expectations. The result is - as we alluded to - always a somewhat unique interpretation of what EA is for that particular organization.

• Step 3 - You'll need to choose, customize or otherwise develop a methodology and align that with the core business objective of your organization. This is one of the hardest parts of setting up an EA group because it usually requires integration with and or rework of existing lifecycle processes.

• Step 4 - You'll need to select what tools you'll be using. On occasion you may actually have the tools you need to start with but that isn't often the case and even when it is those tools are generally underutilized or otherwise lacking in how they're being exploited. Keep in mind that the tool or automation does not in itself make a practice or EA department magically come together - however building an EA group without one is fairly daunting.

• Step 5 - You'll then need to apply and integrate the tools and methodologies with your targeted portfolio of projects and or systems (or clients). This is where the rubber meets the road and where any ROI that might occur should happen. EA that only facilitates strategic planning and isn't fully linked into project architecture is a generally very poor value proposition. EA is ultimately an integration solution but to become that you have to go out and make the necessary connections with all other aspects of the enterprise. 

This of course represents only a very high level overview of what goes into building an EA practice or department - but it does cover some of the most important considerations.


Copyright 2014,  Stephen Lahanas

#ITarchitectureJournal
#StephenLahanas
#Semantech-Inc

Read More