Reinventing Enterprise Architecture

I've seen articles recently proclaiming that Enterprise Architecture (EA) is dead. Of course, I've seen the same proclamations for SOA, for COBOL and any number of other technologies or IT practices over the years. These premature announcements all have one thing in common - they equate the value of a technology (or technology practice) with its current position in the hype cycle rather than its actual utility. There is another thing these death notices fail to grasp - technologies aren't static and they aren't always replaced by something new. But before we dive further into this topic, perhaps it is worthwhile to revisit what the current expectations for Enterprise Architecture.

Sadly, Mark Twain is really dead now, but EA isn't...

EA, as it is currently defined…

The set of high-level design activities necessary to support IT portfolio planning and/or specific IT Transformation initiatives. While EA is typically considered primarily a business-focused activity, it is also often concerned with high-level design of data, application or security solutions. EA is typically distinguished from other types of architecture by the scope or scale of the associated problem-space.

Now, this definition doesn't capture any of the core issues associated with the typical exploitation of EA in most enterprises and the reasons why some people have determined it is no longer relevant. And, like most other aspects of technology, there are legitimate issues associated with EA and a history of many Enterprise Architecture efforts not meeting expectations. Some of the problems that have been associated with EA over the years include:

• A lack of understanding as to what value it provides
• A too narrow focus on feeding (building out) an EA repository
• Lack of connection to the actual IT solution architectures

Sometimes, when we hear the word "Reinvention" we get the impression that whatever needs to be reinvented is somehow broken; that's usually not the case and it is definitely not the case with EA. What Reinvention does imply however is that there is always a need to evolve and improve - that is how any capability, technology or set of skills remain relevant over the long-term. Reinvention allows us to enhance what works, replace what doesn't and redirect all of it in more meaningful ways. That's exactly what I'm proposing for Enterprise Architecture.

So, how does one go about reinventing a major IT field of practice? 

It starts at the beginning - the expectations for that practice. These new expectations must take into account both the lessons learned (the current criticisms) as well as the trajectory of future need. The expectations fall into three categories:

1. Greater Flexibility
2. Greater Agility
3. More Relevance

EA must become more Flexible
Some of the severest criticism of enterprise architecture comes from those who have seen EA become a means unto its own end. In other words, the practice of EA has sometimes been overly focused with the management tools used to implement it. These tools are powerful and typically include EA repositories, technology catalogs and various analytics. These tools are also typically built atop a foundation known as an Enterprise Architecture Framework. These frameworks, which I will talk about in more depth in some future posts, include TOGAF (general), DODAF (military), FEAF (federal) and Zachman (general). The Zachman and TOGAF frameworks are typically better suited for commercial environments while the government ones tend be mandated as part of the larger IT portfolio processes of the agencies involved. 

The key thing to keep in mind with Frameworks is that they merely represent meta-models of what the types of things a typical organization ought to be concerned with. In this respect they are much like ITIL (which is essentially a taxonomy of data center processes) or like standard industry EDW data models (for energy, healthcare etc.). The Framework/model becomes the schema for the EA Repository and then the tool helps architects to build visualizations using data placed into that repository. The problem is that sometimes this process of building out the repository and the enterprise view of what's happening becomes too inwardly focused and too rigid. The flexibility required moving forward is this:

1. The tools are there to help the larger process of assessing the enterprise and supporting planning. That means that the tools must also serve the process and not the other way around.
2. The meta-models are not the end-all, be-all of what can be done with enterprise architecture. Like EA itself, all of these meta-models are evolving and there will often be important aspects of what's happening in the enterprise that aren't properly covered by the framework. EA is more than any given framework and can exist beyond them or even without them if needbe.
3. The work that an Enterprise Architect does is part of a larger continuum. We'll talk about that a little more in a minute, but the key thing to keep in mind here is that the Architect and the organization need to understand where and how the work fits in - and be willing to adjust as necessary (rather than staying on the same rigid course regardless of what is discovered).

Agile EA is possible...

EA must become more Agile
This is also a popular criticism of Enterprise Architecture; that it is often a rather lethargic and abstract exercise. Well, sometimes it is true, but that of course only happens when the activity is not made Flexible (see above) or more Relevant (see below). But what does Agile really mean in the context of Architecture? Many people might think that the two concepts represent a sort of cognitive dissonance. That's not really true though. Enterprise Architecture, like any complex enterprise level activity can be made more Agile by simply changing the expectations associated with it; for example:

• That analysis or assessment activities be time-boxed and aligned with specific goals or initiatives.
• That building EA deliverables or documents serve a specific purpose, such as Portfolio review or creation of Reference Architecture guidelines.
• That EA is not a giant one-off effort, but rather a more or less permanent and iterative process, that allows for continuous innovation as an organization.

Making it Real (or at least Relevant)
This is perhaps the most important change that needs to occur, but it also happens to the be the least recognized by most groups who are actually having issues with EA. This is borne from the problems associated with lack of flexibility - namely the stove-piping off of EA from the rest of what the enterprise is doing. I've been there, seen that - an EA group producing something that is totally ignored by other groups charged with making strategy real. There is one sure way to cure this problem and that is to connect EA to the rest of the IT lifecycle. The first step in doing that is understanding where EA fits in the larger spectrum of IT Architecture and then building the necessary links between all design and governance process for IT capability. 

EA when viewed by itself, represents the strategic or business focus of the larger IT Architecture mission spectrum. It is where enterprise options are assessed and where organizational strategy becomes an actionable plan for achieving organization-wide capability. It is also the starting place for all design. This unique, combined role places it squarely in between business and technical interests and their respective stakeholder groups. The future of EA is with this role empowered to broker issues between the wants and realities of the typical enterprise. EA is not analysis for its own sake - it is an ongoing and collaborative process for ensuring that Digital Innovation or evolution occurs according to plan.

copyright 2014, Stephen Lahanas

Read More

Creating Agile IT Transformation - part 1

IT Transformation has been a buzzword for more than a decade now, but what does it really mean? The first time I heard it used regularly was in relation to specific Department of Defense (DoD) technology initiatives from the early 2000's. I had the opportunity to work on several of those projects and as the years progressed the concept of IT Transformation evolved quite a bit – becoming much more flexible and yes – even somewhat Agile in nature.

At first IT Transformation was viewed from a more comprehensive perspective, sort of an organizational make-over if you will. This initial view involved Transformation at multiple levels and from multiple perspectives. This often included consolidation of organizational functions as well as IT systems and hosting capabilities – all at once. In some cases, IT Transformation was becoming almost synonymous with Enterprise Resource Planning (ERP) initiatives; in fact some people still view it that way.

A holistic perspective of IT Transformation

The problem with the comprehensive view of IT Transformation though is its scope. As hard as it is to even get a project like that off the ground (funding, stakeholder buy-in etc.) successfully executing something that large is even more challenging – sort of the ultimate “Big Bang” approach. Despite that, Transformations are still hard to avoid – organizations that don’t adjust to changing realities and emerging technologies can rapidly become ineffective or redundant.

So, how does one approach Transformation in a way that can actually succeed? First we need to redefine it:

IT Transformation

“The set of activities required for an organization or group of related organizations to successfully adopt emerging capabilities and practice. This emerging technology or practice could be focused on one major capability or may involve multiple technologies and processes associated with a specific initiative.”

This definition allows us to view Transformation differently. Rather than the entire organization changing all at once we’re focusing now on areas of specific significant change.  This Transformation based on significant enterprise change can be further decomposed into segments similar to the previous view of holistic Transformation (the business portion, the data portion, the solution portion etc.).

You might be asking yourself how this new view of Transformation differs from any other type of major IT initiative or project. The primary difference is that while today’s more Agile Transformation can be highly targeted it still exhibits these differentiating characteristics from typical IT projects:

1.  It is designed to fit into a larger set of Transformation goals (e.g. it comes pre-integrated, enterprise-aligned from day 1)
2.  It typically involves the combination of several distinct technologies and processes – moreso than other IT projects (because it is already enterprise or strategic-facing in nature)
3. It typically is more mission-focused than many other IT projects. In other words, it has been selected to tackle a critical business issue, not just a technical concern.

Solution providers that support this new type of Transformation are somewhat more flexible in their perspectives on how to tackle complex Transformations than some of the more well-established consulting firms may be. While an ERP transformation may easily cost several hundred million dollars and still not succeed, Agile Transformation approaches look for smaller chunks of capability with higher ROI and success rates.  We will highlight the primary Use Cases and several case studies for Agile Transformation in the near future.




copyright 2014, Stephen Lahanas

Read More

Making the Case for IT Architecture

There are still quite a few misconceptions in regards to IT Architecture. For many, hearing the term "Architecture" in relation to any IT topic seems to imply Enterprise Architecture (EA). To others, the notion of formal design processes represents the anti-thesis of Agile or responsive problem-solving. These misconceptions are unfortunate because there has never been more architecture connected to IT in actual practice and there has never been so much need for it.

So, let's start at the top - IT Architecture is a continuum and an umbrella for every design process in the enterprise. IT Architecture is the foundation for understanding every enterprise capability as well as being the starting point for all planning and governance. The reason it serves all these roles is because it provides the necessary insight for guiding all of those processes. Without that insight, making decisions and governing IT management or evolution becomes more or less like guesswork.

Why should this be so? Well, it has a lot to do with the disruptive and dynamic nature of Information Technology; it is not all uncommon for large organizations to have incomplete information in regards to their own systems, costs and data. Often times there are redundant, distributed islands of IT that cross business units or may even involve the separation of capability across IT & business communities. Gaining a complete picture of what's going in many organizations is quite a challenge - but is absolutely necessary in order to unify systems operation and to move forward in a coordinated fashion to exploit new opportunities and technologies.

So, let's go back and address those two very common misunderstandings again…

1 - IT Architecture and EA are the same thing (and EA is useless) - now I've paraphrased some of the criticism I've heard over the years about EA, but of course there is a legitimate question there. Is there any real value to EA? First off, EA and IT Architecture are not the same. EA is a top level architecture process and represents perhaps 20% of the total architecture that occurs in the typical enterprise. EA is integrated with IT Strategy and portfolio planning and governance.

Does EA have value? Yes, it does. It represents that portion of IT architecture that helps allow business stakeholders understand their IT capability landscape. It is high level enough to function as a communications tool, a governance framework and a project estimation bench-marking tool. EA is also, when done properly, the bridge to all other solution architecture.

2 - Does Architecture inhibit Agility? This is a thorny question and a complex topic, but I will try to address it briefly. At first blush, it might appear that any formalized design process might not fit within the context of the Agile Manifesto. But there are some important considerations worth noting that tend to contradict that initial assumption:

• Agile, as a methodology, has never been effectively transferred from a system or application scope to an enterprise scope except through the adoption of more iterative, time-boxed approaches to lifecycle management. There is a good reason for this - purist Agile approaches are focused on deriving requirements through experimentation rather than up front design.
• When you move from the application development scope to the enterprise integration domain things change - a lot. The problem at this level is no longer invention, but reconciliation and complexity management. In this context, the discovery associated with design relates to existing capability and introduction of well-defined new technologies. Here, IT Architecture provides the reference point for establishing and maintaining control over what otherwise might become a chaotic environment.
• Agile is all about making the application work by any means necessary - operations is all about making the enterprise work as a whole in the most efficient manner possible.
• And even within Agile, there is a design process and that process is merely one specifically-tailored component of a larger family of IT Architecture practice.

Perceptions are often hard to change and IT Architecture is a large, emerging and complicated field of practice. After working as an architect for 16 years, I can honestly say I've never seen more organizations doing architecture work. More often than not that architecture has become an important part of their overall IT management strategy. Yet, the continued misconceptions tend to hold back the true potential of IT Architecture. Too often, people view it from the perspective of one of the many sub-disciplines (such as Agile design, etc.) rather than seeing it as a holistic enterprise resource. I think that as the realization of its true scope becomes clearer IT Architecture will become even more important in the years to come.

copyright 2014, Stephen Lahanas

Read More

Lifecycle Management, Defined

Like many other aspects of Information Technology over the past 15 to 20 years, Lifecycle Management has evolved quite a bit. In fact, it didn't generally even get referred to as "Lifecycle Management" until about ten years ago. It (both the term and the practice) has become more expansive in nature as time has passed. This is an interesting not just due to the evolution which has occurred to the surrounding practice, it is also the one area in IT where technology and business / management comes closest to being blended into one concern.

Lifecycle Management is also the reason that IT Architecture exists the way it does today - without the driving force of various governance processes and mandates, IT Architecture would have likely never quite crossed the threshold from systems or software engineering and design. We'll return to that in a minute.

So time for definitions - and in this case - a hierarchy.

Lifecycle Management, DefinedLifecycle Management is the practice of coordinating all the related activities for any given selection of IT capability (within or across organizations). There is generally a hierarchy for these activities which falls roughly into the following structure; Portfolio, Program, Project. Generally at the Portfolio level, the capability management spans many systems whereas there may be multiple Projects supporting just one piece of software or one system. In an optimal situation, all Lifecycles should be unified based upon shared methodology, principles, goals and supporting tools. 

Many people have long considered Lifecycle Management to be the province of the manager. However, nothing could be further from the truth. For Lifecycle Management to function as intended there has to be a partnership between management and technology resources. This partnering usually occurs on the tech side through the role of IT Architect.

This view is a more traditional enterprise-focused view (mainly waterfall in nature) 

Why Lifecycle Management isn't just about Management
Let's go over a few things that must have technology input to succeed:

1. System (and project) Scope - While there may be a number of specific expectations from business, the most successful systems (and projects) are those in which the tech side helps to define the boundaries of what can realistically be done. This manifests itself at the highest level in portfolio roadmaps and at lower levels through statements of objectives or similar scope documents for projects (which then get passed through governance processes for approval). 
2. Project Estimation - This is a much more complex process than many realize. Good project estimation requires a precise understanding of the complexity of the technical tasks involved. Whether you're talking about building J2EE-based APIs or deploying a WCMS, the tech experts who have been there / done that are invaluable in helping to define a realistic project estimate. Since much of IT is measured based upon estimated cost versus actual expense - this area is disproportionately important. 
3. Risk Assessments - Sometimes this includes Architecture Assessments, other times it is more of a technology strategy type exercise.
4. Technology Selection - Should we Build or Buy, or if we're going to buy, how do we choose? These are the typical questions that get asked, but who can really answer them? The IT Architect, of course - its part of their job description.
5. Solution Decomposition - This is related to project estimation but more detailed. Generally, somewhere within a planning process, there is a preliminary design exercise which goes as far as determining how best to 'break up' elements of the intended solution into digestible pieces. This also involves prioritization - understanding which parts have to come before or follow others due to various technical dependencies or constraints. 
6. Project Oversight - This is a big topic, one we will dedicate other posts to, but for now we'll just introduce it. IT Architects often play a semi-leadership role within individual projects (as a project or solution architect). This is so technical issues can be tracked, managed and mitigated with continuity across all phases of the individual lifecycle. 
7. Enterprise Integration - This is without a doubt the hardest part of the overall set of potential Lifecycle Management related activities. Now, many think of Enterprise Integration as a purely code focused or networking relating topic - believe me it's not. Enterprise Integration is in reality the intersection of potentially overlapping systems, data and expectations. The most important aspect of Enterprise Integration is inter-operability design  (which comes in many, many forms). It's a study in complexity and not for the faint of heart. 

Let's step back for a moment and explore Lifecycle Management from a different angle. In previous posts we've been talking about methodology a bit; Agile, Waterfall and so on, but what does it all mean? The short answer goes something like this:

An IT methodology quantifies the workflow or lifecycle steps necessary to provide consistent support for system development and management. Most lifecycles share the same common elements; a planning phase, design phase, build phase and implementation phase. And yes even Agile has all of these. Within the common framework, there is a massive amount of variation. So, if you ever had to work with Carnegie Mellon's CMMi (waterfall spiral methodology) and then were put on a Scrum project (Agile variant) it would be like moving from Iowa to Italy - a culture shock to be sure. 

In today's terminology, you often hear the term Application Lifecycle Management (ALM) used - it is essentially the same thing as Software Development Lifecycle (SDLC) except perhaps that today an ALM can encompass both Waterfall and Agile whereas before SDLC was more Waterfall focused. So what are Waterfall and Agile, anyway? 

Waterfall - this refers to both purely linear and 'linear iterative' development methodologies, wherein requirements are more fully defined up front and allocated to segments in a somewhat sequential manner. In some cases, code is completed and tested in iterations - other times no capability is delivered until the very end (traditional waterfall).

Agile - Up front design is minimal, few if any requirements are formally development. The focus is the production of working capability with each (very short) iteration. Design in this model becomes iterative in that core assumptions can be readdressed and correct "in flight" (which is why up front requirements become redundant or unnecessary). 

We will be talking a lot more about 'Agile' and 'Waterfall' as we progress on the blog. 

Lifecycle Management has evolved from a set of competing software development methodologies to become a potentially all encompassing set of technology management processes for the enterprise. And it will continue to evolve with the IT Architect maintaining a key role for some time to come.

Copyright 2014,  Stephen Lahanas

#ITarchitectureJournal
#StephenLahanas
#Semantech-Inc

Read More

Aligning Lifecycle Methodologies - Part 1

In a previous post, I contended that requirements are still an important part of most enterprise environments, even those that might be using Agile lifecycle methodologies. However, I didn't specify exactly how those requirements should be captured — or even what constitutes a requirement. This is an important question when determining how to align multiple lifecycle approaches within one organization (the need for which is much more common than most people realize)

This question encompasses two major lifecycle approaches; Agile and Waterfall methodology. Many organizations feel that User Stories (Agile) are sufficient to build both code and project structures around them. Is it out of the question to consider that Use Cases might be used within an Agile approach?

Before answering that, let’s take a look at User Stories, Use Cases and Requirements in context again:

• A User Story can be thought of as a high level or conceptual scenario, or perhaps just a problem. An example might something like this: A user needs to be able to save a report into multiple file formats for downloading, including both .pdf and .xls. Even though we’ve described two different formats, it still represents the same scenario or problem.
• A Use Case, on the other hand, can be considered a functional requirement. The difference between a functional requirement and technical one is the level of detail the document includes about how specific actions are carried out. For example, a use case would capture the process flow, or steps involved in generating the report and the expected outcomes or alternatives. This helps to build test cases later on. A use case is slightly more structured and detailed and is generally written by someone IT or in combination with the IT group. That way, some of the technical expectations are captured. In other words, the use case describes not just behavior, but how that behavior is achieved.
• A Requirement in this context refers to a technical requirement. This is entirely specific to the elements needed to develop and/or deploy the capabilities described in the previous two documents. It allows us to formalize performance constraints, data elements and other pertinent technical details. This is especially important in environments where EA, service or data governance paradigms are in place. If written properly, requirements can then serve as system documentation.

Mapping across lifecycle approaches begins with requirements (however they're captured)

While many organizations practicing Agile bypass requirements, they may in fact still capture the same information after the fact if they actually document their deployed solutions (after the fact). When you’re considering whether or how to employ user stories and use cases, keep these points in mind:

• They are not the same thing.
• The user story ought to come first.
• The use case ought to be derived from the user story.
• Any requirements managed from this process should be embedded within, or otherwise traceable to, a specific use case and user story.
• User stories could be considered either scenarios, high-level processes or problems.
• Every organization approaches this process somewhat differently. For example, in a less Agile environment, a number of other architecture artifacts could be added to the illustration above:
• At the user story level, many people like to capture a data-flow diagram or a context diagram.
• At the use case level, many folks like to add a UML sequence diagram and conceptual data model.
• At the requirement level, many UML diagrams are often added including activity models, state models, component diagrams or deployment diagrams.
• If a formal EA framework is being applied, those UML or ERD diagrams are often replaced or folded into EA Framework views/artifacts.

Getting back to the original question — should use cases be used with Agile lifecycle management? — my recommendation is yes. Here’s why and how.

Why to Do It

1. It ensures some level of systems engineering is included in the Agile design process.
2. It helps to define the tasks associated with each user story and doesn’t add that much extra effort - it also doesn’t require a use case diagram (which in most cases doesn’t convey that much meaning anyway).
3. It ensures that both IT and business stakeholders capture their expectations in formats tailored to their roles and needs.

How to Do It

• Define your use case template and what it must convey.
• Determine whether user stories require more than one use case. It happens occasionally, although in most situations user stories should be in a 1-1 ratio.
• Ensure that someone from IT helps with, or provides much of the detail, in the use case.
• Ensure that use case development is aligned with both the quality and testing paradigms. This is especially important if no requirements are captured.

The majority of Agile development environments are fairly flexible in regards to storing of related attachments, so the biggest impact to employing use cases along with user stories is in the design and development process. This is the first step towards aligning lifecycle methodologies; we will examine this further in several upcoming posts.

Copyright 2014,  Stephen Lahanas

#ITarchitectureJournal
#StephenLahanas
#Semantech-Inc

Read More

Can Enterprise Architecture be Agile?

IT practice changes quite a bit and certain practices come and go. About 10 years ago, Enterprise Architecture (EA) surged to the forefront of the IT community primarily because it was adopted as part of the key acquisition processes connected to the Federal Government. The law that brought this about was called the Clinger-Cohen Act and it provided portfolio management expectations for IT which were designed to help avoid large-scale IT project failures (ones costing the taxpayers more than $100 million) but also applied to all IT investments. Not too long after this law was passed in 1996, the Department of Defense and other agencies redesigned their IT processes / methodologies to become compliant with Clinger-Cohen. Soon after that the practice of EA became very much in demand.

EA was around before this of course, namely through the introduction of the Zachman framework in the late 80's, early 90's, but most organizations hadn't adopted it yet. The federal mandate provided an excellent incentive both for federal and commercial adoption. It also created some issues however, namely that much of the early work in EA became tied to a more procedurally intense form of practice - in other words, to many it seemed to be a bit bureaucratic in nature. The perception quickly became that EA was not an easy undertaking or something that could be used to produce quick results. This is similar to what happened to E-learning at roughly the same time for about the same reasons - government adoption and stewardship was both a blessing and curse to the industry.

So the following question is now often posed within IT in regards to Enterprise Architecture - is it possible for EA to be Agile? In most quarters, the immediate reaction would be a resounding - no. But perhaps maybe we're being too harsh in our initial judgement. First, though, we need to define what Agile would mean in an EA context:

1. It would require rapid turnarounds for designs and decisions.
2. It would require the ability to integrate both with traditional (waterfall) methodology and Agile development methodology.
3. It would need to support and promote collaborative problem-solving within the organization using it. In other words, an EA group doesn't become another island or silo, it is the mechanism that bridges all of the silos.

Based on these definitions, EA can and definitely is Agile if practiced correctly. The following articles provide examples of how various aspects of EA can be integrated within an organization and remain Agile:

• Aligning User Stories, Use Cases and Requirements
• Integrating Design & Architecture
• SharePoint as an Architecture Repository

Many of the articles included in this blog were produced from projects that followed an Agile EA approach, including the Intelligent Healthcare framework, Semantic COP and Governance as a Service. I've applied Agile EA in the following domains thusfar:

• Defense
• Aerospace 
• Healthcare
• Retail
• Manufacturing 
• Finance

I've also applied to the following IT focus areas as well:

• Cyber-security
• Data Management
• Services / application design
• Portal / CMS

So returning to the original question - can EA be Agile - my answer is yes. The key to making that happen involves the following assumptions:

1. Clearly defined expectations for the both what the EA group or architect must accomplish.
2. A willingness to improvise rather than following industry patterns or methodologies too closely.
3. The ability to work to deadline - to engineer a design around the constraints.
4. A willingness by the organization to allow the silos to actually work with one another (with an architect as mediator) - this is far less common than you might imagine.
5. The ability to develop architecture that is easily translatable and traceable to implementation level designs. In other words, alignment of all design related efforts must be built into the Agile approach.
6. The ability to make decisions quickly. 
7. A willingness to experiment. Many architects and organizations make the mistake of separating EA into a logical or somewhat abstract exercise and not allowing it to be involved in actual prototypes or POCs. The best way to ensure that key design decisions make sense is through an understanding of the technology in question.

Copyright 2014,  Stephen Lahanas

#ITarchitectureJournal
#StephenLahanas
#Semantech-Inc

Read More