February 8, 2010

We are delighted for the residents of greater New Orleans who deserve a celebration after their Saints beat the Indianapolis Colts last night in Super Bowl XLIV. The people of New Orleans and the Gulf Coast region, still recovering from Katrina, have been waiting to take home a Vince Lombardi trophy for the first time since the Saints began to play in 1967.

And while it took the Saints forty-three years to achieve their goals of success, back here in the IT industry it has taken about that long to achieve our version of a Super Bowl win – true business-IT alignment.

There is a lot of talk in the industry about the importance of “aligning IT to business”, “managing IT based on business priorities”, and “monitoring IT infrastructure performance from a business perspective”. The concept of aligning IT with the goals of the business is not new – we’ve been talking about it ever since the first time a computer was used for a business application (by a food manufacturer in the UK!) back in 1950.

Yet, the vendor community has not been able to enable CIOs to meet their goals of B/I alignment. A key factor is that application teams, the folks in IT that liaise with the business and are in the best position to align with their LOB counterparts, are still using infrastructure-based tools to monitor and manage their environment. However, transactions are where IT and the business meet, and only with a transaction-based management approach can IT start to manage services from the business perspective. Business transaction management enables application teams to manage their environment, for the first time, from the perspective of business transactions, rather than from infrastructure metrics. (Many of you will recognize these as the still-unrealized goals of BSM – business service management; see a recent Forrester report on how BTM delivers on the promise of BSM.)

BTM answers the following questions to support B/I alignment:

• Resources: How will moving to the cloud help reduce customer churn or improve employee productivity? What is the business impact of a consolidation project? How will adding new servers or VMs change service levels?
• Process: What is the IT cost per business transaction? Is investment in new infrastructure going towards the most business-critical services? Is business impact factored into decisions for change management and release management?
• Language: Does the LOB care about the performance of silo metrics (e.g. JVM, database) or do they care about the performance of a business transaction — for example, what is the time for a submit order transaction, how long the transaction took to verify the credit card, authorize the user, check inventory, etc., and what will the performance look like when transactions increase by 100%?

Without BTM, you lack the transaction-centric information to answer these questions and are left about as helpless as defending against a Peyton Manning pass. And while the Saints achieved success with 340-pound linemen on their side, all you need is to make the move to BTM.

I have yet to meet a satisfied customer with a CMDB. It’s not like I go looking everyday but somehow the subject always seems to arise and people tend to get quite touchy on the subject. It wasn’t so long ago I worked for a company who spent a fortune acquiring an application discovery and mapping solution. It promised auto-discovery in real-time and all those wonderful things yet it seemed to die a cold death months after the acquisition closed. I listened to the sales pitches around “When applications go down the first question asked is ‘What changed’?” and the famous “75% of application outages are related to change”. To me that’s like stating the bloody obvious, of course something changed, if nothing changed the application would be still running.

The problem these early discovery & mapping solutions lacked is business context. They were inherently built to track IT interactions from server to server and technology to technology. They’d scan server ports, executables and protocols to try and piece together the relationships between IT assets within a data centre. Or they’d do it by listening to the actual interactions. They’d paint pretty little diagrams like the London Tube Map and then give you all the technology, versions and patch levels these assets were running. Some solutions were priced per cpu which can be highly lucrative when vendors told customers to put an agent on every server in their data centre. Unfortunately the one thing they didn’t discover or understand was the business services that ran across these IT assets. Applications and Business Services these days don’t just relate to 4 apache web servers, 8 weblogic servers and 1 Oracle database. The pervasiveness of SOA and mashups these days means a single IT asset can serve one or more business services. Applications and IT Assets no longer have a 1 to many relationship, an application isn’t simply a collection of segmented IT assets.

I did a search on Wikipedia for CMDB and found the quote “A key success factor in implementing a CMDB is the ability to automatically discover information about the CIs (auto-discovery) and track changes as they happen.”. If a business service is a CI then that’s a pretty tough proposition to auto-discover and track change on. How do you discover business context from IT assets? For starters you can stop looking at the IT assets for answers, you’ll get your versions and patch levels of Linux but you won’t get a description of your business services that flow through them.

I’ll put my head on a lance and state that Business Transaction Management (BTM) can add significant value to any CMDB project. When you start to monitor business transactions you start to acquire lots of key intelligence on how your business runs and maps to IT. You auto-discover transaction flows and the IT assets they interact with, all in real-time. It also gets better, you can store all this data historically so that you can report and compare business services and their CI’s before and after a change. You can even visualise how the business and IT asset dependencies change over time using transaction flow/topology diagrams as key evidence. When a change occurs on an IT asset you can instantly report whether this change had a positive or negative impact on your business services or transactions by reviewing related latency and SLA. I’m not claiming BTM is the answer to all CMDB pain but it solves some of the most basic and common challenges:

1. Auto-discovers business services and their IT dependencies in real-time
2. Stores information historically so you can track, report and quantify change when it occurs

Maybe BTM is the pain relief CMDB projects need right now.

Follow me on Twitter

we know our users are complaining, we know we are losing business, we have been trying to identify the issue for days, I am losing credibility, I missed several friends dinner, I work every weekends, I have to leave the office now because I have to jump on a change management conference call while driving with the kids screaming in the back of the car. I have other things on my plate, like launching our new private banking services, budgeting for new servers to address our merger with ABC company, I need to grow my business, we can’t even have a feel on how our services behave nor identifying simple problem such as one out of five times the browser hangs when entering employee badge number. The assumption I made last week on where the problem might have been are now wrong, the change management team applied a patch against that specific application and the problem didn’t go away. I am stress and tired…. I am stress and tired…. I am stress and tired…. I am stress and tired….

IT experts would say: “I have tools several, several, several, several tools, and it is true after triaging all the alerts, the tools were able to isolate issues but I really just care about what impacted my users in company ABC. What is the behavior of my most revenue generating transactions today and what will it be after we merge the two companies’ systems next week, how would I know if it improves or degrades the overall business service?”Familiar with THIS?  What if you would take a peek at introducing Business Transaction Management (BTM) into your IT process?

You would finally see at this moment the IT consumers and IT producers of business transaction information, knowing whom and what is impacted, focusing only on the most important services. What if you knew the exact flow of the information and the behavior of your special revenue generating credit card application transactions? BTM is a source of rich IT information.  It is much more than incident management, you can not only understand the current behavior and plan for growing your business you can see the impact on your services of an unplanned or planned change.

This is the cure to resolve the “THIS”, today, tomorrow, next week, on a constantly changing fluid IT environment. Really who could have predicted that you would transact business via text messages?  With this information on hand feel free to use those specialized tools and apply them appropriately to isolate granular application components issues but change the way you think about managing IT,  It is not always about technical components. Now, I won’t cure all your stress and fatigue as there always be screaming kids, traffic, lines at the coffee shop but one less thing to worry about, getting a little more of your personal life back, one more thing to proudly walk to your management and really feeling good that you know the “THIS” at every moment of the day and I guarantee you will be grooving along a Disco song….

As a vendor of BTM solutions, we know what our costs are, so that part of the equation is fairly simple to figure out.  Trying to put a dollar amount on the value of BTM is where things get trickier.

The common way of quantifying the value is breaking it down into value points that derive from the different ways customers use BTM.  For example, practically every user who deployed a BTM solution in Production has been using it, among other things, to prevent application outages.  If we can determine the monetary cost of an outage – and we can – and if we can show that BTM has reduced outages by a certain percentage (or eliminated them entirely), we will be able to calculate how much money was saved by deploying BTM for outage avoidance.

And outage avoidance is just one of the many use-case scenarios for BTM.  Virtually everyone who implemented BTM has also discovered how useful it can be for problem isolation and resolution.  Just like with outages, it is possible to measure or estimate the cost of the problem determination and resolution cycle.  If it can then be demonstrated that the Mean Time to Repair (MTTR) for performance issues dropped by 70% since a BTM solution was implemented, we can add this windfall as another component in the total value of BTM.

We can come up with more use-case scenarios for BTM and estimate how much money was saved in each one of them.  BTM can cut costs by improving processes such as application optimization, QA testing, SLA management, capacity planning, activity-based costing, application consolidation, and more.  Each one of these cost savings can contribute nicely to the bottom line value of BTM in an organization, but does this total really represent the value of BTM for the organization?

I hereby propose that it does not.

If we examine all the different scenarios in which BTM is used, we can see one common element and that is that BTM provides a new level of visibility that did not exist beforehand.  BTM allows IT to see things that they could not see before.

Consider this analogy: Nine year-old Jack is a vision-impaired kid who, for the first time in his life, gets a pair of glasses that allows him to see clearly.  How much are the glasses worth to him?  We could point out that with his new glasses on, Jack successfully avoided an incoming car as he was crossing the street and saved himself from being badly injured or worse.  Also, Jack now spends only 30 minutes a day doing homework whereas before it used to take him well over two hours.  Furthermore, Jack’s family can finally move out of their expensive house, which has special amenities for the vision impaired, and into a regular house.  This will cut down their rent by almost 20%.

Each one of these observations can be translated into a dollar amount, but is this getting us any closer to determining the true value of glasses for nine year-old Jack?  What about the fact that for the first time, Jack can see what his parents and sister look like?  His amazement when he found out that his cat actually has one blue eye and one green eye?  His pride at finally being able to read books and watch TV just like the other kids?  Isn’t Jack getting far more value out of these simple discoveries and accomplishments than from having to spend less time doing homework?

BTM is a game-changing technology. Being able to see transactions clearly enables IT to stop “flying blind” and start making informed decisions that take service management to the next level.  Just like Jack with his new glasses, once IT starts using BTM regularly, they can no longer manage without it even for one day.  What, then, is the real value of BTM for an organization?  The question will remain open for now, but I can tell you that much – it goes far, far beyond just being able to avoid outages and fix problems faster.

When people in IT learn about Business Transaction Management for the first time, their reaction is often a similar mix of awe and disbelief.  They are awed, because gaining true visibility into the behavior and flow of all business transactions has been an industry holy grail for quite some time now, and they also express disbelief, well, for the exact same reason.  How is it possible to auto-discover and track all business transactions, when transactions are not tangible, manageable configuration elements like servers and routers?  At best, such technology might work in a simple, straightforward environment, but our IT environment is huge, complex, distributed, heterogeneous… can this BTM work for us?

Like other emerging technologies, BTM is climbing up a maturity curve.  From just being able to show round-trip response times and infer some latency breakdown of network versus data center time, superior BTM solutions are now capable of tracking transactions deep into the data center, providing rich, granular topology views that cross hundreds of web, application, authentication, messaging, and database servers.  The ability to show simple request-response sequences has expanded to cover complex, asynchronous flows using pub/sub and “send and forget” messaging protocols.  In addition to showing all transactions, some BTM solutions will now auto-discover and show entire business processes, aggregating many discrete transactions into a “short list” of real business flows.  The ability to measure service quality is also maturing from application and server uptime SLAs (remember five nines?) to transaction-specific response time SLAs and Apdex ratings that represent end-user satisfaction in real-time.

Despite being one of the privileged few who witnessed BTM grow from a mere idea into a full-blown enterprise solution, I still find myself sometimes awed by this technology.  It is, after all, an ambitious attempt to visualize complex, abstract business ideas, and manage them like any other assets of the organization.  How is it possible?  Does it really work?  As my friend and colleague Andy previously noted, the “aha moment” for BTM typically doesn’t arrive until after the customer has already seen it live in their own environment.  And that’s when the fun begins.