Business Transaction Management Blog

by Stephen Burton, May 18th 2010.

I’ve been in the IT management/monitoring space now for almost 6 years. 6 years might not seem a lot but from what I’ve experienced not a lot has changed in the application performance management (APM) market. I glanced at the Gartner MQ a few weeks ago and the same old (and they are old let’s be honest) players were there in the leader quadrant, these vendors being HP, CA, Compuware and Quest. Leaders are normally people I aspire to be like or follow, they should inspire, excite and as their name suggests “Lead”, they should innovate and make the customers life easier. Please therefore excuse my sarcasm while I take a few seconds to think of all the innovation and inspiration these companies have given me…1 millisecond pause…right moving on.

The APM market from what I’ve seen over the last decade was driven from two needs a.) Database Performance Management and b.) J2EE Performance Management. Sure you had things like Road Runners and Profilers but those things were more used in dev/test rather than production environments where pain and business impact were real. And so database and J2EE performance management solutions came to market around the late 90’s with the premise to manage “application performance”. The simple fact is that these solutions were never built to manage applications across the enterprise, they were built to manage pieces of applications (e.g. JVM’s and databases). They became a victim of their own success to a large extent, they were fine at managing up to 100 database or J2EE nodes but they never really provided customers with that “End to End” application or enteprise view. The only end to end view they could provide was the end to end within a Java Virtual Machine (URL to SQL, Servlet to JDBC) or a relational database (SQL to Objects), many tried to correlate J2EE activity with database activity and failed, some succeeded but even seeing J2EE to database calls is not true “End to End” unless you ignore things like web servers, authentication servers, ESB’s, Mainframes, Message Brokers , 3rd party web services calls and other tiers which applications environment still have but are not covered in general by today’s APM vendors.

Yes, many vendors tried to expand their “End to End” coverage and acquired new technology in the space of end user monitoring, synthetic transactions & change mgmt but those solutions were never truly integrated together to make the customers life any easier. The result was a monumental jigsaw puzzle (known as the “APM suite”) held together with super glue and string. These giant jigsaws were put together and managed with professional services and support departments, all of which cost the customer money, patience and frustration. It’s therefore no surprise to learn that these vendors have spent the last 4 years re-writing their architectures to make all the bits work together. The bad news is that most of these vendors forgot about the most important piece of the jigsaw: business transactions.

I recently interviewed around 20 enterprises in the last 6 months for a project I’m working on. The purpose of my research was to learn about the strengths and weaknesses of today’s APM solutions. Pretty much all of the enterprises I interviewed already had solutions from who Gartner classified as “leader”. Therefore you’d expect to hear how APM was transforming the lives of customers and helping them generate more ROI than a city banker (pre 2008). To my surprise the results weren’t encouraging or good, the feedback I got was consistent and almost desperate from a few enterprises. APM solutions were great from a feature and data collection perspective but the physical effort, resource and investment to deploy these solutions was prohibitive. Too many moving parts, points of failure and pieces to deploy, configure, manage and support. Many of the people I met spent more time administering the performance of their APM solution management servers than their own applications! It seems many APM solutions just grew up to be big expensive monsters that were so impossible to manage many customers simply gave up, not because the solutions weren’t any good they just didn’t have enough time and patience to install and configure them across their portfolio of applications. When you’ve got to manage multiple applications across several hundred servers you want to be managing your applications and NOT your APM solution. That is unless you want to work long hours with many sleepless nights.

So what caused this response from enterprises? Well, by nature these APM solutions collect a lot of data to satisfy the needs of the technical people who develop and manage each and every application and database. Everyone has their own way of solving problems which is why you need thread metrics, JMX metrics, OS metrics, session metrics, invocation metrics, JDBC metrics, I/O metrics, CPU metrics, heap metrics, process metrics, SGA metrics, locking metrics, metric this, metric that, my favourite metric, his favourite metric, her favourite metric, my dogs favourite metric and so on. I think many APM vendors lost track of what is relevant when it comes to managing application performance and availability. They focused too much on monitoring application components rather than these things called business transactions that make their customers business run on IT. Customers care about their business, they don’t care about individual JVM’s, databases or other tiers. If the business is hurting they want to know the when, who, what & why so they can make it stop hurting. It’s about time IT was able to feel the business hurting and act accordingly rather than responding to infrastructure alerts, events, log files, exceptions and CPU spikes.

I’ve noticed now a lot of APM vendors talk about business transactions and are moving into the business transaction management, transaction performance management and business transaction performance space. This is a very good thing for customers, maybe we’ll finally see some innovation to help customers manage IT based on the needs and priorities of the business rather than JVM’s or Oracle databases.

Follow me on Twitter