March 7, 2011

We’re proud of the fact that OpTier software powers a variety of critical businesses. Every day OpTier BTM ensures that stock trades execute fast, national train lines keep on schedule, billion dollar procurement systems don’t fail, online bill payments are executed properly, mobile phone service plans are provisioned, and insurance claims are processed. And while it’s not as sexy, we also ensure that citizens are able to pay their taxes on time by managing tax return filing systems.

Our customer, a large, national tax authority, is responsible for collecting all online tax submissions each year. Since few of us prepare our tax forms in advance, it comes as no surprise that most of its traffic arrives in one giant peak just before the deadline.  In fact, more than 80% of its annual traffic occurs during those 3 weeks, and 10-15% of the traffic occurs during the final 8 hours! Of course the annual peak is extremely stressful for the IT department, and in the past, there have been some painful system failures that resulted in submission delays.

This year, (we’re happy to report,) the annual peak was scaled successfully. Our customer used OpTier BTM to monitor all of the key servers, and during the final day, it was processing nearly 5 million transactions per hour.  These transactions are exceptionally complex, with over 200 tiers, and OpTier BTM discovers them all automatically, which is important, since there are changes every year.

During the peak, a customized OpTier BTM dashboard is displayed on a 50” plasma monitor at all times. Around 50 people man the command center 24/7 during the peak, and OpTier BTM is always in focus.

The SOA architecture is developed by a number of different application teams and vendors, so the ability to identify where a problem is occurring and put the resolution into the hands of the correct team is absolutely essential – it saves everybody a lot of finger-pointing and arguing over who’s holding the ball. For example, in the cut-out from the dashboard below, the red block shows a slow-down in the performance of the back-end services. By isolating the problem, OpTier BTM can reduce the time spent on troubleshooting by as much as 90%.

Needless to say, the business impact of any outage is enormous for the authority, the vendors, and the public. So the ability to identify and resolve problems quickly is crucial.

OpTier BTM repeatedly identified significant slow-downs much sooner than other monitors, and proactively identified several different types of incidents. The team also used OpTier BTM to drill down and isolate problems. In more than one case, OpTier BTM was used to halt an all-hands call, and to identify both short-term and long-term solutions.

While OpTier BTM complemented the other monitoring tools in the data center, the team appreciated its business focus and the ability to understand the user impact of IT issues. Where other monitors each showed one piece of the puzzle, OpTier BTM captured the entire picture. To quote one of their operations managers, “We need this stuff! We should be using it to monitor other applications as well. ”

“OpTier, the company that ensures you pay your taxes on time.” As true as it is, we’ll have to mull that one over again as a company slogan…

February 22, 2011

Many companies are developing their strategy for migration of business applications to private and public clouds. During this critical stage, it is vital to ensure that service levels are not impacted by migrating the application from dedicated to shared IT resources. It’s no wonder that according to analyst firm IDC, two of the top three concerns that CIO’s have about private clouds are performance and availability.

We see in the market that enterprises are forming new cloud teams and internal committees, with a diverse set of skills, to plan for an effective organizational cloud strategy. One of their mandates in the organization’s journey to cloud is to plan for how to monitor and manage the performance and behavior of applications after deployment. These organizations undoubtedly have a range of infrastructure monitors in the data center. And most cloud service providers, whether internal or external, will provide services for monitoring cloud resources. Yet these tools typically do not provide an accurate picture of what end users are truly experiencing and how to quickly isolate and fix performance issues in application components located inside and/or outside the cloud.

This blog entry points out a few of the key application performance challenges that you are likely to encounter when pursuing a cloud strategy, so that you can address them proactively. I hope that during my session in the Cloud Performance Summit at CloudConnect (Instrumenting Applications When Access Goes Away on Monday March 7) the esteemed panel will address some of these challenges with a variety of perspectives – it should be informative and thought-provoking!

1. How do you know if an application is ready for the cloud?

Not all applications are ready for “cloud time”, and sometimes one part of an application is cloud ready while other components are not. You need to identify the best components for migration as well as potential problems such as chattiness and latency that are amplified in the cloud.

2. How do you find server-related root causes when performance issues arise?

In fully-dedicated environments, we sometimes use infrastructure metrics and events to diagnose performance issues. But inferring application performance from tier-based statistics becomes challenging – if not impossible – when applications share dynamically allocated resources. In the cloud, you must be able to understand application performance and its correlation with the underlying physical and virtual components.

3. How can you minimize the risk of change to the cloud infrastructure or the application?

In a shared environment, any change to the application, or to the infrastructure, is high risk. Cloud owners, operations staff and application teams must be able to test the impact of change on service delivery – whether that change is in an application before deployment, or in the cloud infrastructure.

4. How do you implement or verify chargeback?

Traditional application performance monitoring (APM) tools do not collect resource utilization per transaction to enable business-aligned costing and chargeback paradigms. For the cloud, you need a solution that monitors consumption for every service across multiple applications and tiers, so you can accurately cost services, decide on appropriate chargeback schemes, and tune applications and infrastructure for better resource utilization and lower cost.

5. How do you ensure that services are allocated according to business priority?

To ensure that SLAs in the cloud are met, you must be able to prioritize the allocation of resources based on measurements of real end user performance and an accurate view of where additional resources can truly alleviate SLA risks. To make that possible, you need a clear picture of resource consumption at the transaction level and business intelligence about the impact of each infrastructure tier on performance.

6. How can you maintain a real-time up-to-date view of how each service flows through the cloud when VMs are moving around dynamically?

In the cloud more than ever, you need a real-time picture of service dependencies that does not need to be manually updated. The environment is simply too dynamic (e.g. so called “VMotion sickness”) to make it feasible to keep manual models and static infrastructure dependency maps up to date.

7. How can you right-size capacity and prevent over-provisioning that undercuts ROI?

In the cloud, a complete history of all transaction instances, including precise resource utilization metrics and SLAs, is essential for making intelligent decisions about provisioning. And with an accurate picture of resource consumption for each business transaction, cloud owners can plan future capacity requirements (e.g. servers, storage, VMs, databases) in the most cost-efficient manner possible.

We've come a long way since the IBM 3270. Or have we?

By Russell Rothstein

December 6, 2010

In today’s economy with sluggish job creation, there’s much talk about the change in skills required in today’s workforce.  Drill down into the world of IT operations management, and there is an even greater shift happening, related not to the economy, but to cloud computing. The rapid adoption of private cloud architectures is creating ripple effects, not only on the way IT delivers services to its customers, but also on the types of skills IT requires to support these new architectures.

Cloud computing is heralding the most significant shift in IT skill sets since we displaced the armies of punch card operators with the IBM 3270. Cloud is a realization of utility computing, where whereby shared resources, software, and information are provided to computers and other devices on demand. As Gartner says in a recent report, private cloud services “will require a cultural and political change inside of IT to see the role of operations move to being more proactive — requiring predefined policies, service levels and automated actions to take on the runtime environment, as opposed to the manual initiation of scripts or workflows. This requires very different skills over time — a shift away from rote work toward more planning, service analysis and a better understanding of service users in order to continually improve how the service is ultimately delivered.” (Source: Gartner “Key Considerations in the Development of a Private Cloud Architecture”, August 23, 2010).

The key phrase used by Gartner is that IT personnel will require “a better understanding of service users”, which means a better understanding the business which is what’s driving the users to consume those IT services. In essence, cloud will necessitate IT to be more business focused. We have been talking about Business/IT alignment for too long now without sufficient progress; with the emergence of cloud models, this is no longer a choice – either IT upgrades to a business-centric service delivery function, or is ultimately to be replaced by outsourced cloud service providers that can provide utility computing services with greater cost efficiencies. That’s why Business Transaction Management, or BTM, must be at the center of your cloud management capabilities, in order to effectively plan for and manage cloud services from a business perspective.  In an upcoming blog post, we’ll get the opinions from CIOs in the industry to understand their plans to address this rapidly changing environment.

To close up, it’s interesting to understand the new roles in IT that Gartner sees as emerging in order to support the delivery of new private cloud services:

• Cloud service architect (new role): Designs and documents the end-to-end cloud platform
• Portal developer: Develops interfaces that cloud consumers use to requisition services
• Workflow specialist: Defines requirements for instantiating automated processes
• Configuration management specialist: Develops consistent packaging and policy-conflict-free service deployment methods

We trust you are already filling these roles in your IT organization. And while these may not be the best the job in the world, but they most certainly beat a career as a roustabout.

April 28, 2010

End-User Experience Monitoring. Real User Montoring. The key words here are Experience and Real. When you are choosing a solution for monitoring the experience of your customers, partners and employees, it’s critical to keep their perspective in mind.

What’s important to your users – pages or transactions? When you are banking online, and it’s taking a long time to view that suspicious-looking check that you supposedly wrote last week, do you care about the rest of the page? The page probably includes more information about your account activity and enables you to conduct additional transactions like transferring funds and tracking automatic payments. But you may not even know what page you are on. Your only concern is the business transaction that you are currently trying to complete – viewing your cancelled check.

Yet most end-user experience monitors track pages or page objects. Pages are significant to application developers, but they are rarely meaningful to users or the business. Of course we need to track information about pages in order to isolate and resolve problems. But in order to understand the user experience, and to triangulate that with SLAs and business priorities, you need to monitor and measure the performance of business transactions.

Part of the problem with monitoring pages lies in the definition of a page. Technically speaking, a page is a URI, a Unique Resource Identifier. But what the user views as a page may actually be a container displaying a number of URIs.  When we are monitoring pages, it’s important to be able to track and analyze the page as a transaction – the way the user experiences it – as well as in its component parts. Given all of this complexity, it’s also important to be able to identify, classify and group the page components automatically, while leveraging as much meaningful information as possible. Otherwise, you are going to spend a lot of time manually “recreating” the application your users experience from an assortment of objects and content items with unrecognizable names.

According to one industry analyst, “One of the key goals in deploying end-user monitoring solutions is to move from an IT-centric view to actually realizing how IT is contributing to business goals. The value of end-user experience information significantly increases if organizations have capabilities for translating application performance metrics into business metrics such as lost revenue opportunities, conversions, the value of transactions failed, application utilization, disruptions of business processes, etc.” By using a Business Transaction Management (BTM) approach to End-User Experience Monitoring, you can evaluate your users “real experience” and measure its impact on your business.

February 8, 2010

We are delighted for the residents of greater New Orleans who deserve a celebration after their Saints beat the Indianapolis Colts last night in Super Bowl XLIV. The people of New Orleans and the Gulf Coast region, still recovering from Katrina, have been waiting to take home a Vince Lombardi trophy for the first time since the Saints began to play in 1967.

And while it took the Saints forty-three years to achieve their goals of success, back here in the IT industry it has taken about that long to achieve our version of a Super Bowl win – true business-IT alignment.

There is a lot of talk in the industry about the importance of “aligning IT to business”, “managing IT based on business priorities”, and “monitoring IT infrastructure performance from a business perspective”. The concept of aligning IT with the goals of the business is not new – we’ve been talking about it ever since the first time a computer was used for a business application (by a food manufacturer in the UK!) back in 1950.

Yet, the vendor community has not been able to enable CIOs to meet their goals of B/I alignment. A key factor is that application teams, the folks in IT that liaise with the business and are in the best position to align with their LOB counterparts, are still using infrastructure-based tools to monitor and manage their environment. However, transactions are where IT and the business meet, and only with a transaction-based management approach can IT start to manage services from the business perspective. Business transaction management enables application teams to manage their environment, for the first time, from the perspective of business transactions, rather than from infrastructure metrics. (Many of you will recognize these as the still-unrealized goals of BSM – business service management; see a recent Forrester report on how BTM delivers on the promise of BSM.)

BTM answers the following questions to support B/I alignment:

• Resources: How will moving to the cloud help reduce customer churn or improve employee productivity? What is the business impact of a consolidation project? How will adding new servers or VMs change service levels?
• Process: What is the IT cost per business transaction? Is investment in new infrastructure going towards the most business-critical services? Is business impact factored into decisions for change management and release management?
• Language: Does the LOB care about the performance of silo metrics (e.g. JVM, database) or do they care about the performance of a business transaction — for example, what is the time for a submit order transaction, how long the transaction took to verify the credit card, authorize the user, check inventory, etc., and what will the performance look like when transactions increase by 100%?

Without BTM, you lack the transaction-centric information to answer these questions and are left about as helpless as defending against a Peyton Manning pass. And while the Saints achieved success with 340-pound linemen on their side, all you need is to make the move to BTM.

October 12, 2009

The debate going on in Washington and the blogosphere around Net Neutrality is reaching fever pitch. If you haven’t been following the issue (no doubt due to spending too much time following your local gas station on twitter), it boils down to whether ISPs have the right to control what flows over their pipes or whether Internet users have the right to get unencumbered access to anything out on the Net. We’re especially intrigued by the debate going on in the Network World blogs between Johna Till Johnson and the dynamic duo of Sevcik and Wetzel. (Disclaimer: OpTier is a member of the Apdex alliance led by Peter Sevcik.)

Which brings us to our point that we can’t wait until someone buys the assets of Clear Corporate and restores the service. Clear is the company that created the speedy security lanes in the airport for those of us who hate to stand in long lines. Bidding is currently on to buy Clear’s assets from Morgan Stanley who took possession of the company when it went bust. (Note to our CFO: Don’t use money invested by Morgan Stanley in OpTier to make bid for Clear.)

You may be asking what Net Neutrality has to do with Clear. The answer, of course, is prioritization. Prioritization of data flowing over the network. Priority of people in airport security lines (very busy and important businesspeople, mind you). We here at OpTier think a lot about prioritization – prioritization of troubleshooting activities when outages occur, prioritization of infrastructure spending to maintain service levels, prioritization of virtualization deployments, and in general, prioritization of IT management in order to best support the goals of the business.

We believe prioritization is at the heart of the BTM approach and that’s what sets OpTier apart from the traditional APM vendors, even as they start to talk about business transactions in their pitches. While these vendors are cobbling together a group of silo monitoring tools using correlation techniques, we believe that true BTM means complete visibility into how every single business transaction executes from the end-user across all IT components with a complete breakdown of latency, resource consumption and SLA compliance.

Now that’s even worth waiting in line for.