Posts tagged ‘Business Service Management’
Online Banking, Still Open for Business!
By Jonathan Williams
A recent incident at a customer site illustrates how OpTier BTM can play a crucial role in detecting, isolating and remediating performance issues before business-critical services are severely affected.
At a large UK bank, OpTier BTM is used to monitor the central internet banking application. With 4 million business customers using the bank’s site, OpTier monitors over 40 million transactions every day. During a recent Friday morning, OpTier BTM detected a marked increase in application response times as well as a large number of errors. It was absolutely critical to address the issue right away, because not only was it the peak time of day, it was also the last Friday of the month – payday for many people – and the last work day before a 3-day bank holiday weekend.
As you can see in the graph above, OpTier BTM showed an increase in average service time (the blue line) and errors (black area) after 9:50 am. Because the timing was so critical, the bank decided to switch over to their remote contingency data center. As you can see in the graph, the performance improves after 10:50 when switch was made. Even after the switch, we still see some errors because a public-facing internet application it is constantly hit by incorrect URLs – from end user typos to automated Trojans and hack attempts.
While the failover was taking place, the team used OpTier BTM to isolate the cause of the problem. In the graph below, the OpTier dashboard shows a marked increase in service time for User Identification and Verification database calls from the application server. Since nearly every transaction in the application makes a call to this database – even after the user is logged in – nearly all application functionality was affected by the slowdown.
In the drill-down to an individual transaction instance, we can see that calls to the identification and verification database were taking almost 2:30 minutes to perform.
When we drill down into the topology of another transaction instance, we can see that there is a very large Inter-tier time of 1:41 between Apache and WebSphere, indicating a communication problem. This behavior is usually an indication that the WebSphere resource has been exhausted while waiting for backend availability. This would be a secondary effect of the slowdown of the database service.
With the information provided by OpTier BTM, the bank was quickly able to identify that the source of the problem was in the database, resulting in very fast problem resolution and preventing an all hands call that would have wasted valuable time for all of the silo teams (i.e. not only DBAs but also architects, Java developers, network teams, and representatives from other IT silos). The bank’s DBA quickly pinpointed the source of the problem using OpTier BTM data – one of the nodes in their database cluster had reached its session limit. Without OpTier BTM, even isolating the problem would be like searching for a needle in a haystack.
Thanks to OpTier BTM, the problem was identified, addressed and resolved as efficiently as possible. Customers were able to deposit their pay and – along with the bank’s support teams – enjoy the holiday weekend.
Maintaining PCI DSS Compliance with BTM
By John Aronson
May 12, 2011
Payment Card Industry Data Security Standard (PCI DSS) compliance is an important consideration for thousands of companies and an integral part of their network security strategy. But what about application performance monitoring? You need to make sure that APM, BTM and other monitoring tools are PCI DSS compliant as well.
Recently, a customer approached us about monitoring an application that executes over a million financial transactions per day. The users of the BTM system are application owners and performance specialists that need an end-to-end perspective on all of the business transactions in the application. In this case, the need is especially critical because the system was developed by a different team which is no longer at the company.
The current team needed to be able to manage the performance of business transactions, without exposing sensitive customer information to unauthorized individuals. They were aware of this issue because other monitoring tools in their environment, from respected vendors, also captured protected customer data and did not provide a mechanism for removing it – resulting in a potentially serious compliance issue.
OpTier BTM features a solution for PCI DSS compliance that scrubs transactions of sensitive data. It masks protected personal information, while leaving the rest of the transaction, which is useful for problem solving, intact. This mechanism is very simple to implement and saves the customer from having to implement a more costly solution for maintaining compliance, such as auditing all BTM users for application access and/or moving the database to a tighter security zone.
Below is an example of how the data would appear without the PCI DSS compliance feature. (The credit card number is circled in red and some of the digits have been blurred.)
And here is an example of a similar transaction with the PCI DSS solution installed. The credit card number is masked, along with names, addresses, and other protected information.
After installing the mechanism, our customer has all of the information they need in order to manage performance and ensure SLAs – while maintaining compliance with the payment card industry standard.
Gotta Love Paying Taxes on Time
By Russell Rothstein
March 7, 2011
We’re proud of the fact that OpTier software powers a variety of critical businesses. Every day OpTier BTM ensures that stock trades execute fast, national train lines keep on schedule, billion dollar procurement systems don’t fail, online bill payments are executed properly, mobile phone service plans are provisioned, and insurance claims are processed. And while it’s not as sexy, we also ensure that citizens are able to pay their taxes on time by managing tax return filing systems.
Our customer, a large, national tax authority, is responsible for collecting all online tax submissions each year. Since few of us prepare our tax forms in advance, it comes as no surprise that most of its traffic arrives in one giant peak just before the deadline. In fact, more than 80% of its annual traffic occurs during those 3 weeks, and 10-15% of the traffic occurs during the final 8 hours! Of course the annual peak is extremely stressful for the IT department, and in the past, there have been some painful system failures that resulted in submission delays.
This year, (we’re happy to report,) the annual peak was scaled successfully. Our customer used OpTier BTM to monitor all of the key servers, and during the final day, it was processing nearly 5 million transactions per hour. These transactions are exceptionally complex, with over 200 tiers, and OpTier BTM discovers them all automatically, which is important, since there are changes every year.
During the peak, a customized OpTier BTM dashboard is displayed on a 50” plasma monitor at all times. Around 50 people man the command center 24/7 during the peak, and OpTier BTM is always in focus.
The SOA architecture is developed by a number of different application teams and vendors, so the ability to identify where a problem is occurring and put the resolution into the hands of the correct team is absolutely essential – it saves everybody a lot of finger-pointing and arguing over who’s holding the ball. For example, in the cut-out from the dashboard below, the red block shows a slow-down in the performance of the back-end services. By isolating the problem, OpTier BTM can reduce the time spent on troubleshooting by as much as 90%.
Needless to say, the business impact of any outage is enormous for the authority, the vendors, and the public. So the ability to identify and resolve problems quickly is crucial.
OpTier BTM repeatedly identified significant slow-downs much sooner than other monitors, and proactively identified several different types of incidents. The team also used OpTier BTM to drill down and isolate problems. In more than one case, OpTier BTM was used to halt an all-hands call, and to identify both short-term and long-term solutions.
While OpTier BTM complemented the other monitoring tools in the data center, the team appreciated its business focus and the ability to understand the user impact of IT issues. Where other monitors each showed one piece of the puzzle, OpTier BTM captured the entire picture. To quote one of their operations managers, “We need this stuff! We should be using it to monitor other applications as well. ”
“OpTier, the company that ensures you pay your taxes on time.” As true as it is, we’ll have to mull that one over again as a company slogan…
Concerned about Application Performance in the Cloud? Ask These Questions First
By Russell Rothstein
February 22, 2011
Many companies are developing their strategy for migration of business applications to private and public clouds. During this critical stage, it is vital to ensure that service levels are not impacted by migrating the application from dedicated to shared IT resources. It’s no wonder that according to analyst firm IDC, two of the top three concerns that CIO’s have about private clouds are performance and availability.
We see in the market that enterprises are forming new cloud teams and internal committees, with a diverse set of skills, to plan for an effective organizational cloud strategy. One of their mandates in the organization’s journey to cloud is to plan for how to monitor and manage the performance and behavior of applications after deployment. These organizations undoubtedly have a range of infrastructure monitors in the data center. And most cloud service providers, whether internal or external, will provide services for monitoring cloud resources. Yet these tools typically do not provide an accurate picture of what end users are truly experiencing and how to quickly isolate and fix performance issues in application components located inside and/or outside the cloud.
This blog entry points out a few of the key application performance challenges that you are likely to encounter when pursuing a cloud strategy, so that you can address them proactively. I hope that during my session in the Cloud Performance Summit at CloudConnect (Instrumenting Applications When Access Goes Away on Monday March 7) the esteemed panel will address some of these challenges with a variety of perspectives – it should be informative and thought-provoking!
1. How do you know if an application is ready for the cloud?
Not all applications are ready for “cloud time”, and sometimes one part of an application is cloud ready while other components are not. You need to identify the best components for migration as well as potential problems such as chattiness and latency that are amplified in the cloud.
2. How do you find server-related root causes when performance issues arise?
In fully-dedicated environments, we sometimes use infrastructure metrics and events to diagnose performance issues. But inferring application performance from tier-based statistics becomes challenging – if not impossible – when applications share dynamically allocated resources. In the cloud, you must be able to understand application performance and its correlation with the underlying physical and virtual components.
3. How can you minimize the risk of change to the cloud infrastructure or the application?
In a shared environment, any change to the application, or to the infrastructure, is high risk. Cloud owners, operations staff and application teams must be able to test the impact of change on service delivery – whether that change is in an application before deployment, or in the cloud infrastructure.
4. How do you implement or verify chargeback?
Traditional application performance monitoring (APM) tools do not collect resource utilization per transaction to enable business-aligned costing and chargeback paradigms. For the cloud, you need a solution that monitors consumption for every service across multiple applications and tiers, so you can accurately cost services, decide on appropriate chargeback schemes, and tune applications and infrastructure for better resource utilization and lower cost.
5. How do you ensure that services are allocated according to business priority?
To ensure that SLAs in the cloud are met, you must be able to prioritize the allocation of resources based on measurements of real end user performance and an accurate view of where additional resources can truly alleviate SLA risks. To make that possible, you need a clear picture of resource consumption at the transaction level and business intelligence about the impact of each infrastructure tier on performance.
6. How can you maintain a real-time up-to-date view of how each service flows through the cloud when VMs are moving around dynamically?
In the cloud more than ever, you need a real-time picture of service dependencies that does not need to be manually updated. The environment is simply too dynamic (e.g. so called “VMotion sickness”) to make it feasible to keep manual models and static infrastructure dependency maps up to date.
7. How can you right-size capacity and prevent over-provisioning that undercuts ROI?
In the cloud, a complete history of all transaction instances, including precise resource utilization metrics and SLAs, is essential for making intelligent decisions about provisioning. And with an accurate picture of resource consumption for each business transaction, cloud owners can plan future capacity requirements (e.g. servers, storage, VMs, databases) in the most cost-efficient manner possible.
February 22, 2011 at 3:47 pm Russell Rothstein Leave a comment
Cloud Requires a New IT Employee (Hint: MBA May Be Required)
We've come a long way since the IBM 3270. Or have we?
By Russell Rothstein
December 6, 2010
In today’s economy with sluggish job creation, there’s much talk about the change in skills required in today’s workforce. Drill down into the world of IT operations management, and there is an even greater shift happening, related not to the economy, but to cloud computing. The rapid adoption of private cloud architectures is creating ripple effects, not only on the way IT delivers services to its customers, but also on the types of skills IT requires to support these new architectures.
Cloud computing is heralding the most significant shift in IT skill sets since we displaced the armies of punch card operators with the IBM 3270. Cloud is a realization of utility computing, where whereby shared resources, software, and information are provided to computers and other devices on demand. As Gartner says in a recent report, private cloud services “will require a cultural and political change inside of IT to see the role of operations move to being more proactive — requiring predefined policies, service levels and automated actions to take on the runtime environment, as opposed to the manual initiation of scripts or workflows. This requires very different skills over time — a shift away from rote work toward more planning, service analysis and a better understanding of service users in order to continually improve how the service is ultimately delivered.” (Source: Gartner “Key Considerations in the Development of a Private Cloud Architecture”, August 23, 2010).
The key phrase used by Gartner is that IT personnel will require “a better understanding of service users”, which means a better understanding the business which is what’s driving the users to consume those IT services. In essence, cloud will necessitate IT to be more business focused. We have been talking about Business/IT alignment for too long now without sufficient progress; with the emergence of cloud models, this is no longer a choice – either IT upgrades to a business-centric service delivery function, or is ultimately to be replaced by outsourced cloud service providers that can provide utility computing services with greater cost efficiencies. That’s why Business Transaction Management, or BTM, must be at the center of your cloud management capabilities, in order to effectively plan for and manage cloud services from a business perspective. In an upcoming blog post, we’ll get the opinions from CIOs in the industry to understand their plans to address this rapidly changing environment.
To close up, it’s interesting to understand the new roles in IT that Gartner sees as emerging in order to support the delivery of new private cloud services:
- Cloud service architect (new role): Designs and documents the end-to-end cloud platform
- Portal developer: Develops interfaces that cloud consumers use to requisition services
- Workflow specialist: Defines requirements for instantiating automated processes
- Configuration management specialist: Develops consistent packaging and policy-conflict-free service deployment methods
We trust you are already filling these roles in your IT organization. And while these may not be the best the job in the world, but they most certainly beat a career as a roustabout.
December 6, 2010 at 3:14 pm Russell Rothstein Leave a comment
Building a next generation BTM UI
by Anonymous, 20th April 2010.
It’s the day after we put the finishing touches to a brand new BTM UI and I feel like I’ve gone 12 rounds with Mike Tyson. It’s true what they say though, No Pain No Gain. I’ve been on the ropes, been knocked down several times but like Elton John says “I’m still standing”. A new baby is born and its certainly not an ugly one. I am expecting a few tears though when our field organization and customers gets their hands on it and start to shake it for the first time.
To be precise or wily it’s taken 18 months, 125 pages, 50+ detailed designs, 65 flights (most to Tel Aviv), 50+ customer meetings/webex’s, thousands of emails, hundreds of arguments, thousands of beers (5 beers per argument) and a good few hundred bugs. All for the sake of Business Transaction Management. Has it been worth it? Absolutely.
About 18 months ago OpTier made a decision to build a next generation BTM user interface. It was becoming increasingly clear from customers and prospects that the value of BTM touched many stakeholders in the business and IT. Our previous GUI was more focused towards the technical audience being application support teams and application developers. When you track all transactions across all tiers all of the time and provide response time, SLA and resource consumption metrics by Application, Transaction, User, Location and Tiers you end up collecting a lot of valuable information. The problem is how do you show all this information to the right user in the right context? What is relevant for an application owner isn’t always relevant for an application support team or a capacity planner. Showing a technical view to a business stakeholder is like an English person receiving an Hebrew menu in an Israeli restaurant (not good when you’re hungry). Therefore one of the first decisions we made was to make our BTM GUI role based so we could support a variety of users and use cases. Simply put, a user logs in and is presented with a series of “activities” relevant to their role. Each activity (e.g. problem isolation) has a series of tailored views which allows the user to perform that activity using information which is only relevant to that user. For example, a business user is typically interested in business transaction volume, activity and SLA compliance – they don’t really care about CPU usage, tier chattiness or how many I/O operations happen when a business transaction is executed.
The second key decision we made was to make the GUI extensible from both an architecture and user perspective. It’s obviously good to provide out of the box roles, activities and views for our customers. However, the needs of every customer and user is different no matter how hard you try to support every use case. Therefore users can simply create their own roles, activities and views to make BTM information more relevant to them incorporating their own context, terminology and processes. With 13 dimensions (application, transaction, users, locations, tiers, …), 100+ available metrics, 3 working modes, 10 different visualizations (table, charts, clouds, GoogleMaps, …) and a slick graphic design built on top of Flex technology the possibilities to our users are now endless. In the current beta release we support 10 BTM activities with over 75 different BTM views, if users want something different they can go right ahead and create whatever activity and view they wish. Over the next 18 months we’ll be adding more dimensions, metrics and visualization that will help customers truly manage IT from a business perspective, the future is bright.
The last key decision was to outsource the whole usability and graphic design aspects of our GUI. OpTier specializes in tracking transactions across tiers, it does not specialize in designing user interfaces. BTM is our business, usability is someone else’s business (Puzzlehead in this case). I can tell you right now though usability is a political hot potato. Get 2 people in the same room and you’ll get two different opinions on what is intuitive, get 25 people in the room and you’ll get 25 different opinions. You’d be surprised how many times I heard the words “When I used to develop GUI we did it like this…”. As Einstein once said “Insanity: doing the same thing over and over again and expecting different results”. When you work with a usability company you’re implicitly trusting them to do the right thing based on your inputs and feedback. Unless your open minded with usability experts you’ll always look to the past instead of looking into the future. What was relevant 10 years ago in GUI might not be relevant in 2010. GUI from my perspective is more art than science, it has to capture the imagination and invite the user to play. Larry Ellison once said “When you innovate you’ve got to be prepared for people to tell you you’re crazy.”
Like anything which is brand new though, we’ll have our issues. I’m not naïve enough to think that our new GUI will be perfect from day one, it’s a significant improvement and departure from our existing GUI. We’ll be able to show more value than we could have possibly imagined 18 months ago with our previous GUI. Our users can access BTM value in a single click depending on what they are trying to accomplish. You want to see your application CPU consumption over time? Click on this view. You want to understand the business impact of the release you’ve just released to production? Click on this view. You want to see the individual transactions user X executed? Click on this view.
I see this GUI as the start of something new and exciting rather than the end of just another GUI project. GUI’s should evolve and provide users with something fresh, unique and exciting. Innovation isn’t about standing back and admiring what you’ve delivered in the past, it’s about constantly listening to what your customers want and coming up with killer ideas so you can deliver that competitive edge to your business and your customers business.
Gartner, Forrester, et al: IT Spending and BTM
By Russell Rothstein
April 19, 2010
It’s no secret that the business transaction management (BTM) market has taken off big time. Whereas just a couple of years ago, OpTier was single-handedly pioneering and educating the market about BTM, today just about every application performance management (APM) vendor is pitching some type of transaction profiling capabilities. The marketing barrage and hype are reaching fever pitch, oftentimes confusing enterprises about what truly constitutes BTM. In a market with inflated expectations, a good industry analyst will help end-user organizations separate the wheat from the chafe and identify the vendors that can deliver the goods. (Whereas the less proficient analyst will focus on creating snazzy yet vapid infographics.)
The good analysts aren’t afraid to ask tough questions and dig down deep into the details of technology, strategy and vision to get to their conclusions. So it’s not without a bit of amusement that we find that the major analyst firms are themselves out of sync in their predictions about the size of overall IT budgets in 2010. Gartner predicts that IT spending will remain flat in 2010, Forrester says IT spending will rebound strongly in 2010, and to round things out, Ovum sees that IT spending will grow only slightly.
And while the analyst groups differ regarding 2010 IT budgets, they all agree about one thing – BTM is growing fast and consuming a larger part of the IT software budget. Forrester has reclassified the APM market to include both APM and BTM, reflecting the increasing role that BTM is playing in managing application performance. And Gartner says that BTM is fundamental because it is the very center of application management.
So it’s no surprise that OpTier closed out Q1 with another blow-out quarter, penetrating into new geographies and vertical markets. Go no further than recent tweets by our SVP of worldwide sales replete with CAPS and lots and lots of exclamation points:
Stay tuned for public announcements of new customer wins and exciting news – we promise to keep the punctuation marks to the bare necessity… (!!!)
Transaction Madness
This article will compare the NCAA Men’s Basketball Tournament with BSM challenges that are met by Business Transaction Management (BTM).
Continue Reading March 22, 2010 at 6:33 pm Andy Wetzel 2 comments
The Oscars of Private Cloud Management
By Russell Rothstein
March 7, 2010
It was fun to watch the Academy Awards this year – a glitzy evening replete with glamorous movie stars, stylish attire, and most importantly to those of us from the software industry, totally cool computer-generated characters. (Unless you preferred to spend the evening watching your favorite Super Bowl commercial for the umpteenth time or the latest OK Go video instead.) It’s a night where aspiring actors and screenwriters silently wish that next year they’ll get that break to make it to the Oscars, and we wish we had taken that extra course in computer graphics in college, observing the box-office proceeds from Avatar and Alice in Wonderland.
And while OpTier won’t be releasing a 3-D feature film any time soon, we do believe a multi-dimensional approach is critical to that new genre of enterprise IT service management – managing private clouds. Private clouds are becoming more pervasive – according to Gartner, by 2012 enterprises will spend more than half of their cloud dollars on private cloud services because of improvements in cost and management efficiency.
OpTier BTM provides performance management, resource management, and cost-based accounting for private clouds
That’s where the 3-D approach comes in. Some enterprises erroneously assume that since virtualization is the enabling technology for clouds, then virtualization-centric management systems are sufficient for managing cloud-based services. In effect, they are extending the flawed silo-based APM approach to IT management by applying another one-dimensional toolset to manage virtual hosts and guests in the cloud. (It’s no surprise that in these cases, when a performance issue occurs in the cloud – everyone first blames the “VM guy”.)
OpTier takes a multi-dimensional approach to the cloud. In order to provide true end-to-end service management in the cloud, you need to include visibility into both virtual and physical metrics. Only with a business transactional-based approach do you have 3-D visibility to cover the three key dimensions of transactions that flow through the cloud:
- Time per transaction (i.e. cloud performance management)
- Resource utilization per transaction (i.e. cloud capacity management)
- Cost per transaction (i.e. chargeback and activity-based costing for the cloud)
Frost and Sullivan found that two of the top three concerns about cloud are loss of control and availability. OpTier’s business transaction management approach to private clouds is the most efficient manner to address these concerns – enabling organizations to take control and assure 24/7 availability of services in the cloud.
So the OpTier 3-D feature film may be off in the future, but for some time already, OpTier customers have been using OpTier BTM’s 3-dimensional approach to manage services in the cloud on a daily basis. Now that’s worth an Oscar speech.
Virtualized in Vancouver
By Russell Rothstein
February 22, 2010
We’re fascinated by what’s going on in Vancouver this month. Sure, there’s a lot of drama, action and suspense going on over at the Winter Olympics (such as the valiant efforts of the Jamaican Freestyle Skier to be the first to restore the pride of winter sports to his homeland since the 1988 Jamaican Bobsled team) but we’re even more intrigued by the less mesomorphic young singles cross-town in Vancouver engaging their social skills in a flittering party – flirting virtually with one another through the twitter platform.
For those of you not previously familiar with flittering, it means sending a flirtatious tweet to someone that interests you in the room. And while having the option of flirting online is generally a good thing as it provides a new way for people to find their match, it presents a management challenge – keeping up with the fast pace of the tweets and achieving the visibility to link the virtual (witty tweets) with the physical (cute guy or gal – hopefully).
We find these issues remarkably similar to the management challenges companies face when deploying virtualization. While virtualization presents huge benefits in efficiency gains, it also creates three application management challenges:
- Lost Visibility: Applications become more difficult to manage because virtualization masks the underlying infrastructure layers. It becomes difficult to isolate the problematic tier when a problem occurs, or even understand that a problem is starting to occur before users are impacted. Likewise, it is difficult to determine the impact of VM changes on performance.
- Dynamic Environment: If becomes difficult to keep up with speed of changes in virtual infrastructure. As the folks at EMC call it – “VMotion Sickness”. Then when application performance begins to degrade, people tend to first blame the VM administrator, even if the problem lies elsewhere.
- Overprovisioning: Due to lack of visibility, enterprises often overprovision infrastructure in order to assure performance. Overprovisioning physical hosts (e.g. extra CPU) reduces the cost savings of virtualization.
Virtualization-aware Business Transaction Management addresses these challenges. With OpTier BTM you can regain the visibility lost from virtualization, keep up with changing VMs, and rightsize to maximize cost efficiencies. And while that last sentence would fit into a 140-character tweet, we’re not sure it would get us any dates at the next flittering party.
February 22, 2010 at 2:14 pm Russell Rothstein Leave a comment
OpTier
Assaf Amit
Andy Wetzel
Russell Rothstein
Motti Tal
Linh C. Ho
Diego Lomanto
