Case study: One medical center's experience with IT thresholds

Wherever you go, you can see technology's impact on our society. Household bills are paid online, satellites can locate your home from space, and cell phones have the ability to send and receive e-mail. Another area of society seeing a surge in technological advancement is the healthcare sector. From applying transactional data management for disaster recovery to implementing real-time analysis software to improving systems management, hospitals are adopting technology to ensure that their most important information-patient data is accessible and secure.

Opened in 1902, Cedars-Sinai Medical Center is a nonprofit major academic medical center that includes 1,800 affiliated physicians, 8,900 employees and 2,000 volunteers. Cedars-Sinai has a complex IT environment that is not immune to performance challenges, including how to accurately manage the performance of our messaging systems because of miscalculated performance thresholds.

A threshold defines acceptable tolerance limits for an application resource or system resource. The value of the resource is then measured, and when the value is greater than or less than the maximum value that is set, an exception is raised and IT is notified. Thresholds are useful for monitoring and managing the performance of a system. Not only were we at Cedars-Sinai struggling to correctly calculate and set performance management thresholds, but the time it took to filter through and research false alerts was taking away from our more important IT tasks.

Several of our administrators were tapped with the task of qualifying the alerts coming in around the clock. After combing through the information and finding relevant data, they then had to determine how to respond to what might in the end, turn out to be a false alert. Although performance management tools provide alerts when enterprise systems exceed established performance thresholds, many of them rely on manually set static thresholds which can inundate an IT staff with hundreds or thousands of false alerts every single day.

To meet this challenge, we decided to shift from traditional performance "management" solutions to real-time analysis software to ensure accurate insight into performance and security problems.

Our messaging department turned to NetIQ, a leading provider of integrated systems and security management solutions. NetIQ then introduced us to their partner Netuitive and its real-time analysis solution. With Netuitive we gained the ability to self-learn a system, network, application or security performance. This real-time capacity facilitates quick and time-appropriate decision making and forecasting of future problems, enabling our messaging department to take corrective measures before problems occur.

We quickly discovered that our environment was not a typical messaging environment when it came to performance monitoring. Because of the complexity of our system, typical thresholds did not cut it. The real-time solution automates the threshold administration process and in turn decreases false alerts. Our IT department can now better monitor the performance of our messaging systems and when there is a real problem, we can quickly diagnose them using real-time analysis.

Once Cedars-Sinai began automating the threshold administration process, there was a significant decrease in false alerts. The technology integrated seamlessly into our existing performance management solution and deployed within hours. We have successfully automated our threshold administration, dramatically speeding problem diagnosis and resolution, while reducing false alerts.

James Brady is the senior messaging architect and engineer at Cedars-Sinai Medical Center in Los Angeles.