MULE

Monitoring Utilisation of Large Environments

Understanding organization-wide hardware utilization is currently a difficult challenge for industry. Current monitoring tools tend to focus on monitoring critical servers and databases and have not been designed to scale to manage entire IT infrastructures including desktops, laptops, servers, routers etc. where the number of devices can be in the order of tens of thousands. This is an issue for many different domains (e.g. organizations with large IT infrastructures, cloud computing providers, software as a service providers) where an understanding of how computer hardware is being utilized is essential for understanding business costs and future investment requirements. There is currently no automated standard solution for understanding whether current hardware resources are being utilized efficiently such that current resource utilization can be maximized and potential investment in further hardware resources reduced.

This work programme will investigate, develop and deliver a solution to track utilization and efficiency of assets in heterogeneous environments. The proposal will build on current monitoring and analysis technologies (e.g. real time data collection infrastructures, enterprise monitoring technologies, expert analysis systems and profile based analysis) to develop a system for determining how efficiently resources are being used given their business purpose, processing capabilities and energy efficiency.

What is the question that this WP addresses?

Identifying critical system production level issues requires an understanding of how the IT environment is being utilised. This research proposes to develop expert tools, techniques and algorithms to allow the automated capture of systems utilisation data. These tools will contribute to problem determination in critical systems.

The objective is to investigate and develop tools, techniques and algorithms to track utilization of IT assets in heterogonous IT environments. This will analyse the efficiency of IT asset usage and will define metrics to measure efficiency. For accurate and commercially meaningful efficiency metrics, the business purpose of the asset usage will also be considered. A catalogue of business usage profiles will be developed as part of this research. The research will consider all measurable IT equipment to be within the scope of the work programme, for IT resources deployed to address a wide range of different business purposes. Expert tools for systems utilisation and efficiency analysis will be developed such that over/under utilised systems can be identified.

Why is this question significant?

Utilisation of current critical resources is currently difficult to determine as it relies on human expert analysis. However understanding utilisation efficiency of critical IT resources is a key requirement for industry planning and cost analysis. For example overutilization of critical system IT resources can lead to service downtime with catastrophic consequences in some cases.

How will the question be addressed?

Understanding critical system hardware utilization is a difficult challenge.

Large organizations considering whether further capital investment is required in critical hardware resources typically rely on requests from, and judgment of, management and technical staff. There is currently no automated standard solution for understanding whether current hardware resources are being utilized efficiently such that current resource utilization can be maximized and potential investment in further hardware resources reduced. This is particularly problematic for systems that are critical for business operations and where accurate resource planning is required. Similarly, for cloud computing providers there is a need to monitor customer resource usage such that service charges/invoices can be sent to customers. Currently cloud providers use in-house solutions for monitoring this information. Current approaches also do not allow for real time billing information for customers of cloud providers; this represents a major issue for organizations trying to track day to day costs.

Profile-based efficiency analysis will be applied whereby the business use of the system will be considered as part of the efficiency analysis. Today systems used for different purposes are currently compared on a single common scale. This generally implies that the systems that are working the hardest are 'well utilized'. This is often not a fair comparison as a production systems with high CPU consumption may have excess capacity, while a test environment with low CPU consumption may be fully utilised (due to the nature of tests being carried out for example). Thus, to assess utilisation efficiency, business use must also be considered. Business profiles will be developed as part of this work for application as part of efficiency analysis techniques.

An IT utilization infrastructure is also required so that resource usage levels can be monitored in a standard manner. Views of the data collected are required at various levels of abstraction to meet the requirements of different end users (e.g. managers, technical staff), business use cases, and domains (internal infrastructure analysis e.g. cloud computing billing). Such an infrastructure would be a valuable asset and applicable to a wide range of organizations in a large number of domains. The infrastructure would be able to:

collectresource usage information from a wide range of heterogonous devices in a low overhead manner

evaluatethe utilization of an extremely heterogeneous collection of systems to determine how efficiently systems are being used, given their business purpose, processing capabilities and energy efficiency – all measurable IT equipment is in scope, every possible type of use is in scope and every possible business purpose

defineefficiency metrics for utilisation levels such that a measure of resource usage efficiency can be obtained

applyprofile-based usage evaluation such that business use will be considered as part of the utilisation analysis. A range of business-use analysis profiles will be developed for common business use cases.