Xharru Ltd.

AKMA Labs

Z1 Monitor Experimental
Advanced Performance Monitoring utility

Z1 Monitor is Experimental Advanced Performance Monitoring utility with ability to model complex interdependencies between multiple monitored variables and monitoring result (combined meaning of all those variables that you monitor). It is experimental because some of the concepts used could becontroversial and could potentially be hard for user to understand and/or use in practice in their current implementation.

How it works

As part of this Experimental release we would like to get answers for following questions:

1. Do you find it understandable how to map multiple input variable to monitoring results using methods used in this application?

2. Do you feel that you need ability to have more then single conditions on the left side of the rules, for example not just IF (A) THEN (B) but IF (A) AND (C) AND (E) THEN B etc.?

3. Do you think that if this application would have ability to learn all these rules and give you a chance to adjust them later it would be very useful or more useful then in current form?

Now lets explain briefly what this utility does by looking at simple example.

Say you would like to have single monitoring result to mean something like System Load about performance and load of your system possibly running some critical application. In this case we could achieve this by monitoring following performance counter (number of monitored parameters is limited to 6 for simplicity of example):

• CPU Utilization;
• System Queue Length;
• Available Memory;
• Pages/sec;
• Current Disk Queue Length;
• Avg. Disk sec/Transfer;

These variables represent main elements of system performance (CPU Load, Memory Subsystem Load and I/O Subsystem Load).

Now imagine that in your head you could say something like this about results of you monitoring of all these variables:

and second rule

These 2 (or could be little bit more) generic rules do really sound like the way most/many people would think about monitoring and interpreting meaning of what they get by looking at monitoring results in real time. Unfortunately with all available commercial or non-commercial monitoring tools they do not have a way to do it in the way we think in our head (like rules above). Usually user would be presented with many thresholds for every monitored parameter, time etc., which they must configure and which are hard to combine among themselves to have results as combination ofobservations of many variables exceeding thresholds etc.. Also these tools would have hard time to deal with time as part of observation (for how long certain value is above some threshold and why this long and not that long etc.).

If you find that monitoring must be much simpler then what is currently offered and think that two rules shown above (show in bold in red) is more logical and intuitive way to think about monitoring problem, then you should try using Z1 Monitor, which is fully functional experimental version at this time. Even so it is experimental version it can do most of what you would need to do to perform monitoring of real world applications and systems in the way expressed by two rules above. It is experimental because more functionality is coming to this tool in stages please visit our web site periodically.

In order to better understand how to use this tool please read Step by Step Example below.

Step by Step example

To illustrate all the steps you need to perform in order to do monitoring using this utility we will take, as an example, simplified problem of monitoring CPU Bottleneck conditions on the computer.

Before we go to actual steps to perform in Z1 Monitor software lets perform mental steps in our heads that we would take without such tool (steps later performed in Z1 Monitor would try to mimic as close as possible our mental steps, so it is important to try to formalize mental steps). Most of these steps are done in our heads almost instantaneously and not as well defined, but steps must be well defined for computer program to execute..

First, we must formulate what we would like to do:

We would like to monitor multiple system performance parameters related to identification of CPU Bottleneck and based on all the parameters monitored to tell what level of CPU Bottleneck is at current time on monitored system.

Second, we need to decide on performance counters that we must monitor(using Performance Monitor) to help us to determine level of CPU Bottleneck on monitored system. For purposes of this exercise we will limit number of monitored parameters to just these two:

"Processor\%Processor Time" – amount of time CPU spends doing some work, this counter is most obvious candidate. But it is clear that it is not enough to have just this one, because system could have high CPU utilization with single application utilizing CPU time and no other application having to wait for time to execute.

System\Processor Queue Length – is number of threads waiting in the ready queue. Number substantially higher then 2 would indicate that we have threads that have to wait in order to get execution time.

Now we need to define intervals in CPU Bottleneck monitoring result, say we would like to have 3 of them as follows:

Low CPU Bottleneck – this would mean that there is no CPU bottleneck.

Medium CPU Bottleneck – this would mean that there is some CPU Bottleneck.

High CPU Bottleneck – which would mean that there is high level of CPU Bottleneck and we must try to investigate reasons for this and take some actions.

Next we would define intervals for each monitored parameter (not necessary same number of them and ranges as in monitored result). For CPU Utilization counter: Low CPU Utilization, Medium CPU Utilization and High CPU Utilization.
And for Processor Queue Length: Low Processor Queue Length, Medium Queue Length and High Queue Length.
Please keep in mind that there could be more intervals then those defined, for example you could define interval as Very High CPU Utilization and Very Low CPU Utilization if you need them for your monitoring.

Also since we would be making our conclusion not just by looking at single observation taken say every second, but instead on number of observations taken overperiod of time (say over 30 seconds), we would need to define Frequency with which monitored variables take value that fall in one or the other interval.
Good names with corresponding meaning for these intervals are: Mostly – which would mean that more then say 50% of observations take values in particular interval, Frequently and Rarely.
So let's define following terms in order to reason about time, call it Frequency: Rarely, Frequently and Mostly. We will use them to be able to say things like: High CPU Utilization Most of the Time.

Now with all the terms defined we could build monitoring rules. Please keep in mind that we reason about each monitored variable and its relation to result as if there are no other variables to help to make our decision about level of result(Comb's method). So you need to take monitored variable one at the time and try to build rules as you don't have any other monitored variables. Later Z1 Monitor will combine results of all the rules to produce final result. So for our example we could produce following rules:

IF CPU Utilization is Low Most ofthe Time THEN CPU Bottleneck is Low;

IF CPU Utilization is Medium Mostof the Time THEN CPU Bottleneck is Medium;

IF CPU Utilization is High Most ofthe Time THEN CPU Bottleneck is High;

IF Processor Queue is Low Most ofthe Time THEN CPU Bottleneck is Low;

IF Processor Queue is Medium Mostof the Time THEN CPU Bottleneck is Medium;

IF Processor Queue is High Most ofthe Time THEN CPU Bottleneck is High;

Please keep in mind that in our example we only take in account observation of input values that they take most of the time and ignore others (mainly for simplicity of example).

After this we would begin collecting performance counters and apply our rules in order to determine current level of CPU Bottleneck. So as result of execution we would periodically say that CPU Bottleneck is High, Low or Medium.

This is what we would do in our heads (intuitively) if we use just use any performance collecting tools and run it and then think about observations that we see over time.

As we have indicated in the beginning, steps we would take in our Z1 Monitor application are very close to explained above. Lets see now all the step we need to do in Z1 Monitor utility to do same thing.

Step 1: On the page called "Counters" add "Processor\% Processor Time" and "System\Processor Queue Length" counters one at a time. For "Processor\% Processor Time" specify range from 0 to 100, since CPU utilization can take any value from 0 to 100%. For "System\Processor Queue Length" specify range value from 0 to 12. It is possible that queue can even get larger then 12 but it would clearly indicate totally "dead" system and we only focus on relevant value range.



click on image to enlarge

Step 2: On the page called "Frequency" click to add new frequency variable. For our case remove 2 intervals called "Rarely" and "Frequently" and keep interval called "Mostly" (for simplicity of example).



click on image to enlarge

Step 3: On the page called "Results" add new result variable and call it "CPU Bottleneck". As of this time all the result variables are defined from 0 to 100 to be interpreted as %, so we could say CPU Bottleneck of 90%. Keep all defaults values for intervals (you can change it for yourself but keep them for this example).



click on image to enlarge

Step 4: On "Dependencies" page select "CPU Bottleneck" result and add dependencies to it: "% Processor Time" and "Processor Queue Length".



click on image to enlarge

Step 5: On the "Rules" page add following rules:

IF %Processor Time is Mostly High THEN CPU Bottleneck is High;

IF %Processor Time is Mostly Med THEN CPU Bottleneck is Med;

IF %Processor Time is Mostly Low THEN CPU Bottleneck is Low;

IF Processor Queue Length is Mostly High THEN CPU Bottleneck is High;

IF Processor Queue Length is Mostly Med THEN CPU Bottleneck is Med;

IF Processor Queue Length is Mostly Low THEN CPU Bottleneck is Low;

You can also assign confidence to difference rules, but for now use default confidence of 1.0 for every rule, later when you get more experience you can experiment with rule confidences.



click on image to enlarge

Step 6: Go to page "Monitor" and click Start button, Z1 Monitor will start collecting your data and after about 15 seconds will present you with result of what the current CPU Bottleneck level is given current rules that you have provided. With number of application demanding high CPU running in background onour system results are shown on image below.



click on image to enlarge

Good Luck using Z1 Monitor!!!

Download the latest vesrion of Z1 Monitor