December 9, 2008

    Statistical methods (average, mean, standard deviation, etc.) are the bread and butter of performance analysts and capacity planners. Not a day goes by without having to study a graph or a set of measurement data. According to Ray Wicks of IBM, we continuously process such data and draw conclusions about the meaning. The processing is often numeric and there is both a conceptual and sensual component at work here.

    The conceptual component is what we have been taught about how numbers work, such as the average is equal to the sum divided by the number of observations.

    The sensual component is the result of the evolution of our visual cortex. For example, we perceive circular-shaped objects with the shadow on the underside as raised bumps, not dimples, because our visual system is used to light coming from the sky and projecting the shadow on the underside of objects.

    Not surprisingly, the conceptual and visual components influence each other so that what we think and see are not independent. In other words, we can influence the conclusion of the data by how we present the data. Consider these two separate graphs:

    TeamQuest image


    Our visual system tells us that the two are different. We reflexively draw the conclusion that there is not much variability in the data in the first graph compared to the second. But conceptually, and upon closer inspection, we realize that we are looking at the same data at different scales. (BTW, both TeamQuest IT Service Analyzer and Reporter let you control the scaling of your graphs.)

    We can be tricked by statistics without the help of our visual system. Consider the fact that the average age of orchestra conductors is 73 compared to 68.5 for the rest of us. Are those guys healthier? Perhaps living a major part of your life waving your hands in the air is good for your health. The pitfall here is that the average is based on the population of orchestra conductors, and that population consists mostly of white, healthy males above the age of 65. The other population contains people of all ages, all walks of life, men and women.

    Thus if you are an orchestra conductor and make it beyond 65 years of age, chances are that you'll make it to 73, but not thanks to all those hours you've spent waving your hands in the air. The two averages are based on two very different populations, and thus not comparable.

    At TeamQuest we have long understood the treachery of simple averages. That's why we use weighted averages, for the appropriate data, when aggregating the data in our performance database.


    Category: cmg-2008