Chapter 3 of 8

Visual Exploration and Descriptive Summaries

Learning objectives

  • Choose summary statistics that match the shape of a distribution rather than defaulting to the mean
  • Apply the quartile fence rule to identify outliers and decide what to do with them
  • Diagnose the common ways a chart misleads, including truncated axes and hidden aggregation

Look at the distribution before you summarise it

A single number summarising a column is a compression, and every compression discards something. The question is whether it discarded anything the decider needed. Business data is rarely symmetric. Revenue per customer, order size, time to resolution, and deal value are almost always right-skewed, because there is a floor at zero and no ceiling. A long upper tail can pull the mean above the median. Compare both with a plot of the observations; the size of the gap has no universal cutoff for deciding whether concentration matters. For skewed data the median answers the question a manager usually means when they say typical, while the mean answers the question that matters for budgeting, because the mean times the count is the total. Both are legitimate; using one while implying the other is not. Beyond centre, report spread and shape. The interquartile range, the range from the twenty-fifth to the seventy-fifth percentile, is robust and interpretable: half your observations fall inside it. Report a measure of spread alongside the mean so the reader can see how much observations vary.

Outliers, fences, and the decision about what to do next

The conventional quartile fence rule flags an observation as an outlier if it lies more than one and a half interquartile ranges below the first quartile or above the third. The multiplier is a convention, not a law of nature; it is calibrated so that for roughly bell-shaped data only a small fraction of honest observations get flagged. For heavily skewed business data the rule will flag many legitimate points, which is why the fence is a trigger for investigation rather than a licence to delete. Once a point is flagged, classify it. Data errors, such as a quantity of one million units caused by a unit-of-measure mix-up, should be corrected or removed with the removal documented. Genuine but out-of-scope observations, such as a single enterprise contract inside a small-business analysis, should be excluded with the scope restriction stated in the conclusion. Genuine and in-scope extremes are the most important case, because they usually carry the business signal: in many firms a handful of accounts drive most of the profit. Deleting them to make a chart look tidier deletes the finding. Whichever you choose, report the result with and without the point so the reader can see how much rests on it.

How charts mislead, mostly without anyone intending it

Three failure modes cover most misleading business charts. The first is the truncated axis: starting a bar chart's value axis above zero exaggerates differences, because the eye compares bar areas and the areas no longer stand in the ratio of the values. Bars encode magnitude and must start at zero; lines encode change over time and may be truncated, provided the axis is clearly labelled. The second is hidden aggregation. Any average over groups can reverse direction when the groups are separated, because group sizes weight the totals. If a company-wide conversion rate rises while the rate in every single channel falls, the mix shifted toward a high-converting channel, and the honest chart shows the channels separately with their volumes. Always plot at the level where the decision will be taken. The third is the wrong encoding for the question. Comparing categories calls for a bar chart sorted by value; showing a trend calls for a line; showing a distribution calls for a histogram or box plot; showing a relationship between two continuous variables calls for a scatter plot. Pie charts fail at their nominal job, comparing slice sizes, because angle is a weak visual channel; a sorted bar chart does it better in almost every case.

Worked example

Problem

A hypothetical retail chain has twelve stores with the following weekly revenues in thousands of dollars: 18, 21, 22, 24, 25, 25, 27, 28, 30, 33, 36, 91. Compute the mean, median, quartiles, interquartile range, and outlier fences. Determine which observations are flagged, then recompute the mean and median with the flagged store excluded and state which summary a regional manager should be given.

Step by step

  1. The values are already sorted. n = 12. Sum = 18 + 21 + 22 + 24 + 25 + 25 + 27 + 28 + 30 + 33 + 36 + 91. Running total: 39, 61, 85, 110, 135, 162, 190, 220, 253, 289, 380.
  2. Mean = 380 / 12 = 31.67 thousand dollars per week.
  3. Median with n = 12 is the average of the 6th and 7th values = (25 + 27) / 2 = 26.00 thousand dollars per week.
  4. First quartile as the median of the lower six values (18, 21, 22, 24, 25, 25) = (22 + 24) / 2 = 23.00.
  5. Third quartile as the median of the upper six values (27, 28, 30, 33, 36, 91) = (30 + 33) / 2 = 31.50.
  6. Interquartile range = 31.50 - 23.00 = 8.50 thousand dollars.
  7. Upper fence = Q3 + 1.5 x IQR = 31.50 + 1.5 x 8.50 = 31.50 + 12.75 = 44.25. Lower fence = Q1 - 1.5 x IQR = 23.00 - 12.75 = 10.25.
  8. Flagged: 91 exceeds 44.25, so it is an outlier by the rule. The value 36 is below 44.25 and is not flagged. No value falls below 10.25.
  9. Excluding the 91: sum = 380 - 91 = 289 over n = 11. Mean = 289 / 11 = 26.27 thousand dollars per week.
  10. Median of the remaining 11 values is the 6th value in order (18, 21, 22, 24, 25, 25, 27, 28, 30, 33, 36), which is 25.00 thousand dollars per week.
  11. Compare the movement. Excluding one of twelve stores moved the mean by 31.67 - 26.27 = 5.40, or 17 percent, but moved the median by only 26.00 - 25.00 = 1.00, or 4 percent.

Answer. Mean 31.67, median 26.00, Q1 23.00, Q3 31.50, IQR 8.50, fences 10.25 and 44.25, all in thousands of dollars of weekly revenue. Only the 91 store is flagged. Excluding it, the mean falls to 26.27 and the median to 25.00. The regional manager should be given the median of 26.00 as the typical store, together with the note that one flagship store at 91 accounts for 91 / 380 = 24 percent of chain revenue on its own. The mean of 31.67 is not wrong and is the right figure for forecasting chain totals, since 31.67 x 12 = 380, but it describes no actual store: nine of the twelve fall below it and only the 33, 36, and 91 stores sit above. The flagged store should not be deleted, because a quarter of the revenue is the finding, not the noise.

Practice

Work each question before opening the solution.

  1. Support ticket resolution times in hours are 1, 1, 2, 2, 3, 3, 4, 6, 9, 49. Compute the mean and median and state which one belongs in a customer-facing service commitment, with a reason.

    Show solution for question 1

    Sum = 1 + 1 + 2 + 2 + 3 + 3 + 4 + 6 + 9 + 49 = 80, so the mean is 80 / 10 = 8.0 hours. With n = 10 the median is the average of the 5th and 6th values = (3 + 3) / 2 = 3.0 hours. Neither belongs in a commitment on its own. A commitment should be stated as a percentile, for example 90 percent of tickets resolved within 9 hours, because customers experience their own wait, not the average, and a mean of 8.0 hours driven by one 49-hour case would be both pessimistic for most customers and silent about the worst case.

  2. A bar chart shows regional sales of 96, 98, and 104 units with the value axis starting at 95. Describe what a reader takes away and what the same data looks like on a zero-based axis.

    Show solution for question 2

    With the axis starting at 95 the bars have visible heights of 1, 3, and 9, so the third region appears roughly nine times the first, suggesting a dramatic gap. On a zero-based axis the bars are 96, 98, and 104, which are visually near-identical, correctly showing a spread of about 8 percent between best and worst. Bars encode magnitude by area, so truncating their axis breaks the encoding; if the small differences are the point, use a dot plot or plot the differences from the mean directly and label the axis as such.

  3. Company-wide trial-to-paid conversion rose from 18 percent to 19 percent, but conversion fell in each of the three acquisition channels. Explain how both can be true and state what to plot instead.

    Show solution for question 3

    The channel mix shifted toward the channel with the highest conversion rate. The company-wide figure is a volume-weighted average of the channel rates, so a change in weights can move it opposite to every component. Plot the three channel conversion rates as separate lines over time, with a stacked area or bar chart of channel volumes underneath, so the mix shift and the within-channel deterioration are both visible. Reporting only the blended number would hide a real decline in every channel.