Practice Test

Why it is hard

The method is standard, but the key rests on reading the scale between its labels. The axis is ticked only every 4 degrees, and the minimum and the upper quartile land off the ticks, so rounding either one to the nearest label moves the result by a degree or two with nothing to flag it. Students who find the range correctly can also stop there, entering 45, the larger of the two spreads, instead of the difference the question asks for.

1. Reading a figure, table or graph major Understanding the question

   Two of the five values fall between ticks spaced 4 degrees apart: the left whisker ends just short of the 40 tick, at 39, and the box’s right edge sits midway between 76 and 80, at 78. (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

2. Many steps minor Working it out

   Four values are read and combined in three subtractions, the range, the interquartile range and then their difference, so one misread value changes the final entry. (Daniel & Embretson, 2010; Enright, Morley & Sheehan, 2002)

3. Something you must already know minor Working it out

   Neither measure is defined on the page: the range spans the whiskers, and the interquartile range is the width of the box, the upper quartile minus the lower. (Anderson & Krathwohl, 2001)

4. An answer in an unexpected form minor Choosing the answer

   The result is typed with no choices to compare, so an entry one or two degrees off, from rounding a value to the nearest tick, looks just as finished as the right one. (Enright, Morley & Sheehan, 2002)

Strategy: Read the minimum, both quartiles and the maximum, judging where each mark falls between the ticks, then subtract for the range and for the IQR, and subtract those two results.

Traps

• Misreads the figure: Reads a value, label or tick spacing in a figure, table or graph wrongly.
• Stops one step early: A choice is a number from partway through the solution.

The wrong answers, classified

• 14 Misreads the figure (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

  Reveals a slip Expected pull: strong

  Reads the left whisker’s end as 40, the nearest labeled tick, instead of 39, so the range becomes 84 − 40 = 44 and 44 − 30 = 14.

• 45 Stops one step early (King et al., 2004; Gierl et al., 2017)

  Reveals partial knowledge Expected pull: moderate

  Finds the range, 84 − 39 = 45 degrees, and enters it, stopping before the interquartile range of 30 is subtracted from it.

• 17 Misreads the figure (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

  Reveals a slip Expected pull: weak

  Reads the box’s right edge as 76, the tick just left of it, instead of 78, so the IQR becomes 76 − 48 = 28 and 45 − 28 = 17.

How the questions are analyzed, and the research behind it

Why it is hard

The plot is drawn on whole-hour ticks, and the three sentences lay out the steps, so this is a direct check of two definitions. The main risk is taking a value off the plot, such as the lower quartile of 8, for a spread between two values. Because the four tiles allow only one use each, a student who confuses the measures can still fill every box, which hides the error.

1. Something you must already know minor Working it out

   The sentences name the range and the IQR but never define them; students must know the range spans the whiskers and the IQR spans only the box. (Anderson & Krathwohl, 2001)

2. Many steps minor Working it out

   Three boxes must all be right, the third depends on the first two, and each tile can be used once, so one mix-up spreads to a second box. (Daniel & Embretson, 2010; Enright, Morley & Sheehan, 2002)

Strategy: Read the five key values off the plot, subtract the least from the greatest for the range, subtract the lower quartile from the upper for the IQR, then subtract those results.

Traps

• Misunderstands the concept: A wrong belief about a concept: that percents can be added, that area grows like length, that averages can be averaged, that a bigger denominator means a bigger fraction.
• Wrong formula: Uses a related but wrong formula: perimeter for area, diameter for radius, area for volume, or a misremembered one.

The wrong answers, classified

• DCB Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: moderate

  Takes the lower quartile, 8, as the interquartile range, reading a value off the box instead of its width, so the difference becomes 15 − 8 = 7.

• BDC Wrong formula (King et al., 2004)

  Reveals a misconception Expected pull: weak

  Swaps the two measures, giving the box’s width, 7, as the range and the whiskers’ spread, 15, as the IQR, while the difference stays 8.

What was corrected from the DOE handbook, and why

• Explanation: The DOE’s explanation runs two values together: “Maximum value = 20 hr Minimum value = 5 hr”. Its next line is “Upper Quartile Value = 15 hr, Lower Quartile Value = 8 hr”, and Question 32’s explanation has “Maximum value = 84° F, Minimum value = 39° F”. The site adds the comma.

How the questions are analyzed, and the research behind it

Why it is hard

No calculation here is long; the challenge is knowing what a box plot does and does not show. Two of the wrong statements sound reasonable because they borrow real features of the plot, its long left whisker and its middle line, and give them the wrong meaning. Remembering that the plot splits the days into four groups of about equal size makes the true statement easy to confirm.

1. Something you must already know moderate Working it out

   Each statement draws on a box plot fact the test does not supply: what the middle line marks, how the IQR is found, and when a value counts as an outlier. (Anderson & Krathwohl, 2001)

2. Every choice must be checked moderate Choosing the answer

   The four statements test four separate ideas, an outlier, the mean, the IQR and a quartile's share, so ruling out one says nothing about the next. (Embretson & Wetzel, 1987)

3. Reading a figure, table or graph minor Understanding the question

   The scale is labeled every 2 fares with the numbers crowded together, so the five values 2, 12, 16, 24 and 32 must be read off with care. (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

Strategy: Read the minimum, quartiles, median and maximum off the plot, then test each statement in turn, computing the IQR and the outlier cutoffs rather than judging by eye.

Traps

• Misunderstands the concept: A wrong belief about a concept: that percents can be added, that area grows like length, that averages can be averaged, that a bigger denominator means a bigger fraction.
• Wrong formula: Uses a related but wrong formula: perimeter for area, diameter for radius, area for volume, or a misremembered one.

The wrong answers, classified

• A Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: moderate

  Takes the long left whisker as a sign that 2 is an outlier, but that whisker holds a quarter of the days, and 2 is above the cutoff 12 − 1.5 × 12 = −6.

• B Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: strong

  Takes the median line at 16 for the mean, though a box plot shows the median and quartiles and cannot show the mean.

• C Wrong formula (King et al., 2004)

  Reveals a misconception Expected pull: moderate

  Subtracts the minimum from the maximum, 32 − 2 = 30, which is the range, not the interquartile range of 24 − 12 = 12.

How the questions are analyzed, and the research behind it

Why it is hard

This is a question about evidence: the right statement is the only one the box plots guarantee, while the others may or may not be true. Students who treat the plots as pictures of every member, assuming ages fill each whisker or spread evenly through each box, find support for claims that cannot be checked. Asking what must hold for any 30 ages with these summaries separates the verifiable claim from the merely plausible ones.

1. Every choice must be checked major Choosing the answer

   Three of the statements are not false but unverifiable, so each must be tested against what the five-number summaries guarantee, and none can be dismissed at a glance. (Embretson & Wetzel, 1987)

2. Something you must already know moderate Working it out

   Students must know that the minimum and maximum are actual ages, that each quarter of a plot holds about a quarter of the 30 members, and that a whisker shows nothing in between. (Anderson & Krathwohl, 2001)

3. Reading a figure, table or graph moderate Understanding the question

   The scale is labeled only every 3 years, so values such as 17, 19, 26 and 29 have to be read from unlabeled ticks on two separate plots. (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

Strategy: List each group's minimum, first quartile, median, third quartile and maximum, then test every statement against only what those five values guarantee about the 30 ages.

Traps

• Misunderstands the concept: A wrong belief about a concept: that percents can be added, that area grows like length, that averages can be averaged, that a bigger denominator means a bigger fraction.
• Numbers taken from the question: Combines the numbers in the question without a model of the situation, or is simply a number that appears in the question.

The wrong answers, classified

• A Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: moderate

  Sees that Group Q's maximum is exactly 29 and that 29 also lies in Group P's upper whisker, assuming someone in Group P is that age.

• C Numbers taken from the question (Hegarty, Mayer & Monk, 1995)

  Reveals a surface strategy Expected pull: weak

  Matches the two medians, 21 and 24, and assumes the stretch between them holds the same number of members in both groups.

• D Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: strong

  Knows Group P's box from 19 to 27 holds half its members, then assumes ages spread evenly so Group Q has fewer there, which its plot cannot show.

How the questions are analyzed, and the research behind it

Why it is hard

Reading the five values off the plot is quick; the work is knowing what a box plot can and cannot say. It shows where quarters of the data begin and end, not how many students share any one age, and a long whisker is not an outlier unless it passes the 1.5 × IQR cutoff, here 13. The true statement uses the third quartile: about a quarter of the students are 10 or older.

1. Something you must already know moderate Working it out

   The statements rely on box plot facts the question does not give: each section holds about a quarter of the data, the middle line is the median, and outliers lie beyond 1.5 IQRs. (Anderson & Krathwohl, 2001)

2. Every choice must be checked moderate Choosing the answer

   Four statements test four separate ideas, an outlier, the share at one age, a quarter of the data and two spreads, so ruling one out says nothing about the rest. (Embretson & Wetzel, 1987)

3. Reading a figure, table or graph minor Understanding the question

   The five values, 7, 8, 9, 10 and 12, are read off a 5-to-13 scale, where the longer right whisker draws the eye more than the box does. (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

Strategy: Read the five key values off the plot, then test each statement: compute the range, the interquartile range and the outlier cutoffs, and recall what share of the data each part holds.

Traps

• Misunderstands the concept: A wrong belief about a concept: that percents can be added, that area grows like length, that averages can be averaged, that a bigger denominator means a bigger fraction.
• Wrong formula: Uses a related but wrong formula: perimeter for area, diameter for radius, area for volume, or a misremembered one.

The wrong answers, classified

• A Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: moderate

  Takes the long right whisker, reaching 12, as a sign of an outlier, but 12 is below the cutoff 10 + 1.5 × 2 = 13.

• B Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: strong

  Reads the median line at 9 as the age of half the students, though a median only splits the ages into a lower and an upper half.

• D Wrong formula (King et al., 2004)

  Reveals a misconception Expected pull: weak

  Swaps the two measures, taking the box's 2 years as the range and the whiskers' 5 years as the interquartile range, so the IQR seems 3 greater.

How the questions are analyzed, and the research behind it

Why it is hard

The numbers are given outright, so the work is knowing what each statement claims and testing it. The outlier statements sound right because 98 looks far from 73 and 29 is the smallest value, but the 1.5 × IQR rule clears both. The true statement needs only one subtraction, 73 − 31, and a look at the median, which happens to have the same value.

1. Something you must already know moderate Working it out

   Neither the interquartile range, the third quartile minus the first, nor the outlier rule, more than 1.5 × IQR beyond a quartile, is defined on the page. (Anderson & Krathwohl, 2001)

2. Every choice must be checked moderate Choosing the answer

   The four statements test different ideas, two possible outliers, a value for the IQR and a comparison of the IQR with the median, so ruling out one says nothing about the next. (Embretson & Wetzel, 1987)

Strategy: Compute the interquartile range from the two quartiles, use it to set the outlier cutoffs 1.5 IQR beyond each quartile, and test every statement against those numbers.

Traps

• Misunderstands the concept: A wrong belief about a concept: that percents can be added, that area grows like length, that averages can be averaged, that a bigger denominator means a bigger fraction.
• Wrong formula: Uses a related but wrong formula: perimeter for area, diameter for radius, area for volume, or a misremembered one.

The wrong answers, classified

• A Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: weak

  Calls 29 an outlier just because it is the minimum, though it sits only 2 below the first quartile and far inside the cutoff 31 − 63 = −32.

• B Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: strong

  Judges 98 an outlier because it lies 25 above the third quartile, without testing it against the cutoff 73 + 1.5 × 42 = 136.

• C Wrong formula (King et al., 2004)

  Reveals a misconception Expected pull: moderate

  Subtracts the first quartile from the median, 42 − 31 = 11, instead of from the third quartile, measuring only the lower half of the box.

How the questions are analyzed, and the research behind it

Why it is hard

The arithmetic is one subtraction, so the question tests whether students know which parts of a box plot to use. The whiskers' ends give the range, the line inside the box gives the median, and both are easy to grab in place of the quartiles. The story about the carnival and the 15 ages plays no part in the answer.

1. Something you must already know minor Working it out

   The plot shows five values with no labels, so students must know that the box's ends mark the quartiles and that the interquartile range is the box's width, not the whiskers'. (Anderson & Krathwohl, 2001)

2. Reading a figure, table or graph minor Understanding the question

   The values 7, 8, 10, 17 and 18 are read off a scale ticked at every whole number, with the box's ends at 8 and 17 and the median line at 10. (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

Strategy: Find the first and third quartiles on the plot by recalling which parts of a box plot mark them, then subtract the smaller quartile from the larger.

Traps

• Wrong formula: Uses a related but wrong formula: perimeter for area, diameter for radius, area for volume, or a misremembered one.
• Misunderstands the concept: A wrong belief about a concept: that percents can be added, that area grows like length, that averages can be averaged, that a bigger denominator means a bigger fraction.
• Information you don't need: The question gives numbers or facts that play no part in the answer.

The wrong answers, classified

• A Wrong formula (King et al., 2004)

  Reveals a misconception Expected pull: weak

  Subtracts the median from the third quartile, 17 − 10 = 7, measuring only the part of the box right of the middle line.

• C Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: moderate

  Gives the median, 10, read off the line inside the box, taking a single value on the plot for the spread between the quartiles.

• D Wrong formula (King et al., 2004)

  Reveals a misconception Expected pull: strong

  Subtracts the minimum from the maximum, 18 − 7 = 11, which is the range shown by the whiskers, not the width of the box.

How the questions are analyzed, and the research behind it

Why it is hard

The question turns three sentences into three features of a box plot, and a plot that matches two of them is easy to accept. The upper-quartile fact is the conceptual one: students who do not know that the box’s right edge marks 75 percent put 82 at the end of the whisker. The other two facts are quick visual checks, but only if every plot is actually checked.

1. Every choice must be checked moderate Choosing the answer

   Each wrong plot fails a different fact, so all four must be tested against all three bullets: the left whisker against 54, the right whisker against 100, and the box’s right edge against 82. (Embretson & Wetzel, 1987)

2. Reading a figure, table or graph moderate Understanding the question

   The axis is labeled every 5 points, and the deciding features fall between labels: one left whisker starts near 51 instead of 54, and box edges sit near 82.5. (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

3. Something you must already know minor Working it out

   Nothing explains that “75% of the students scored less than 82” locates the upper quartile, the right edge of the box, rather than the highest score. (Anderson & Krathwohl, 2001)

Strategy: Turn each fact into a feature to check: where the left whisker starts, where the right whisker ends, and where the box ends. Rule out any plot that breaks one.

Traps

• Misreads the question: Misses or misreads a condition or a word in the question: "at least", "each", "remaining", "not".
• Misunderstands the concept: A wrong belief about a concept: that percents can be added, that area grows like length, that averages can be averaged, that a bigger denominator means a bigger fraction.
• Misreads the figure: Reads a value, label or tick spacing in a figure, table or graph wrongly.

The wrong answers, classified

• A Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: strong

  Puts 82 at the end of the right whisker, as if it were the highest score, when 75 percent below 82 marks the box’s right edge.

• C Misreads the figure (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

  Reveals a slip Expected pull: weak

  Accepts the left whisker as starting at the lowest score, 54, when it begins near 51, just past the 50 tick.

• D Misreads the question (Hegarty, Mayer & Monk, 1995)

  Reveals partial knowledge Expected pull: moderate

  Matches the lowest score and the box’s right edge near 82 but overlooks that a whisker reaching 100 means someone scored 100.

How the questions are analyzed, and the research behind it

Why it is hard

Most of the work happens before the plots are even read: the weights must be sorted, and the median of ten values is the average of two of them. After that, three plots look almost identical, so a careless median, taken from the unsorted list or placed halfway between the extremes, settles on a wrong one. The careful student checks the median line and the box’s left edge against values that fall between the tick marks.

1. Many steps moderate Working it out

   Ten unsorted decimals must be ordered, then five values found: the median as the mean of the fifth and sixth, 4.6 and 4.8, and each quartile as the middle of its half. (Daniel & Embretson, 2010; Enright, Morley & Sheehan, 2002)

2. A very close wrong answer moderate Choosing the answer

   Three plots share whiskers at 3.5 and 6.5 and a box ending at 6; they differ only in the median line, at 4.5, near 4.75 or at 5, and in where the box starts. (Embretson & Wetzel, 1987; Gorin & Embretson, 2006; Ascalon et al., 2007)

3. Reading a figure, table or graph moderate Understanding the question

   The axis is ticked every half ounce, and the correct plot’s median, 4.7, and lower quartile, 3.9, both fall between ticks, so each line’s position must be judged by eye. (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

4. Slow to do carefully minor Choosing the answer

   Sorting ten decimals, computing a five-number summary and comparing four plots takes well over a minute and a half when done carefully. (Lu & Sireci, 2007)

Strategy: Sort the ten weights, then find the least and greatest, the median as the mean of the fifth and sixth values, and each quartile as the middle of its half.

Traps

• Wrong formula: Uses a related but wrong formula: perimeter for area, diameter for radius, area for volume, or a misremembered one.
• A faulty procedure: A systematic but wrong procedure: adding fractions straight across, applying an exponent to each term of a sum, dropping a sign when moving a term.

The wrong answers, classified

• A Little reason to choose it (Haladyna & Downing, 1993; Rodriguez, 2005)

  Reveals a surface strategy Expected pull: weak

  No working from these weights gives whiskers near 4.1 and 6.3 or a median of 5.5, since the data run from 3.5 to 6.5, so it mostly draws guesses.

• B A faulty procedure (Brown & Burton, 1978; Tatsuoka, 1983)

  Reveals a misconception Expected pull: moderate

  Skips sorting and takes 4.5, the fifth weight as printed, for the median, so the plot with its median line on 4.5 seems to fit.

• D Wrong formula (King et al., 2004)

  Reveals a misconception Expected pull: strong

  Finds the median with the midrange formula, (3.5 + 6.5) ÷ 2 = 5, halfway between the lightest and heaviest packages instead of the middle of the sorted weights.

How the questions are analyzed, and the research behind it

Why it is hard

Once the two edges of the box are read, one subtraction finishes the question, so the work is reading the scale and knowing which part of the plot is meant. The upper quartile falls on an unlabeled tick, which costs care. The wrong answers measure the whole spread or only half of the box, all common mix-ups of what the interquartile range covers.

1. Reading a figure, table or graph moderate Understanding the question

   The axis is labeled only every 6 units but ticked every 3, and the box’s right edge sits on the unlabeled tick for 27, so it has to be counted from 24. (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

2. Something you must already know minor Working it out

   Nothing defines the interquartile range; it is the width of the box, the upper quartile minus the lower, not the span of the whiskers. (Anderson & Krathwohl, 2001)

Strategy: Find the values at the left and right edges of the box, counting tick marks in steps of 3 where there is no label, and subtract the smaller from the larger.

Traps

• Wrong formula: Uses a related but wrong formula: perimeter for area, diameter for radius, area for volume, or a misremembered one.

The wrong answers, classified

• A Wrong formula (King et al., 2004)

  Reveals a misconception Expected pull: weak

  Gives the median minus the lower quartile, 15 − 6 = 9, as the interquartile range, measuring only the left part of the box.

• B Wrong formula (King et al., 2004)

  Reveals a misconception Expected pull: weak

  Gives the upper quartile minus the median, 27 − 15 = 12, as the interquartile range, measuring only the right part of the box.

• D Wrong formula (King et al., 2004)

  Reveals a misconception Expected pull: strong

  Subtracts the ends of the whiskers, 30 − 3 = 27, giving the range of the data instead of the interquartile range.

How the questions are analyzed, and the research behind it

Why it is hard

The arithmetic is easy, since 25% of 420 is a quarter of it; the points are lost on the phrase "at least 3". Each common misreading, more than 3, exactly 3 or fewer than 3, gives a listed choice, so a wrong reading never leads to a dead end. Adding the percents before converting saves time and avoids three separate multiplications.

1. Turning words into math moderate Understanding the question

   The bold "at least 3" has to become the rows for 3, 4 and "5 or more", taking in both the 3-person row and the open-ended last row. (Hegarty, Mayer & Monk, 1995; Koedinger & Nathan, 2004)

2. Reading a figure, table or graph moderate Understanding the question

   The table gives percents, not counts, so the rows have to be added as percents and then taken of the 420 vehicles named only in the text below the table. (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

Strategy: Add the percents for every row that meets the condition first, then take that single percent of 420 instead of working row by row.

Traps

• Counting the ends: Inclusive or exclusive ends, fence posts, consecutive numbers: an answer that is one too many or too few, or an endpoint drawn open where it belongs closed (or the reverse).
• Answers a different question: A choice is right for another quantity: the other unknown, the rest instead of the part, the part instead of the whole.

The wrong answers, classified

• A Counting the ends (Brown & Burton, 1978)

  Reveals a misconception Expected pull: strong

  Reads at least 3 as more than 3 and leaves out the 3-person row, adding only 7% + 3% = 10%, and 10% of 420 is 42.

• B Misreads the question (Hegarty, Mayer & Monk, 1995)

  Reveals a surface strategy Expected pull: moderate

  Matches the 3 in the question to the row for exactly 3 people, 15% of 420 = 63, missing the 4-person and 5-or-more rows.

• D Answers a different question (King et al., 2004; Hegarty, Mayer & Monk, 1995)

  Reveals partial knowledge Expected pull: weak

  Adds the 40% and 35% rows to get 75% and takes 75% of 420 = 315, which counts the vehicles with fewer than 3 people.

How the questions are analyzed, and the research behind it

Why it is hard

The method is short: subtract the percents first, or find both groups and subtract, and both reach the same number. What makes the question less safe than it looks is the tight spread of choices, all within 500 of each other, so rounding 18% or slipping on one product lands on a wrong answer. Subtracting the percents first leaves one multiplication to get right instead of two.

1. A very close wrong answer moderate Choosing the answer

   2,700 sits only 100 from 2,800 and 300 from 3,000, so estimating 18% of 15,000 as about a fifth cannot pick it; only the exact product does. (Embretson & Wetzel, 1987; Gorin & Embretson, 2006; Ascalon et al., 2007)

2. Reading a figure, table or graph minor Understanding the question

   The Walk 22% and Bicycle 4% labels must be picked out of five sectors, and the 15,000 total sits in a caption below the circle graph. (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

3. Error-prone arithmetic minor Working it out

   The percent gap has to become a count, 0.18 × 15,000, a decimal times a five-digit number worked by hand without a calculator. (Brown & Burton, 1978; Junker & Sijtsma, 2001)

Strategy: Subtract the two percents first, then take that single percent of 15,000, instead of finding each group and subtracting.

The wrong answers, classified

• A Little reason to choose it (Haladyna & Downing, 1993; Rodriguez, 2005)

  Reveals a surface strategy Expected pull: weak

  No common working gives 2,500, which is 1/6 of 15,000 and matches no sector or difference of sectors; it mostly catches estimates.

• C An arithmetic slip (Junker & Sijtsma, 2001)

  Reveals a slip Expected pull: weak

  Takes 4% of 15,000 as 500 instead of 600 and subtracts it from the 3,300 walkers, getting 2,800.

• D Rounds too early (King et al., 2004)

  Reveals a slip Expected pull: strong

  Rounds the 18-point gap up to 20% to make the multiplication easy and takes 20% of 15,000, which is 3,000.

How the questions are analyzed, and the research behind it

Why it is hard

Neither the tallest bar nor the highest rate wins, which is the point of the question: total students depend on both. The explanation's shortcut, comparing only M, P and Q because N and R each share a rate with a taller bar, cuts the work to three products. The top two totals are close, so the products must actually be computed rather than estimated.

1. A very close wrong answer moderate Choosing the answer

   City M's 4,000 students and City Q's 4,200 differ by only 200, so judging from bar heights and rates cannot separate them; only computing 8 × 500 and 6 × 700 does. (Embretson & Wetzel, 1987; Gorin & Embretson, 2006; Ascalon et al., 2007)

2. Much to keep track of minor Working it out

   The rates come in three groups in the stem, 500 for M and N, 400 for P, 700 for Q and R, and each must be matched to its bar. (Sweller, 1988; Carpenter, Just & Shell, 1990)

3. Reading a figure, table or graph minor Understanding the question

   The number of schools comes only from the bar heights, read against gridlines from 0 to 10; all five bars end exactly on a gridline. (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

Strategy: For each city, multiply the number of schools read from the graph by its students per school and compare the products; cities that share a rate need only their school counts compared.

Traps

• Stops one step early: A choice is a number from partway through the solution.
• Answers a different question: A choice is right for another quantity: the other unknown, the rest instead of the part, the part instead of the whole.

The wrong answers, classified

• A Stops one step early (King et al., 2004; Gierl et al., 2017)

  Reveals partial knowledge Expected pull: strong

  Computes M's 8 × 500 = 4,000 and P's 9 × 400 = 3,600 and stops, never working out Q's 6 × 700 = 4,200.

• B Answers a different question (King et al., 2004; Hegarty, Mayer & Monk, 1995)

  Reveals a surface strategy Expected pull: moderate

  Picks the tallest bar, P's nine schools, which answers which city has the most schools rather than the most students.

• D Numbers taken from the question (Hegarty, Mayer & Monk, 1995)

  Reveals a surface strategy Expected pull: weak

  Picks a city with the stem's highest rate, 700 students per school, without noticing that R has only 4 schools to Q's 6.

What was corrected from the DOE handbook, and why

• Explanation: The handbook says “Cities M and N have the same number of students”. They have the same number of students per school (500), not the same total: M has 8 × 500 = 4,000 and N has 5 × 500 = 2,500, which is why only City M, with more schools, needs to be calculated. The sentence now says “the same number of students per school”.

How the questions are analyzed, and the research behind it