Practice Test

Why it is hard

The arithmetic is tiny; the difficulty is knowing that arrangements are counted by multiplying, and that the no-repeat rule removes one digit for each later place. Students who have met the counting principle finish in seconds, while those who start listing numbers run out of time or guess a product they half remember. With no choices to check against, the 3,125 that comes from ignoring the rule looks as convincing as the right count.

1. Something you must already know moderate Working it out

   Nothing in the stem names the counting principle: students must already know to multiply the digits left for each place, 5 × 4 × 3 × 2 × 1, because listing 5-digit numbers by hand cannot finish. (Anderson & Krathwohl, 2001)

2. An answer in an unexpected form minor Choosing the answer

   As a grid-in there are no choices to compare with, so a count of 3,125 or 25 cannot be caught by noticing that it is missing from a list. (Enright, Morley & Sheehan, 2002)

Strategy: Fill the five places from left to right, write how many digits are still available for each place, and multiply those five counts together.

Traps

• Without replacement: In a second pick there is one fewer item, which changes the probability.

The wrong answers, classified

• 3125 Without replacement (Fischbein & Schnarch, 1997)

  Reveals partial knowledge Expected pull: strong

  Uses the counting principle but lets digits repeat, multiplying 5 × 5 × 5 × 5 × 5 instead of using one fewer digit for each later place.

• 25 Numbers taken from the question (Hegarty, Mayer & Monk, 1995)

  Reveals a surface strategy Expected pull: moderate

  Multiplies the five digits by the five places, 5 × 5, as if each digit simply took each position once.

• 15 Wrong operation (Hegarty, Mayer & Monk, 1995; King et al., 2004)

  Reveals a misconception Expected pull: weak

  Sees that the choices shrink from 5 to 1 but adds them, 5 + 4 + 3 + 2 + 1, instead of multiplying.

How the questions are analyzed, and the research behind it

Why it is hard

This is a two-move equation that most students finish quickly: multiply both sides by 12, then find what must be taken from 147 to leave 144. Points are lost only at the last move, where the minus sign in front of x invites 144 − 147 or 147 + 144. Substituting the result back into the fraction catches either slip in a few seconds.

1. Error-prone arithmetic minor Working it out

   After multiplying by 12, the unknown sits behind a minus sign in 147 − x = 144, so students must ask what is taken from 147 rather than subtract in the order the numbers appear. (Brown & Burton, 1978; Junker & Sijtsma, 2001)

Strategy: Multiply both sides by 12 to clear the fraction, then ask what number must be taken away from 147 to leave that product.

Traps

• Negatives and direction: Arithmetic with negative numbers, the order of a subtraction, or direction on a number line. (Moving a term across the equals sign without changing its sign is a faulty procedure.)

The wrong answers, classified

• -3 Negatives and direction (Brown & Burton, 1978)

  Reveals a slip Expected pull: strong

  Reaches 147 − x = 144 but subtracts in the order the numbers appear, writing x = 144 − 147.

• 146 Wrong operation (Hegarty, Mayer & Monk, 1995; King et al., 2004)

  Reveals a slip Expected pull: moderate

  Divides the right side by 12 instead of multiplying, so sets 147 − x equal to 1 and gets 146.

• 291 A faulty procedure (Brown & Burton, 1978; Tatsuoka, 1983)

  Reveals a misconception Expected pull: weak

  Moves 147 across the equals sign without changing its sign and drops the minus on x, so x = 144 + 147.

How the questions are analyzed, and the research behind it

Why it is hard

Every step is familiar, but the expression packs four of them with negatives and decimals, and a single sign slip anywhere gives a clean-looking wrong number. The subtlest point comes last: absolute value bars make each part positive, yet the difference of two absolute values can still be negative. Students who believe bars always produce a positive answer reverse the final subtraction, and the grid-in format offers no choices to expose it.

1. Error-prone arithmetic moderate Working it out

   Three sign decisions stack up: (−6) − (−5) must become −6 + 5, then −1 + 4.2 crosses zero, and 3 − 9.6 is negative before its bars make it positive. (Brown & Burton, 1978; Junker & Sijtsma, 2001)

2. Many steps moderate Working it out

   Each pair of bars must be simplified inside and then made positive, and only after both does the outer subtraction happen; doing the bars and the subtraction in the wrong order changes the result. (Daniel & Embretson, 2010; Enright, Morley & Sheehan, 2002)

3. An answer in an unexpected form minor Choosing the answer

   The result is a negative decimal to be gridded with its sign, and with no choices on screen nothing warns a student who has dropped the minus or flipped the subtraction. (Enright, Morley & Sheehan, 2002)

Strategy: Simplify inside each pair of absolute value bars first, rewriting minus a negative as plus, then take each absolute value, and only then subtract the second from the first.

Traps

• Negatives and direction: Arithmetic with negative numbers, the order of a subtraction, or direction on a number line. (Moving a term across the equals sign without changing its sign is a faulty procedure.)

The wrong answers, classified

• 3.4 Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: strong

  Believes an expression built from absolute values cannot be negative, so reports 6.6 − 3.2 instead of 3.2 − 6.6.

• 9.8 A faulty procedure (Brown & Burton, 1978; Tatsuoka, 1983)

  Reveals a misconception Expected pull: moderate

  Treats the bars like parentheses, leaving 3 − 9.6 as −6.6, then subtracts that negative: 3.2 + 6.6.

• 0.2 Negatives and direction (Brown & Burton, 1978)

  Reveals a slip Expected pull: moderate

  Reads (−6) − (−5) as −6 − 5 = −11, so the first bars give |−6.8| = 6.8, and 6.8 − 6.6 = 0.2.

How the questions are analyzed, and the research behind it

Why it is hard

The question is short and the arithmetic is one division, so it rests on reading the second sentence the right way round. Students who see 20% next to 60 and compute 20% of 60 answer a different problem, and a grid-in gives no list of choices to make them doubt it. A quick check that 60 is one fifth of the result catches every one of these errors.

1. Turning words into math moderate Understanding the question

   The sentence “This is 20% of the total number of pages” makes 60 the part and the total the unknown, the reverse of the familiar “find 20% of a number” pattern. (Hegarty, Mayer & Monk, 1995; Koedinger & Nathan, 2004)

Strategy: Write the finished pages as 20 percent of an unknown total and solve for the total, or think about what fraction of the whole book 20 percent is.

Traps

• Percent of the wrong amount: Finding the whole from a part, or a percent change measured from the wrong starting amount.
• Answers a different question: A choice is right for another quantity: the other unknown, the rest instead of the part, the part instead of the whole.

The wrong answers, classified

• 12 Percent of the wrong amount (Parker & Leinhardt, 1995)

  Reveals a misconception Expected pull: strong

  Takes 20% of 60 instead of finding the number that 60 is 20% of, applying the percent to the wrong amount.

• 240 Answers a different question (King et al., 2004; Hegarty, Mayer & Monk, 1995)

  Reveals partial knowledge Expected pull: moderate

  Sees that 60 pages is one fifth of the book but multiplies 60 by 4, giving the pages Tyler still has left.

• 3 A faulty procedure (Brown & Burton, 1978; Tatsuoka, 1983)

  Reveals partial knowledge Expected pull: weak

  Divides 60 by 20 instead of by 0.20, treating 20% as the whole number 20.

How the questions are analyzed, and the research behind it

Why it is hard

The arithmetic is one subtraction; the work is in seeing the figure correctly. Students have to notice that a fourth angle sits between the 30° angle and y, recognize it as the vertical partner of the 25° angle, and then use the straight line. The drawing is not to scale: y is drawn close to the 60° angle across the vertical line, which rewards anyone who computes rather than eyeballs.

1. Reading a figure, table or graph major Understanding the question

   Four lines make eight angles around P but only three are labeled; above the horizontal line sit 60°, 30°, an unlabeled angle and y°, and that unlabeled angle is easy to overlook. (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

2. Combines ideas moderate Working it out

   The solution joins two facts: the unlabeled angle is vertical to the 25° angle below P, and the four angles above the horizontal line make a straight angle of 180°. (Embretson, 1998; Daniel & Embretson, 2010)

3. An answer in an unexpected form minor Choosing the answer

   As a grid-in, nothing confirms the result, and the drawing shows y at about 60°, so a reading of the picture produces a number that looks right. (Enright, Morley & Sheehan, 2002)

Strategy: Pick a straight line through P and list the angles on one side of it; replace the unlabeled one with the labeled angle vertical to it, then subtract from 180.

Traps

• Misreads the figure: Reads a value, label or tick spacing in a figure, table or graph wrongly.
• Stops one step early: A choice is a number from partway through the solution.
• Trusting the picture: The figure suggests something it does not state; work from the given facts.

The wrong answers, classified

• 90 Misreads the figure (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

  Reveals partial knowledge Expected pull: strong

  Misses the unlabeled angle between the 30° angle and y, so subtracts only 30 and 60 from 180.

• 115 Stops one step early (King et al., 2004; Gierl et al., 2017)

  Reveals partial knowledge Expected pull: moderate

  Adds the three known angles beside y, 25 + 30 + 60, and grids that sum without subtracting it from 180.

• 60 Trusting the picture (Lesh, Post & Behr, 1987)

  Reveals a surface strategy Expected pull: moderate

  Judges from the drawing, where y looks like a mirror image of the 60° angle across the vertical line, instead of computing it.

How the questions are analyzed, and the research behind it

Why it is hard

Substitution itself is routine, so the question tests one thing: subtracting a negative inside the parentheses. A student who turns 9 − 2(−7) into 9 − 14 gets −45 and may then choose 45, since it is the only choice that looks close. Working the parentheses first and writing −(−14) as +14 before multiplying by 9 avoids every wrong choice.

1. Error-prone arithmetic moderate Working it out

   The middle step is 9 − 2(−7): the product 2(−7) is −14, and subtracting it means adding 14, so two negatives must be handled in one short move. (Brown & Burton, 1978; Junker & Sijtsma, 2001)

Strategy: Work inside the parentheses first: find 2y, subtract it from x by rewriting minus a negative as plus, and multiply that result by x last.

Traps

• Negatives and direction: Arithmetic with negative numbers, the order of a subtraction, or direction on a number line. (Moving a term across the equals sign without changing its sign is a faulty procedure.)

The wrong answers, classified

• A Negatives and direction (Brown & Burton, 1978)

  Reveals a slip Expected pull: weak

  Drops the factor 2 and adds y instead of subtracting it, working out 9(9 + (−7)) = 9 × 2 = 18.

• B Negatives and direction (Brown & Burton, 1978)

  Reveals a misconception Expected pull: strong

  Writes 9 − 2(−7) as 9 − 14 = −5, gets 9 × (−5) = −45, and picks 45 as the nearest choice.

• C Misreads the question (Hegarty, Mayer & Monk, 1995)

  Reveals a slip Expected pull: moderate

  Handles the negative correctly but drops the coefficient 2, computing 9 − (−7) = 16 and then 9 × 16 = 144.

How the questions are analyzed, and the research behind it

Why it is hard

Few steps are needed, but the figure misleads. It draws triangle PQT as isosceles, so the angle at P looks the same as the marked 70°, and the angle at R looks the same again; a student who trusts the drawing never computes. Those who do compute the angle at P must still choose the opposite angle of the parallelogram, not the angle next to it, which is where 120° comes from.

1. Reading a figure, table or graph major Understanding the question

   The figure carries facts the stem leaves out: T lies on side PS, the 50° is only part of the angle at Q, and the 70° is at T, not at a corner of the parallelogram. (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

2. Combines ideas moderate Working it out

   Two separate facts link up: the angles of triangle PQT sum to 180°, and the angle at P, once known, equals the opposite angle at R of parallelogram PQRS. (Embretson, 1998; Daniel & Embretson, 2010)

3. A very close wrong answer moderate Choosing the answer

   60° and 70° are close enough that judging the angle at R by eye cannot separate them, and the drawing even favors 70°; only computing the triangle's third angle does. (Embretson & Wetzel, 1987; Gorin & Embretson, 2006; Ascalon et al., 2007)

4. Something you must already know minor Working it out

   Neither the triangle angle sum nor the rule that opposite angles of a parallelogram are equal is given; students must bring both to the figure. (Anderson & Krathwohl, 2001)

Strategy: Find the third angle of triangle PQT from the angle sum, work out which angle of the parallelogram it matches, then use the parallelogram's angle facts to reach angle QRS.

Traps

• Trusting the picture: The figure suggests something it does not state; work from the given facts.
• Answers a different question: A choice is right for another quantity: the other unknown, the rest instead of the part, the part instead of the whole.

The wrong answers, classified

• B Trusting the picture (Lesh, Post & Behr, 1987)

  Reveals a surface strategy Expected pull: strong

  Trusts the drawing, where triangle PQT looks isosceles, so takes the angle at P, and its opposite angle at R, to be 70° like the angle at T.

• C Little reason to choose it (Haladyna & Downing, 1993; Rodriguez, 2005)

  Reveals a surface strategy Expected pull: weak

  No standard step gives 80°; at most a student treats the triangle as having two 50° angles and computes 180 − 100, which nothing in the figure supports.

• D Answers a different question (King et al., 2004; Hegarty, Mayer & Monk, 1995)

  Reveals partial knowledge Expected pull: moderate

  Finds the 60° angle at P but answers with the angle at Q, 180 − 60 or 50 + 70 = 120, the consecutive angle instead of the opposite one.

How the questions are analyzed, and the research behind it

Why it is hard

The question rewards a strategy, not a calculation: once the shared value is named, each variable follows in one line and R is clearly largest. Without that step, students judge by how the expressions look, and the look is backwards, since the 3 in 3N makes N small and the 7 under R makes R large. The positivity condition at the end is easy to skip but is what fixes the order.

1. Variables instead of numbers major Understanding the question

   There are no numbers to compute: five variables are tied together in the one chain M = 3N = P/4 = Q + 5 = R/7, and each must be compared through the shared value. (Koedinger & Nathan, 2004; Enright, Morley & Sheehan, 2002)

2. No obvious method moderate Working it out

   Nothing says how to compare; the efficient move is to name the common value, or pick one such as 12, and rewrite each variable, which students must think of themselves. (Schoenfeld, 1985)

3. Much to keep track of minor Working it out

   Five relations and the condition > 0 must be held at once; the condition matters, since with a negative common value the order of M, N, P and R would reverse. (Sweller, 1988; Carpenter, Just & Shell, 1990)

Strategy: Call the shared value k, a positive number, and rewrite each variable as an expression in k; or substitute a convenient positive value such as 12 and compare the results.

Traps

• Equation written backwards: Writes a comparison the wrong way round, word for word: "six times as many students as professors" as 6S = P. The bigger quantity gets the bigger number.

The wrong answers, classified

• A Equation written backwards (Clement, Lochhead & Monk, 1981)

  Reveals a misconception Expected pull: weak

  Reads P/4 and R/7 as a quarter and a seventh of M, the reverse of the truth, so M looks largest of all.

• B Equation written backwards (Clement, Lochhead & Monk, 1981)

  Reveals a misconception Expected pull: strong

  Reads the 3 in 3N as making N three times as large, when 3N = M actually makes N one third of M.

• C Stops one step early (King et al., 2004; Gierl et al., 2017)

  Reveals partial knowledge Expected pull: moderate

  Correctly sees that P is four times M but stops there without checking R, which at seven times M is larger.

How the questions are analyzed, and the research behind it

Why it is hard

The rate is simple once it is turned into 32 square feet per bundle, and 416 ÷ 32 comes out exactly. Trouble starts for students who divide 416 by 96 first: 4.33 sections of roof invites rounding, and rounding before multiplying by 3 gives a neighbor of the right answer. Keeping the fraction exact, or using the unit rate, avoids the problem.

1. A very close wrong answer minor Choosing the answer

   12 and 14 bundles sit on either side of the right 13, so a rough estimate of 416 ÷ 32 cannot pick among them; only the exact division, 32 × 13 = 416, does. (Embretson & Wetzel, 1987; Gorin & Embretson, 2006; Ascalon et al., 2007)

2. Error-prone arithmetic minor Working it out

   The division 416 ÷ 96 does not come out even, about 4.33, so students who use it rather than 32 square feet per bundle must carry a repeating decimal. (Brown & Burton, 1978; Junker & Sijtsma, 2001)

Strategy: Find how many square feet one bundle covers, then divide the roof's 416 square feet by that amount.

Traps

• Answers a different question: A choice is right for another quantity: the other unknown, the rest instead of the part, the part instead of the whole.

The wrong answers, classified

• A Answers a different question (King et al., 2004; Hegarty, Mayer & Monk, 1995)

  Reveals partial knowledge Expected pull: moderate

  Divides 416 by 96 and rounds 4.33 up to 5, giving the number of 96-square-foot sections instead of the number of bundles.

• B Rounds too early (King et al., 2004)

  Reveals partial knowledge Expected pull: strong

  Rounds 416 ÷ 96 down to 4 sections before multiplying by 3 bundles, leaving the last 32 square feet uncovered.

• D An arithmetic slip (Junker & Sijtsma, 2001)

  Reveals a slip Expected pull: weak

  Slips in dividing 1,248 by 96, or adds a spare bundle to be safe, landing one bundle above the exact count.

How the questions are analyzed, and the research behind it

Why it is hard

The computation leads straight to 70, and the whole question is about what to do with that number. Students who treat the boundary as the answer forget that equal cost is not less cost, while the other choices only catch those who misdivide or ignore the word largest. The question rewards checking the boundary case in the story: 70 sheets cost exactly the package price.

1. A very close wrong answer moderate Choosing the answer

   70 and 69 sheets differ by one, and 70 is exactly where the two costs meet at $10.50; only the strict “less expensive” separates them, ruling out 70. (Embretson & Wetzel, 1987; Gorin & Embretson, 2006; Ascalon et al., 2007)

2. Turning words into math minor Understanding the question

   The words “less expensive than buying the package” must become a strict inequality, 0.15x < 10.50, and “largest value” must then pick the biggest whole number below the boundary. (Hegarty, Mayer & Monk, 1995; Koedinger & Nathan, 2004)

3. Error-prone arithmetic minor Working it out

   Dividing 10.50 by 0.15 means shifting the decimal point in both numbers, a step where a misplaced digit gives a wrong boundary. (Brown & Burton, 1978; Junker & Sijtsma, 2001)

Strategy: Divide the package price by the price of one sheet to see where the two costs meet, then find the largest whole number of sheets that still costs less than $10.50.

Traps

• Counting the ends: Inclusive or exclusive ends, fence posts, consecutive numbers: an answer that is one too many or too few, or an endpoint drawn open where it belongs closed (or the reverse).

The wrong answers, classified

• A Misreads the question (Hegarty, Mayer & Monk, 1995)

  Reveals partial knowledge Expected pull: weak

  Checks a round number, 60 sheets for $9.00, sees it is cheaper than the package, and stops without seeking the largest such number.

• B An arithmetic slip (Junker & Sijtsma, 2001)

  Reveals a slip Expected pull: weak

  Divides only the $10 by $0.15, about 66.7, ignoring the 50 cents, and settles on the nearest smaller choice.

• D Counting the ends (Brown & Burton, 1978)

  Reveals partial knowledge Expected pull: strong

  Solves 0.15x = 10.50 and keeps the boundary 70, but 70 sheets cost exactly $10.50, the same as the package, not less.

What was corrected from the DOE handbook, and why

• Explanation: The handbook sets up 0.15x ≤ 10.50 and x ≤ 70, which would allow 70 sheets, and then says that 70 sheets cost exactly $10.50. Buying the sheets has to be less expensive than the package, so the site uses strict inequalities, 0.15x < 10.50 and x < 70, which make 69 the largest number of sheets.

How the questions are analyzed, and the research behind it

Why it is hard

The arithmetic is light; the question turns on counting the hours from 1:00 to 7:00 correctly. Counting the clock times instead of the gaps between them gives seven hours, starting the drop an hour late gives five, and both errors are built into the choices. Subtracting the times, 7 − 1, is safer than counting on fingers.

1. Error-prone arithmetic minor Working it out

   The total drop of 18 degrees is larger than the starting 8 degrees, so the subtraction crosses zero and the result must be negative. (Brown & Burton, 1978; Junker & Sijtsma, 2001)

Strategy: Count the hours that pass from 1:00 to 7:00 by subtracting the times, multiply by the 3-degree drop, and subtract that total from 8.

Traps

• Counting the ends: Inclusive or exclusive ends, fence posts, consecutive numbers: an answer that is one too many or too few, or an endpoint drawn open where it belongs closed (or the reverse).

The wrong answers, classified

• A Counting the ends (Brown & Burton, 1978)

  Reveals a slip Expected pull: strong

  Counts the clock times 1, 2, 3, 4, 5, 6, 7 as seven hours, so subtracts 21 degrees from 8.

• C Counting the ends (Brown & Burton, 1978)

  Reveals a slip Expected pull: moderate

  Starts the drop at 2:00 instead of 1:00, counting five hours and subtracting 15 degrees from 8.

• D Numbers taken from the question (Hegarty, Mayer & Monk, 1995)

  Reveals a surface strategy Expected pull: weak

  Subtracts a single 3-degree drop from 8, combining the two numbers in the question without counting the hours.

How the questions are analyzed, and the research behind it

Why it is hard

Each step is easy because both ratios share the 7 for blue, so the work is about keeping the chain going without stopping early. The main danger is doing the swap on the ratio numbers or forgetting it altogether. Two of the wrong choices, 5:1 and 15:3, are the same ratio, so a student who notices that can rule out both before doing any arithmetic.

1. Many steps moderate Working it out

   The chain is long: merge 15:7 and 7:3 into 15:7:3, find that 25 parts make 75 marbles, scale to 45, 21 and 9, then add 2 to the green count. (Daniel & Embretson, 2010; Enright, Morley & Sheehan, 2002)

2. Much to keep track of minor Working it out

   Three colors' counts must be tracked through the swap, which lowers blue and raises green while red stays where it was. (Sweller, 1988; Carpenter, Just & Shell, 1990)

3. Turning words into math minor Understanding the question

   The phrase “2 blue marbles are removed and replaced with 2 green marbles” must be read as changing the actual counts, not the ratio numbers 7 and 3. (Hegarty, Mayer & Monk, 1995; Koedinger & Nathan, 2004)

Strategy: Merge the two ratios into one red-to-blue-to-green ratio, scale it so the parts total 75, make the swap on those counts, then write red to green.

Traps

• Stops one step early: A choice is a number from partway through the solution.

The wrong answers, classified

• A Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: moderate

  Treats the ratio's 3 as the actual number of green marbles and adds the 2 swapped marbles to it, getting 15:5, or 3:1.

• B Stops one step early (King et al., 2004; Gierl et al., 2017)

  Reveals partial knowledge Expected pull: strong

  Finds the counts 45 red and 9 green correctly but forgets the swap, giving the original ratio 45:9 in lowest terms.

• C Numbers taken from the question (Hegarty, Mayer & Monk, 1995)

  Reveals a surface strategy Expected pull: moderate

  Reads red and green straight off the combined ratio 15:7:3, with no counts and no swap; it equals 5:1 anyway.

How the questions are analyzed, and the research behind it

Why it is hard

This is a direct relative-frequency question with friendly numbers, and most students finish it quickly. The risk is using a number from the table as if it were already a percent, or treating the four desserts as equally likely. Remembering that a probability compares one count with the total of all counts prevents both mistakes.

1. Reading a figure, table or graph minor Understanding the question

   The four counts sit in a table, and the probability needs all of them: the ice cream row gives the numerator and the sum of the column gives the denominator. (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

2. Error-prone arithmetic minor Working it out

   Adding 42 + 23 + 47 + 48 by hand and then simplifying 48/160 to 3/10 are small steps where a careless total spoils the percent. (Brown & Burton, 1978; Junker & Sijtsma, 2001)

Strategy: Add the four counts in the table to get all the orders, divide the ice cream count by that total, and convert the fraction to a percent.

Traps

• Answers a different question: A choice is right for another quantity: the other unknown, the rest instead of the part, the part instead of the whole.

The wrong answers, classified

• A Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: moderate

  Treats the four desserts as equally likely, one chance in four, ignoring how often each was actually ordered.

• C Answers a different question (King et al., 2004; Hegarty, Mayer & Monk, 1995)

  Reveals partial knowledge Expected pull: weak

  Divides 48 by the 112 orders of the other desserts instead of all 160, getting about 43%, and takes the nearest choice.

• D Numbers taken from the question (Hegarty, Mayer & Monk, 1995)

  Reveals a surface strategy Expected pull: strong

  Reads the 48 ice cream orders as 48 percent without dividing by the total number of desserts ordered.

How the questions are analyzed, and the research behind it

Why it is hard

Finding a least common multiple is routine, and listing multiples of 24 reaches the answer by the third one. The wrong choices are built to catch incomplete checking: each passes the test for some of the numbers but fails for one, or passes for all and is simply not the smallest. Testing every candidate against all three numbers is the whole skill.

1. Every choice must be checked minor Choosing the answer

   Each wrong choice passes part of the test: 36 is a multiple of 6 and 18, 48 of 6 and 24, and 144 of all three, so every candidate needs checking against all three numbers. (Embretson & Wetzel, 1987)

2. Something you must already know minor Working it out

   Students must know what “least common multiple” means and see that 6 can be dropped, since it already divides both 24 and 18. (Anderson & Krathwohl, 2001)

Strategy: Note that 6 divides both 24 and 18, so only those two matter; list multiples of 24 and stop at the first one that 18 also divides.

Traps

• Stops one step early: A choice is a number from partway through the solution.

The wrong answers, classified

• A Stops one step early (King et al., 2004; Gierl et al., 2017)

  Reveals partial knowledge Expected pull: strong

  Finds the least common multiple of 6 and 18 and stops, never checking that 36 is not a multiple of 24.

• B An arithmetic slip (Junker & Sijtsma, 2001)

  Reveals a slip Expected pull: weak

  Lists multiples of 24, stops at 48 and misjudges it as a multiple of 18, though 48 ÷ 18 is not a whole number.

• D Misreads the question (Hegarty, Mayer & Monk, 1995)

  Reveals partial knowledge Expected pull: moderate

  Multiplies 24 by 6 to get 144, which all three numbers divide, overlooking that the question asks for the least such multiple.

How the questions are analyzed, and the research behind it

Why it is hard

The proportion is simple, since 300 customers is one and a half times 200, and the only judgment is what to do with 22.5 dozen. The choices are spread wide enough that 23 is clearly nearest once 22.5 is found. Errors come mostly from students who skip the proportion and combine the numbers 300, 200 and 15 in some quicker way.

1. An answer in an unexpected form minor Choosing the answer

   The scaled amount is 22.5 dozen, which is not a choice; the word “approximately” signals that the answer is the nearest whole number of dozens. (Enright, Morley & Sheehan, 2002)

2. Error-prone arithmetic minor Working it out

   Scaling 15 dozen by 300/200 needs the factor 1.5, and 15 × 1.5 = 22.5 brings in a half that students must handle before matching a choice. (Brown & Burton, 1978; Junker & Sijtsma, 2001)

Strategy: Compare 300 customers with 200 to get the scale factor, multiply the 15 dozen eggs by it, and choose the closest answer.

The wrong answers, classified

• A Numbers taken from the question (Hegarty, Mayer & Monk, 1995)

  Reveals a surface strategy Expected pull: moderate

  Divides 300 customers by 15 dozen, combining two numbers from the question without a model of eggs per customer.

• C Little reason to choose it (Haladyna & Downing, 1993; Rodriguez, 2005)

  Reveals a surface strategy Expected pull: weak

  No standard route gives 25; at most a student who finds 22.5 rounds it to the nearest five instead of the nearest whole dozen.

• D Rounds too early (King et al., 2004)

  Reveals partial knowledge Expected pull: moderate

  Rounds the scale factor 300 ÷ 200 = 1.5 up to 2 before multiplying, doubling 15 dozen to 30.

What was corrected from the DOE handbook, and why

• Explanation: The handbook says to round 22.5 up to 23 “because you can't have half an egg.” But 22.5 dozen is 270 eggs, a whole number; the half is half a dozen, or 6 eggs. The site gives the real reason: 22 dozen eggs would not be enough for 300 customers, so the amount rounds up to 23 dozen.

How the questions are analyzed, and the research behind it

Why it is hard

The counting is easy, 14 bottles in all, so the question mainly tests careful reading. The word “not” turns the event around, and the probability of the opposite event is waiting among the choices. The other wrong choices catch students who count kinds of juice instead of bottles, or who build a fraction with the wrong total.

1. Turning words into math minor Understanding the question

   The event “not apple juice” must be turned into grape plus orange, 3 + 6 bottles over all 14, so the word “not” decides which counts go in the numerator. (Hegarty, Mayer & Monk, 1995; Koedinger & Nathan, 2004)

Strategy: Add all the bottles for the denominator, then count only the grape and orange bottles for the numerator, or subtract the chance of apple juice from 1.

Traps

• Misreads the question: Misses or misreads a condition or a word in the question: "at least", "each", "remaining", "not".

The wrong answers, classified

• A Numbers taken from the question (Hegarty, Mayer & Monk, 1995)

  Reveals a surface strategy Expected pull: weak

  Puts 1 over the 9 bottles that are not apple juice, as if one particular bottle had to be picked.

• B Misreads the question (Hegarty, Mayer & Monk, 1995)

  Reveals a slip Expected pull: strong

  Finds the probability of choosing apple juice, 5 out of 14, missing the word “not” in the question.

• D Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: moderate

  Counts two of the three kinds of juice as not apple and treats the kinds as equally likely, ignoring how many bottles each has.

How the questions are analyzed, and the research behind it

Why it is hard

The question describes the two plates in different measures, one by radius and one by circumference, so students must convert before they can compare. The trap is a halving argument: the small circumference is half the large one, but the area it covers is only a quarter. Students who keep both areas in terms of π and subtract finish quickly; those who reason from the circumferences alone go wrong.

1. Many steps moderate Working it out

   The chain runs through four stages: the large area 400π, the small radius 10 from the circumference 20π, the small area 100π, then the uncovered difference. (Daniel & Embretson, 2010; Enright, Morley & Sheehan, 2002)

2. Something you must already know moderate Working it out

   Neither formula is given: students need C = 2πr to recover the small plate's radius and A = πr² for both areas. (Anderson & Krathwohl, 2001)

Strategy: Use C = 2πr to get the small plate's radius, find both areas with A = πr², keep everything in terms of π, and subtract.

Traps

• Answers a different question: A choice is right for another quantity: the other unknown, the rest instead of the part, the part instead of the whole.
• A formula you must know: Needs a formula the test does not give: area, circumference, volume, probability rules.

The wrong answers, classified

• A Wrong formula (King et al., 2004)

  Reveals a misconception Expected pull: moderate

  Works with circumferences instead of areas, subtracting the small plate's 20π from the large plate's 40π.

• B Answers a different question (King et al., 2004; Hegarty, Mayer & Monk, 1995)

  Reveals partial knowledge Expected pull: strong

  Finds the area of the smaller plate, the part that is covered, instead of the part of the large plate left uncovered.

• C Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: moderate

  Reasons that a circumference half as long covers half the area, so takes half of 400π, not seeing that area scales with the square.

How the questions are analyzed, and the research behind it

Why it is hard

Solving for the number of ads is the obvious goal, and finding it feels like the finish, but the question asks for page space, which needs one more pass through the table. The awkward wording about buying pages in equal numbers slows students before any arithmetic starts. The wrong choices mark each place to stop: one set of ads, the number of each ad, or a slip in adding the fractional pages.

1. Many steps major Working it out

   The work has two phases: turn the money into a count, 1,150x = 11,500 so x = 10, then turn the count into space by multiplying 10 by 1/4, 1/2 and 1 and adding. (Daniel & Embretson, 2010; Enright, Morley & Sheehan, 2002)

2. Turning words into math moderate Understanding the question

   The phrase “purchased 1/4 pages, 1/2 pages, and full pages of page space in equal numbers” has to be read as the same number of ads of each size, so the three prices add to one coefficient. (Hegarty, Mayer & Monk, 1995; Koedinger & Nathan, 2004)

3. Reading a figure, table or graph minor Understanding the question

   Both columns of the table are needed: the price column builds the equation, and the page-space column turns the count of ads into the final total. (Lesh, Post & Behr, 1987; Kirsch & Mosenthal, 1990)

Strategy: Let x be the number of each ad size, add the three prices to write one equation for the total, solve it, then multiply x by the page space in one set of ads.

Traps

• Stops one step early: A choice is a number from partway through the solution.
• Answers a different question: A choice is right for another quantity: the other unknown, the rest instead of the part, the part instead of the whole.

The wrong answers, classified

• A Answers a different question (King et al., 2004; Hegarty, Mayer & Monk, 1995)

  Reveals partial knowledge Expected pull: moderate

  Adds 1/4 + 1/2 + 1, the page space in one set of the three ads, without using the $11,500 to find how many sets were bought.

• B Stops one step early (King et al., 2004; Gierl et al., 2017)

  Reveals partial knowledge Expected pull: strong

  Solves 1,150x = 11,500 for the number of ads of each size and stops there instead of converting the count to page space.

• C An arithmetic slip (Junker & Sijtsma, 2001)

  Reveals a slip Expected pull: weak

  Takes ten quarter pages as 1 1/2 pages instead of 2 1/2, so adds 1 1/2 + 5 + 10.

What was corrected from the DOE handbook, and why

• Explanation: The handbook adds 200x + 350x + 600x and writes 1,500x = 11,500. The sum is 1,150x, and the handbook’s next line, x = 10, only follows from 1,150x = 11,500. The site shows 1,150x = 11,500.

How the questions are analyzed, and the research behind it

Why it is hard

Solving the inequality is routine, and that is the trap: x ≤ 28 looks like an answer and 28 is among the choices. The real task is counting odd numbers from 1 to 27, where the usual shortcut, (27 − 1) ÷ 2, misses the first term. Pairing each odd number with the even number after it, up to 28, gives the count directly and avoids the fencepost slip.

1. A very close wrong answer moderate Choosing the answer

   13 and 14 differ by one, and a rough count of the odd numbers up to 27 cannot separate them; only counting both ends, 1 and 27, gives the exact 14. (Embretson & Wetzel, 1987; Gorin & Embretson, 2006; Ascalon et al., 2007)

2. Combines ideas minor Working it out

   Solving 3x + 8 ≤ 92 is only half the work; counting the positive odd whole numbers up to 28 is a separate number-sense task. (Embretson, 1998; Daniel & Embretson, 2010)

Strategy: Solve the inequality as you would an equation, keeping the direction of the sign, then count odd numbers from 1 up to the boundary by pairing or halving.

Traps

• Counting the ends: Inclusive or exclusive ends, fence posts, consecutive numbers: an answer that is one too many or too few, or an endpoint drawn open where it belongs closed (or the reverse).
• Stops one step early: A choice is a number from partway through the solution.

The wrong answers, classified

• A Counting the ends (Brown & Burton, 1978)

  Reveals a slip Expected pull: strong

  Counts the odd numbers from 1 to 27 as (27 − 1) ÷ 2 = 13, forgetting to add one for the first term.

• C A faulty procedure (Brown & Burton, 1978; Tatsuoka, 1983)

  Reveals a slip Expected pull: weak

  Moves the 8 across without changing its sign, solving 3x ≤ 100, and counts the odd numbers up to 33.

• D Stops one step early (King et al., 2004; Gierl et al., 2017)

  Reveals partial knowledge Expected pull: strong

  Solves the inequality to x ≤ 28 and gives the boundary 28 instead of counting the positive odd numbers up to it.

What was corrected from the DOE handbook, and why

• Explanation: The handbook solves the inequality to x ≤ 28 and then lists “the positive odd numbers less than 28.” The solutions include 28 itself, so the site says “less than or equal to 28.” The count does not change, because 28 is even: the odd numbers are still 1, 3, 5, …, 27, which is 14 numbers.

How the questions are analyzed, and the research behind it