Practice Test

Why it is hard

The menus do much of the setting up, since each sentence already shows the shape of its equation. The second menu is the hard one: the 8 stamps leave Liam and join Kevin before the gap of 12 is measured. Solving gives Kevin's 28 first, and K + 20 then equals 48, Liam's count after the gift, which waits on the last menu beside his starting 56. Testing each starting count against the story is a quick check on the last menu.

1. Turning words into math moderate Understanding the question

   The gift changes both counts, so "Liam now has 12 more stamps than Kevin" compares Liam's L − 8 with Kevin's K + 8, and the second menu's K + 20 holds a 20 that appears nowhere in the stem. (Hegarty, Mayer & Monk, 1995; Koedinger & Nathan, 2004)

2. Much to keep track of moderate Working it out

   Each boy has a count before and after the gift, four numbers in all, and all three menus must be right; the last asks for Liam's start, not the 48 he has after giving 8 away. (Sweller, 1988; Carpenter, Just & Shell, 1990)

Strategy: Let K be Kevin's starting count, write each boy's count after the gift in terms of K, and set Liam's 12 above Kevin's; testing each number in the last menu is a fast check.

Traps

• Answers a different question: A choice is right for another quantity: the other unknown, the rest instead of the part, the part instead of the whole.
• Follows a key word: Picks the operation from a single word rather than the situation: "removed" so subtract, "of" so multiply, "more" so add, when the situation needs something else.
• Equation written backwards: Writes a comparison the wrong way round, word for word: "six times as many students as professors" as 6S = P. The bigger quantity gets the bigger number.

The wrong answers, classified

• BCB Answers a different question (King et al., 2004; Hegarty, Mayer & Monk, 1995)

  Reveals partial knowledge Expected pull: strong

  Sets up both equations and solves for Kevin's 28, then takes the value of K + 20, 48, which is Liam's count after the gift, not at the start.

• BBC Follows a key word (Hegarty, Mayer & Monk, 1995)

  Reveals a surface strategy Expected pull: moderate

  Follows the words "12 more" to L + 12, adding 12 to Liam's own count instead of Kevin's, then finds 56 by testing the three counts against the story.

• ACC Equation written backwards (Clement, Lochhead & Monk, 1981)

  Reveals a misconception Expected pull: weak

  Writes "twice as many stamps as Kevin" backwards, as 2L = K, and picks 0.5, giving Liam half of Kevin's stamps, while the other two menus are right.

How the questions are analyzed, and the research behind it

Why it is hard

The question turns a statement about perimeters into a ratio of sides, and the natural guess, that the sides keep the perimeters’ 2 to 1, ignores that the square shares its perimeter among 4 sides and the triangle among 3. Choosing a triangle perimeter such as 12 makes the comparison concrete. The order of the ratio still has to be checked at the end, because its reverse is listed too.

1. Turning words into math moderate Understanding the question

   “The perimeter of square W is twice the perimeter of equilateral triangle X” must become 4s = 2 × 3t, with the 2 on the triangle’s side; written backward, it gives the triangle the larger perimeter. (Hegarty, Mayer & Monk, 1995; Koedinger & Nathan, 2004)

2. Variables instead of numbers moderate Understanding the question

   No side length is given, so students must either work with letters for the two sides or choose a convenient perimeter themselves, such as 12 for the triangle. (Koedinger & Nathan, 2004; Enright, Morley & Sheehan, 2002)

3. A very close wrong answer minor Choosing the answer

   The ratios 3:2 and 2:3 use the same two numbers, so only keeping the square’s side first, as the question orders it, separates the answer from its reverse. (Embretson & Wetzel, 1987; Gorin & Embretson, 2006; Ascalon et al., 2007)

Strategy: Pick an easy side length for the triangle, find its perimeter, double that for the square, divide by the square's number of sides, and write square to triangle.

Traps

• Misunderstands the concept: A wrong belief about a concept: that percents can be added, that area grows like length, that averages can be averaged, that a bigger denominator means a bigger fraction.
• Wrong operation: Adds instead of multiplying, divides the wrong way round, turns a ratio or rate upside down, or undoes a step in the wrong direction (subtracts what should be added back).
• Equation written backwards: Writes a comparison the wrong way round, word for word: "six times as many students as professors" as 6S = P. The bigger quantity gets the bigger number.

The wrong answers, classified

• A Equation written backwards (Clement, Lochhead & Monk, 1981)

  Reveals a misconception Expected pull: weak

  Writes the comparison backward, 2 × 4s = 3t, giving the triangle the larger perimeter, so the ratio of the sides comes out 3:8.

• B Wrong operation (Hegarty, Mayer & Monk, 1995; King et al., 2004)

  Reveals partial knowledge Expected pull: moderate

  Finds the right relationship but turns the ratio upside down, giving the triangle’s side to the square’s instead of the square’s side first.

• D Misunderstands the concept (Smith, diSessa & Roschelle, 1993; De Bock et al., 1998; Ni & Zhou, 2005)

  Reveals a misconception Expected pull: strong

  Assumes the sides are in the same ratio as the perimeters, 2:1, ignoring that the square has 4 sides and the triangle only 3.

How the questions are analyzed, and the research behind it

Why it is hard

The question is all setup: no solving is needed, just an equation that says what the story says. The trade moves games out of one count and into the other, and the comparison has to be written with the 2 on the smaller side. The wrong equations reverse the comparison, forget Jeff's gain, or drop the parentheses, and all four look alike. Plugging in sample counts is the surest check.

1. Turning words into math moderate Understanding the question

   After the gift both counts change, Ken's to k − 6 and Jeff's to j + 6, and “twice as many video games as Jeff” must double Jeff's whole new count, putting the 2 on his side. (Hegarty, Mayer & Monk, 1995; Koedinger & Nathan, 2004)

2. A very close wrong answer moderate Choosing the answer

   All four equations share the pieces k − 6, j + 6 and a 2; the right one differs from 2(k − 6) = j + 6 only in where the 2 sits, and from k − 6 = 2j + 6 by parentheses. (Embretson & Wetzel, 1987; Gorin & Embretson, 2006; Ascalon et al., 2007)

3. Variables instead of numbers minor Understanding the question

   There are no numbers to check against, only k and j, so testing an equation means inventing counts that fit the story, such as Jeff with 4 games and Ken with 26. (Koedinger & Nathan, 2004; Enright, Morley & Sheehan, 2002)

Strategy: Write each boy's number of games after the exchange as an expression, then translate "Ken will have twice as many as Jeff" into an equation and test it with sample numbers.

Traps

• Equation written backwards: Writes a comparison the wrong way round, word for word: "six times as many students as professors" as 6S = P. The bigger quantity gets the bigger number.
• Order of operations: Working left to right, or distributing a minus sign wrongly, gives a choice.

The wrong answers, classified

• B Order of operations (Brown & Burton, 1978)

  Reveals a misconception Expected pull: moderate

  Leaves out the parentheses, doubling only Jeff's old count j and then adding 6, instead of doubling his whole new count j + 6.

• C Equation written backwards (Clement, Lochhead & Monk, 1981)

  Reveals a misconception Expected pull: weak

  Puts the 2 on Ken's side, writing the comparison backwards, and also forgets that Jeff gains the 6 games, two errors in one equation.

• D Equation written backwards (Clement, Lochhead & Monk, 1981)

  Reveals a misconception Expected pull: strong

  Writes the comparison backwards word for word, doubling Ken's new count, which would mean Jeff ends with twice as many games as Ken.

How the questions are analyzed, and the research behind it

Why it is hard

There is little arithmetic, but the relationships have to be read the right way round and chained. Students who think a larger count means a more valuable coin rank the coins backward, and students who stop after one or two links pick a coin that beats only its neighbor. Writing each coin's worth in lorgs, 4 for a sind, 10 for a harp, 12 for a dalt and 30 for a plunk, makes the ranking plain.

1. Turning words into math moderate Understanding the question

   In "2 plunks = 5 dalts" the coin with the larger count is worth less, the reverse of what the bigger number suggests, and the same holds in all four equations. (Hegarty, Mayer & Monk, 1995; Koedinger & Nathan, 2004)

2. Much to keep track of moderate Working it out

   Five coins and four exchange rates have to be linked into one ranking; dalts connect only through plunks, and lorgs only through sinds. (Sweller, 1988; Carpenter, Just & Shell, 1990)

3. No obvious method minor Working it out

   Nothing says how to compare coins that never appear in the same equation; converting every coin to lorgs, or chaining inequalities, has to be chosen. (Schoenfeld, 1985)

Strategy: Rewrite every coin in terms of lorgs, which only ever appear as the smaller unit, following the rates one link at a time, then compare.

Traps

• Equation written backwards: Writes a comparison the wrong way round, word for word: "six times as many students as professors" as 6S = P. The bigger quantity gets the bigger number.
• Stops one step early: A choice is a number from partway through the solution.

The wrong answers, classified

• A Equation written backwards (Clement, Lochhead & Monk, 1981)

  Reveals a misconception Expected pull: moderate

  Reads every equation backward, giving more value to the coin with the larger count, so 5 sinds = 2 harps puts sinds above harps and 1 plunk = 3 harps puts harps above plunks.

• B Equation written backwards (Clement, Lochhead & Monk, 1981)

  Reveals a misconception Expected pull: strong

  Reads 2 plunks = 5 dalts backward, giving the bigger value to the coin with the bigger count, so the dalt seems to outrank the plunk.

• C Stops one step early (King et al., 2004; Gierl et al., 2017)

  Reveals partial knowledge Expected pull: moderate

  Sees from 5 sinds = 2 harps that a harp beats a sind and stops, missing the last line, 1 plunk = 3 harps.

How the questions are analyzed, and the research behind it

Why it is hard

The question rewards a strategy, not a calculation: once the shared value is named, each variable follows in one line and R is clearly largest. Without that step, students judge by how the expressions look, and the look is backwards, since the 3 in 3N makes N small and the 7 under R makes R large. The positivity condition at the end is easy to skip but is what fixes the order.

1. Variables instead of numbers major Understanding the question

   There are no numbers to compute: five variables are tied together in the one chain M = 3N = P/4 = Q + 5 = R/7, and each must be compared through the shared value. (Koedinger & Nathan, 2004; Enright, Morley & Sheehan, 2002)

2. No obvious method moderate Working it out

   Nothing says how to compare; the efficient move is to name the common value, or pick one such as 12, and rewrite each variable, which students must think of themselves. (Schoenfeld, 1985)

3. Much to keep track of minor Working it out

   Five relations and the condition > 0 must be held at once; the condition matters, since with a negative common value the order of M, N, P and R would reverse. (Sweller, 1988; Carpenter, Just & Shell, 1990)

Strategy: Call the shared value k, a positive number, and rewrite each variable as an expression in k; or substitute a convenient positive value such as 12 and compare the results.

Traps

• Equation written backwards: Writes a comparison the wrong way round, word for word: "six times as many students as professors" as 6S = P. The bigger quantity gets the bigger number.

The wrong answers, classified

• A Equation written backwards (Clement, Lochhead & Monk, 1981)

  Reveals a misconception Expected pull: weak

  Reads P/4 and R/7 as a quarter and a seventh of M, the reverse of the truth, so M looks largest of all.

• B Equation written backwards (Clement, Lochhead & Monk, 1981)

  Reveals a misconception Expected pull: strong

  Reads the 3 in 3N as making N three times as large, when 3N = M actually makes N one third of M.

• C Stops one step early (King et al., 2004; Gierl et al., 2017)

  Reveals partial knowledge Expected pull: moderate

  Correctly sees that P is four times M but stops there without checking R, which at seven times M is larger.

How the questions are analyzed, and the research behind it