Aren’t everyone answering the same questions? How can an exam be biased?

First of all, the bias we are discussing in this context is a Statistical Bias. This means that the exam is being influenced by factors that do not relate to what the exam writer intended to measure.

For example, if an exam is given only on Saturdays ( like the SHSAT was until recently ). And there were a group of students who absolutely couldn’t do anything on the weekend for religious reasons. Then the exam results may show a Statistical Bias against that group of students.

It’s important to understand. Statistical Bias IS NOT Intolerance. It’s often due to sloppiness by the test designers, but it’s quite common and usually it’s simply fixed when detected.

The SHSAT exam does a poorer job predicting high-performing students than 7th grade GPA. That’s a statistical fact. What this fact means is that we can choose students who are more likely to succeed if we use the state scores already available for every student.

Not using the better predictor is considered biased education policy against girls and Black/Latinx students because these students perform better when measured by GPA/State scores. The superior selector anyway.

Here are some ways the SHSAT exam may introduce bias…

1. The SHSAT is optional
   1. Students are informed of the exams at different rates.
   2. Students may have more responsibility on the weekend at different rates.
2. The SHSAT is Multiple-Choice
   1. There are decades of evidence showing gender biases in high-stakes multiple-choice exams.
   2. Girls who perform better than boys tend to earn a lower score on these exams. Even while being the academically superior student.
3. The SHSAT material isn’t taught in most schools
   1. Not all schools get to all SHSAT material before the student can take the exam. E.g. Algebra, etc.
   2. For instance, two students in the same public school class. In class, they’re both taught Math ratios after the SHSAT exam. Unfortunately, ratios came up on the exam. Student A‘s parents invested in a high-quality SHSAT prep program. The instructors taught Student A math ratios a few weeks before the exam. Guess who passes? Even as student B may be the superior student, she learns about the math concept just weeks to late to affect her SHSAT score.
   3. A student learning material before another obviously does not make that student any smarter or more deserving.
4. The Student self-image may affect SHSAT participation
   1. Some students who would perform well in specialized high schools do not take the SHSAT. Because they feel they’ll fail when they wouldn’t. These students see SHSAT schools as not for their in-group. Maybe due to low historic participation and other self-selecting factors.
   2. By using transparent, and more accurate multiple-measures, the student would receive the offer and then decide on their future.
5. SHSAT is affected by Testing Strategies
   1. Expensive test prep services coach students on being more practiced and savvy test takers. These students are not academically better than other students. Just more practiced.
   2. Simple advice, such as answering all questions because this is multiple-choice and there’s no penalty for guessing. Many 12 year-olds may forget this when they’re nervous. Also how to spot high-value questions, or starting in the middle of the exam. What topics almost always comes up, etc, etc.
   3. An exam that’s influenced by coaching is not measuring some inherent in the student.