faq

Short answer: A policy being race-neutral does not preclude it from being racist.

Long answer: A race-neutral policy is one that does not take race directly into account. And yes, the SHSAT exam is race-neutral. Also, for a long time now our courts have strongly preferred race-neutral approaches to diversity through a long list of rulings.

A race-neutral approach to diversity typically uses a race proxy to diversify its student population. E.g. recently, the state of Connecticut switch from race quotas to considering a myriad of non-racial student attributes for admissions. Attributes such as family-income and other socio-income factors. The effects of these new race-neutral admissions criteria are expected to be the very similar to race-quotas.

It stands to reason that if we can use race-neutral policies to increase diversity, that we can also use different race-neutral policies to DECREASE diversity.

The trick in both cases is to find an effective race proxy. In the SHSAT Exam’s case, the proxy is exposure to high-quality exam specific preparation. Whether that prep is available at school, at home or through a third party for thousands of dollars per month.

But it’s also important to remember that exam-specific prep ( aka “teaching to the test” ) does not improve the student’s academic ability, which is what is supposed to be predicted with the SHSAT. In the end, the Black and Latino students left behind are often 1-2 multiple-choice questions lagging behind the prepped students.

Specialized high schools are demographically wealthier than almost all other NYC public schools. In other words, the SHSAT exam has found a way to favor wealthier students over poorer students.

But the mayor’s 2018 proposal statistically increases the number of poor students at specialized high schools.

According to DoE data, out of NYC’s over 500 public high schools, specialized high schools ALL fall in the top 92% ( percentile ) for wealth.

Further, the NYC non-partisan IBO states…

Students in the specialized high schools came from census tracts where the median household income averaged $62,457 compared with $46,392 for students in other high schools. (All dollar amounts are reported in 2012 dollars).

If we rank the census tracts by their median income and then divide the tracts into equal fifths (quintiles), we observe large differences between the share of students in specialized high schools and other high schools from each quintile.

NYC IBO Report on Specialized high schools

Source: https://ibo.nyc.ny.us/cgi-park2/2015/04/do-a-larger-share-of-students-attending-the-citys-specialized-high-schools-live-in-neighborhoods-with-higher-median-incomes-than-those-attending-the-citys-other-high-schools/

It’s important to realize that New York families are not very wealthy in aggregate.

With test prep ranching from $1,000 to $5,000 do we really want to continue a system that just about forces poor families to invest sometimes 5-10% of their annual income into test-prep for a single exam?

This one seems counter-intuitive. That GPAs from many different schools tend to predict student success much better than a multiple-choice exam. After all, students take the same exam.

Multiple papers on the SHSAT put the exam’s accuracy at about 20%. This means that a middle school student’s score accounted for about 20% of their high school outcomes. But GPA and state scores have a greater than 40% accuracy.

The problem at least seems to be that we under-estimate the variation in a single seating sample of a student’s academic ability. Compared to our over-estimation of how different the top students are.

But we have decades of research showing that this really is the case. It’s also easy to see when we look at college students then compare their past GPAs along with their past high-stakes test scores.

Although they both correlate with success, the successful students’ GPAs MORE accurately predicted their academic success.

Research

Note that much of the research involves high-school GPA predicting college success because the idea of using high-stakes testing in middle school to get INTO high school is pretty much unheard of in most of the nation.

1.Predicting College Success with High School Grades and Test Scores: Limitations for Minority Students also ( researchgate )

2.VALIDITY OF HIGH-SCHOOL GRADES IN PREDICTING STUDENT SUCCESS BEYOND THE FRESHMAN YEAR: High-School Record vs. Standardized Tests as Indicators of Four-Year College Outcomes* ( PDF )

3. I’ve spent years studying the link between SHSAT scores and student success. The test doesn’t tell you as much as you might think.

4. What Predicts College Completion? High School GPA Beats SAT Score

5. GRE fails to identify successful Ph.D. students

6. When It Comes to College, High School Grades Reveal More Than Just Academics

7. GPA Versus Exam Scores: What’s Better in Predicting College Success?

8. GRE fails to identify successful Ph.D. students

9. It’s GPAs Not Standardized Tests That Predict College Success

There’s the common argument that students who get SHSAT offers have worked very hard, hence it’s unfair to replace the SHSAT exam.

But that argument ignores the fact that students who excel throughout the entire year at school to be in the top 7% of their entire school probably worked just as hard or even harder. And to be considered for the current SHSAT proposal, these high-achieving students would have also worked hard to excel in their state scores.

Regardless of what measure we use for determining a student’s academic merit, these students would have worked very hard. And I’d argue that maintaining a top 7% rank at school is harder than passing a single 3-hr multiple-choice exam.

Aren’t everyone answering the same questions? How can an exam be biased?

First of all, the bias we are discussing in this context is a Statistical Bias. This means that the exam is being influenced by factors that do not relate to what the exam writer intended to measure.

For example, if an exam is given only on Saturdays ( like the SHSAT was until recently ). And there were a group of students who absolutely couldn’t do anything on the weekend for religious reasons. Then the exam results may show a Statistical Bias against that group of students.

It’s important to understand. Statistical Bias IS NOT Intolerance. It’s often due to sloppiness by the test designers, but it’s quite common and usually it’s simply fixed when detected.

The SHSAT exam does a poorer job predicting high-performing students than 7th grade GPA. That’s a statistical fact. What this fact means is that we can choose students who are more likely to succeed if we use the state scores already available for every student.

Not using the better predictor is considered biased education policy against girls and Black/Latinx students because these students perform better when measured by GPA/State scores. The superior selector anyway.

Here are some ways the SHSAT exam may introduce bias…

1. The SHSAT is optional
   1. Students are informed of the exams at different rates.
   2. Students may have more responsibility on the weekend at different rates.
2. The SHSAT is Multiple-Choice
   1. There are decades of evidence showing gender biases in high-stakes multiple-choice exams.
   2. Girls who perform better than boys tend to earn a lower score on these exams. Even while being the academically superior student.
3. The SHSAT material isn’t taught in most schools
   1. Not all schools get to all SHSAT material before the student can take the exam. E.g. Algebra, etc.
   2. For instance, two students in the same public school class. In class, they’re both taught Math ratios after the SHSAT exam. Unfortunately, ratios came up on the exam. Student A‘s parents invested in a high-quality SHSAT prep program. The instructors taught Student A math ratios a few weeks before the exam. Guess who passes? Even as student B may be the superior student, she learns about the math concept just weeks to late to affect her SHSAT score.
   3. A student learning material before another obviously does not make that student any smarter or more deserving.
4. The Student self-image may affect SHSAT participation
   1. Some students who would perform well in specialized high schools do not take the SHSAT. Because they feel they’ll fail when they wouldn’t. These students see SHSAT schools as not for their in-group. Maybe due to low historic participation and other self-selecting factors.
   2. By using transparent, and more accurate multiple-measures, the student would receive the offer and then decide on their future.
5. SHSAT is affected by Testing Strategies
   1. Expensive test prep services coach students on being more practiced and savvy test takers. These students are not academically better than other students. Just more practiced.
   2. Simple advice, such as answering all questions because this is multiple-choice and there’s no penalty for guessing. Many 12 year-olds may forget this when they’re nervous. Also how to spot high-value questions, or starting in the middle of the exam. What topics almost always comes up, etc, etc.
   3. An exam that’s influenced by coaching is not measuring some inherent in the student.

There’s a common argument against replacing the SHSAT with multiple-measures.  It goes something to the effect, “We concede this test is a problem, but why focus on the test when we have bigger issues?“.

Not all Black or Hispanic students need the same thing

Just as not all White or Asian students need the same thing.  We’re fighting for fair measurement of merit because we know there are many students out there earning the grades for Specialized High Schools but not getting SHSAT offers.  Mainly because they didn’t study for a single multiple-choice test.

These Black students are not the same students that are scoring below proficiency in state tests. 

This Argument is an “Appeal to Worse”  

An Appeal to the Worse suggests we fix unrelated problems before we can consider fixing a relatively smaller issue.  Even as both these issues are separate and important.  

We can fix the Specialized High School entrance to include multiple-measures without fixing anything else.  There doesn’t need to be any prioritization or competition.  Especially if fixing the SHSAT does not compete with resources to fix other issues, which it doesn’t.

We absolutely should fix rigor in public schools.  Giving every classroom access to improved rigorous instruction.  It’s not ok that 70% of Black students in NY public schools are not proficient.  But that statistic says nothing of the roughly 30% of Black students that are proficient. 

In their ranks, there are many students who do get the grades for Specialized High Schools but are not considered because they didn’t focus on a single exam.

E.g. This student’s account: “I was the valedictorian of my eighth-grade class and earned a special honor for never missing a day of school…“

Why try to “fix” the SHSAT?  Isn’t the issue of the fact that only about 30% of Black students read at grade-level?

The above question is a common, yet incorrect argument.  That some demographics “in aggregate” are not prepared enough to do well at specialized high schools.  Hence the test weeds them out.

The argument is confusing descriptive statistics that describe demographics, with an individual student’s potential.  In other words, it fails to consider that within that 30% of black students who perform at grade level, the top percentiles perform comparatively to the top percentiles of the roughly 60% Asian and White students who perform at or above grade level.

It’s a stretch, but with all else equal there should be twice as many Asian students at Stuyvesant than Black students, taking proficiency into account alone.  But instead, we have close to SIXTY times.  And that’s 2x versus 60x.

The use of the SHSAT is an access problem and a rigor problem.  It’s not just a rigor problem, as the original question implies.  The SHSAT exam limits individual high-performing students’ access to highly sort after resources.  In other words, even a school that has never sent a student to any specialized high schools may unknowingly have a student that deserves admittance to Stuyvesant.  But without specific attention to SHSAT prep, that child will never make it there.