SHSAT Sunset

Category: research

  • Pathways to an Elite Education Exploring Strategies to Diversify NYC’s Specialized High Schools (2015)

    This brief examines students’ pathways from middle school to matriculation at a specialized high school, and simulates the effects of various admissions criteria that have been proposed as alternatives to the current policy. Analyzing data from 2005 to 2013, we found that while the SHSAT is (by design) the most important factor determining who attends the specialized high schools, it is not the only factor. Many students—including many high-achieving students—do not take the SHSAT at all, and some of those offered admission decide to go to high school elsewhere.

    Even when comparing students with the same level of prior academic achievement (based on state tests), we noted disparities at each stage of the pathway into a specialized school. For example, among students with comparable prior achievement, girls and Latinos were less likely to take the SHSAT. And girls, Latino, and Black students who took the test were less likely to receive an offer of admission. These findings suggest that there are opportunities to increase access, even within the confines of SHSAT-based admissions. Interventions that ensure that well-qualified students sit for the SHSAT—and have adequate resources to prepare for it—could help make the specialized schools more diverse.

    https://steinhardt.nyu.edu/research_alliance/publications/pathways_to_an_elite_education

  • High Stakes, but Low Validity? A Case Study of Standardized Tests and Admissions into New York City Specialized High Schools

    This is a study of the admissions process at a select group of New York City public high schools. It offers the first detailed look at the admissions practices of this highly regarded and competitive group of schools, and also provides a window into the broader national debate about the use of standardized tests in school admissions. According to New York State law, admission to these schools must
    be based solely on an exam. The exam used is called the Specialized High Schools Admissions Test (SHSAT). This study makes use of the individual test results from 2005 and 2006.
    Several key findings emerge:
    1. The SHSAT has an unusual scoring feature that is not widely known,
    and may give an edge to those who have access to expensive test-prep tutors. Other reasonable scoring systems could be constructed that would yield different results for many students, and there is no
    evidence offered to support the validity of the current system.
    2. Thousands of students who are not being accepted have scores that are statistically indisti nguishable from thousands who are granted admission. And these estimates are de
    rived using the less precise, classical-test-theory-based measures of statistical uncertainty, which may understate the problem. The New York City Department of Education (NYCDOE) fails to provide the more accurate, item- response-theory-based estimates of the SHSAT’s standard error of measurement (SEM) near the admi
    ssion cutoff scores, which would offer a clearer picture of how well the test is able to differentiate among students who score close to the admi ssion/rejection line. This omission
    violates generally-accepted testi ng standards and practices.
    3. Students who receive certain versions of the test may be more likely to gain admission than students who receive other versions. No evidence is offered on how accurate the statistical equating of different test versions is. The mean scaled scores vary across versions much more than would be expected given the ch
    ance distribution of ability across large random samples of students, suggesting that the scoring system may not be completely eliminating differences among test versions.
    4. No studies have ever been done to see if the SHSAT is subject to
    prediction bias across gender and ethnic groups (i.e., if SHSAT scores
    predict things for different groups).

  • VALIDITY OF HIGH-SCHOOL GRADES IN PREDICTING STUDENT SUCCESS BEYOND THE FRESHMAN YEAR: High-School Record vs. Standardized Tests as Indicators of Four-Year College Outcomes

    High-school grades are often viewed as an unreliable criterion for college admissions, owing to differences in grading standards across high schools, while standardized tests are seen as methodologically rigorous, providing a more uniform and valid yardstick for assessing student ability and achievement. The present study challenges that conventional view. The study finds that high-school grade point average (HSGPA) is consistently the best predictor not only of freshman grades in college, the outcome indicator most often employed in predictive-validity studies, but of four-year college outcomes as well.

    A previous study, UC and the SAT (Geiser with Studley, 2003), demonstrated that HSGPA in college-preparatory courses was the best predictor of freshman grades for a sample of almost 80,000 students admitted to the University of California. Because freshman grades provide only a short-term indicator of college performance, the present study tracked four-year college outcomes, including cumulative college grades and graduation, for the same sample in order to examine the relative contribution of high-school record and standardized tests in predicting longerterm college performance.

    Key findings are: (1) HSGPA is consistently the strongest predictor of four-year college outcomes for all academic disciplines, campuses and freshman cohorts in the UC sample; (2) surprisingly, the predictive weight associated with HSGPA increases after the freshman year, accounting for a greater proportion of variance in cumulative fourth-year than first-year college grades; and (3) as anadmissions criterion, HSGPA has less adverse impact than standardized tests on disadvantaged and underrepresented minority students. The paper concludes with adiscussion of the implications of these findings for admissions policy and argues forgreater emphasis on the high-school record, and a corresponding de-emphasis on standardized tests, in college admissions.

    https://files.eric.ed.gov/fulltext/ED502858.pdf

  • Why Gifted and Talented Schools are the Wrong Approach: To diversify schools, reimagine G&T: A bill to expand segregated programs moves in exactly the wrong direction

    But we’ve already tried this, and it didn’t work. Back in 2009, Mayor Bloomberg tried to expand gifted programs and switched from multiple measures to a single test score for gifted admission. The result was actually more segregation, and reduced access for black and Latino students: The percentage of black and Latino students entering such programs in kindergarten was cut in half, from 46% of program entrants to just 22%, while the percentage of white and Asian students climbed from 53% to over 70%.

    […]

    Furthermore, the city should improve its strategy for serving students once they are in a gifted program. G&T program proponents often refer to them as a form of special education — but this ignores that research on special education overwhelmingly points to the benefits of placing students of all types in regular classrooms as much as possible.

    The city’s approach to serving students with disabilities reflects these best practices: Many schools across the city offer integrated co-teaching programs where students with disabilities are educated alongside their general education peers with special education support.

    http://www.nydailynews.com/opinion/ny-oped-to-diversify-schools-reimagine-gt-20180803-story.html

  • Exam High Schools and Academic Achievement: Evidence from New York City

    Publicly funded exam schools educate many of the world’s most talented students. These schools typically contain higher achieving peers, more rigorous instruction, and additional resources compared to regular public schools. This paper uses a sharp discontinuity in the admissions process at three prominent exam schools in New York City to provide the first causal estimate of the impact of attending an exam school in the United States on longer term academic outcomes. Attending an exam school increases the rigor of high school courses taken and the probability that a student graduates with an advanced high school degree. Surprisingly, however, attending an exam school has little impact on Scholastic Aptitude Test scores, college enrollment, or college graduation — casting doubt on their ultimate long term impact.

    This was done in 2011, but still important peer-reviewed research.

    http://www.nber.org/papers/w17286

  • SHSAT Invalid: I’ve spent years studying the link between SHSAT scores and student success. The test doesn’t tell you as much as you might think.

    First, that requires defining merit. Only New York City defines it as the score on a single test — other cities’ selective high schools use multiple measures, as do top colleges. There are certainly other potential criteria, such as artistic achievement or citizenship.

    However, when merit is defined as achievement in school, the question of whether the test is meritocratic is an empirical question that can be answered with data.

    https://www.chalkbeat.org/posts/ny/2018/06/22/ive-spent-years-studying-the-link-between-shsat-scores-and-student-success-the-test-doesnt-tell-you-as-much-as-you-might-think/

  • Bronx High School of Science Accused of Bias in Admissions

    A Manhattan community school board charged yesterday that an admissions policy based on entrance examinations made the Bronx High School of Science the most segregated school in the city.

    Members of the board, of District 3 on the West Side, said they would meet with lawyers on Monday to plan legal action aimed at nullifying the entrance examinations given at the school this week.

    The community board and its superintendent, Alfredo Mathew Jr., said the Bronx High School of Science was a privileged educational center for children of the white middle class because “culturally” oriented examinations worked to “screen out” black and Puerto Rican students who could succeed at the school. They said that 90 percent of the school’s 3,200 students were white.

    https://www.nytimes.com/1971/01/22/archives/bronx-high-school-of-science-accused-of-bias-in-admissions.html