Mistakes on Standardized Tests, Teachers Ordered Not To Alert Students Unless They Ask

The state of New York makes a baffling, unfair decision in response to an error on a standardized test it’s administering starting today:

On the eighth grade test, one question had no correct answer, and schools are instructed to alert students.

And on the fourth grade exam, one question has two correct answers. But in this case, schools are directed to tell students about the problem only if they ask questions about the item.

Elizabeth Phillips, principal of Brooklyn Public School 321, called it “completely unfair” not to warn fourth-graders.

“That means that in some rooms, where a child asks, the children will be at an advantage as they will know that they don’t have to keep deliberating, while in other classrooms, where students don’t ask, some are likely to waste a lot of time on this question,” she said.

Read more.

Your Thoughts?

About Daniel Fincke

Dr. Daniel Fincke  has his PhD in philosophy from Fordham University and spent 11 years teaching in college classrooms. He wrote his dissertation on Ethics and the philosophy of Friedrich Nietzsche. On Camels With Hammers, the careful philosophy blog he writes for a popular audience, Dan argues for atheism and develops a humanistic ethical theory he calls “Empowerment Ethics”. Dan also teaches affordable, non-matriculated, video-conferencing philosophy classes on ethics, Nietzsche, historical philosophy, and philosophy for atheists that anyone around the world can sign up for. (You can learn more about Dan’s online classes here.) Dan is an APPA  (American Philosophical Practitioners Association) certified philosophical counselor who offers philosophical advice services to help people work through the philosophical aspects of their practical problems or to work out their views on philosophical issues. (You can read examples of Dan’s advice here.) Through his blogging, his online teaching, and his philosophical advice services each, Dan specializes in helping people who have recently left a religious tradition work out their constructive answers to questions of ethics, metaphysics, the meaning of life, etc. as part of their process of radical worldview change.

  • John Morales

    Elizabeth Phillips, principal of Brooklyn Public School 321, called it “completely unfair” not to warn fourth-graders.

    “That means that in some rooms, where a child asks, the children will be at an advantage as they will know that they don’t have to keep deliberating, while in other classrooms, where students don’t ask, some are likely to waste a lot of time on this question,” she said.

    Surely the whole point of testing is to determine who can pass — all tests are evidently “completely unfair” to those who will fail them.

    (That they may be poorly designed just means they’re not testing for what they think it’s testing, but a test it remains)

    • http://freethoughtblogs.com/camelswithhammers Daniel Fincke

      What an absurd definition of unfair that is, John. It’s completely fair that someone who does not know the material get a corresponding grade that indicates they did not know the material. Any test is fair which leads to scores that correlate with knowledge of what is being tested.

    • http://nojesusnopeas.blogspot.com James Sweet

      Yeah, this is just equivocation, and a rather inane bit of it at that. Clearly, what “fair” means in this context is that your passing/failing is primarily determined by what the test claims that it is measuring.

      Certainly if the test claimed that it was measuring whether students had the courage and outside-the-box thinking to bring up the possibility of an error on the test, then the test would be quite “fair” by this definition. But since that is not supposed to be what is being tested here, we would have to call it “unfair”.

    • John Morales

      Yeah, this is just equivocation, and a rather inane bit of it at that.

      In your opinion.

      Clearly, what “fair” means in this context is that your passing/failing is primarily determined by what the test claims that it is measuring.

      By that criterion, it remains fair both with and without the flaws.

      Certainly if the test claimed that it was measuring whether students had the courage and outside-the-box thinking to bring up the possibility of an error on the test, then the test would be quite “fair” by this definition. But since that is not supposed to be what is being tested here, we would have to call it “unfair”.

      I note the test meets your criterion for fairness (as noted in the first quotation) yet you claim it unfair (as noted in the second quotation).

    • John Morales

      Daniel:

      [1] What an absurd definition of unfair that is, John. [2] It’s completely fair that someone who does not know the material get a corresponding grade that indicates they did not know the material. [3] Any test is fair which leads to scores that correlate with knowledge of what is being tested.

      1. It relates to the test testing.

      2. This will still be the case for this test, even with its flaws.

      3. This also will still be the case for this test, even with its flaws.

      (Your point?)

  • http://twitter.com/nicoleintrovert Nicole

    I think the principal is correct. In the 4th grade I would have probably sat there anxious over the damn question for 10 minutes before asking. (If I even had the courage to ask at all not wanting to seem like an idiot). Wasting a lot of time and mental energy that would be better spent on the remainder of the test.

    What would be wrong with announcing “Question number 8 has two correct answers. I will not give you which ones they are, but if you pick either one of the correct answers you will be given credit.”

    • John Morales

      Test-taking is its own competence, and learning to prioritise is important.

  • unbound

    What would be wrong with announcing “Question number 8 has two correct answers. I will not give you which ones they are, but if you pick either one of the correct answers you will be given credit.”

    I agree completely, and that is the approach being taken with the 8th graders. I have no idea why they same approach would not be applied to the 4th grade test.

    Alternatively, just have the students skip the question. Does it really matter if there is one less question on the test than was planned? 4th grade is a bit young to messing with kids.

  • Kierra

    So if I 4th grader asks about the question, the whole room is told about the mistake? or is just that student told? Telling the whole room seems really unfair, as it means that a given student’s knowledge of the mistake will be based on what other students do. Telling just the student(s) who asks seems slightly more fair. On the other hand, I recall that asking questions during standardized tests was generally discouraged, as in you usually got the standard “just answer the question as best you can” response rather than useful information. So I would worry that students that notice the problem wouldn’t bother to ask, but would continue to debate the question.

    I would assume that their logic in not telling the students beforehand is that many students are going to see a correct answer, choose that one, and move on without reading the other options. But it still seems weird not to just tell the students.

    • https://twitter.com/nicoleintrovert Nicole

      You are right, as I also recall asking questions during a standardized test was very frowned upon. It was not the same environment as a regular old weekly vocaublary test. More than likely any question asked received a non-answer.

      Exactly why I said 4th grader me would have sat there for at least 10 minutes lamenting before attempting to muster up the courage to ask.

  • http://ogremk5.wordpress.com ogremk5

    As a professional in the assessment industry, I can tell you regardless of what the teachers say or do, that item will be ignored in the statistical analysis of the exam. And it will be done so for exactly the reason mentioned… because it’s skewing the data and no-one will really know how the data is skewed.

    We do something called a key-check, where we get a content specialist who has never seen the test or the items on that test to take the test… as if they were really taking it.

    Then they compare their answers to the ‘correct’ answers and provide a rationale or evidence to support their answer, if different from the listed answer.

    When that happens, a senior reviewer examines the question and everyone’s comments and decides if there is an issue with the item or not. If so, then item is removed from the stats for the test.

    So, again, if there is a problem that was discovered after the test book had been printed, then that item will not be counted in any way, shape, or form.

    Finally, if that item is a field test item (which is probably is), then it’s not counted on the test anyway. The data from that item will show skewed results and the item will be tossed out because of that.

  • R Johnston

    This reminds me of when I took the bar exam. On one of the multiple choice questions the Supreme Court had issued a ruling changing the law in the weeks before the exam. The change wouldn’t have been in anyone’s study materials and almost certainly happened after the exam was written, but it was a significant enough matter to make the news and lots or most of the people taking the exam would have heard about it (I don’t remember the specific issue now). The before and after state of the law both appeared as multiple choice answers. There was no announcement made either regarding the specific question or more generally how changes of law over recent weeks would affect scoring.

    I didn’t worry about the question too much because I knew I’d done fine with the rest of the multiple choice, but it would have driven me batty if I had thought I was borderline to pass.

  • http://replacinggod.blogspot.com/ Fil Salustri

    I saw the problem as relating to the inconsistency between the two classes, not among the students themselves. Within each of the two classes in isolation, the students appear to have been treated the same – and thus fairly. But the inconsistency of policy regarding announcements of errors _between_ the two classes might be considered “unfair.” (I’d suggest that “inconsistent” would have been a better word to use anyways.)

    Then there’s the matter of whether students were, in part, being trained to deal with problematic problems. If they were, then they’d know that the questions _might_ be wrong. IE: we tend to assume test questions were devised by a “perfect reasoner,” which is not a good assumption. Perhaps these students were too young to have had that in their curriculum.

    There might have been some politics too. The 8th grade students would have been off to middle school, and so their performance may have reflected more on the elementary school?

  • http://naaaaaaa..... Ed Oleen

    Well, this is one of the reasons we have a free press. Given that almost all of the children who take the test will be exposed to TV, hopefully the news as well as the kiddy-junk, the news casts could certainly devote a few seconds to a warning about the exam. Not the actual questions, of course, but telling fourth graders that the problem exists, and that they should ASK at the beginning of the exam.

    And if the ed racket objects, fuck-em…


CLOSE | X

HIDE | X