August 08, 2003
Does not compute

The designers of the new Texas Assessment of Knowledge and Skills (TAKS) test have admitted that they screwed up on one of the 10th grade math tests, though it appears from their explanation that they still don't fully grasp the nature of their screwup. Here's the problem they presented, which was to find the perimeter of the regular octagon shown:

As the Texas Education Association states in its press release, they thought that students would use the Pythagorean Theorem to solve the problem, which yields an answer of 36 cm. However, if a student noted that the angle from the center of the octagon is 45 degrees, then using the Law of Sines would have led to an answer of 27 cm. Given that, they gave everyone credit for the question.

The real problem, as the various mathematicians and engineers quoted in the story note, is that the problem is impossible on its face. Here's why:

In any regular polygon, the sum of the exterior angles is always 360 degrees. For an octagon, that means each exterior angle is 45 degrees, which in turn means that each interior angle is 135 degrees.

In the triangle drawn within the octagon, the two sides that go from the center of the octagon to one of the interior angles bisects that angle. That means that the angle formed by the bottom segment of the octagon (whose length we need to calculate in order to find the perimeter) and the line whose length is given as 4.6 cm is 67.5 degrees.

(Even if you didn't know all that, if you observe that the angle from the center of the octagon to the two corners as drawn must be 45 degrees, then the other two angles in the isoceles triangle must be 67.5 degrees each.)

Now then. In a right triangle, one can calculate the sines of its angles by the formula sine = opposite/hypotenuse. (Cosine is adjacent/hypotenuse and tangent is opposite/adjacent, leading to the mnemonic SOHCAHTOA.) In the drawing, that would make the sine of 67.5 degrees about 0.870. Unfortunately, the angle whose sine is 0.87 is about 60.4 degrees. The actual sine of 67.5 degrees is about 0.923. In other words, a right triangle with a 67.5 degree angle cannot have a side measuring 4 cm and a hypotenuse measuring 4.6 cm. It's impossible.

Shame on the TEA for getting this wrong. They claim that "Each test item goes through a rigorous review process that includes a field test of the items and two separate review sessions by professional educators who have subject-area and grade-level expertise and who are recommended by their district", and they say that this sort of error has "rarely been used" in the history of these tests, but this is still inexcusable.

As it happens, I'm not reflexively opposed to standardized testing. I think standardized tests are a useful tool that can help to objectively evaluate how much a student has really learned. What I do object to is the amount of emphasis being put on standardized tests. I believe they should be a means to an end, and not an end in and of themselves. I won't indict an entire system based on one case of quality control failure, but given what's at stake for the students, the least the TEA can do is make damn sure this sort of thing doesn't happen again.

Finally, if you have a few minutes to kill, check out this page, which has over 40 different proofs of the Pythagorean Theorem, including links to pages that have animated demonstrations. Did you know that President James Garfield published an original proof of the Pythagorean Theorem in 1876? Corollary jokes about our current President are left as an exercise for the reader.

UPDATE: Mike T speaks from experience working for a textbook publishing company in the comments. He doesn't think much of the TEA's claims of "rigorous review".

Posted by Charles Kuffner on August 08, 2003 to Technology, science, and math | TrackBack

Color me shocked. No really. Then again, I'm a bit burned on standardized testing lately. Pretty much since the moment I realized that the "No Child Left Behind" act is at least partially responsible for my wife's difficulty finding a teaching position.

You wouldn't know any district that needs a 4-8 Generalist with a heavy math background, would ya? :)

Posted by: Morat on August 8, 2003 10:58 AM

Hmm. What if the drawing made it clear that the intersection of the two lines was not the center? Is it possible to draw the figure with the sides they propose?

Let's see, given the length of a side, the distance across the octagon from the bottom to the top should be (side)/sqrt(2) +(side)+(side)/sqrt(2). The side is 4.5cm according to the state and pythagoras, so that's about 11.

So, it could be a problem in drawing the figure.

Posted by: Michael on August 8, 2003 12:00 PM

Morat, What exactly about LNCB puts your wife out of a job?

Posted by: Tx Bubba on August 8, 2003 12:25 PM

Doesn't put her out of a job, not exactly. The local districts are interpreting certain requirements in a way that makes it more difficult for those undergoing alternative certification programs to get a job.

Basically, my wife got a degree in education. She didn't go into teaching for a number of years, however, because she got better pay and benefits as a secretary. So now that we're in a place where she can teach, she had to go through an alternative certification program.

She took passed her Excet tests, but isn't fully certified until she completes her year long intership. (In her case, it's "teaching for a year, and spending two or three saturdays in boring classes). She's got three years to do it.

The way most of the Region IV schools are viewing the act (beats me which. I've just heard this from three seperate administrators) as more or less requiring you hire certified teachers before uncertified teachers, even if the uncertified teachers have a more relevant degree or even more experience. *shrug*.

I don't pretend to understand exactly what they're talking about, but the people who tried to explain it were in a position to know about hiring practices for their district, and didn't have any reason to lie to me (they weren't trying to hire my wife).

Ironically, my wife might end up teaching kindergarten for a year. It won't count towards her certification, but it gets her hired...which makes transferring to 4-8 position easier. Basically, schools always underestimate the number of students, and each district ends up frantically and desperately looking for teachers the first few weeks of school.

Still, I'd prefer she was already hired....

Posted by: Morat on August 8, 2003 2:24 PM

Let me lend my voice to the anti-testing crowd. I spent two years working as a math-website coder for a major textbook publishing company, and the Q&A was atrocious. We developed standardized tests for students, and I was the only person in my group with any significant mathematical background. By 'significant mathematical background' I mean that I took honors level high-school math at a prestigious prep school, up to and including calculus. Given that I was twelve years removed from high school at the time, I was regularly confronted with tasks that I was ill-suited to perform. In addition, I wasn't even a full-time employee of the pubco, merely an independent contractor working from home in my underwear, and I was making critical decisions about what content to include in online exams. Remember, I was the MOST EXPERIENCED MEMBER OF MY TEAM. We were able HTML-coders with a ton of experience in that niche, but the bulk of my team had absolutely no understanding of the mathematical concepts we were working with. After I left, I learned that the company began outsourcing much of our work to Russian programmers to cut down on expenses. I believe their mathematical skills were sharper, but from discussions with a friend that still works there I've learned that the program now suffers from major language translation problems. In short, I have little confidence that the product they develop is error-free, and I shudder to think that this product could be used to determine whether a student passes or fails.

Posted by: Mike T on August 8, 2003 7:15 PM

The first Michael makes the key point: nowhere dose the problem state that the tope vertex of the triangle is the center of the hexagon. Therefore, there is no basis to assume that the angle involved is 45 degrees, and the law of sines cannot be applied.

In fact, since the two methods give different answers, that vertex can't be the center of the hexagon.

The (only) correct answer is 36 cm.

Posted by: Another Michael on August 11, 2003 2:03 AM

Another Michael, you are technically correct, but the problem tells the test-taker to look at the regular octagon "drawn below". Since the drawing is inpossible, the problem is wrong. Either that or you have to consider it a trick question.

Posted by: Laura on August 11, 2003 2:22 AM

The drawing and the test convey the following facts about the situation:

1) The hexagon is regular
2) The dotted line segment is perpendicular to one side and has length 4 cm
3) The hypotenuses of the two right triangles that ensue have length 4.6 cm

The top vertex is not at the center of the hexagon. (You can check this by printing the image and measuring for yourself -- it is a little closer to the bottom side that to the top side.)

One important aspect of geometry (or mathematics in general) is to distinguish between the things you know and assertions that still need to be proven. If you look a the picture and assume that the vertex is in the center, and then proceed on that basis to an incorrect answer, you should not get credit for the problem. Keeping what you know separate from what you think you know is part of doing mathematics.

The TEA should have just said that, but apparently whoever ruled on the appeal wasn't thinking very clearly. Maybe he/she would have gotten the wrong answer as well. :)

Posted by: Another Michael on August 11, 2003 7:39 AM

The drawing also conveys the following "facts":
The dotted line segment goes to the center of the image in the drawing.

One of the main disclaimers on the SAT that I always remembered is "figure are not drawn to any scale and are not accurate representations" or something like that. They always disclaimed that their figures were not accurate representations.

Something like that wasn't stated. They were told to look at the "regular octagon drawn below". They didn't state a disclaimer. It's their own damn fault for putting in a wrong question.

Posted by: Trickster Paean on August 11, 2003 11:36 AM

The drawing shows that the vertex is near the center of the octagon. Without an infinitely precise measuring tool (and who has one of those?), that's all you can infer.

By the way, if you take the diagram literally, there is a much more serious problem with it. That is not a drawing of an octagon at all! Why? Because the "sides" are drawn as long thin rectangles, about 1/2 mm wide. Everyone knows a line segment has zero width! (I'll bet you that the paper wasn't perfectly flat, either.)

Seriously, if the stated assumptions of a problem are inconsistent with an implied, unstated one, then the unstated one must be disacarded.

Posted by: Another Michael on August 11, 2003 6:02 PM

Sorry, one last thing. If the instruction had said, "consider a regular octagon with three segments meeting at it's center, as depicted below..." then I would agree that the description of the problem was inconsistent.

And to be clear, I'm not disagreeing with the critics of standardised testing, just the critics of this particular test question.

Posted by: Another Michael on August 11, 2003 6:13 PM

I would agree that it's nothing but an assumption that the triangle goes to the exact center. Given a choice between assumptions and stated facts, I'd go with stated facts. Then again, if it did not say it was NOT drawn to scale, and the question called for a rounded answer, the other answer is reasonable to some extent. One could assume that if it's reasonably close to center, the answer would be close enough when rounded, and if it were far off enough to change the answer, it would be perceptibly off center. If one considers the full ramifications, they prove that it was not drawn to scale, though, but that's beyond reasonable expectations for that type of test. It should not be up to the student to have to determine if the test is problematic. It's still a bad question, unless of course the whole test had a disclaimer about scale.

As for Another Michael, if the test had said, "consider a regular octagon with three segments meeting at it's center, as depicted below..." then I'd say that they have an even bigger problem with English than they do with math.

It's ironic that there are those who decry the lack of mathematics knowledge in this country, yet have not mastered the apostrophe by third grade. If we are going to complain about education, we need to get a lot more basic than math.

I don't mean that as a personal attack, but there is a real problem with education in this country and I would not rank the nuances of this problem up there with the more serious ones.

Posted by: Hagrinas on May 11, 2004 11:14 AM