Observations on the SAGE sample exam

Earlier this year I wrote a multi-part series on the PARCC test samples, picking at potential pitfalls and interface issues. This was done with the assumption this would be my state’s new test.

Then (allegedly) the price tag on the bid came in too high, and we went with the American Institutes for Research instead. They have a contract to administer the Smarter Balanced test (so for the part of the US doing that one, this should interest y’all) but the test we will be seeing is customized for Arizona, presumably out of their test banks. This is close to the situation in Utah, which has a sample of what they are calling the SAGE test. Since there is no Arizona sample yet I decided to try my hand at Utah’s.

I’d like to think my approach to PARCC was gently scolding, but there’s no way around it: this test is very bad. One friend’s comment after going through some problems: “I’m starting to think this is an undergrad psych experiment instead.”

Question #1 is straightforward. Question #2 is where the action starts to happen:

Adding a point on the line gets the r-value closer to 1, but with no information on the exact coordinate points (those are nowhere near the grid lines) or the original r-value I believe this problem is impossible as written.

Question #3 is fairly sedate although they screwed up by neglecting to specify they wanted positive answers; (17, 19) and (-17, -19) both work but the problem implies there is only one valid pair. I’d like to draw attention to the overkill of the interface, which includes pi and cube roots for some reason. There seem to be multiple “levels” to the numerical interface, with “digits and decimal point and negative sign” being the simplest all the way up to “including the arctan if for some reason you need that” but without much rhyme or reason to the complexity level for a particular problem.

Case in point:

The percents in the problem imply the answer will also be delivered as x%, but there is absolutely no way to type a percent symbol in the line (just typing % with the keyboard is unrecognized). So something like 51% would need to be typed as .51. Fractions are also unrecognized.

Here’s the Common Core standard:

CCSS.MATH.CONTENT.HSA.SSE.B.4
Derive the formula for the sum of a finite geometric series (when the common ratio is not 1), and use the formula to solve problems. For example, calculate mortgage payments.

I could linger on the bizarre conditional clause that makes this problem (*why* would one ever need to have a line with a y-intercept greater than one given in table form yet also perpendicular to some other particular line is beyond me) but instead I’ll point out the interface to the right, which is how all lines are drawn. (Just lines: there seems to be no way to draw parabolas and so forth like in the PARCC interface.) To add a line you click on “Add Arrow” (not intuitive naming) and click a starting point and an ending point. Notice that the line does not “complete” itself but rather hangs as an odd fragment on the graph. Also, fixing mistakes requires clicking “delete” and then the line, except if you click right on the line the points do not disappear so you have to repeat delete-click-delete-click on each of the points to clear everything out.

Oh, and the super-tiny cursor button is what you click if you want to move something around rather than delete and add. There was not enough room to have a button called “Move”?

First off, “objective function” is not a Common Core vocabulary word and linear programming is not in Common Core besides, at least not as presented in this question.

CCSS.MATH.CONTENT.HSA.REI.B.3
Solve linear equations and inequalities in one variable, including equations with coefficients represented by letters.

CCSS.MATH.CONTENT.HSA.REI.D.12
Graph the solutions to a linear inequality in two variables as a half-plane (excluding the boundary in the case of a strict inequality), and graph the solution set to a system of linear inequalities in two variables as the intersection of the corresponding half-planes.

Besides that, the grammar is very sloppy. It should say “the objective function z = -3x + 4y” without lumping it in a set of five statements where the student has to fish and presume the function being meant is the first line because it is the only one that includes all three variables in function form.

I include this problem only to indicate how wide the swerves in difficulty of this test are. First linear programming, then a simple definition, and then…

I came up with sqrt(2) and 2, but notice how the number line only accepts “0 1 2 3 4 5 6 7 8 9 . -” in the input. There is no way to indicate a square root.

Fractions are also right out, so one possible answer that does work (0.25 and .5) is very hard to get to. (I confess I was stumped and had to get hinted by a friend.)

I drove my eyes crazy trying to get the right numbers on the axis to match up, especially on Survey 1 which is not even placed against the number line. I thought the PARCC snap-to-half-grid was bad, but this is a floating snap-to-half grid which means it is very unclear if one has in fact aligned one’s graph with 6.7.

My average is going to be imaginary. The level of input that each problem allows is again quite erratic.

Incidentally, I found no “degrees” button which I guess means all arcsines and so forth are supposed to be in radians. (I was incidentally taught arcsin means the unrestricted inverse — that is, it is not a function and gives all possible answers — but they’re using it here to mean the function with restricted domain.)

This (very easy) geometry problem requires a very nasty number of clicks (I ended up using 12) for something that can be done by hand in 10 seconds. With practice I could do it in 30 but my first attempts involved misclicks. Couldn’t the student just place the point and that would be enough? Why is the label step necessary? How many points are deducted if the student forgets to drag C somewhere semi-close to the point? How close is close enough?

Since this is a small experimental probability set, I just made sure there were 10 trials. I do not believe this is what the test makers intended.

Is my letter “D” close enough? I could easily see this being accepted by a human but the parameters of the computer-grader are uncertain.

This question is extremely vague. What is considered acceptable here? Does it have to just look slightly bell-curvy? Since there is no axis label one could just claim the y-axis maximum is very high and the graph is normal distribution without clicking any squares at all.

First, note how it is an undocumented feature the arrows will “merge” to a point if they are on the same position. I was first confused by this problem because I had no idea how to draw it.

Also, notice how I’m having trouble here affecting a slope of 1 and -1 if I attempt to make the graph look “correct” by spanning the entire axis.

The correct side to shade is indicated by a single dot, which is puzzling and potentially confusing.

Their logic here is if the digits repeat, it is a rational number. It took me several read-throughs to discover that the first number does, in fact, repeat. By their same logic if I wrote

2.718281828…

it would be a repeating number, but of course it is e. The repeated digits should use a bar over them to reduce both the ambiguity and the scavenger-hunt-for-numbers quality of the problem as it stands.

I am fairly certain the Common Core intent is to only have linear inequality graphs, not absolute value:

CCSS.MATH.CONTENT.HSA.REI.D.12
Graph the solutions to a linear inequality in two variables as a half-plane (excluding the boundary in the case of a strict inequality), and graph the solution set to a system of linear inequalities in two variables as the intersection of the corresponding half-planes.

This standard could be stretched, perhaps

CCSS.MATH.CONTENT.HSA.REI.D.10
Understand that the graph of an equation in two variables is the set of all its solutions plotted in the coordinate plane, often forming a curve (which could be a line).

but the intent is for the overall conceptual understanding that graph = solutions, not permission to run wild with inequality graphs.

I am fairly certain this is computer-graded. I can think of many ways to phrase what I presume is the intended answer (“at the least either a or b has to be irrational”) but the statement is open enough other answers could work (“neither a nor b can be zero”).

…

It is possible I am in error on something, in which case I welcome corrections. (I don’t know Utah too well — it is possible Utah made some additions to the standards which null out a few of my objections.) Otherwise, I would plead with the companies working on these tests to please check them carefully for all the sorts of issues I am pointing out above.

Telling left from right

I had a discussion last week when reviewing slope that went like this:

Student: Wait, how can you tell if the slope is positive or negative just by looking?

Me: Well, if you imagine traveling on the line from left to right, if you’re moving up the slope is positive and moving down the slope is negative.

Student: …What?

Me: (points) So, starting over here … (slides hand) … and traveling this way … this slope is moving up. Starting over here … (slides hand) … this slope is moving down.

Student: But I don’t understand where you start.

Me: You start on the left.

Student: I’m still confused.

Me: (delayed enlightenment) Wait … can you tell your right from your left?

Student: No.

This isn’t the picture that was up at the time, but it’s in the same genre.

Left-right confusion (LRC) affects a reasonably large chunk of the population (the lowest estimate I’ve heard is 15%) but is one of those things teachers might be blissfully unaware is a real thing. (Note that LRC is at something of a continuum and affects women more than men.)

My own mother (who was a math teacher) has this problem, and has to use her ring finger whenever she needs to tell her right from her left. She reports that thinking about the graph as “reading a book” lets her get the slope direction correct.

Teaching the strange phrasing of technical mathematics

I’ve given my first geometry test this year (yes, I’m back on geometry, high five to all my geo-buds) and this is the first test I’ve given with the PARCC in mind.

Specifically, I made the phrasing match the technical language of PARCC questions, and I had my first experience with what happens when the students encounter something truly alien to them, like:

The point R is at (0, 3) and the point S is at (16, 7). Draw the line segment .

Find a point L on the line such that is four times as long as .

The first line didn’t go so bad. In a way students can plow through without reading it (draw a line segment? ok there must be points …. there they are!) but the second line had lots of bafflement.

It is a circumstance where vocabulary isn’t the issue, but phrasing is.

The issues seemed to be
a.) Not being clear where the question was; in this case it was directive to “find L”
b.) Students who got past the first hurdle were unclear in juggling the phrasing after; essentially the student brain seemed to go — first I find L, but to do that I need to worry about RS and RL, and somehow RS is — wait what?
c.) That rather than being told what to do (find the midpoint between X and Y) they had to hold a conditional in their head and fuss with a bit before they even understood they wanted a quarter-point to answer the problem.

This sort of conditional indirection seems to be common in PARCC questions: rather than being told what needs to be accomplished, be given some geometric object to assign THEN be told what will happen once that geometric object is in place THEN try to start unpiling what needs to be accomplished for those conditions to hold.

This sort of thing is routine in formal math texts but does not seem to be in the high school experience at all.

(From the PARCC sample Geometry test.)

Does anyone have experience teaching this sort of thing? How does one get students — both fluent and English language learners included — to read statements like the one above without blanking out?

Observations on the PARCC sample Geometry exam

Part 1: Observations on the PARCC sample Algebra I exam
Part 2: Observations on the PARCC sample Algebra II exam
Part 3: Observations on the PARCC sample Geometry exam

Calculator part: 18 of 25

Use the information provided in the animation to answer the questions about the geometric construction.

To pause the animation, select the animation window.

The students are supposed to watch a video of a construction and then say things about the proof enacted through the constructions. This is a very specific skill that needs to be practiced. Daniel Schneider kindly sent me a link to a website with a large number of construction animations (along with proofs) in case you need more to use in class.

However, there’s a serious interface problem. Here’s what the video looks like when paused, as well as a question to go with it:

Point “C” is completely covered. Whoops.

Non-calculator part: 6 of 7

This is one of those simple-looking questions which has enough of a trick to it I’m not sure how many students will get it right.

Part A requires students to work a double-completing-the-square manipulation, hopefully not getting sidetracked by the presence of b on the right hand side:

Part B requires noticing that a radius of 7 means the right hand side will be 49, so and thus .

In principle this problem is solvable, but the lack of partial credit on a problem with a “trick” that I worry a student who can normally complete the square would still get no points due to the indirection.

Calculator part: 13 of 25

This problem’s rough for three reasons:

a.) Even with the phrase “the pipe is open at both ends” placed in there, this is something of a background knowledge problem; the students need to know the “outer surface” excludes the circles on the top and bottom.

b.) There are volume formulas on the formula sheet but not surface area formulas. Thus the students need to have memorized or be able to extrapolate it, and know enough to exclude the circles.

c.) If is set to be 3.14, there answer comes out to be 1356.48. If is set to be 3.14159 or something with more digits (not unusual since graphing calculators have a “pi” button) the answer comes out to be roughly 1357.168. Rounding to the nearest integer thus can give either 1356 or 1357 as an answer.

Non-calculator part: 7 of 7

Out of all the problems on the PARCC final exam for geometry, 28% are related to transformations.

I can understand a transformational emphasis in general: it leads to a function transformation understanding of graphs (which is far more powerful and useful than looking at each kind of graph individually). However, why do so many of the dilation and rotation problem — 4 out of the 9 — involve centers not at the origin? This is not rhetorical; I really want to know where the utility is.

Non-calculator part: 20 of 25

This is one of the easier problems on the test, but assumes background the students don’t necessarily have. I can assume what a “collar” means here (even though I’ve never heard the word used in this context) but my ELL students are more likely interpret it as gibberish.

Calculator part: 10 of 25

This is very similar to the other problem in relying somewhat on background knowledge. Technically speaking one can ignore all the external stuff about merchant vessels and probes and focus on the math, but the brain of the ELL student doesn’t have an easy time removing the context.

Also note the weirdness of the rounding; in problem 20 the rounding needed to be done to the nearest tenth, while in this problem the rounding needs to be done to the nearest integer off the list.

Calculator part: 3 of 25

I’m noting this one because nothing in my current textbook (Carnegie Learning, written for Common Core) has anything resembling this kind of problem. Anyone have a source with problems that are similar?

Calculator part: 23 of 25

I don’t think I’ve ever give this much emphasis to the vocabulary of proof. Getting my students to keep the reflexive, symmetric, and transitive properties of congruence straight is going to be a nightmare and a half.

Ok, one last problem, from HS Sample Math Items, 7 of 10 (so not the final exam, but the open response part):

Here the angle bisector video returns (complete with unhelpful play button covering the diagram when paused) but the student is supposed to free-write a proof.

Here is how you type the first line as given:

1.) Pick “geometry” on the side and pick the short line in the upper right; that’s a “line segment” and will give you a blank box under a line segment so you can type letters.

2.) Type the letters you want under the line segment. If you accidentally type more than two letters any extra keypresses will be ignored.

3.) Go to “relations” above “geometry” and find the congruence symbol. Pick that. This will give the congruence symbol and a blank box.

4.) Pick the “line segment” and it will take the blank box that just appeared and put a line segment over it.

5.) Type the letters you need for the other line segment.

Now you have one step of the proof, now you just need to give a reason and then do four more steps.

(What would constitute a valid reason here, by the way? The mathopenref site I linked to early in this post just states “They were both drawn with the same compass width” — would this be considered valid by graders?)

Robot maze puzzle

The puzzle above has a robot (marked with an arrow pointing up, or “north”) that you can control with a set of command cards that either move the robot forwards a set number of spaces (the + numbers) or backwards (the – numbers). After using a command card, the card is used up and can’t be used again.

If the robot hits either the border of the grid or one of the black spaces, the robot stops moving and any remaining steps on the command card being used are ignored.

Landing on one of the spaces marked with circles causes your robot to turn 90 degrees. (That is, if the robot faces north it turns east, if east it turns south, if south it turns west, and if west it turns north.) The robot starts facing north. Can you get the robot to the star?

Observations on the PARCC sample Algebra I exam

Similar to my breakdown of the PARCC Algebra II, here’s some question-by-question comment on the sample PARCC Algebra I Final Exam. I’m not too discouraged by the actual items on this one, but the interface needs work.

This is the second problem of the test. You click points to set a line, then click “solution set” to shade in one side. Note (just like the Algebra II test) the points snap to half-grid points, not to grid points, a circumstance I find hazardous. Also, if you click on solution set to color a side, then realize your line was wrong, you have to click back on “line” again but the color goes away, so you have to add the color back again after the line is fixed. This is true even if switching the line from solid to dotted.

I should add this is one case where I see the superiority of open response to multiple choice. Here’s an inequality graphing problem from our old state test:

The lack of choices makes the problem a dotted-or-solid / above-or-below question where the actual shape of the graph is given away.

The question here is fine, but what if a student drags in the wrong number and wants to fix their answer? Removing a number only works if you drag the number back to the original number boxes, just “tossing” the number to a random position outside the answer box doesn’t work.

One common technique in the PARCC interface is for students to fill in sentences with a drag-down menu. By my eye, though, the interface doesn’t look much like a sentence, and I could imagine a student not understanding they are placing words between f(2) and g(2) and so forth to produce something that is meant to be read from left to right.

I guess 4 answer boxes — clear overkill — is better than the situation with 2 answer boxes where the suggestion seems strong to fill both of them even if one of the answers turns out to be extraneous.

I’m pretty sure logarithms aren’t supposed to be on the Algebra I test? Also, the graph is drawn automatically through the points, unless it can’t like in the example above. It took me a bit of deciphering to realize there’s an asymptote on there (right on the y-axis) and the asymptote can be slid around, so the reason the graph wasn’t showing up is the points were on opposite sides of the asymptote.

Do the blanks really have to be so large? I admit to getting confused because the symbols spread out in a single function looked to me like function-break-really small expression-break-random parenthesis and I had to do a double-take before I realized what was going on.

I hope students have their window large enough to realize (or least deduce from there being a “Part A”) that there is a “Part B” to the question.

There’s a truly weird option to change colors of things. Sometimes I can get it to trigger but I’m not sure how. The upper right inequality in pink shows what things look like after you’ve messed with the color.

There’s even an interface for systems of linear inequalities. Notice how there’s still a snap-to-half-grid feature even when the y-axis goes up by 5.

Why does one “find the zeros” question have a drag-and-drop interface, while this one gives a list?

Observations on the PARCC sample Algebra II exam

Fresh off the presses at

http://practice.parcc.testnav.com/

you can now find complete PARCC sample exams. Since it’s the one I most likely have to worry about next year, I tried out the Algebra II exam.

While I haven’t checked the other tests yet, the Algebra II has laid off the “show your work” type question. I’m going to put this in the positive column (I was deeply worried about open responses being graded by an army of interns; plus the turnaround needs to be very fast because this is technically the final for our Algebra II class. Imagine having to shave off an extra two months from the end to give time for the test to be graded). [ADD: According to this comment, this only applies to the final exam, and there are still “show your work” type questions on the other tests.]

Some question-by-question comments:

I admit I was baffled enough by this one to check Wolfram Alpha (I got m = -15/2 where m = 1 was extraneous), until I realized the key phrase “you may not need to use all of the answer boxes”. We’re going to have to specially train students for that trick, I think.

I tested this one on my students. The majority put the value for x which makes the statement true (1) not the extraneous solution (-3/4). I can understand the motivation here: forcing students to go through the factoring rather than just test numbers or even eyeball the thing, but putting the extraneous solution as an answer is rare and I can imagine a legion of students being confused by this.

I always considered the trick of rejiggering a number’s base (turning 4 into 2^2) to be something of a side trick; I’ve never had to use it outside a college algebra book.

Because saying “complete the square” would be too easy. In all seriousness, this is one of those problems I would have many students know how to do but get intimidated by the language of the question.

So the first 8 questions (which I was just quoting from) are in the non-calculator portion of the test. The remaining 26 are from the calculator portion. The students will have access to a TI-84 emulator.

I’ve got mixed feelings on this one. In a way it’s a neat bit of factoring (yank out the (x-y) terms, and what remains will be a difference of squares, tossing an extra (x-y) on the pile) but it also looks nothing like the kind of factoring from any textbook I’ve used; I’m fairly sure our Carnegie Learning book (which was written from scratch specifically for Common Core) has nothing like it.

The number of fancy widgets to enter answers has dropped considerably, but here’s one; you drag and drop in order.

Also noteworthy: there are two problems involving average rate of change. I think in our textbook there was enough material to squeeze out a single day. The PARCC writers must be putting rather more emphasis on it.

Here’s another widget: the dots are dragged around to form the graph. I was somewhat thrown by the dots not just snapping to the grid but also the halfway points. I believe this is a bad idea. A student could easily think they have a correct answer but leave the dot halfway rather than right on the appropriate point.

My students would be thrown off by the wording here. I think the intermediate step is actually helpful, but the phrase “Product of greatest common factor and binomial” looks so technical many of them would shut down.

I’m used to tests that make the statistics so easy students can answer them with no training whatsoever, but this is what I’d call a real statistics question.

This is a widget that lets you select an interval on the number line. In the process of trying to enter an answer the number line disappeared. Then going back a question and returning I was unable to get anything to appear at all.

Mathematics classes do not teach mathematics literacy

Both the phrases “mathematics classes” and “mathematics literacy” can be ambiguous, so I am meaning–

“Mathematics classes”: The high school curriculum delivered to a typical student in the United States. This usually excludes classes like “Statistics” or “Financial Math”.

“Mathematics literacy”: The kind of “good citizen” math that people refer to in articles like Headlines from a Mathematically Literate World. The word can also mean “ability to problem solve”, but that’s not what I mean here.

Taking the Headlines article and the classes of a typical high school mathematics student, how many of the headlines would a a student understand?

At the very least, understanding the entire list requires knowing about: correlation vs. causation, inflation, experimental replication, estimation of large numbers, incompatibility of comparisons with different conditions, understanding how tax brackets work, meaninglessness of predictions within a margin of error, statistically unlikely events, and reversion to the mean.

None of these will ever occur in an traditional math class. In other words, in the list of supposed math literacies, the typical math student in the US receives zero of them. (Some might possibly show up in a class labelled “Economics” or “Free Enterprise”, but those don’t get called Math Classes).

It’d be fair to argue I’m being highly specific in my starting definitions, but I often see the “good citizen” argument used during a general “why are we teaching math” type discussion which assumes a traditional math class track. That sort of argument only works if people are prepared to also overhaul the curriculum (by putting, for example, statistics before calculus as Arthur Benjamin discusses at TED).

Why is a negative divided by a negative a positive?

So there’s a whole lot of posts, including one from this very blog, which give intuitive explanations of why a negative times a negative is a positive.

I haven’t seen nearly as much material for a negative divided by a negative. One can certainly appeal to the inverse — since , . Google searching leads to answers like that, but I’ve found nothing like the multiplication picture above.

Can anyone explain directly, at an intuitive level, why a negative divided by a negative is a positive? Or is the only way to do it to refer to multiplication?

The evolution of mathematical exposition

More rambly and unsubstantiated than usual, apologies —

[Source.]

Theory: Mathematical exposition has evolved just like fiction writing has. However, tradition has held stronger in mathematics (likely due to a need for precision) and it means that clarity in writing is if not actively discouraged at least passively devalued.

Theory: We are not anywhere near the threshold of simplest and clearest explanations in the exposition of mathematical subjects.

Still, what used to be difficult is now considered easier. Various subjects have shifted their supposed level. For instance, not long ago College Algebra was the prestige class at the top of the high school level.

Furthermore, our raw definitions of what each class is has shuffled the actual content of subjects; Algebra I from the 1940s is not the Algebra I of today.

Theory: It would be possible to take a “hard” subject like group theory or transcendental number theory and make it comprehensible at a lower level. However, as there is no requirement to do so there is little motivation to make the subject easier. When a curriculum shift happens to move topics to a lower level, mathematical exposition evolves to catch up.