Sunday, January 20, 2008

Backwards Design - Test Creation

Nick asked about assessment. He opened Pandora's Proverbial Box of Everything, so blame him when you start reading this and get bored. In an effort to save all you poor souls from reading, I will give the short answer here, and the long answer in a second post.

"This is interesting. How did you create a target quiz/test score from a needed score on what I expect is a different type of test entirely. Unless these test/quizzes are in the same format as the TAKS examine. Not questioning your methodology. I just wanna know how it was done, so I can use it in other applications." - Nick

There are four released TAKS tests. I also have textbook materials from 4 different publishers that are supposed to be aligned with the TAKS test, although any given learning objective has many interpretations, so there is a fair amount of variability between publishers. With these examples, I know what a specific learning objective looks like (in terms of assessment) for any of the 34 or so that my kids are supposed to know by the end of the year.

For any instructional unit, my assessment will have questions that scaffold from basic knowledge based questions up to the level that TAKS requires. I will have usually 4 questions per learning objective, with at least half of them being TAKS (literally off the TAKS test) or TAKS equivalent questions.

An example:
I have an objective: 7.1A compare and order integers and positive rational numbers.

My students need to be able to compare:

  1. Fractions with fractions
  2. Decimals with decimals
  3. Percents with percents
  4. Integers with integers
  5. Fractions with decimals
  6. Fractions with percents
  7. Decimals with fractions
  8. All 3 at the same time
  9. All 3 with integers
So I might ask 1 greater than/less than question for the first 7 items. These will be easy, low-level. Then I will ask maybe four questions with #8 and four questions with #9. Probably 6 of the 8 questions would be TAKS or TAKS equivalent, with 2 being "put these in order from greatest to least".

The end grade doesn't exactly match with the TAKS test because of the easy questions, but it does give an accurate view of how well the student knows that particular learning objective. And really, the goal is not, necessarily, to predict how the student will do, since the type of questions on the test change almost every year. The point is to see what areas the student knows and doesn't know so that I can give targeted remediation to sub-groups of students so that they have the tools to be able to pass regardless of questions.


3 comments:

Nick Haywood said...

Interesting indeed.

Also, I hate you. I noticed that I asked a question in that comment and forgot to put question mark. THEN, I was going to correct it, but since I had made a previous mistake in the first post, I would have had to delete the first TWO posts. THEN, I thought, "Oh big deal, I'll just leave my typo."

Now the whole world (those that don't check out my website already)know my deficiencies with grammar. In fact, this comment could have multiple mistakes because I loath reading what I write, so I don't.

Nick Haywood said...

So, I guess in regards to my previous question. You attempt to make the quizzes and tests similar to the TAKS. Which seems like it would be the proper thing to do considering that is the entire goal of your teachings.

Which begs the question, should that be the goal of teaching? A standardized test. Or should it be about "learning". In practice, very few students would learn just because that is what education is about. Those are probably already the top students, such as a Mr. Blair as a child. Do the standardized tests help the laggards though? Probably not, nothing probably helps. So you have this middle group. Does it help the middle group? This is not a simple question, and one that I think we can all have opinions to but no right answers. I have teacher friends who loath No Child Left Behind (I have not discussed with Mr. Blair). I don't have a real opinion about it, but I can say with confidence that there was a reason it was created.

Maybe standardized tests are more of an accountability measure for teachers?

Mr. Blair said...

Man nick, you're making me work for this. time for another post re:NCLB and whether the test is actually a good thing to be focusing on.