Anthony Reibel is a creative, vibrant, and caring school leader who hails from Adlai E. Stevenson High School in Illinois, where he is the current director of assessment, research, and evaluation.

Before We Get to Work: Foundational Questions of Quality Assessment Design

Think of a recent assessment design conversation you had with a colleague. What aspect of the assessment process did you discuss? Did you consider which standards to assess? Did you talk about how many questions, or tasks, were needed to determine student mastery? Or, did you examine the content that you would evaluate?

As the director of assessment at a large public high school in the Midwest, I engage in these assessment conversations often with teachers and collaborative teams. While we discuss all aspects of the assessment process, the most common question I hear from teachers is, “What should my assessments look like?”

While this question is vital to developing quality assessments, it is still too myopic of an approach to assessment design. By focusing on only this question, teachers are more likely to create assessments that evaluate knowledge retention instead of assessments that develop a student’s proficiency in course standards. To expand the focus of an assessment design conversation, teachers should consider the following three questions first before beginning any structural design of assessments. These questions are:

  1. What is the purpose of the assessment?
  2. What type of event should the assessment be?
  3. What is the reliability of the assessment evidence?

These questions set the foundation for quality assessment design, effective use of assessment, and accurate feedback.

What is the purpose of the assessment?

If teachers don’t know the purpose of an assessment, the assessments may feel disconnected from the overall learning process. Students may think that the assessment is meant only to evaluate their learning instead of growing their learning.

I maintain that there are three primary purposes of any assessment: to produce evidence of proficiency, to expose thinking, and to promote growth conversations with students.

Produce Evidence of Proficiency

For assessments to create credible evidence of proficiency, teachers should consider using more performance-based assessments rather than forced-choice. A teacher may find the evidence of learning less reliable if the assessment is only evaluating outcomes—in other words, looking for right or wrong answers, which forced-choice often does. When a student demonstrates their learning through performance, there is more of a chance the student produces more authentic evidence.

Expose Thinking

Certain times, a teacher may want to administer an assessment that not only provides proof of proficiency but evaluates the student’s thinking as well. Imagine, as was the case in my classroom, that a student is showing evidence of proficiency, but is guessing or simply mimicking a quality performance! Teachers need to remember that there are many wrong ways to produce the correct result.

Teachers need to remember that there are many wrong ways to produce the correct result. Therefore, it is essential that teachers have all the information available to judge the performance accurately. When teachers use assessment to expose student thinking, they can better assess whether the student’s proficiency is real or an illusion.

Promote Growth Conversations with Students

In my experience, teachers who use assessment as a chance to converse with students about their emerging proficiency and development create more enduring learning. Teachers can connect assessments to essential learning standards as well as add reflective components to help students talk about their growth more effectively. When students can see their growth, they are more likely to take productive risks in their learning and embrace failures as learning experiences.

What Type of Event Should the Assessment Be?

A teacher should also begin the assessment design conversation by asking, “What type of event do I intend to create?” When teachers explore this question before the design, they are more likely to create assessments that align with the method of instruction and the nature of the content. There are six major types of assessment events that a teacher can create: thematic, aligned, reasoning, reflective, connected, and process. It is up to the teacher to choose which of these types best aligns to the method of instruction.

Thematic Assessment

Thematic assessments are not connected directly to learning outcomes and gradations of learning. Instead, they are related to the content, themes, or topics. Thematic assessments usually contain questions that are categorized by content topic or theme, and are most effective for content-heavy units that ask for merely the recall of information. The most common format for thematic assessments is forced-choice.

Aligned Assessment

Aligned assessments are events that teachers relate to course standards. These assessments usually contain questions that are categorized by a learning gradation (4, 3, 2, 1) and provide evidence of proficiency in the learning standards. Teachers can use the evidence for collaboration, feedback, timely intervention, and instructional change. Aligned assessments are most effective for standards-based grading courses where achievement of standards is the goal. The most common format for this type of assessment is still forced-choice. However, a teacher typically includes performance-based aspects.

Reasoning Assessment

Reasoning assessments contain all the aspects of aligned assessments but also ask students to articulate process thinking. With this type of assessment, teachers evaluate how a student thinks through solving a problem as well as their answers. These assessments are most effective for process-based courses where students are often asked to show their work. The most common format for this type of assessment can be either performance-based or forced-choice, but they must include areas for students to show their work and thinking.

Reflective Assessments

With reflective assessments, teachers understand that what a student thinks about their work provides useful information to assess their level of understanding. Here, teachers evaluate the student’s reflective thinking along with their answers and process thinking. These assessments are useful in any course and should be used often to provide the full context of a student’s learning. The most common formats include performance-based and forced-choice, and include sections that gather student’s reflective thoughts before, during, and after the assessment.

Connected Assessment

When a teacher uses assessments to demonstrate how a student is growing in a learning standard over time, they are using connected assessment. Essentially, teachers use connected assessments that shows a student’s mastery of standards, as well as provide insight into their social-emotional development. This type of assessment is ideal for all courses and subject areas. The most common format for connected assessments is performance-based or forced-choice, but the teacher also includes sections where reperformance of previous unit material is required.

Process Assessment

Here a teacher considers all moments in the classroom viable moments of assessment. A warm-up, an exit slip, an instructional activity, or a project are all used to gather evidence of student learning. If an observer entered the classroom using process assessments, they would find it hard to tell the difference between an instructional event and an assessment event. When teachers use process assessments, students are more likely to co-construct learning, collaboratively engage with classmates, and value their continuous development. Process assessments are ideal for all courses but are imperative for standards-based grading courses. The most common format for these assessments is performance-based, with the teacher using forced-choice for intervention purposes.

In the following chart, I articulate how assessment purposes interplay with assessment types. As you can see, thematic and aligned assessments are more diagnostic, searching primarily for right or wrong answers. Reasoning and reflective assessments are more reactive, using the students’ thoughts and reflections to create a more contextualized assessment process. Connected and process assessments are proactive, engaging the student in a co-constructed learning process.

It is important for teachers to consider that these two assessment factors (purpose and type) can lead to a more effective design of assessments.

Assessment Type Assessment Purpose
Thematic Outcomes Only
Aligned Proficiency Only
Reasoning Proficiency & Thinking
Reflective Proficiency & Thinking
Connected Proficiency, Thinking, and Growth
Process Proficiency, Thinking, and Growth

What is the Reliability of the Assessment Evidence?

When teachers administer assessments that yield unreliable evidence, they are more likely to make assumptions about a student’s learning. When teachers, students, parents, or administrators make assumptions about student learning, they are more likely to not meet the academic as well as the emotional needs of the student. The goal with any assessment should be to produce highly reliable evidence of student learning. Below is a cross-section of assessment types, assessment purposes, and the reliability levels for each type of assessment:

Assessment Type Assessment Purpose Evidence Reliability
Thematic Outcomes Only Low
Aligned Proficiency Only Medium
Reasoning Proficiency & Thinking Medium-High
Reflective Proficiency & Thinking Medium-High
Connected Proficiency, Thinking, and Growth High
Process Proficiency, Thinking, and Growth High

Not only is reliable evidence the desired result of any assessment, but it also leads to more accurate feedback, more effective collaboration, and more meaningful grading practice when they have better evidence.

When teachers explore these foundational questions with fidelity, they can more effectively embed assessment into the learning process. When this happens, it can lead to more independent student learning, as well as generate undeniable evidence of proficiency.


Comments

  1. lorry dickinson

    Can you speak to the efficacy of using the same pre and post assessments in English. Our thought is that they need to be the same to assess the skill, the curriculum. We are concerned that if we change the reading passages, inaccuracies in the data could happen with students having difficulty with the text and then not being able to show mastery on the skill. In research, pure data comes from using the same pre and post assessment. We are having a professional disagreement. Thanks for your help.

    Reply

Leave a Reply

  • (will not be published)