Tagged: assessments


Responding to Trauma and Responding to Evidence – Both Needed, Both Possible

Responding to Trauma and Responding to Evidence – Both Needed, Both Possible

As regular readers of the STAC blog posts offered by my colleagues and me, you won’t be surprised to read the next sentence. Assessment is one of the most stress-inducing activities educators put students through. Perhaps some of you might even have some uncomfortable reactions when you recall some of your own test experiences. I should qualify this with the notion that I’m talking about assessment done poorly – the type of assessment I define as a number chase, instead of effective assessment which I believe is an evidence chase. But first, let me connect the dots to the title of this post.

In our newly released book Trauma-Sensitive Instruction: Creating a Safe and Predictable Classroom Environment, John Eller and I share a definition of trauma that really stopped us in our conversations because it was so powerful. The definition is from the work of Kathleen Fitzgerald Rice and Betsy McAlister Groves (2005) and states: “Trauma is an exceptional experience in which powerful and dangerous events overwhelm a person’s capacity to cope” (p. 3). The current stress we are all experiencing (and to varying degrees) brought on by the health pandemic far outweighs the stress induced by ineffective assessment practice. However, the combination of these two – poor assessment practice and additional trauma from the pandemic – may combine to negatively impact students to the point that their progress and academic growth might never recover in their remaining school years. It doesn’t have to be that way.

In my work with teachers, I’ll often hear that the constant stress and lack of stability sets up students for difficulties in calming down in order to feel safe, learn, and give their best when it comes time to performing on assessments. Teachers, then, would be wise to invest in assessment design that does not depend on a “one shot, achieve or fail to” test. Instead, formative assessment – the practice before the performance – must be a part of the evidence gathering, not just for students experiencing trauma, but for all students.

The impact of the pandemic was not evenly distributed nor evenly felt. For some of our students (and colleagues) it reinvigorated past traumatic experiences almost incapacitating any opportunity for progress. For some students, the extended trauma exposure resulted in what Jim Sporleder and Heather Forbes (2016) refer to as toxic stress. Toxic stress can lead to issues that can impair students’ normal development and success in the classroom, including their ability to focus and respond appropriately to teacher requests. The potential for assessment to be inaccurate or incomplete is very high. In Trauma-sensitive Instruction we offer many scenarios like this one:

“Laura, a seventh-grade student, lives in a home where her father drinks excessively and comes home drunk. When he gets home, he is both verbally and physically abusive to Laura’s mom and any of the children he sees. Laura normally knows that when he comes home, it’s a good idea to stay out of his way and try to be invisible. She usually withdraws from the situation and tries not to cause a lot of issues.

In her classes, Laura uses similar behavior. Even though she may not understand what she is learning, she is reluctant to ask questions or get clarification. When working in groups, Laura contributes little to the conversation and goes along with the ideas of the group. She is reluctant to make eye contact with people (adults and peers) and appears to be disconnected and isolated.”

Relationships are critical
How might the teacher respond to Laura’s actions while also committing to gathering good evidence to assist her on her educational journey? If Laura is disengaged and disconnected, her teacher may not be able to assess what she knows. Somehow, her teacher has to be able to reduce her stress to be able to get an accurate read on her progress.

Again, the focus on why we assess comes into play. One of the powerful outcomes of a fair an equitable assessment process is the development of a positive relationship between teacher and student. The more assessment is viewed as an opportunity to demonstrate learning and to progress from “not yet” to “proficient”, the greater the view that teacher and student are on a journey together. In our research for Trauma-sensitive Instruction one of the keys that emerged to help buffer against adversity is having warm, positive relationships, which can prompt the release of anti-stress hormones. The choice to have assessment as a stress inducer versus a stress buffer comes down again to how the evidence is used by both the teacher and the student. If teachers can help the student see how assessment data can help them learn, it may cause less of a stressful reaction. If the student thanks that assessment is being used only to label or sort them, it will not be seen as positive or productive, and they certainly won’t capitulate.

We also know that the trauma that came with the health pandemic did not arrive on every doorstep equally. The Center for Infectious Disease Research and Policy (CIDRAP) suggests that “COVID-19 exposures were significantly different across race and household income strata, with Black, Latino, and low-income families reporting higher rates of COVID-19–related stressors, which they attributed to systemic racism and structural inequities…” The trauma-aware schools project further suggests, “Symptoms resulting from trauma can directly impact a student’s ability to learn. In the classroom setting, this can lead to poor behavior, which can result in reduced instructional time and missed opportunities to learn.” It’s important then, that educators avoid overemphasizing the importance of tests and exercise caution to avoid overemphasizing the consequences of failure. The messages educators use to communicate about tests matter, and efforts should be made to reduce students’ anxiety and increase students’ self-efficacy beliefs. The message should focus on the role of assessments as a measure of students’ knowledge and ability at this moment.

Know what to look for–and how to react
One of the key body reactions to trauma occurs as a result of the fight-or-flight response. Educators often see this reaction during test time with those students who arrive and seem to “power down” immediately upon receiving their tests. They may quickly do as much as they can and turn in an incomplete exam or they may just put their name at the top and stop there. When this level of trauma is occurring, the body may be releasing cortisol which keeps the body on alert and primed to respond to the threat. It is important to note here that while the body is primed to respond to the threat, the control center of the brain, the pre-frontal cortex, is shut down. This is the place for logic and reasoning, key skills needed for assessments.

In a paper that focused on high-stakes testing, the authors (Jennifer A. Heissel, Emma K. Adam, Jennifer L. Doleac, David N. Figlio, Jonathan Meer) found that “Students whose cortisol noticeably spiked or dipped tended to perform worse than expected on the state test, controlling for past grades and test scores.” So, if summative tests unfairly penalize students who are experiencing high levels of trauma, it might be reasonable to conclude those tests aren’t generating the evidence we need them to, and they might not be aligned with the formative evidence we already have. The authors go on to state “A potential contributor to socioeconomic disparities in academic performance is the difference in the level of stress experienced by students outside of school.” This means as educators we have to be mindful that students will react differently to similar traumatic experiences and may need different kinds of support from us.

Let me summarize by going back to the title of this post. It’s important for educators to recognize the need to pair trauma-informed teaching with assessment processes to ensure we are not adding more trauma to our students’ lives. Strategies to consider include communicating the purpose for assessment, providing a calm and predictable classroom environment, building and leveraging positive relationships with students, and recognizing when students are under stress and helping them to relax in order to make assessment a natural part of their learning journey. These and other trauma-informed practices will not only help them do better, but will also help them build the resilience they need to be productive and well-rounded adults. Adults who have the capacity to break the trauma cycle for their own children. By the way, these practices, when fully implemented, will benefit ALL learners not only those whose lives have been impacted by trauma.

https://www.cidrap.umn.edu/news-perspective/2021/04/covid-studies-note-online-learning-stress-fewer-cases-schools-protocols

https://traumaawareschools.org/impact

Testing, Stress, and Performance: How Students Respond Physiologically to High-Stakes Testing
https://justicetechlab.github.io/jdoleac-website/research/HADFM_TestingStress.pdf

Sporleder, J., & Forbes, H. T. (2016). The trauma-informed school: A step-by-step implementation guide for administrators and school personnel. Boulder, CO: Beyond Consequences Institute.


Shift Away from Learning Loss and Focus on Relevant Assessments that Emphasize Creativity and Critical Thinking

There is a buzz in education circles and districts raising concerns about the significant learning loss affecting students after the absence of in-person learning and the multiple shifts between learning models. It is important to shift away from this idea of learning loss, as it focuses on the deficits of students and does not recognize the strengths and assets students developed during the pandemic. The Spencer Foundation and The Learning Policy Institute created six principles to consider when addressing this concept of learning loss. One pillar identified is to focus on creative inquiry forms of learning. This was further explained as:

“Learning environments can engage in disciplinary and interdisciplinary inquiries through a variety of contexts and by using artistic forms of expression. All students deserve the opportunity to be treated as creative thinkers and makers. Learning should be an opportunity for play and authentic meaning-making, with the focus on how students are learning instead of drilling content in isolation. Children should be inspired, their curiosity encouraged, and their dreams fed. Research demonstrates that such environments provide rich opportunities for deep learning. Such forms of learning can support meeting multiple learning goals and create opportunities for students to imagine and contribute to thriving post-pandemic worlds that serve themselves and their communities” (Bang et al., 2021).

Refocusing
This report started the wheels turning in my brain. Summer is the time for educators to step away, refresh, and reflect. It is also the time when intentional and rich collaborative conversations take place around curriculum and assessment among teaching teams. They may be formal or informal conversations but what is so powerful is the creativity that comes from these collective voices coming together.

What if this understanding of creative inquiry was shared with teams in advance of planning together? Would this guide teams to think differently about formative and summative assessments? Would it spark new ways to allow students to show their knowledge? Instead of defaulting to traditional assessment practices such as multiple choice, a short essay, or matching, the sky’s the limit in the ways that teams can collaborate to create assessments that are meaningful, relevant to their students’ own lived experiences, and allow students to show what they know. Stephanie Woldem, a brilliant math teacher in a south Minneapolis urban high school, asked students to analyze tables of data showing numbers of homicides, assaults, and arrests. She asked students to find out which police precincts had the most positive interactions with police compared to arrests and crimes. (Star Tribune, Mattos, 2015). Interactions between the community and police were a growing concern. Woldem was very aware of these concerns among the students and created a relevant learning task with an assessment in her freshman algebra class. Policing impacted many of the young scholars in the community around the school. Woldem knew that having a deeper understanding of the data and helping the students make meaning of the data was far more impactful than practicing a few algebra problems.

Bringing lessons to life
The teachers at this school also asked students to participate in data tours, using relevant city data and students created questions to dig into the data. Often they came away with more questions than answers when analyzing the information. What this teacher learned is that in a subject matter such as math where students struggled to make meaning and find relevance, math jumped alive for these students. The teacher found a way to use content and assessments to motivate, engage and inspire the students to become critical thinkers using the data in ways they have not done before.

Through understanding what is going on in the lives of her students and the community in which the students live, the teacher helped them develop the skills of critical and creative thinking. The three great assessment gurus which taught me much of what I know about assessments today, Cassie Erkens, Tom Schimmer and Nicole Dimich, wrote a book called Growing Tomorrow’s Citizens. In the text they outlined critical competencies necessary for success in a changing world. Two of these tenets that Woldem also used in her math classroom were critical thinking and creative thinking. While the authors argue that all seven are critical for deeper and more meaningful learning for student’s today, I want to focus on critical thinking and creative thinking when developing assessments for students.

Reimagining assessments
The authors identify the skill of critical thinking and to also consider the dispositions or how to help students behave as critical thinkers. Students need opportunities to engage in practicing this skill prior to providing evidence of their learning through a formative or summative assessment. As did the math teacher, “it is preferred that learners emerge as partners and key decision makers in their own experiences, allowing them to create relevance while exploring those areas and topics that naturally pique their curiosity” (Erkens et al., p.77). Inquiry and project based learning can be valuable examples of helping learners to practice these skills. As teachers working in collaborative teams in the summer, how can teams reimagine a unit summative assessment to build in critical thinking skills in a way that is relevant for the students in their school community?

The second of seven critical competencies is creativity. Erkens et al., state that “creativity is the backbone of innovation, and humanity thrives on innovation; indeed, it is what improves the quality of living across all age and continents” (144). Creativity may be difficult to measure and far too often students do not have an opportunity to imagine, invent or develop original ideas. Schools are looking for right and wrong to prepare students for standardized tests, which also do not assess creativity. Thus, teaching this critical competency is left behind. A mentor once said to me what is measured is what identifies the values of an organization. If the school is only looking at standardized tests, then students often would not have experiences to produce or create and collaborate to develop new solutions. However, if educators were to ask employers what are the top skills they are looking for in employees, often creativity and developing new, innovative solutions rises to the top. Schools are not preparing students well for the skills they need in our ever-changing world. Again, as educators work together to develop new curriculum and assessments in the summer, consider embedding teaching the skills of creativity into units of study. Allow students an opportunity to practice these skills. In fact, work together to create new opportunities for students as you get your own creative juices flowing. One recommendation from Erkens et al., is to teach the creative process. Melissa Purtee came up with this visual to assist in helping educators understand the creative process.

Inspiration design creation reflection and presentation flowchart

(Purtee, 2017).

One middle school classroom in a STEM school had a specific design course. The students were asked to design musical instruments that were required to make a noise at a certain level of frequency. The students spent time walking through each step of the creative design process. They kept journals documenting and reflecting on each stage of the design process. This was also used as a formative assessment for the teacher. The teacher read a few journal entries each day to get a grasp of how students were making sense of each step in relation to their instrument design. In the end, the students were asked to play the instrument. If the criteria was not met, the teacher provided feedback and the students were able to have another attempt, as failure and revision is a necessary part of the design process. However it is not necessary to have a separate course to teach creativity and this can be embedded into any course or subject matter. To further understand the creative process, visit go.SolutionTree.com/assessment for a free reproducible of a table called” Instructional Questions for Teaching the Creative Process”.

Growing tomorrow’s citizens is important to engage in relevant and meaningful assessments that tap into creativity and critical thinking. This aids in motivation and engagement for the learners when they find assessments where they can make personal connections to prior knowledge, new learning, and develop new skills. With an emphasis on creativity and critical thinking with assessment practices, this will aide in the acceleration of learning instead of focusing on this concept of learning loss. The time is now to engage in critical and creative thinking while collaborating with your own teams during the summer to develop authentic and relevant assessments that are meaningful for students.

Matos, Alejandro (2015, December 6). “South High teachers illustrate inequities through math.” The Star Tribune. www.startribune.com
Bang, Megan, et al. The Learning Policy Institute, 2021, Summer Learning and Beyond: Opportunities for Creating Equity.
Erkens, C. et al. Growing Tomorrow’s Citizens in Today’s Classrooms: Assessing 7 Critical Competencies. Solution Tree Press. 2019.
Purtee, M. “The Essential Framework for Teaching Creativity.” The Art of Education. 2017, theartofeducation.edu/2017/08/15/essential-framework-teaching-creativity/.


The 4×2 of Student Self-Reflection

The story self-reflection within the context of my own career is a good news/bad news story. First, the good news. While there is much I’m not proud of when I reflect on the early part of my career – especially from an assessment and grading perspective (e.g. zeros, no reassessment) – the one thing I did do is have my students engage in some self-reflection.

The bad news? Well, I didn’t engage my students in self-reflection very often and when I did, it was awful; I didn’t know what I was doing, there was no structure to it, and it, more often than not, ended up being a waste of time.

Self-reflection is essential to the development of our students’ metacognitive awareness that allows them to plan, monitor, and assess their learning and themselves, as learners (Clark, 2012; Jones, 2007). Metacognition entails both the knowledge and regulation of one’s cognition (Pintrich, 2002). Through the processes of self-judgment and self-reaction (Zimmerman, 2011), self-reflection plays an essential role in the cyclical process of metacognitive awareness before, during, and after the learning. Read more


Uncovering Implicit Bias in Assessment, Feedback and Grading

Education is a noble profession. It is a profession that aims to cultivate diverse thinkers and aspires to nurture personal growth. It is a profession that can lift humanity’s spirits and help humankind strive to be the best version of itself—the “great equalizer of the conditions of men,” as Horace Mann famously stated in the 19th century. However, even with great people, a worthy goal, and an admirable vision, the opposite can often be the case.

Unfortunately, education can also be the great unequalizer, where personal biases can inform practice and policy development, stifle student growth, enforce discriminatory policies, and even socially isolate students. According to some researchers, implicit educator biases may contribute to a racial achievement gap; precisely, the negative impact of teacher assumptions on students’ ability based on race, culture, or values (van den Bergh et al. 2010). Unknown to an educator, these personal biases may create imagery of an ideal student, which is often seen through a white privilege lens because of society’s tendency toward whiteness, distorting our interactions with students of color.

Without social awareness and continuous self-monitoring, educators may let their implicit bias become an influential factor in their pedagogy, influencing everything from assessment to grading. This blog will discuss how personal biases can appear in our teaching and learning practices if educators are not diligent. I will focus on three of the more considerable teaching and learning modes: assessment, feedback, and grading.

Assessment Bias

Without attention, teachers may create assessments that reflect their values and experiences and ignore those of their students. The teacher may use language that they are more familiar with in their queries or create prompts influenced by their personal experiences. Ultimately students may find it hard to relate to the questions—potentially leaving some students unable to perform to the best of their abilities. For assessments to be less biased, a teacher must consider all backgrounds, ethnicities, genders, and identities to ensure that their lived experience isn’t the only one represented on an assessment.

By being more introspective when developing assessments, teachers can lessen the chance they produce a personally irrelevant assessment for their students. The impact of this irrelevancy could be low student performance, disinterest in the task, and even apathy toward school.

To help teachers be more aware of their biases when creating assessments, they can ask themselves the following questions as they develop questions:

  1. Does the assessment give the sense that the teacher has unwavering support and is a partner in a student’s success?
  2. Are the questions seeking to understand the student or judge them?
  3. Does the teacher draw on the students’ life situations, interests, and curiosities when creating problems/prompts? – Adapted from Tomlinson (2015).

Microaggressions in Feedback 

Suppose we ignore our implicit biases when speaking with students. In that case, we run the risk of putting both parties into what social psychologist Albert Bandura calls “a downward course of mutual discouragement” (Bandura 1997, 234). A student’s reaction to deficit-based feedback may result in the teacher reacting in kind. Once this cycle starts, a student’s self-belief is now at risk.

Microaggressions and subtle discriminations can exist in the feedback process, and when they do, they may severely limit feedback acceptance.

Teachers can limit bias in their feedback to use the student’s thinking to grow the student. For example, let’s look at the following examples:

Example A

Feedback that Uses Teacher Thinking: You didn’t include [these details] about [person] in your essay. Try [these words].

Feedback that Uses Student Thinking: Tell me more about these words [here]. I am interested to know why you think [this word] didn’t work instead? Oh, okay, that would work. You should add what you just said to your paragraph, and it was perfect.

Example B

Feedback that Uses Teacher Thinking: When I write, I try and think about [detail]. Remember when I taught you the three-step process? No? The one that is in your textbook? That’s the most effective process.

Feedback that Uses Student Thinking: What did you think about when you wrote [this]? Seems like that interests you? Yeah, I can see you are passionate about [that]. What are the first three things you did when you wrote [this]? That is an interesting place to start. Could I convince you to start here? No? Okay, that makes sense. Have to make this work for you.

Example C

Feedback that Uses Teacher Thinking: In this graph, I would start [here] because this information is important. Has anyone heard of the [rhyme name] to remember the key features of a graph? No? Oh, this helped me a lot.

Feedback that Uses Student Thinking: In this graph, what information did you think was essential for you to begin this problem? I’m surprised to hear you say that because yesterday you said something different, what changed? Interesting, I saw you smile as you were talking. Why? Yeah, I agree you are getting this concept more. Are you using any strategies to help you learn this? Yes. Well, [that strategy] is undoubtedly helping you.

These scenarios are fictional, but the point here is teachers should always be aware of their language. Otherwise, they can inadvertently make the student feel like an unequal and devalued student in the class and even the school. In short, words matter.

Race Bias in Grading Practices

Teachers must judge student performance fairly and accurately. It is our professional duty. Inaccurate judgments have the potential not only to alter grades but could negatively affect teacher-student relationships, distort a student’s self-concept, or reduce opportunities to learn (Cohen and Steele 2002). One factor that can lead to a misrepresentation of a grade is teacher race and ethnicity bias. A student’s racial or ethnic group, socioeconomic class, or gender can substantially bias a teacher’s judgment of student performance. Any internalized racial biases can activate stereotypes and lead teachers to utilize discriminatory performance evaluations (Wood and Graham 2010).

For example, several studies found substantial differences in students’ performance judgments from various racial subgroups when the teacher subconsciously subscribed to the general stereotype that African American and Latino students generally don’t perform as well as their White and Asian counterparts (Ready and Wright 2011).

Different minority statuses can affect teacher perceptions in performance evaluation, leading to inaccurate grades, potentially harming students’ perception of their academic experience. (Ogbu and Simons 1998). In other words, students may feel like school is insignificant, unsupportive, or even harmful.

To help lessen the likelihood that implicit personal bias influences the grading process, teachers can democratize the grading process. They can use learning evidence instead of points and employ a modal interpretation of gradebook scores instead of averages. They can use a skills-focused curriculum instead of a content-focused one. Perhaps most important, they can involve the student in the grading process by infusing more self-evaluation moments into their instruction.

School leaders should explore ways to evaluate pedagogical practices through a racial and equity lens and observe classroom interactions between teachers and students. School leaders should also continue training on white privilege and its influence over the status quo, and teachers should evaluate student performance to judge its fairness and accuracy.

The Work Ahead

Although we may feel like we are objective and rational people, we all have biases.  We all have values, beliefs, and assumptions that help us make sense of what is happening in our lives and guide our interactions with others. For the most part, these values, beliefs, and assumptions guide us in positive and productive directions, but the interplay between these same values and social interactions can produce implicit biases that distort our decisions, perspectives, and actions. We must notice, monitor, and manage these distortions to achieve a goal of racial equity in school and life. If we don’t, we are at risk of our unconscious biases harming our pedagogy, our relationships with students, and our perception of their needs.

 

Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: W. H. Freeman & Co, Publishers.

Cohen, G. L., & Steele, C. M. (2002). A barrier of mistrust: How negative stereotypes affect cross-race mentoring. In J. Aronson (Ed.), Improving academic achievement: Impact of psychological factors on education (pp. 303–327). San Diego, CA: Academic.

Ogbu, J. U., & Simons, H. D. (1998). Voluntary and involuntary minorities: A cultural-ecological theory of school performance with some implications for education. Anthropology and Education Quarterly, 29(2), 155-188.

Ready, D.D., & Wright, D. L. (2011). Accuracy and Inaccuracy in Teachers’ Perceptions of Young Children’s Cognitive Abilities: The Role of Child Background and Classroom Context. https://doi.org/10.3102/0002831210374874

Tenenbaum, H. R., & Ruck, M. D. (2007). Are teachers’ expectations different for racial minorities than for European American students? A meta-analysis. Journal of Educational Psychology, 99(2), 253–273.

“Teaching Up” – Carol Ann Tomlinson: Teaching for Excellence in Academically Diverse Classrooms (2015).

van den Bergh, L., Denessen, E., Hornstra, L., Voeten, M., & Holland, R.W., (2010). The implicit prejudiced attitudes of teachers: Relations to teacher expectations and the ethnic achievement gap. American Educational Research Journal, 47, 497–527.

Wood, D., & Graham, S. (2010). “Why race matters: social context and achievement motivation in African American youth.” In Urdan, T. and Karabenick, S. (Eds.) The Decade Ahead: Applications and Contexts of Motivation and Achievement (Advances in Motivation and Achievement, Vol. 16 Part B) (pp. 175-209). Emerald Group Publishing Limited, Bingley.


Data as a Flashlight: Using the Evidence to Guide the Journey (Yours and Theirs)

My granddaughter was struggling with the latest topic in her grade 3 math class and her recent assessment result validated that she did not fully understand the learning target of patterns and the equations that supported them. Determining patterns is not always an easy process as this example would indicate:

2, 6, 3, 9, 6, 18, 15…

With a big test coming up, my daughter-in-law reached out to me to help get my granddaughter past the block and gain some confidence in her ability to master the concept. We connected online a few times over the days before the test and worked through a lot of questions and strategies. As she grasped the concepts and different ways to get to the solution, I could see her confidence soar. By the time we concluded all of the practice and she routinely got every solution, she was excited to demonstrate her skills on the assessment. As I write this post it’s been well over a week since the assessment was completed and my granddaughter has not received any feedback.  Read more


How Do We Do Common Formative Assessments in a School of Singletons?

We define singletons as those teachers who are the only one who teaches a grade level or subject area in their school.  

When schools define themselves as professional learning communities, one of the hallmarks of that work is to work with a collaborative team with whom teachers learn together. Read more


#Winning: Anticipating Errors as Preventative Instruction

I was recently working with a team to develop common formative assessments, and they were having some difficulty generating appropriate questions and tasks for the standards being addressed in this particular unit of study. 

We talked through the standards, as well as the learning targets, and brainstormed assessments methods to match the various elements of learning required for students to master the concept. Still, we were in a pause. 

So, I decided to change course. 

Instead of further belaboring the learning targets and what student proficiency would need to look like, I engaged the team to consider what was preventing student learning in the first place. Read more