Posts by Cassandra Erkens


What’s in a 4? DOK and Proficiency Scales

When it comes to measurement, four is a popular number; rather, a range of 1 to 4 is a common scheme. Two different powerful measurement systems use a range of 1 through 4 scores to clarify levels of quality. The Depth of Knowledge [DOK] framework by Webb, 2005, uses a 1 through 4 scale to rank the cognitive complexity of an assessment task. The Proficiency Scale framework by Marzano and Kendall (2008) uses a 1 through 4 scale to rank students’ performance levels on individual standards.

Both assessment measurement systems have much in common:

  • Both systems range from 1 to 4
  • Each number in the range of both systems represents a degree of sophistication
  • The 1 represents the lowest degree of sophistication and the 4 represents the highest degree, or the “best” number

But that’s where the similarities stop. The DOK scale is used to measure the rigor level of a task (the source), whereas a proficiency scale is used to measure the quality of student work (the result) on an academic standard. A side by side summary of both frameworks highlights the differences in each level:

Depth of Knowledge Levels Proficiency Levels
(The measure of cognitive complexity in a task) (The measure of proficiency found in student work)
4: Extended Thinking—Requires an investigation, time to think and process multiple conditions of the problem (Synthesize, Analyze, Prove, Connect, Design, Apply Concepts). 4: Advancing—The student has met standard expectations and advances the standard requirements with in-depth inferences and/or extensive sophisticated connections, etc.
3: Strategic Thinking—Requires reasoning, developing a plan or a sequence of steps, some complexity, more than one possible answer (Assess, Revise, Critique, Draw Conclusions, Differentiate, Formulate, Hypothesize, Cite Evidence). 3: Achieving—The student is independent and demonstrates accuracy (no major errors or omissions) on standard expectations.
2: Skill/Concept—Use information or conceptual knowledge, two or more steps, etc. (Infer, Identify Patterns, Modify, Predict, Distinguish, Compare). 2: Developing—The student is independent, but demonstrates only partial accuracy on standard expectations
1: Recall—Recall of a fact, information, or procedure (Recite, Recall, Label, Naming, Define, Identify, Match, List, Draw, Calculate). 1: Initiating—The student is dependent on scaffolding and support to demonstrate minimal or inaccurate / incomplete understandings of standard expectations

 

Even though one system measures the input (task) and one measures the output (result), it’s a very common and alluring misstep to equate the two systems as synonymous. The argument could be made, for example, that a student is advancing (proficiency level 4) if he/she can engage in extended thinking (DOK level 4) regarding the standard(s).

However, it’s important to remember that the two 4-point schemes are meant to serve very different purposes. All four levels of proficiency can apply to any task that is not binary (there are no gradations of quality, so evaluation is solely based on right/wrong, yes/no, or present/absent responses); and, a single performance task can be assigned any one level of cognitive complexity. In other words, a task that requires strategic thinking (DOK level 3), can generate all 4 levels of proficiency in student responses.

The 3 Dangers

There are a few dangers in tying the two measures together.

Trapping Students at the Lower Levels

The first involves the limitations such a combination would place on students: Students who struggle to earn high levels of proficiency would become trapped in lower level tasks. Obviously, this is a significant concern. The joy and the big picture of learning often rest in DOK 3 and 4 level tasks which many struggling learners need, but if deemed a 1 or a 2 in proficiency would not be allowed to experience.

Forcing Students to Only Engage at Level 4

The second danger is the reverse of the first: Students wouldn’t be able to earn a 4 level of proficiency unless they were always engaged in extended thinking tasks.

Oversimplification

The third danger is the oversimplification of two different sophisticated measurement systems. Turning complex evaluation processes into streamlined, simple algorithms compromises the accuracy of teacher decision making and exempts educators from needing to think carefully about what constitutes deep learning.

So, what’s in a 4? It turns out there is a lot of important information behind any assigned score … but the information will vary based on the specific measurement system being employed and the items being measured. Let’s use each system as it was intended and avoid the temptation to merge and simplify them.

“I wouldn’t give a nickel for the simplicity on this side of complexity, but I would give my life for the simplicity on the other side of complexity.” —Einstein

 

References:

Marzano R. and Kendall J. (2008) Designing and assessing educational objectives: Applying the new taxonomy. Thousand Oaks, CA: Corwin Press

Webb, N. L., Alt, M., Ely, R., and Vesperman, B. (2005). Web alignment tool (WAT) training manual. Washinton, DC: Council of Chief State School Officers


Comments: 1

Achievement – Relationships, Relevance, and Rigor (Post 3 of 4 on Using Assessment to Improve Achievement)

Author’s note: In the spring of last school year, I offered 2 posts from a 4 part series on using assessments to increase achievement. The first was in regards to defining ‘learning’ beyond numerical indicators and the second was on holding high expectations for all learners. The third and fourth posts will be offered this fall.

Read more


Achievement – High Expectations for All

Post 2 of 4 on Using Assessment to Improve Achievement

We cannot solve our problems with the same thinking we used when we created them
Einstein

As noted in my first post of this series regarding using assessment to support achievement, the primary mission of schools is to help kids learn. Schools write mission statements toward that same end:  all students will be successful. But, have those mission statements become routine and somewhat cliché?   Read more


Achievement—It’s More Than a Number

Post 1 of 4 on Using Assessment to Improve Achievement

A person who never made a mistake, never tried anything new.
Einstein

The primary mission of schools is to help kids learn. That’s it. That’s the bottom line. It stands to reason, then, that the primary indicator of success will always be achievement scores. But our work with making decisions about learners must remain far more humane then making decisions about learners based on a set of cold, calculated scores (and it doesn’t matter if those data come from the grades in our gradebooks or external test scores). The measure of achievement should never mask the face of the learner. This is personal. And, it’s very serious work. Read more


Assessments Must Build Efficacy

When you come upon a seemingly insurmountable challenge, do you give up or do you persist?  If you tackle such moments, then you have a strong sense of efficacy.  Efficacy requires belief (I can do this) and action (I will take the risks, even though failure is a possibility).  It is the foundation learners require if they are to develop deep understanding and personal skill. Read more


Assessments as Hope

At its core, assessment has always been a process that was meant to support learning; in other words, teachers and learners would use the information from assessments to make decisions about what comes next. In order for teachers and especially learners to even fathom ‘what’s next’ they must maintain a sense of hope. Read more





Groupings for Collaborative Learning

Collaboration is critical to everyone’s success – especially learners in the classroom. In the early days of exploring brain research implications for the classroom, Pat Wolff was adamant that the speaker is always the learner (Wolff, 2010). Read more