Depth of Knowledge: What Webb’s Four Levels Actually Measure

By Clay Shumate

Key Takeaways

• Depth of knowledge describes the task, not the student and not the difficulty. Webb’s own 2009 guide says plainly that DOK “describes the kind of thinking required by a task.” A punishing recall question is still level 1.
• It was built to audit tests, not to plan Tuesday. Webb (1997) created it to check whether a state assessment was as cognitively demanding as the standards it claimed to measure. Everything teachers do with it now is a borrowed use — which is why no district should fund a DOK initiative expecting test scores to move. There are no controlled trials behind that promise.
• A verb is not a level. Webb’s guide states that the central verb alone “is/are not sufficient information to assign a DOK level.” Verb wheels are a shortcut around the judgment the framework actually requires.
• The levels are hard to agree on, and the people who built the method said so. Webb’s 2002 alignment study of three states found reviewer agreement on DOK was weaker between groups than within them, with most of the disagreement sitting on the line between level 1 and level 2.
• Most tasks land low, and that is the finding worth acting on. Hess, Carlock, Jones and Walkup analyzed more than 200,000 samples of student work from about 200 schools in Nevada and Oklahoma. English assignments clustered in one low-to-middle cell; math sampled the lowest level even more heavily.

What is depth of knowledge?

Depth of knowledge is a four-level classification of how deep into the content a task forces a student to go. It was developed by Norman Webb, a researcher at the University of Wisconsin, and it answers one question about an assignment: how much thinking does this actually require?

That is the whole idea, and it is worth saying what it is not. DOK is not a measure of how hard a question is. It is not a ranking of students. It is not a lesson-planning template, and it was never designed as one. Webb (1997) built it as a criterion for alignment — a way of asking whether the test a state gave was as demanding as the standards the state had written. The original phrase is “depth-of-knowledge consistency,” and consistency is between two documents, not between a teacher and a class.

Webb defined depth of knowledge as something “related to the number and strength of the connections within and between mental networks.” In plain terms: a task is deeper when more ideas have to be held together at once, and when the path from the question to the answer is not already laid down.

What are Webb’s four DOK levels?

The four levels run from recall to extended investigation. Level 1 is retrieving a fact or running a known procedure. Level 2 is processing past recall — choosing a method, taking multiple steps, interpreting data. Level 3 is reasoning you have to defend. Level 4 is an investigation that takes time and multiple sources.

Level	What the student does	Example task	How long it takes
DOK 1 Recall & reproduction	Retrieves a fact, term or definition, or runs a routine procedure the same way every time.	Name the three branches of government. Identify the independent variable in a given setup.	Seconds to a minute.
DOK 2 Skills & concepts	Chooses a procedure, works through steps with decision points, organizes or interprets data.	Summarize an argument in your own words. Build a graph from a data table and describe the trend.	Minutes.
DOK 3 Strategic thinking	Plans an approach, justifies it, defends a claim with evidence. Usually more than one defensible answer.	Argue which of two causes mattered more, and say what evidence would change your mind.	A class period or a short assignment.
DOK 4 Extended thinking	Investigates over time, across multiple sources or conditions, often across subjects.	Design and run an inquiry, gather your own evidence, and publish a conclusion to a real audience.	Days or weeks.

One line from Webb’s guide is worth copying onto a sticky note, because it settles most arguments in a department meeting: a level should reflect “the level of work students are most commonly required to perform” to produce an acceptable answer. Not the best answer. Not what a strong student might do unprompted. What the task requires to be counted correct.

How is depth of knowledge different from Bloom’s taxonomy?

Bloom’s taxonomy classifies the mental operation — remember, understand, apply, analyze, evaluate, create. Depth of knowledge classifies how far into the content the student has to travel. They are not two names for the same ladder, and they do not line up.

Hess, Carlock, Jones and Walkup built a Cognitive Rigor Matrix precisely because of this. They crossed Bloom’s levels against Webb’s and reported that there is “no simple one-to-one correspondence” between the two. Their complaint about Bloom’s is the one every teacher has had at some point: many verbs “appear at multiple levels and do not clearly articulate the intended complexity.”

Here is the practical version. “Analyze this chart” is a Bloom’s analyze-level verb either way. If the chart is labeled, the trend is obvious and the answer is one sentence, you have written a level 2 task. If the student has to decide which of three charts is even relevant to the question and say why the other two mislead, you have written a level 3 task. Same verb. Different depth. The verb told you nothing.

Which is also why this matters for writing a target you can actually check. A target and a task at different depths make the target decoration.

Why isn’t depth of knowledge the same as difficulty?

Because difficulty is about how many students get it right, and depth is about what kind of thinking the question asks for. Those move independently. A level 1 question can have a 20 percent success rate and a level 3 question can have an 85 percent one.

Ask students to name the Secretary of State in 1952. That is pure recall — level 1 — and almost nobody will get it. Now ask a class that has spent two weeks on an argument to pick a side and defend it with two pieces of evidence from the documents in front of them. Level 3, and most of the room will manage it, because they have been taught to.

This is the most useful thing DOK gives a teacher and the first thing lost when a school adopts it. “Raise the rigor” usually becomes harder recall: more terms, more obscure terms, a longer list. That raises difficulty and leaves depth where it was. The Cognitive Rigor paper records a teacher saying DOK levels “do not correlate to her notion of ‘difficulty,’” which is the realization the whole framework is trying to produce.

It is also the honest answer to a question families ask at curriculum night. When a school says it is increasing rigor, the fair follow-up is whether the work asks students to think about more at once, or just to memorize more. Those are different promises, and only one of them is what depth of knowledge describes.

How do I assign a DOK level to a question I already wrote?

Read the task, not the verb, and ask what a student must do to earn credit. Five steps get you to a defensible level in under a minute, and the last one is Webb’s own tiebreaker.

1. Cover the verb. Read the rest of the prompt and ask what the student has to produce. If the verb is doing all the work in your judgment, you are classifying the wording rather than the task.
2. Ask whether the answer is already known to the student. If it is retrieval from memory or a procedure practiced to automaticity, you are at level 1 and no amount of wording changes that.
3. Ask whether there is a decision point. If the student has to choose a method, select which information matters, or take steps in an order they determine, you are at level 2 or higher.
4. Ask whether the student has to justify. If an acceptable answer requires reasoning that could be challenged — and if a different defensible answer exists — you are at level 3.
5. Ask how long it runs. Level 4 needs time, multiple sources or conditions, and usually work the student plans themselves. If it fits in a period, it is not level 4, whatever the project is called.
6. When you are stuck between two levels, take the higher one. That is Webb’s instruction, not a convenience: his guide says when unsure, “it is appropriate to select the higher of the two levels.”

At under a minute an item, a twenty-item quiz is about a fifteen-minute job — that is arithmetic from the step above, not a research finding, but it is the right order of magnitude to plan around. And if a department is going to use DOK at all, calibrate before you argue: have two people level the same five items independently, then compare. Webb’s own reviewers disagreed most on the line between level 1 and level 2, so expect that to be where your department lands too, and settle it with examples rather than with the wheel.

What does the research actually say about depth of knowledge?

Less than the training slides suggest. DOK has a strong record as an audit tool for alignment and almost no record as a classroom intervention. There is no body of controlled trials showing that teaching teachers DOK raises student achievement.

What the evidence does show is worth having anyway. Webb’s 2002 alignment study of three states applied the framework to state math standards and tests and set the bar at 50 percent — an objective passed if at least half the matching items sat at or above its DOK level. Most standards failed. In the one grade where item-level numbers are reported in full, 46 percent of items were at or above the objective’s level, 39 percent at it, 7 percent above it, and 54 percent below. Webb’s conclusion was that “more than half of the objectives under the standards required a more complex depth-of-knowledge than the corresponding items.”

Translated: the standards asked for more thinking than the test measured — the same gap that opens between a unit plan and the quiz at the end of it.

The honest caveats belong here too. Webb’s 2002 alignment study of three states reported that agreement between reviewer groups on DOK was weaker than agreement within them, and that most of the trouble was coding level 1 against level 2 — the two levels teachers also argue about most. Webb’s recommendation was more reviewer training, which is a polite way of saying the scale is not self-evident. None of that makes DOK useless. It makes a single teacher’s rating an opinion with reasons behind it, which is the right way to hold it.

Where does depth of knowledge go wrong in schools?

Four ways, all of them from treating a classification scale as a quality scale: the verb wheel, the difficulty mix-up, labeling students instead of tasks, and chasing level 4 on an ordinary school day.

The verb wheel is the most common and the most explicitly ruled out: Webb’s guide says the central verb alone is not sufficient to assign a level. A wheel hands you a level without reading the task, which is the error the framework exists to prevent.

Labeling students is the one that does real damage. The level belongs to the task. There is no such thing as a DOK 2 kid, and once a department talks that way, a scale built to audit a test has become a way to describe children. If a class is not managing level 3 work, the answer is supports you plan to remove on a schedule, not a lower ceiling.

The one an administrator should watch hardest is DOK showing up in walkthroughs. A level is a property of the task, and a five-minute visit catches whatever the task happened to be at that moment — which on any honest Tuesday is often level 1, because students cannot reason about content they have not learned yet. Writing up a teacher for a level 1 observation punishes the part of the lesson that was supposed to be there. If a building wants to use DOK, use it on assessments and unit plans, where the sample is the whole thing rather than a slice of it.

And the daily level 4 chase produces a fake. Level 4 means extended investigation across time and multiple sources. If it happened between the bell and the exit ticket, it was not level 4. The Cognitive Rigor teachers landed somewhere sensible on this: their working plan blended levels 1, 2 and 3 on assessments and reserved level 4 for in-class projects. That is the right shape. Most of the week is 1 through 3, and level 4 shows up a handful of times a year with the calendar cleared for it.

What this is actually good for

Here is where Clay comes down on it, as a grades 6–12 teacher rather than a researcher. DOK is a diagnostic, not a goal, and it is at its best pointed backward at work you already assigned: over the last two weeks, what did my students mostly do?

Pull your last five assignments and the last quiz and put a level on each item. Not to feel bad about the level 1s — recall is load-bearing, and nobody reasons about content they do not know — but to see the shape of the week. If every item is a 1 or a 2, the rigor problem is in the task design, not in the kids. If the unit plan promised level 3 reasoning and the test is all recall, you have reproduced Webb’s three-state misalignment inside one classroom.

That is also a cheaper fix than it sounds. Raising the depth of a question usually means removing something — the labeled chart, the pre-sorted evidence, the step list — rather than adding. Some of that is simply leaving students in the difficulty on purpose, for a stretch that has a floor under it.

One warning on that, and it is the thing students notice first. If you raise the depth of a task by taking away the labeled chart or the step list, say so out loud. A teenager who gets last week’s assignment back with the supports quietly stripped out does not read that as rigor; they read it as the teacher making it harder and not saying why, and the reasonable response to that is to stop trying. Name the removal, name what you are now asking them to do without, and tell them the difficulty is the point. Students will take a harder task on those terms. They will not take it on terms nobody explained.

What to do next

Take one upcoming assessment and level every item before you give it. If more than three-quarters of the points sit at levels 1 and 2, rewrite two items to level 3 by removing scaffolding rather than adding words — then check the result still matches what you told students the unit was about.

After that it is a habit, not an initiative. The next step is making sure the thing you are leveling is the thing you meant to measure, which is the difference between a check and a grade. And if you want the day-to-day version, the quick in-lesson reads are covered in checking for understanding.

Frequently Asked Questions

Sources

• Webb, N. L. (1997). Criteria for Alignment of Expectations and Assessments in Mathematics and Science Education. Research Monograph No. 6. National Institute for Science Education, University of Wisconsin–Madison, and the Council of Chief State School Officers. Full text (ERIC ED414305). Source of the origin of depth-of-knowledge consistency as an alignment criterion, the “number and strength of the connections within and between mental networks” definition, and the “full” and “acceptable” qualitative thresholds. Note that this monograph is about mathematics and science standards and assessments, not about classroom instruction.
• Webb, N. L. (2009). Webb’s Depth of Knowledge Guide. Full text. Source of the four level descriptors quoted here, the statement that DOK “describes the kind of thinking required by a task,” the warning that the central verb alone is not sufficient to assign a level, the “most commonly required to perform” standard, and the instruction to select the higher of two levels when unsure.
• Webb, N. L. (2002). An Analysis of the Alignment Between Mathematics Standards and Assessments for Three States. Paper presented at the American Educational Research Association Annual Meeting, New Orleans. Full text. Source of the 50 percent criterion, the 46 percent at-or-above item figure and its 39/7/54 breakdown, the “more than half of the objectives” conclusion, and the finding that between-group reliability on DOK was weaker than within-group reliability, concentrated on level 1 versus level 2. The item-level percentages come from one grade in one of the three states; the rest of the paper reports the share of standards meeting the criterion rather than item counts.
• Hess, K. K., Carlock, D., Jones, B. S., & Walkup, J. R. Cognitive Rigor: Blending the Strengths of Bloom’s Taxonomy and Webb’s Depth of Knowledge. Full text (ERIC ED517804). Source of the Cognitive Rigor Matrix, the “no simple one-to-one correspondence” finding, the criticism that Bloom’s verbs appear at multiple levels, the teacher quotations about difficulty, and the work-sample analysis. The paper carries no publication date and its sample size is reported inconsistently — 200 schools in the text, 205 in a figure caption — which is noted in the article rather than smoothed over.