How to run an exercise for maximum learning?
To make this concrete, let's say we want to use practice exam problems as a Courselets exercise prior to a Friday midterm exam. When should we make it due? What are the time-critical events for making it succeed?
- the problem: Within the crowded schedules of modern university students, the number of hours required to study for a midterm cannot (reasonably) much exceed the number of spare hours a student has in one day (8 hours, say). This produces an unfortunate corollary: if a student can study for a midterm in one day, when will they begin studying? For too many the answer is, the day before. But if their studying reveals misconceptions, that will be too late to fix them.
- best practice: If we assign it to start on Monday and be due Wednesday (for a student to receive its credit-for-completion), this make students start the practice exam by Tuesday. And once they begin, an interesting captive-audience effect kicks in: since they are obligated to spend this time on the courselet no matter what, they might as well treat this as their midterm study time (which they would not otherwise have started till as late as Thursday). In effect this "backports" the midterm's urgency from Thursday to Tuesday, while giving the courselet the special combination of safety + urgency that is essential for learning.
- Courselets.org automatically alerts the instructor when the first ten students have answered the exercise, so the instructor can immediately identify misconceptions in this small sample. Our data show that that will on average address 70-80% of errors in the rest of the class, who will find their errors addressed as soon as they answer the question (since the error models were entered into Courselets before they even answered the question!). And since only 50% of students on average have a misconception on a given problem, this protocol means the Courselet achieves 90% student coverage (correct, or error addressed) just based on the initial small sample — before even proceeding with any follow-up.
- impact multiplier / efficiency: based on Courselets data from large courses (700 students), the top 4 error models cover 400 students with errors = 100x impact multiplier. I.e. identifying an error in one student on average helps 100 more students.
In practice the 10% rule uses a simple weekly timeline:
- Monday: the practice exam courselet link is made available to students.
- later Monday: Courselets automatically notifies the instructor that 10 students have answered. The instructor writes Error Models for their mistakes, and the first 10 students receive back that individualized help.
- Tuesday: most students are completing the practice exam and immediately receiving help with the specific kind of mistake they made. After receiving each of these error explanations, the student rates whether that explanation was sufficient, or whether they want further materials on that specific point.
- Wednesday: all students must complete the courselet. The instructor now has a statistical prioritization of the frequencies of all misconceptions in the class, and for each one how many students (if any) are still confused and want further material to resolve it. The instructor writes resolution lessons for the highest priority confusions, and those students are automatically notified.
- Thursday: as students work through each resolution lesson, they re-rate whether it resolved their questions. As major blindspots are gradually overcome, the changing stats show the instructor what the remaining priorities are, and the instructor continues to add resolution lessons to address them.