Review: Make It Stick, by Peter C. Brown, et al.

[Russ Allbery]

Make It Stick
by Peter C. Brown, et al.

Author:    Peter C. Brown
Author:    Henry L. Roediger III
Author:    Mark A. McDaniel
Publisher: Belknap Press
Copyright: 2014
ISBN:      0-674-72901-3
Format:    Kindle
Pages:     255

Another read for the work book club.

"People generally are going about learning in the wrong ways." This is
the first sentence of the preface of this book by two scientists
(Roediger and McDaniel are both psychology researchers specializing in
memory) and a novelist and former management consultant (Brown). The
goal of Make It Stick is to apply empirical scientific research to the
problem of learning, specifically retention of information for
long-term use. The authors aim to convince the reader that subjective
impressions of the effectiveness of study habits are highly deceptive,
and that scientific evidence points strongly towards mildly
counter-intuitive learning methods that don't feel like they're
producing as good of results.

I have such profound mixed feelings about this book.

Let's start with the good. Make It Stick is a book containing actual
science. The authors quote the studies, results, and scientific
argument at length. There are copious footnotes and an index, as well
as recommended reading. And the science is concrete and believable, as
is the overlaid interpretation based on cognitive and memory research.

The book's primary argument is that short-term and long-term memory are
very different things, that what we're trying to achieve when we say
"learning" is based heavily on long-term memory and recall of facts for
an extended time after study, and that building this type of recall
requires not letting our short-term memory do all the work. We tend
towards study patterns that show obvious short-term improvement and
that produce an increased feeling of effortless recall of the material,
but those study patterns are training short-term memory and mean the
knowledge slips away quickly. Choosing learning methods that instead
make us struggle a little with what we're learning are significantly
better. It's that struggle that leads to committing the material to
long-term memory and building good recall pathways for it.

On top of this convincingly-presented foundation, the authors walk
through learning methods that feel worse in the moment but have better
long-term effects: mixing practice of different related things
(different types of solids when doing geometry problems, different
pitches in batting practice) and switching types before you've mastered
the one you're working on, forcing yourself to interpret and analyze
material (such as writing a few paragraphs of summary in your own
words) instead of re-reading it, and practicing material at spaced
intervals far enough apart that you've forgotten some of the material
and have to struggle to recall it. Possibly the most useful insight
here (at least for me) was the role of testing in learning, not as just
a way of measuring progress, but as a learning tool. Frequent, spaced,
cumulative testing forces exactly the type of recall that builds
long-term memory. The tests themselves help improve our retention of
what we're learning. It's bad news for people like me who were
delighted to leave school and not have to take a test again, but
viewing tests as a more effective learning tool than re-reading and
review (which they are) does cast them in a far more positive light.

This is all solid stuff, and I'm very glad the research underlying this
book exists and that I now know about it. But there are some
significant problems with its presentation.

The first is that there just isn't much here. The two long paragraphs
above summarize nearly all of the useful content of this book. The
authors certainly provide more elaboration, and I haven't talked about
all of the study methods they mention or some of the useful examples of
their application. But 80% of it is there, and the book is
intentionally repetitive (because it tries to follow the authors'
advice on learning theory). Make It Stick therefore becomes tedious and
boring, particularly in the first four chapters. I was saying a lot of
"yes, yes, you said that already" and falling asleep while trying to
read it. The summaries at the end of the book are a bit better, but you
will probably not need most of this book to get the core ideas.

And then there's chapter five, which ends in a train wreck.

Chapter five is on cognitive biases, and I see why the authors wanted
to include it. The Dunning-Kruger effect is directly relevant to their
topic. It undermines our ability to learn, and is yet another thing
that testing helps avoid. Their discussion of Daniel Kahneman's two
system theory (your fast, automatic, subconscious reactions and your
slow, thoughtful, conscious processing) is somewhat less directly
relevant, but it's interesting stuff, and it's at least somewhat
related to the short-term and long-term memory dichotomy. But some of
the stories they choose to use to illustrate this are... deeply
unfortunate. Specifically, the authors decided to use US police work in
multiple places as their example of choice for two-system thinking, and
treat it completely uncritically.

Some of you are probably already wincing because you can see where this
is going.

They interview a cop who, during scenario training for traffic stops,
was surprised by the car trunk popping open and a man armed with a
shotgun popping out of it. To this day, he still presses down on the
trunk of the car as he walks up; it's become part of his checklist for
every traffic stop. This would be a good example if the authors
realized how badly his training has failed and deconstructed it, but
they're apparently oblivious. I wanted to reach into the book and shake
them. People have a limited number of things they can track and follow
as part of a procedure, and some bad trainer has completely wasted part
of this cop's attention in every traffic stop and thereby made him less
safe! Just calculate the chances that someone would be curled up in an
unlocked trunk with a shotgun and a cop would just happen to stop that
car for some random reason, compared to any other threat the cop could
use that same attention to watch for. This is exactly the type of
scenario that's highly memorable but extremely improbable and therefore
badly breaks human risk analysis. It's what Bruce Schneier calls a
movie plot threat. The correct reaction to movie plot threats is to
ignore them; wasting effort on mitigating them means not having that
effort to spend on mitigating some other less memorable but more likely
threat.

This isn't the worst, though. The worst is the very next paragraph,
also from police training, of showing up at a domestic call, seeing an
armed person on the porch who stands up and walks away when ordered to
drop their weapon, and not being sure how to react, resulting in that
person (in the simulated exercise) killing the cop before they did
anything. The authors actually use this as an example of how the cop
was using system two and needed to train to use system one in that
situation to react faster, and that this is part of the point of the
training.

Those of us who have been paying attention to the real world know what
using system one here means: the person on the porch gets shot if
they're black and doesn't get shot if they're white. The authors
studiously refuse to even hint at this problem.

I would have been perfectly happy if this book avoided the unconscious
bias aspect of system one thinking. It's a bit far afield of the point
of the book, and the authors are doubtless trying to stay apolitical.
But that's why you pick some other example. You cannot just drop this
kind of thing on the page and then refuse to even comment on it! It's
like writing a chapter about the effect of mass transit on economic
development, choosing Atlanta as one of your case studies, and then
never mentioning race.

Also, some editor seriously should have taken an ax to the sentence
where the authors (for no justified reason) elaborate a story to
describe a cop maiming a person, solely to make a cliched joke about
how masculinity is defined by testicles and how people who lose body
parts are less human. Thanks, book.

This was bad enough that it dominated my memory of this chapter, but,
reviewing the book for this review, I see it was just a few badly
chosen examples at the end of the chapter and one pointless story at
the start. The rest of the chapter is okay, although it largely
summarizes things covered better in other books. The most useful part
that's relevant to the topic of the book is probably the discussion of
peer instruction. Just skip over all the police bits; you won't be
missing anything.

Thankfully, the rest of the book mostly avoids failing quite this hard.
Chapter six does open with the authors obliviously falling for a string
of textbook examples of survivorship bias (immediately after the
chapter on cognitive biases!), but they shortly thereafter settle down
to the accurate and satisfying work of critiquing theories of learning
methods and types of intelligence. And by critiquing, I mean pointing
out that they're mostly unscientific bullshit, which is fighting the
good fight as far as I'm concerned.

So, mixed feelings. The science seems solid, and is practical and
directly applicable to my life. Make It Stick does an okay job at
presenting it, but gets tedious and boring in places, particularly near
the beginning. And there are a few train-wreck examples that had me
yelling at the book and scribbling notes, which wasn't really the cure
for boredom I was looking for. I recommend being aware of this
research, and I'm glad the authors wrote this book, but I can't really
recommend the book itself as a reading experience.

Rating: 6 out of 10

Reviewed: 2017-06-30

-- 
Russ Allbery (eagle at eyrie.org)              <http://www.eyrie.org/~eagle/>