Breakthrough — Disaster — Redemption:
Part 3 of the Prop 8 Story and the discovery of deep canvassing
Breakthrough
By September and October 2009, our team had tried many of the script ideas from the January 25 meeting. Most failed quickly (including my own). But far from being discouraged, our team was psyched. It’s like we had put together half of a jigsaw puzzle. We were getting closer.
Just as motivating: we were changing. We were becoming more present, more patient, more curious—comfortable enough to more often simply be ourselves at the door.
Before we quite realized it, we were increasingly treating the voters less like strangers and more like friends. When we disagree with a friend, or think they’re making a mistake, we usually aren’t dismissive or cruel. We try to make sense of what we’re hearing. We’re kind. With a friend, even when a conversation is uncomfortable, we catch our breath. We try to listen.
Even if we lose our composure, we don’t usually abandon the person. If we do abandon the person, we often regret it. We certainly remember and regret it if someone close to us abandons us.
I had not anticipated that at age 54 going door-to-door would push me to be a more decent person. But it did. I was trying hard to be my best self more of the time. I think that was true for many of us.
As we became better listeners, we also realized that many of our early ideas were built on a fatal assumption: that there was something we could tell the voters that would change their mind. We hoped maybe a fact could change them, such as “marriage confers over 1,000 rights that LGBT couples need as much as straight couples do.” Or maybe an argument: “how would you feel if you weren’t allowed to get married?” Or maybe a philosophy: “love is love.”
All of these made sense to people who already agreed with us. None were compelling on their own to voters not yet with us.
Yet one idea from January 25 partly succeeded. That was telling our coming out story, or a story about ourselves or someone we love. Voters listened intently when we were this vulnerable. Occasionally, they even moved. Sometimes, our story was enough—but not often enough for us to feel like we had fully figured out persuasion.
From our months of conversations, we had new hunches—and one was especially provocative because it reversed the idea of us telling our story. What if we asked each voter for their story? Who do they love? Are they married? How important is marriage to them? No-one had mentioned this idea at the January meeting. None of us had ever thought of this before, or heard of another group pursuing it.
But we repeatedly noticed that voters were most fully engaged and focused when they were talking, not us, especially when we got past their opinions on religion, tradition, kids, and politics. Those opinions were real, but not what voters cared about most. Their opinions paled in import compared to the stories they told us about their own life and its most important, emotional moments.
As these ideas went into the training and script, in rapid succession two of us had breakthrough conversations. Jackson Darling-Palacios, at this time just beginning to come out as transgender, had a great conversation with a voter, Lana, on October 17, 2009. Lana started the conversation explaining her vote against marriage equality:
Lana: I was just against it, just religious reasons.
Jackson: Religious reasons? Okay. So your faith is something that's pretty important to you.
Lana: Yeah, right.
Jackson: And I'm curious, are you Christian, or what's your...?
Lana: Yeah.
Jackson: Okay. And are you married, or have you ever been?
Lana: I'm separated.
Jackson: You're separated? Okay. Well, when you got married in the first place, why did you choose to get married?
Lana: For love.
Jackson: For love.
Lana: Okay, I see where you're going.
Jackson: Yeah, I mean, gay and lesbian couples...
Lana: Right, right.
Jackson: They want to get married for the same reason. And you know, I'm in a relationship myself, and I'm not ready to get married yet, but I really love my partner, Frankie. She and I can't legally get married, but someday we'd like to, and we can't. And it's really hurtful. We were both really hurt when Prop 8 passed, because it took away a fundamental, core thing that we really wanted in our lives, to really share a partnership with somebody and have rights and protections and really be a partner for the rest of our lives. So we are really affected by it.
Lana then talked about a friend of hers, a lesbian co-worker. As she thought about what marriage might mean to her friend, she began to change her mind.
Lana: Maybe I was looking at it from a religious point, but now I see where you’re coming from.
Jackson: Yeah, yeah—
Lana: I mean, it’s about being happy. Just think, and I’m not even with a guy, I’m not happy, so [laughs]. You know, happiness is really important, so maybe I’ll look at it different now.
Jackson: We’re definitely going to be voting on this again—
Lana: OK—
Jackson: So can we count on your to vote in favor of marriage for gay and lesbian couples next time?
Lana: I will. I’m a person to my word, I will.
Jackson: That's fantastic. I really love to hear that.
Lana: I will. You made me really look at it different. I did look at it different now talking to you. Everybody deserves to be happy.
My conversation with Alberto, the Mustang Man—detailed in my August 20 Substack post—happened even earlier, on September 26, 2009. These two videos—Lana and Jackson, Alberto and me—raised the whole team’s confidence about talking with people who didn’t vote with us on Prop 8, and also talking with voters whose race, ethnicity, and religious convictions were different than ours. As voters told us their stories, the conversations got better: more specific, more emotional, more surprising, and more focused on love.
Almost every voter, whether for or against us, knew that their own marriage was the best decision they ever made in their lives. This was true not only for married people, but also divorced people; even when a marriage didn’t work out, they remembered how it felt at the beginning, not just the end. Even people who had never married felt sure that when they did get married, it would be the best decision of their lives.
If we helped the voter notice the gap between the happiness they found when they got married and the same opportunity prohibited to us, that’s when they might revise their opinions—because they believed their lived experience much more deeply than their opinions.
We were learning something truly new. In January, we had assumed that to change a voter, we had to make a case the voter couldn’t deny. But voter persuasion turned out not to be something we do to someone else. Instead, voter persuasion was helping voters decide for themselves they want to change an opinion because it doesn’t align with who they want to be and how they treat other people.
Our emerging persuasion “recipe”
In our emerging approach—our “recipe,” as Jackson and Laura called it—we told a story, then elicited their story on “gay.” We asked each voter if they knew someone LGBT, how they knew them, how recently they’d seen them, what the relationship was like, and how much they cared about them.
In the second part of the recipe, we told and elicited stories on “marriage.” We asked each voter, were they married?; and how was it working out for them? Once we and the voters told our “LGBT” and “marriage” stories to each other, we realized we might be more similar than different.
The recipe was more than the stories, however; what also mattered was our reaction
Harder to grasp at first was that our success moving voters also depended on whether and how we responded to the voters’ stories. It wasn’t enough for us to show up and initiate a conversation; we also had to react to what the voter told us—and specifically react in the way that someone only can if they are listening not just to the words but also for the feelings connected to them. For example, we realized that when a voter uses a distinctive word, or says something with emotion, it’s important for us to repeat it back to them and invite them to say more about it. This way, if it was important to them, they would say more. At the very least, when we repeated back their words or feelings, they knew we were listening.
The key to success: to listen well, don’t rush
In future Substack posts, I’ll describe how we learned to listen better. But one key takeaway was that our ability to listen is at its best when we don’t rush the conversation along.
If you have ever canvassed, this may seem counter-intuitive. Most canvassers want to rush. They’re a little desperate to plow through the script before the voter terminates the conversation. The big fear for canvassers: rejection, a door slammed in their face.
The temptation to rush is compounded by typical campaign training that says stick to the script and make it as fast as you can. I’ll never forget the organizer who told me about a role play training he’d co-led. He put volunteers in two lines on opposite sides of the room. One line would play the canvasser; the other line, the voter. On a signal, the first person in each line would bound into the center of the room and the canvasser would immediately start talking. With a stopwatch, the trainer timed how fast the conversation was over. Everyone got to play the canvasser. What the trainers were trying to get across is that the “winner” was the fastest, usually clocking in in under a minute.
Dear reader, please think about this for a moment. In real life, if you want to have a substantive or memorable conversation—with a family member, a friend, even someone you’ve just met—you would never behave this way. “Getting the conversation over with fast” is self-defeating. It guarantees that you’ll extract the minimum from the interchange and be most quickly forgotten.
Consider also: have you ever changed your mind about something important in a minute?
I talk to canvass directors sure that voters will close the door after 1-2 minutes. But it’s much more likely that the voter will close the door because the fast-talking canvasser seems more like a robot than a person.
Our experience, once voters realized we were listening to them, was our conversations ran 10, 15 or 20 minutes. Voters stayed with us because they enjoyed being heard and engaged.
The easiest way I can paint the picture for you of what this looks like is to share one more of my conversations on marriage, with . . .
Feather Boa Man
In 2013, while talking one-on-one with voters in a conservative, middle-class Los Angeles neighborhood, I encountered a beefy, white guy in his 50s. He answered my door-knock with a smile. But when I said “gay marriage,” his eyes got big, his shock of white hair jiggled, his face got red. He took a hop toward me; I took a half-step back, ready for an earful.
Which he delivered. Gay people had just ruined his wedding anniversary. He and his wife had tickets to Disneyland to celebrate. It was “Gay Day,” the one day each year when the park hosts a large LGBT crowd. He and his wife had no idea “Gay Day” existed.
As they entered the park and walked down Main Street, three gay men strolled by. One wore an enormous feather boa. They made an enthusiastic racket about meeting Mickey or Goofy who, they speculated, might be gay.
I listened as all of the above burst out of him, along with the words “flaming” and “effeminate.” Then I said, “It sounds like you are still a little shaken up by the experience. Look, I’m a gay guy, and I hope I won’t disappoint you. I came here tonight without my feather boa.”
He looked at me—paused—and started to laugh. We both laughed pretty hard.
I said, “I can see why you felt surprised by your day at Disneyland. It sounds like you felt surrounded. Had you ever met a gay person before?” He didn’t think so.
I asked, “Did you talk with the gay people you saw?” He hadn’t.
I asked, “May I tell you a little about me? When I was 29, I met Dave Nimmons and we were together for 17 years, romantic partners. We are broken up now, but our anniversary was March 15. So anniversaries are a big deal, I know. They were a big deal for us.”
“And breaking up was hard. Dave and I found our way to a friendship, but it took a few years. The most surprising thing to me is that, though we had to break up, I am still grateful for our 17 years together. Before Dave, I was scared of being gay. I honestly wondered if I would ever fall in love and be loved.”
Feather Boa Man’s face was no longer red.
What anniversary was it for him? Also double digits, he said.
I asked, “How is marriage working out for you?” Suddenly he turned bashful. She was the best thing that ever happened to him. In his 20s, he was awkward, no good at dating. She was the first woman, he felt, who gave him a chance.
I asked, “How did that make you feel?” “Emotional” was his one-word answer. His tone of voice told me which emotions: gratitude and relief that he wouldn’t be alone.
I said, “I think it’s possible that, even though I’m gay and you’re straight, we have something in common.” He nodded yes. I asked, “Is gay marriage making more sense to you now?” It was.
2009 year-end results
2009 was a gigantic investment in learning.
Part of the investment was money, including payroll for our small crew of organizers. I am grateful that Lorri L. Jean, Darrel Cummings and the board of the Los Angeles LGBT Center thought our effort was worth it.
Part of the investment was time: the decision by so many good people to do something truly different and initially scary or at least unknown.
The most crucial investment was leadership. The people I’ve named individually as I’ve told you the Prop 8 Story not only got involved at the beginning; they also stuck with it. They continually stepped up. Their commitment to “radical hospitality” helped even first-timers quickly feel at home.
We were not done, of course. It took until 2012 for our side to assimilate all the lessons from Prop 8 and begin to win multiple statewide ballot measures on marriage. It wasn’t until 2015 that the Supreme Court, in Obergefell v. Hodges, ruled that the Constitution protects the right of gay and lesbian couples to marry.
But in 2009, we had begun. By the time we completed the first 5,000 conversations, we had tried a lot of the ideas for which we initially had high hopes. Overwhelmingly, they didn’t work. In the process, however—by trial and error—we came up with better ideas, and then best practices that helped us find common ground with more voters. We had discovered deep canvassing.
Of course deep canvassing is a funny thing to “discover.” The idea that one human being can affect another human being by taking time to connect with them goes back to when our species first began to grunt.
Yet we had come up with something decidedly not normal in 21st century American politics. Our 15-minute conversations were almost the opposite of the 1-or-2-minute, transactional quickies that conventional campaigns offer voters, if they meet voters in person at all.
Measurement of our results
First, our improved self-measurement; then, disaster; then, vindication.
For the Leadership LAB, the years 2009-13 were focused mostly on marriage. It was also the time when we began to get more accurate at measuring our own impact, thanks to two improvements.
First, at the beginning of the conversation, instead of simply asking voters if they were for or against marriage for LGBT couples, we added a follow-up question to help us identify voters with conflicting feelings.
Jeff Rosenfeld, one of our volunteers, came up with the idea of asking each voter to rate themselves on a scale of zero to ten, where 0 meant they were definitely opposed to LGBT marriage, 10 meant they definitely supported LGBT marriage, and the numbers in the middle indicated different degrees of mixed feelings. Jeff also came up with a great follow-up question as well, where we asked the voter: why is that number right for you? These two questions showed voters we weren’t rushing to put them in a box. Often, this was the first moment in the conversation where they took a breath and paused to think. They were no longer rushing either.
Second, by 2010, we asked the same 0-10 rating question not only at the beginning of the conversation but also at the end. We found that 20-30% of the voters with whom we spoke chose a different rating number by the end of the conversation; they moved on the 0-10 scale, usually closer to us.
Of course, that didn’t answer the question we cared about most: what did the voter think after we left? Did they stay moved? Or were they just being polite, going along with us to end the conversation on a friendly note? Or did some truly move but only temporarily? The last may sound strange, but many voter persuasion tactics change people’s minds only briefly; voters quickly revert back to their original point of view.
All these questions led us to begin our “Second Contact Phonebanking,” where we followed up with voters on the phone six to nine months after we spoke with them in person. The results were both exciting and sobering. We learned that about half of the voters who moved at the door stayed moved. They stayed changed. On the one hand, that meant our 20-30% movement rate on canvass day probably was 10-12% in terms of lasting impact.
Our team were grateful to have a more accurate picture of our actual impact. A 10% voter persuasion rate was still very high by social science standards; and extraordinarily high compared other campaign tactics, many of which obtain a 1% or smaller result.
Disaster
As our LAB leadership team grew increasingly interested in measurement, I met periodically with Professor Don Green. Now at Columbia, before that at Yale, Don and his colleague Alan Gerber are the fathers of a political science subdiscipline focused on accurate measurement of the value of campaign tactics. Their first book for a general readership, “Get Out the Vote!” attracted attention in 2004 when it found that some tactics (robocalls) had literally zero impact, and others (canvassing) had high impact.
As I kept Don apprised of our progress, he was generous with his time. He was curious about what we were doing and watched our videos with great care. He was skeptical; after all, many smart campaign people fail at voter persuasion. He was also impressed by our staying power and our desire to keep learning.
But if we wanted to know for sure whether we were having an impact, Don was unequivocal: we needed an independent researcher to measure us in a randomized controlled trial (RCT). RCT’s are “the gold standard” for judging whether a persuasion tactic—political scientists call any attempt to persuade a “voter treatment”—works, because they compare people who receive treatment with other people, otherwise similar, who only receive a placebo instead of the voter treatment.
Don suggested I talk with Michael J. LaCour, a Ph.D candidate at UCLA’s highly regarded political science graduate school. Don knew LaCour because he was a student in an experiments workshop that Don taught. Don recommended him because he was interested in persuasion research; energetic; and based in Los Angeles, where we were canvassing.
LaCour met with us, then came to one of our canvasses. Impressed, he promised to compare voters who received our “voter treatment”—deep canvassing on same-sex marriage—with voters who received a placebo conversation on recycling. LaCour was passionate about reducing anti-gay prejudice. We felt lucky he wanted to study us.
The next six months were a blur of hard work, as we canvassed frequently complying with the rules LaCour laid out to keep the RCT unbiased.
When LaCour told us his results, we felt amazing: he found we had an impact similar to what we would have guessed based on our Second Contact Phonebanking.
Don Green then got on the phone with me. He said, while the numbers were very encouraging, they were so surprising that we needed to have LaCour measure us a second time, to be sure it wasn’t a fluke.
We said OK and LaCour measured us again. His second study confirmed the results of the first. At this point, Don agreed to serve as a co-author to help LaCour write up the findings.
The resulting paper, published in the prestigious, peer-reviewed journal Science, was a sensation. Media coverage was extensive and generous. LaCour got a job offer to become an Ivy League professor.
Then two political scientists, David Broockman and Joshua Kalla, discovered the truth. They figured out that LaCour had never actually interviewed the voters we had canvassed. He therefore never measured whether we affected them. Instead, his numbers were made up.
Broockman and Kalla uncovered the fraud because they were in the process of trying to replicate LaCour’s work. As they prepared to measure our deep canvassing in Miami—conversations to reduce prejudice against transgender people—they were unable to replicate the protocols LaCour said he used to enroll a sufficient number of voters in follow-up measurement. When they tried, almost no-one enrolled.
Frustrated by LaCour’s slow responses to their questions, Broockman and Kalla contacted the firm that LaCour had hired to do the actual ongoing surveying of voters after we had canvassed them.
What a shock: Broockman and Kalla learned that the firm had never worked with LaCour; never heard of him; never received money from him; and LaCour’s supposed contact person at the firm had never been employed there. This spurred them to dive into LaCour’s data: there they found atypical patterns and serious problems.
Broockman and Kalla then did three remarkable things. First, they worked hard and fast with a colleague to write a paper laying out the irregularities they had discovered.
Second, they took that paper to Don Green. They came to him on a Friday afternoon; convinced him an investigation needed to be launched immediately; and so it was, on Monday morning. When confronted, LaCour had little to offer in his defense (a blanket denial that he had made up data). But LaCour admitted making up the funding; which meant there had been no surveys of the voters the LAB had canvassed. Don urged LaCour to join him in asking Science for a retraction. When he refused, the next day Don made the request himself.
Third, Broockman and Kalla reached out to me. I immediately supported the request for retraction. A day later, they went public. I personally called every reporter who had covered our work to explain our astonishment and distress that LaCour chose to commit fraud rather than measure us for real.
A second round of media coverage, comparable in scale to the first, now trumpeted the fraud and the decision by Science to retract the paper. Many at the LAB were crushed. We had trusted LaCour; now he had brought into question whether deep canvassing had any impact at all.
Vindication and redemption
Luckily, Broockman and Kalla figured out a new set of procedures to enroll voters in the Miami randomized controlled trial. The Leadership LAB team and leaders in SAVE Dade, the local partner partnering with us on the deep canvassing in Miami, worked incredibly hard to make the canvassing happen, even in the wake of the media coverage of LaCour. Two LAB organizers, Ella Barrett and Steve Deline, and SAVE’s organizer, Charo Valero, went above and beyond. Their leadership was inspiring; the project could not have succeeded without them. Justin Klecha at SAVE was similarly essential, never losing faith in the collaborative effort.
The result: when Broockman and Kalla crunched the numbers, it turned out the LAB’s deep canvassing results were actually a bit better than the ones LaCour had made up.
A third tidal wave of media coverage was capped by Broockman and Kalla’s paper being published in 2016, yes, in Science. Deep canvassing was again recognized for its proven value, this time on the basis of exemplary research. Subsequent studies of deep canvassing have continued to find it has high impact compared to other tactics that hope to change voters’ hearts and minds. (To read their full paper and the other academic reports mentioned in this telling of the Prop 8 Story, see the links at the very end of this post.)
For the LAB team, it’s impossible to adequately thank Josh Kalla and David Broockman for their honesty, their intellectual rigor, and their not giving up on us. Without them, the possibility of voter persuasion through deep canvassing might well remain unknown.
Instead, their core finding was that the LAB and SAVE Dade’s deep canvassing to reduce prejudice against transgender people had remarkable impact, and the impact lasted. Two findings stand out:
* 1 in 10 of the conservative voters we deep canvassed changed their minds and their minds stayed changed for the full 9 months that Broockman and Kalla followed up. This duration of impact is extraordinarily unusual, to the best of my knowledge far greater than any measured for other tactics. By comparison, the persuasive impact of TV ads, if any at all, evaporates in 2-3 days.
* The decline in prejudice against transgender people we achieved in 10-15 minutes was comparable to the decline in prejudice against gay and lesbian people that took over a decade.
You may be thinking, changing the minds of 10% of people against us doesn’t sound like much. And there’s no question I wish we changed 100%. But almost every other approach to persuasion struggles to change any minds at all in a way that lasts. 10% is a lot better than 0%; and if we start and persist with it, the cumulative impact year after year could help save the country.
Take-aways from The Prop 8 Story for the coming election
1. Anticipate non-stop disinformation. It is all our opposition has, and it is coming our way.
2. Some voters are more susceptible to disinformation than others. The best way I know to inoculate against disinformation, or to rebut it, is to visit with the voters most susceptible to it. This is particularly true with low-information voters and when disinformation reinforces a widespread existing prejudice.
3. Very few people, if any, are terrific deep canvassers the first time they try it. I sure wasn’t. Give yourself several tries to get the hang of it. Canvass at least three times before assuming it’s not for you.
4. If you try deep canvassing once and are not getting the hang of it, ask for help. Training and coaching make a huge difference for any of us.
5. Deep canvass with a team. Teams get better faster, particularly if they make it easy to ask for the help you need.
6. When you reach out to people you know—family, friends, neighbors, colleagues, classmates—and ask them to get involved with you in a campaign, often you discover that you have the ability to create a team.
7. Being part of a team will continually surprise you, particularly if everyone on the team is both a think-er and a do-er. Better ideas emerge when more people are doing the work and then evaluating it together. Being part of a high-functioning team creates a sense of community that accelerates learning and makes conversations increasingly satisfying.
8. If you try deep canvassing and start to get the hang of it, be prepared to be surprised by how many good people are out there. Non-voters, like conflicted voters, often feel disconnected from politics. They’re waiting for us, not even knowing that they are waiting for us. They need us. And we need them. That is fundamentally why deep canvassing matters.
Links to academic studies and articles referenced in this post
Betsy Levy Paluck commentary in Science on Broockman and Kalla’s 2016 study: https://static1.squarespace.com/static/5186d08fe4b065e39b45b91e/t/5706b7c12b8ddeb7c2e27f3a/1460058050153/Paluck2016.pdf
The Broockman, Kalla, and Aronow paper on the irregularities in LaCour (2014). https://osf.io/preprints/metaarxiv/qy2se/
Broockman and Kalla, 2016, “Durably reducing transphobia: a field experiment on door-to-door canvassing.” Science, 352(6282): 220-224. This was their full evaluation of our team’s work.
You can read their article without encountering a pay wall at https://www.ocf.berkeley.edu/~broockma/broockman_kalla_transphobia_canvassing_experiment.pdf
Also, two addition studies on deep canvassing co-authored by Joshua Kalla and David Broockman are well worth reading, though beyond the scope of the Prop 8 Story:
Their 2021 paper on different narrative strategies,at https://onlinelibrary.wiley.com/doi/10.1111/ajps.12657; and
Their 2020 paper on 3 additional field experiments at https://www.cambridge.org/core/journals/american-political-science-review/article/abs/reducing-exclusionary-attitudes-through-interpersonal-conversation-evidence-from-three-field-experiments/4AA5B97806A4CAFBAB0651F5DAD8F223.
Great post Dave and I appreciate the kind words. I’m glad we have had the opportunity to work together!