Exit Polling Report

We ran a total of five exit polls Tuesday, three in Wichita.  We were outside, so we were hoping for fair weather.  Although the morning started out damp and dreary, it was nice most of the day, albeit a bit chilly after dark.  Setting up to be operating at 6:00 and running until the close of polls made for an exhausting day although I managed a nap in the early afternoon.  Everything went fairly smoothly at all locations.  In SE Wichita, we gave away 10 dozen donuts before 9:00 am with candy after that to bribe voters to talk with us.  I had to make a run for more candy at 4:00, but no major impediments or problems at the S.E. location.

The most interesting thing that happened was regarding a young black woman who had declined to participate in our exit poll.  Later that evening, near to the polls closing, she was back.  Apparently her mama had voted there early that day and filled out our survey.  She had insisted her daughter return to do so. I was delighted!  That young woman has a mama who cares about her and about making her voice heard by voting.  I wanted to thank her and tell her she’s an awesome mom!  But I was too busy handing out survey forms to other voters.  We had excellent participation rates!

I took both Tuesday and Wednesday off work, and was able to spend all day counting ballots Wednesday.  I’m also an experienced survey counter, so I managed to get the 925 surveys organized and counted by the end of the day.

My volunteers are working together to get the other sites counted, but they have other things to do too.  Altogether, we have collected thousands of exit poll surveys.  Counts are continuing even as I write this.  That I want the surveys from each site to have two independent counts doesn’t make the task any easier.

I also need to verify the results we wrote down at the polling stations, but was informed today that they won’t be available until after the canvassing is complete, approximately a week to ten days from now.  I’m glad I asked my site managers to get the totals that night from their polling stations.  I can go ahead and work on my analysis, updating it with any corrections needed when the data is available.  But I’ll hold off publishing the results until I can verify the numbers.

I will share some general stuff from my initial numbers for my site with the caveat that these results are considered preliminary until the data has been verified.

The scanned paper ballot official counts and our exit poll results are close, with nothing falling outside reasonable statistical bounds on any of the races.

The voting machine counts and our exit poll results are not as close, with a couple of results that bear looking into, but I had three results flagged earlier.  One has already turned out to be a data input error on my part.  (That’s why I want to verify the numbers before publishing).

I will present the exit poll provisional ballot results though.  I’m reasonably confident there are no large errors in my counts and small changes won’t alter these results.

In general, people who had voted provisionally were more likely to have time to take our survey.  We had 79 out of 92 provisional voters fill out our survey for a response rate of over 85%, nearly 20% higher than the 66% response rate of voters whose ballots were counted that day.

rep-share-of-provision-versus-counted-exit-poll-ballots-countrysideIn our exit poll survey, I found that provisional voters were approximately 10% less likely to vote for republican candidates.  I think it is reasonable to take this along with the additional data from the other exit poll locations, as a measure of the effectiveness of Kris Kobach’s efforts to disenfranchise non-republican voters.  I have already contacted the league of women voters to see if this data will be of help with their lawsuit regarding his practices.

 

Analyzing Exit Poll Results

It’s important to state up front what data will be collected, what analysis to perform on that data and what constitutes evidence of a serious problem versus random errors in any poll of this nature.

Polling stations allow voters access to the results after the polls are closed when the votes have been tallied.  In Sedgwick County, they will have separate reports printed for the electronic voting machines and the scanned paper ballots.

We will also get a count of the number of provisional ballots collected from the polling location.  Theseballots will not be opened until the voter’s registration is verified and there will never be an official tally of the provision votes for polling location.  But we can look at the results we have for voters who submitted provisional ballots and compare them with the votes that were counted at the polling location.  If there are significant differences, this is evidence of the voter suppression effect of Kris Kobachs voter registration rules.

I have created a general data collection and analysis EXCEL spreadsheet.  Multiple precincts vote at each polling location and the results are reported for each precinct, not the polling location, so I’ve set up a spreadsheet to sum the numbers up and compute the appropriate probabilities.

I will be customizing this EXCEL file for each exit poll location in Kansas, but I am happy to share a general version of this worksheet with anyone who is interested in running an exit poll for their own area.  All you will have to do is input the official results and your exit poll results.    This is an example of of the output.

 

Example Data Analysis
Presidential race Chi-Squared Result:  NA
Candidates Exit Poll Official Results Binomial Probability
Clinton (D) 52 60 0.0638
Trump (R) 38 30 0.0530
Johnson (L) 6 6 0.5593
Stein (G) 2 2 0.5967
Other 2 2 0.5967
Total 100 100

There are two different analyses than can be used in this situation.  The chi-square test will give an exact probability that the actual results differed from what would be expected under the assumption of random chance.  EXCEL has this test as a built in function: CHISQ.TEST.  But the chi-squared test has minimum data requirements which were not met in this example, hence “NA” or Not Applicable as the result of this test.

Since the chi-squared test will not work for every set of possible data, I also show the individual binomial probabilities for each of the candidate.  The minimum probability from this set of five computations is a reasonable approximation to the exact computation using the extension of the binomial distribution and can be easily computed using built-in Excel formula BINOM.DIST.

How to interpret this:

We judge the probability of machine manipulation of the vote by evaluating the probability of our results assuming no manipulation of votes is occurring.  This is referred to as the “null hypothesis”.  All probabilities shown are made under this assumption.  If this probability is above 0.05 (5%), we can reasonably conclude that the differences between the machine vote share and the exit poll vote share are typical of random variation due to the normal errors in the process.

If this value lies between 0.05 and 0.001, raise an eyebrow and give the numbers for that race a little extra scrutiny and consider it in concert with the other exit polling results.

If this values lies below 0.001, that is evidence of fraud.  Personally, I would like to see a recount of any race with results that fall this far from normal.  But only a candidate can request a recount in Kansas.

In this example, I have contrived to show Trump with a questionably low # of votes in the official count compared to the exit poll results.  Hillary has a slightly elevated value.  But these results are not unexpected as the minimum probability of results this far off is above 0.05.

But if the other sites have similar values and they are all benefitting the same candidate, it would be concerning.  If 2 or 3 sites out of 5 show the same beneficiary of the differences, that’s reasonable.  But if 5 out of 5 sites show the same beneficiary, it’s evidence of rigging.

If we see multiple races with low odds and the same slate of candidates are benefiting, we have solid evidence of machine manipulation of our official votes.  If we see only the normal expected errors, then we have solid evidence it is NOT being manipulated.

While a single location and a single race might show evidence of manipulation, savvy cheaters will try to avoid this method of detection by establishing a maximum shift that falls beneath the 0.05 probability results.  But looking at multiple races and sites, we can establish whether even small shifts show evidence of cheating.

We can define a slate of candidates by party and check the probability of getting the results we got using a similar binomial analysis.  Under the null hypothesis of no manipulation, the probability of an error that benefits a candidate is 50%.   There are three races with candidates and five judges we are asking about, for a total of 8 results for each polling location.  Governor Brownback would like to see 4 of the 5 judges lose their jobs and replace them.  We can also presume he supports the Republican Party candidates for President, Senate and Representative.

We will have data from 5 different locations for a total of 40 random samples with approximately 50% probability.  (For example, let X be the number of errors that were the opposite of the Brownback administration preferred candidates. If we have 40 random samples as defined above, the probability of getting errors in the opposite direction of his preferred result is computed with the following excel formula:  BINOMDIST(X, 40, .5, 1)

If this value is extremely low (less than .001), we conclude that the Republican Party has unduly benefited and further investigation would be appropriate.

How to interpret the Provisional Ballot Data:

We cannot know the final count of the provisional ballots collected at a polling location.  They are polled at the county level and only those that are shown to be registered voters are opened and counted.   What we can do is compare the results of the provisional ballots with the other responses to our exit poll.   If there is a major difference between those asked to fill out provisional ballots with the automatically counted votes, we have a measure of the effect of the voter ID laws and if it made a difference to the outcome.

For each race, we can use the chi-square test if we have sufficient data.   Otherwise, we can use the binomial approximation similar to the one used to compare the official count to the exit poll survey results.

electioneering and instructs them regarding what they can and cannot do.  While permission is not required to run an exit poll, we do need permission from the property owner to set up a booth to collect our ballots and provide chairs and shade for our volunteers.  Mainly, we want everyone to know what we are doing to avoid any issues arising on on election day.

How to Run an Exit Poll Part 1

How to Run an Exit Poll Part 2

Creating an Exit Poll Ballot

Creating an Exit Poll Ballot

This is part 3 of my “How-to-Run-an-Exit-Poll-Series.

The exit poll survey ballot is important, but not complicated. The only question of interest, other than their ballot choices, is the method of voting.   Data will be available at the end of the day with separate totals for the machine cast votes and the scanned paper votes.  There will be no official count of provisional votes at this station, so we can only compare those votes to overall total for the polling station. But that comparison allows us to evaluate whether the giving people provisional votes amounts to a voter suppression tactic.

Since space on our survey form is at a premium, and because including that information makes their response less anonymous, I do not recommend including questions about age, race or gender.  Generally speaking, you want to keep the words to minimum.  (Not an easy task for me.)

Here is an example survey I have developed for exit polls in Sedgwick Co.  I included a short paragraph at the top because I feel it’s important to let people know why you want this information and reassure them that results are anonymous, just like their vote.

Sample Exit Poll Survey

The first question is really too long, but I wanted to be as clear as I can about this question.  In Sedgwick County Kansas there are three possible options:  A vote cast via electronic voting machine, a paper ballot that the voter feeds into a scanner for on-site electronic counting or a provisional ballot – a paper ballot that is sealed into an envelop to be counted later (maybe).

Asking about the specific races is straightforward.  State the office and then list the candidates.  Circling answers reduces the need for a blank or box to check.  It saves space on the page.

Staggering the answers for questions with more than one line of answers (ex: Pres) makes it easier to discern the voters intent.  When they are stacked one above another, the answer may easily become ambiguous.

Since a single polling location will have multiple precincts voting there, it’s a problem asking about races where different precincts will be voting  for different candidates.   Generally, I want to confine the questions to races that will appear on every ballot at the polling location.   On the other hand, my site managers for the SW Wichita location are very interested in the county commissioner races.  We arrived at the following:

Who did you vote for your County Commissioner Race? (Select one for District 2 OR  District 3)   –  sw-wichita-nov-8-exit-poll-ballot

I have hopes that we won’t get too many voters identifying their choices for both district 2 and district 3, but I expect we will get some.   OTOH, it’s the only question that would be spoiled and I’m reasonably comfortable in assuming that such mishaps are equally likely to occur regardless of which candidate they support.  I think we will get good data from this exit poll.

How to Run an Exit Poll Part 1

How to Run an Exit Poll Part 2

 

 

How to Run an Exit Poll Part 2

Here are more instructions with links to help people who want to run an exit poll.  This is all pre-election day stuff to do but doesn’t include developing your sample survey form.  I’m going to devote a separate post to that.  I’ll link to it when it’s up.  I’m also planning an excel template that people can download, input their exit poll results and official polling location results to get an output of the probability of the difference between the occurring by chance alone.

Training:  I highly recommend running a short training session with your volunteers.  I think 30 minutes is sufficient, an hour if it’s a large group.   Go over the  instructions,  let them see the supplies (ballot box, survey forms, etc.), and have each participant go through some practice interactions with their fellow students.  It also gives everyone a chance to get acquainted prior to election day.

Schedule:  You’ll need to prepare a schedule in advance and co-ordinate with the volunteers to make sure they can and will be there for their appointed hours.   Here is the example-volunteer-schedule I used in my August primary exit poll.  Notice that some volunteers are tasked with bringing in fresh supplies.  Everyone got a copy of this ahead of time, so they knew when they were working and with whom.  Always have at least two volunteers manning the exit poll.  This is a basic safety precaution.  If one volunteer has a heart attack, the other will notice and call 911.

Tally Sheet:  You need a form for volunteers to record the number of refusals. Here is what I used in my August primary exit poll.  The time categories related to the hours different volunteers worked.  I taped a copy of this to the table and had the volunteers record their refusals.  Some volunteers kept separate tallies for themselves when it was busy and then later recorded them onto that page, which is also fine.

Supplies:  You’ll need to arrange for all supplies to be on site, including refreshments.  I had different volunteers bring refreshments through out the day. Suggested Supply List

Costs:   While many of the supplies are things you may already own or can be borrowed at no cost, printing hundreds of survey copies to be filled out will cost money.  I am estimating costs for the sites I am coordinating at $50 – $100 per site, including refreshments.

Another option is an electronic app.   I’m not using it for my sites because the advantage of voter marked hand-counted paper surveys is the same as for paper ballots.  On the other hand, I have donations sufficient to fund the needed supplies for exit polls I am coordinating, so my out-of-pocket costs are minimal.  If costs are a constraint, this is an acceptable alternative for the 2016 election.

 Permission Slip:  It’s not strictly required, but in addition to notifying the appropriate person regarding setting up your exit poll at their location, having documentation of permission by that person isn’t a bad idea.  I phoned and introduced myself, sent the person an email, and picked up the permission-slip in person weeks ahead of time.  I kept it on site at the exit poll.  No one confronted me about my right to be there, so it wasn’t needed.  But if someone does, it’s nice to have it.

How to run an Election Exit Poll

I’m working to set up multiple exit polls here in Kansas in November.  I thought that people in other areas might be interested in setting one up themselves. You need just one dedicated individual along with a few additional volunteers working a few hours apiece can pull this off.  You may also need to expend from $50 to $150 on supplies like making copies of your surveys and refreshments to offer voters.

The dedicated individual is the exit poll site manager.  The additional volunteers only need to spend a few hours on election day soliciting voters to complete surveys.  Two people should be manning the exit poll booth at all times, just in case any emergency situations arise.  This post outlines what an exit poll site manager needs to do to run a successful exit poll.

The approach I recommend is called cluster sampling.  Each site provides an independent check on the accuracy of the official counts at that polling station. If we combine information from multiple sites, the data can provide excellent precision for determining whether the discrepancies found are reasonable and evenly distributed or if they show evidence of systemic bias, which would indicate problems with our votes having been counted accurately.

This approach means that you need to try to contact all voters at that location to request they complete an exit poll survey.  The reason for this is that the purpose of this exit poll is to validate the official results at that location.   It is not to make predictions prior to the close of polls.  It is not to analyze for demographic information afterwards.  It is to validate the official results.  By concentrating our efforts at relatively few polling stations, we can attain a higher level of confidence that the results of our survey are representative of the polling location.

The first thing to ask is method they voted.  In my location, there are three options – by machine, by a scanner paper ballot, or by provisional ballot.  At the end of the day, I get the counts of votes cast for each candidate by machine and my scanned paper ballot.  Provisional votes are not counted until a determination is made of whether that person was allowed to cast the ballot. The exit poll results for provisional ballots can be compared to the accepted ballots.  If significant deviations occur, that is a measure of the impact of voter suppression attempts, such as voter ID laws.

Before finalizing your design of the exit poll survey, you will need to select a location.  Contact your local elections office and get a list of all the polling locations.  Let them know you are planning a citizens exit poll and ask if they have any regulations or laws that would affect that.  In Sedgwick County, the only significant rule was that we could only approach people after they voted, not before.   There is also a law regarding distance for any electioneering, but as long as we only approach exiting voters, this would not be a concern.

Site Manager Duties:

The site manager is the point person for everything to do with an exit poll at polling place.  They will be there in the morning to set everything up and they will wait at the polling location to get the official results when the machines have finished printing out the records.  They will not need to be there the entire day, but they do need to be available if any problems arise.

Prior to Election Day:

  1. Select polling location:  Site managers need to consider the polling places available near their home, perhaps even scout the locations to make a determination about which they would prefer to exit poll.  You will need to contact the owner/manager of that location and inform them of what you will be doing.  Find out if they have any concerns and address them or refer them back to me.  Obtain written permission to be on their premises to conduct this survey when appropriate.
  2.  Finalize Survey:  While there will be three races applicable to all of Sedgwick County, because site managers decide what locations they will monitor, they have the option to add additional questions specific to their polling place such as state legislators or judges.
  3. Prepare Supplies:  The site manager will decide on and arrange for all supplies to be there.  Tables, Chairs, Refreshments, survey forms, ballot box, etc. Suggested Supply List
  4. Schedule volunteers:  We’ll need to meet together to accomplish this. I’ll keep a list of volunteers and we can discuss where volunteers are needed and when.  I’m also going to see if I can get some student help for the times before and after school, which is often the busy times at polling locations as well.

Election Day – It’s a long day, but this could be split up between two people, say a morning manager and an evening manager.   

  1. Set up:  The site manager arrives half an hour before the polling station opens.  They set up the exit poll booth, making sure everything is ready for the first voter of the day.
  2. Maintain:  The site manager is the person on call for any issues that arise.  But they should be close by, available to take care of whatever issue might arise.  Run out of survey forms?  The site manager will bring more.  Volunteer calls in that they can’t make it after all? The site manager either fills in or finds someone there who can.
  3. Close down:  The site manager will be responsible for securing the completed survey forms and counting them.  The site manager will ensure the booth area is cleaned up and all borrowed equipment is returned.
  4. Official Results:  The site manager will need to remain on the premises to collect the official results for that polling location.  Meet with election officials for your location sometime in the morning, letting them know you will doing this.  They should allow you to examine the results tape for yourself.  However, if they object, you can ask them to fill out your survey form for the machine results and the scanned paper ballots results from the printouts.  In addition, ask them to give you the total number of provisional ballots turned in for that location.

 

 

After Election Day:

  1. Count your results
  2. Publicly post both the official results and your exit poll results for your polling location or email me your results and I’ll post them on my site