Back in the primary we pulled together a team of volunteers to manually enter the election returns for the Republican Primary for Governor so that we could break down that data into usable chunks and draw conclusions in real time about how the race was unfolding. We will not be doing that for the general election next month, it's just not possible. Even if I wanted to I couldn't pull together the volunteers to perform that full real time analysis for even one race and to do it right it would need to be done for both the Governor's race and the Treasurer's race since recent polling shows that both of those races could go either way. (If you would like to do this yourself I'm giving away my tools for free below.)
Someone who is getting a live stream of election data needs to take the lead and format their output to give us the kind of useful data we need. My goal back in the primary was to demonstrate what could be done with all the election return data that comes in on election night if someone turned that data into something usable. These are the most important questions on election night:
- Who is leading and by how much?
- How much of the vote is in? How much is outstanding?
- Which candidates are overperforming/underperforming expectations? By how much? And where?
- Of the vote that is not yet reported which candidate does it favor?
Of the four questions above it has long been my frustration that news organizations really only attempt to answer questions 1 and 2 even though they have the means to answer questions 3 and 4 as well. They really only attempt to tell you the entire statewide totals and the number or percentage of the precincts reporting even though the means to provide much more information is easily available to them. You can't necessarily do this for every race but you can pretty easily put the statewide races into the proper context to either answer questions 3 and 4 or at least provide a pretty good guide.
News organizations can buy a live data stream from the AP, this is where most news organizations get the data they display on their websites on election night. I cannot do this, I tried, I asked for a quote to purchase this data and/or subscribe to the AP's service and I was explicitly told they would not allow me to be one of their customers because I was not a news organization. Clearly that's a stupid policy but it's not exactly shocking news that media people are bad at business.
But since news organizations have access to this data stream it's not much more work to format this data and run some basic mathematical calculations on the data to display the numbers in a usable format. They already have some developer write code to take the data out of the AP's data stream and then format it to display correctly on their election night web pages, just add a little extra effort to then display this info in a way that's helpful for people to understand.
News organizations I IMPLORE YOU use this as a guide and take these tools I am giving away for free and show us the data in a way that allows you to answer all four questions above instead of just the first two. Take the historical data found on this website (and below), take it, I mean literally take it, copy and paste it into your own election night websites or just use iframes and take the data directly from here. Let people compare the real time data with the historical data so they can draw conclusions about which candidates are beating/missing expectations and which candidates are likely to improve their position when the not yet reported election returns come in.
Campaigns, if you're looking to run your own election night vote counting operation and you want some help you can download the spreadsheets (and instructions) that we used in the primary here:
Download Election Night Vote Counting Spreadsheets and Instructions
About Illinois Statewide Elections
There are 102 counties in the state but there are 110 election authorities (102 county and 8 municipal). The eight municipal election authorities are Chicago, Aurora, Galesburg, Bloomington, Peoria, East St. Louis, Danville and Rockford. The county election authorities in the counties where these cities are located only cover the precincts outside of these cities. So for example if you wanted to know how a candidate is performing in Cook County you would have to add that candidate's totals from both the Cook County Election Authority and the Chicago Board of Elections. So to get totals by county you have to account for these 110 individual inputs of data and add them correctly to be able to display the 102 county totals.
If you have all of the inputs above everything else is pretty easy. You can then add up the totals by media market and/or regions. Media markets are comprised of whole counties so this is an easy calculation. You can see a map of media markets here.
You can also calculate the totals by region. Typically in a general election the City of Chicago will be just under 20% of the total statewide vote and the Cook County suburbs will also be just under 20% of the total statewide vote. The five traditional collar counties (Lake, McHenry, Kane, DuPage and Will) will account for just under 25% of the total statewide vote and the remaining downstate 96 counties will account for between 35% - 40% of the total statewide vote. These segments have distinct historical voting patterns from one another and the speed of these election returns can vary so it's useful to see it broken down this way.
Also, if you'd like you can break the vote down by region using the expanded 11 collar counties instead of the traditional 5, for more on that see here.
Most importantly, if you don't want to take the time to segment all of these counties into these various definitions (by media market, by region, etc.) just download my spreadsheet from the link above, each of the 110 election authorities is already defined for you.
The whole point here is that if you can segment the returns coming in by media market or region and you have historical data to compare it to with the same geographical boundaries you can compare this partial data to the historical data and begin to answer the questions listed in #3 above: "Which candidates are overperforming/underperforming expectations? By how much? And where?"
Displaying Data by Media Market
Broadcast TV ads for statewide campaigns are segmented by media market. When campaigns purchase ads they do not necessarily purchase ads in every media market and different media markets are likely to see different ads. Some markets are also more efficient than others, for example the Champaign/Springfield/Decatur market and the Peoria media market are both entirely contained within the state's borders whereas a market like St. Louis bleeds over into Missouri so you would be paying there to air ads to many people (Missouri residents) who cannot vote in the election. Voters who live in inefficient media markets are likely to see fewer ads than voters in more efficient markets. Since campaigns don't communicate with all of the state's voters in the same way or with the same frequency in their paid media it is useful to segment the election returns in a way that mirrors the segmenting used when broadcast TV ads are purchased.
Copy/Paste Historical Media Market Data
You can either copy/paste this data for your own use or you can use the iframes below:
| Entire Tab Control: | <iframe src="https://illinoiselectiondata.com/analysis/mmgentabs.php" width="575" height="350" ></iframe> |
| Dem Perf: | <iframe width='500' height='250' frameborder='0' src='https://docs.google.com/spreadsheet/pub?key=0Ar6RxFH3Vvz9dEtwYV9zWE5IakxkY2pqOGVITTNvNWc&single=true&gid=0&range=a1%3Aac13&output=html'></iframe> |
| Rep Perf: | <iframe width='500' height='250' frameborder='0' src='https://docs.google.com/spreadsheet/pub?key=0Ar6RxFH3Vvz9dEZWTlBYYnh4cF8tdFVvbTNBVHljSkE&single=true&gid=0&range=a1%3Aac13&output=html'></iframe> |
| Vote Share: | <iframe width='500' height='250' frameborder='0' src='https://docs.google.com/spreadsheet/pub?key=0Ar6RxFH3Vvz9dG91VHJxZDBKS3hHVzlGeXpmVml4N1E&single=true&gid=0&range=a1%3Aac13&output=html'></iframe> |
| Turnout: | <iframe width='500' height='250' frameborder='0' src='https://docs.google.com/spreadsheet/pub?key=0Ar6RxFH3Vvz9dHM1MkdxbEZpWDNFMmUzWHMxR2tqZXc&single=true&gid=0&range=a1%3Ag12&output=html'></iframe> |
Displaying Data by Region
Bruce Rauner has publicly stated a goal of 20% in the City of Chicago. Prior to 2010 the Cook County suburbs had been a perfect bellweather for the state as a whole, but then 2010 Kirk and 2010 Rutherford became the first two candidates to win statewide without winning the Cook County suburbs for at least the last couple of decades. In the last Governor's race the downstate vote was historically interesting. Each of these regions has a story to tell.
You can decide for yourself if you'd rather show the regions using the traditional collars or the expanded collars (or both), I'd probably recommend using traditional collars (for an explanation of the difference see here).
Region - Traditional Collars
Copy/Paste Historical Region - Traditional Collars Data
You can either copy/paste this data for your own use or you can use the iframes below:
| Entire Tab Control: | <iframe src="https://illinoiselectiondata.com/analysis/tradcollarsgentabs.php" width="575" height="275" ></iframe> |
| Dem Perf: | <iframe width='500' height='160' frameborder='0' src='https://docs.google.com/spreadsheet/pub?key=0Ar6RxFH3Vvz9dEx3ZmpmVnN0eGtMMW03RjlvNEk5Ync&single=true&gid=0&range=a1%3Aac8&output=html'></iframe> |
| Rep Perf: | <iframe width='500' height='160' frameborder='0' src='https://docs.google.com/spreadsheet/pub?key=0Ar6RxFH3Vvz9dHVyV0x1Y1RHUGRkLUV2eTVmRW1lM0E&single=true&gid=0&range=a1%3Aac8&output=html'></iframe> |
| Vote Share: | <iframe width='500' height='160' frameborder='0' src='https://docs.google.com/spreadsheet/pub?key=0Ar6RxFH3Vvz9dERUSF9yYzdKZzBTRGpUU1pkQk9NM2c&single=true&gid=0&range=a1%3Aac8&output=html'></iframe> |
| Turnout: | <iframe width='500' height='160' frameborder='0' src='https://docs.google.com/spreadsheet/pub?key=0Ar6RxFH3Vvz9dG5oRGRGc3hsakY2NHdLNDBIY1Bobnc&single=true&gid=0&range=a1%3Ag7&output=html'></iframe> |
Region - Expanded Collars
Copy/Paste Historical Region - Expanded Collars Data
You can either copy/paste this data for your own use or you can use the iframes below:
| Entire Tab Control: | <iframe src="https://illinoiselectiondata.com/analysis/expcollarsgentabs.php" width="575" height="275" ></iframe> |
| Dem Perf: | <iframe width='500' height='160' frameborder='0' src='https://docs.google.com/spreadsheet/pub?key=0Ar6RxFH3Vvz9dEd2ZmhSV3VCNnZqUTdFUGZTak92WlE&single=true&gid=0&range=a1%3Aac8&output=html'></iframe> |
| Rep Perf:: | <iframe width='500' height='160' frameborder='0' src='https://docs.google.com/spreadsheet/pub?key=0Ar6RxFH3Vvz9dGNPWVFDaWZoVVBCa3NzOXJJV1pucXc&single=true&gid=0&range=a1%3Aac8&output=html'></iframe> |
| Vote Share: | <iframe width='500' height='160' frameborder='0' src='https://docs.google.com/spreadsheet/pub?key=0Ar6RxFH3Vvz9dGxNVXlnRTItZ2RFUUpNeElrd3lqU2c&single=true&gid=0&range=a1%3Aac8&output=html'></iframe> |
| Turnout: | <iframe width='500' height='160' frameborder='0' src='https://docs.google.com/spreadsheet/pub?key=0Ar6RxFH3Vvz9dG1WTlRuOUtwRF95WnhNUjBXVW16cmc&single=true&gid=0&range=a1%3Ag7&output=html'></iframe> |
Projecting the Uncounted Vote
The last question that everyone has on election night but very few even attempt to answer has to do with the votes that haven not yet been counted/reported. Which candidate does it favor?
There are any number of really complicated ways you can attempt to project an answer to that question, I'm going to go over two pretty simple methods. However they each have their drawbacks.
Method 1 - Extend the Current Voting Patters
I used this method when we covered the 2014 Republican Primary for Governor last spring. For each of the 110 different local election authorities I simply applied the current vote ratio and extrapolated those numbers as if 100% of that area's precincts had reported. For example, if in one area 5 out of 10 precincts have reported and Candidate A had 100 votes and Candidate B has 80 votes the extrapolated Projected totals would be that Candidate A is projected to receive 200 votes and Candidate B is projected to receive 160 votes in that area. Do that for all 110 election authorities and add up the results to get the projected totals.
It's a very simple projection formula and it performs poorly very early in the night when only a few precincts have reported. It obviously gets better with more data. It's the sum of 110 separate calculations which is helpful but it also assumes an even partisan (or candidate support) distribution within each of those 110 election authorities which in some instances is an incorrect assumption. But it's a pretty basic calculation that is easy to perform and will get you a decent projection.
However as I learned the hard way on election night last spring early voting can cause some trouble in the calculation. There were many counties that were reporting only their early voting totals and listed zero precincts reporting for a few hours before any of their returns started showing up. This will cause some problems with your calculations so you'll need to account for this.
You can use this method to both project what the final results might look like based on current vote totals and also to predict which candidate is favored to win the support of the yet uncounted vote.
Method 2 - Use A Generic Partisan Baseline
Method 1 creates projections based on actual candidate performance, this method is based on a baseline of generic partisan expectations.
For example, let's say you started with the Cook Partisan Voting Index for each of Illinois' 18 congressional districts and assigned them a partisan score. You could then take a weighted average of each district's uncounted vote weighted by this partisan score. Let's say that the only uncounted precincts remaining in the entire state were in the 5th and 6th CD's. Let's say that 25% of the vote in the 5th CD was still not reported and 50% of the 6th CD. If the 5th is D+16 and the 6th is R+4 then you could use these figures and get a weighted average of the expected generic partisan makeup of the yet unreported vote. Obviously you wouldn't want to use this method if you'r expecting a strong 3rd party candidate vote.
The other problem with this methodology is the Cook Partisan Voting Index itself, you'd probably want to normalize it to more realistically match the current election. The Cook PVI combines the last two Presidential elections by congressional district and scores them relative to the national average. So if a district is listed as R+4 it means that the Presidential vote in that congressional district over the last two Presidential elections was 4 points more Republican than the national average. If you were to take an average of all 18 congressional districts in Illinois you'd come up with a Cook PVI for the State of Illinois at a little over D+8, this means that in the last two presidential elections the State of Illinois was about 8 points more Democratic than the country as a whole.
Well obviously there are a few problems with that, 1) this isn't a Presidential year election, it's an off year election and the electorate is going to be more Republican leaning than in Presidential years, and 2) in the last two Presidential elections Illinois elected a favorite son so that would artificially inflate the Democratic support. If you were going to try to use the PVI in this way you'd want to normalize it to offset both of the issues above.
Conclusion
Please, please, please give us better, more useful data on election night. Please. If you have any questions you can find my contact info here.
