You know the winners.
You know the spread.
How did the polls do in the final tally for the 2022 Pennsylvania Senate and gubernatorial races?
The folks at FiveThirtyEight say “Let’s give a big round of applause to the pollsters. Measuring public opinion is, in many ways, harder than ever — and yet, the polling industry just had one of its most successful election cycles in U.S. history. Despite a loud chorus of naysayers claiming that the polls were either underestimating Democratic support or biased yet again against Republicans, the polls were more accurate in 2022 than in any cycle since at least 1998, with almost no bias toward either party.”
In their minds, “the best way to measure a poll’s accuracy is to look at its absolute error — i.e., the difference between a poll’s margin and the actual margin of the election (between the top two finishers in the election, not the poll). For example, if a poll gave the Democratic candidate a lead of 2 percentage points, but the Republican won the election by 1 point, that poll had a 3-point error.”
So we’ll do the same.
First, a look at the U.S. Senate race won by Democrat John Fetterman over Republican Mehmet Oz and the polls over the final month.
Fetterman won 51.25 percent of the vote, while Oz captured 46.33 percent – a difference of 4.92%.
- Marist (Oct. 31-Nov. 2): Fetterman +6.0
- SSRS / CNN (Oct. 13-17): Fetterman +6.0
- Franklin & Marshall (Oct. 14-23): Fetterman +5.0
- Siena / NY Times (Oct. 24-26): Fetterman +5.0
- Monmouth (Oct. 27-31): Fetterman +4.0
- Beacon / Fox News (Oct. 26-30): Fetterman +3.0
- Trafalgar (Oct. 8-11); Fetterman +2.4
- Suffolk / USA Today (Oct. 27-30): Fetterman +2.0
- CBS / YouGov (Oct. 21-24): Fetterman +2.0
- Rasmussen (Oct. 19-20): Fetterman +2.0
- Fabrizio / AARP (Oct. 4-12): Fetterman +2.0
- Muhlenberg (Oct. 24-28): Even
- Emerson (Oct. 28-31); Oz +1.8
- Insider Advantage (Oct. 26): Oz +2.7
- Co/Efficient (Oct. 26-28): Oz +3.0
The overall average of 43 polls during the campaign gave Fetterman a +5.6 percent advantage, while the average of the last five polls showed Fetterman up +2.7 points.
Now, a look at the race for Pennsylvania governor won by Democrat Josh Shapiro over Republican Doug Mastriano and the polls over the final month.
Shapiro won 56.49 percent of the vote, while Mastriano captured 41.71 percent – a difference of 14.78%.
- Franklin & Marshall (Oct. 14-23): Shapiro +23.0
- Beacon / Fox News (Oct. 26-30): Shapiro +16.0
- SSRS / CNN (Oct. 13-17); Shapiro +15.0
- Marist (Oct. 31-Nov. 2): Shapiro +14.0
- Muhlenberg (Oct. 24-28): Shapiro +14.0
- Suffolk / USA Today (Oct. 27-30): Shapiro +12.0
- Fabrizio / AARP (Oct. 4-12): Shapiro +11.0
- Co/Efficient (Oct. 26-28): Shapiro +10.0
- Trafalgar (Oct. 8-11): Shapiro +9.3
- Emerson (Oct. 28-31): Shapiro +9.2
- Insider Advantage (Oct. 26): Shapiro +8.4
- Rasmussen (Oct. 19-20): Shapiro +3.0
The overall average of 36 polls during the campaign gave Shapiro a +10.3 percent advantage, while the average of the last five polls showed Shapiro up +13.1 points.
Looking at the differences between the 12 pollsters who surveyed both races and the two outcomes:
- SSRS / CNN: Fetterman (1.1) | Shapiro (0.2) | average difference 0.65
- Marist: Fetterman (1.1) | Shapiro (0.7) | average 0.9
- Beacon / Fox News: Fetterman (1.9) | Shapiro (1.2) | average 1.55
- Suffolk: Fetterman (2.9) | Shapiro (2.8) | average 2.85
- Muhlenberg: Senate (4.9) | Shapiro (0.8) | 2.85
- Fabrizio / AARP: Fetterman (2.9) | Shapiro (3.8) | average 3.35
- Trafalgar: Fetterman (2.5) | Shapiro (5.5) | average 4.0
- F&M: Fetterman (0.1) | Shapiro (8.2) | average 4.15
- Rasmussen: Fetterman (2.9) | Shapiro (11.8) | average 7.35
- Emerson: Oz (6.7) | Shapiro (5.6) | average 6.15
- Co/Efficient: Oz (7.9) | Shapiro (4.8) | average 6.35
- Insider Advantage: Oz (7.6) | Shapiro (6.4) | average 7.0
Among the national pollsters that only surveyed one race:
- Siena / NY Times: Fetterman (0.1)
- Monmouth: Fetterman (0.9)
- CBS / YouGov: Fetterman (2.9)
Now that the exam has been turned in, what were the new and improved grades given out to the pollsters that covered PA in 2022?
A+ – Siena / New York Times
A – Beacon, Cygnal, Marist, Monmouth
A- – Emerson, Muhlenberg, Suffolk
B+ – Co/Efficient, Public Opinion Strategies, Susquehanna Polling, YouGov
B – Insider Advantage, Rasmussen, Trafalgar
C+ – SSRS
C – Survey Monkey
Pollsters with a relatively small numberof polls receive a provisional rating rather than a precise letter grade.
A/B – Echelon Insights, Fabrizio, Lee & Associates
B/C – Blueprint, Franklin & Marshall, Impact Research, Phillips Andover, RABA Research
FiveThirtyEight’s ratings are letter grades that we assign to each pollster based on historical accuracy and transparency. (You can read exactly how it calculates pollster ratings here.) They’re one of many tools you should use when deciding how much stock to place in a poll.