The popularity of nonprobability online surveys is increasing. Since these survey data rely on samples drawn from commercial opt-in online panels with self-selected respondents, it is often questioned whether research based on such survey samples yields comparable results to research based on probability-based surveys. These rely on a random selection of respondents. This study compares a nonprobability and probability sample to answer the question of whether results from both samples are comparable in the analysis of voting behavior.
We use two surveys - a nonprobability and a probability sample survey - of the German Longitudinal Election Study (GLES) fielded prior to the German federal election 2017. We compare the two samples (1), regarding estimates' accuracy of characteristics with available external population benchmarks (e.g. turnout); (2), regarding their distribution in a set of over 100 variables, covering a large range of measures of political attitudes and behavior (e.g.polint) and (3), concerning differences in multivariate analyses through a multimodel comparison with voting intention as dependent variable.
We show that (1) the probability-based survey sample performs slightly better for estimating characteristics with available external benchmarks. Concerning the (2) comparison of distributions and the (3) multivariate comparisons, we find no conclusive evidence for the type of sample affecting variables' distributions, the model´s goodness of fit and their main effects.
This study provides new insights into the usage of nonprobability survey samples for studying voting behavior and exceeds previous research by applying a multimodel comparison and thus quantifying differences between probability and nonprobability survey samples.