Beginning in 2024, Polco adjusted the standard (default) methodology for The National Community Survey; rather than mailing hard copy surveys to randomly selected households, we now mail postcards to randomly selected households inviting them to take the survey online. We did not make this change lightly! We conducted our own research to first understand potential ramifications, and what we found supported this move toward digital survey data collection. Read more details about the study that led to this decision here.
Benefits
As broadband access and internet usage continues to expand, conducting surveys exclusively online grows even more attractive, for many reasons. Compared to traditional mailed surveys, digital data collection:
- Allows you to see survey responses in real time
- Reduces the time period for data collection (and thus, overall survey implementation)
- Reduces ever-increasing costs to perform the survey
- Helps build a reliable panel of subscribers that you can reach out to for future engagement
- Reduces paper waste
...all while having no discernible effect on respondents’ demographics or responses to survey questions, according to our research!
Research
From 2020 to 2022, Polco conducted nearly 100 implementations of The National Community Survey (The NCS) with a split sample methodology. In each of these communities, households were randomly selected to participate in the survey following our standard sampling process. A portion of these households were contacted using traditional mailed outreach (which included a hard copy paper survey), while the remainder were contacted using postcards with a link to the online survey (no hard copy). Our in-house data scientists then analyzed these results to answer three key questions about the two groups:
- Were the response rates different?
- Were respondent characteristics different?
- Were the opinions and behaviors of the survey respondents different?
While response rates were found to be lower for the postcard invitations (digital data collection), the demographic profile of respondents AND survey responses were both nearly identical between the two groups.
Note: These findings were presented in length at the 2022 national convention of the American Association of Public Opinion Research, and a more detailed written summary of the research and methodology can be found here.
Considerations
During this study, Polco heard a few concerns from customers around two main topics: lower response rates and residents with limited internet access or familiarity.
While you can anticipate seeing a lower response rate with digital data collection than with traditional mailed survey outreach, we will account for that with a larger sample size. Mailing postcard invitations to more households will help us continue to achieve a 4-6% margin of error at the 95% confidence level, maintaining the same precision and confidence in your results as in hard copy survey implementations.
To address the concerns around internet access and usage, we strongly recommend including a phone number and email address on the survey invitations specifically for technical questions and assistance. For Polco-led implementations of The NCS, Polco will provide this technical support to residents, to alleviate any burden on government staff. Additionally, if you’d like to offer printed surveys to accommodate those with difficulty accessing the survey online, you could consider making hard copy surveys available for residents upon request, with staff individually mailing and manually entering the paper responses online afterward. We’d be happy to talk through these considerations, and any questions you may have about methodology, as you plan your next survey project.
Overall, our research gives us the confidence to recommend digital survey data collection, despite these minor drawbacks. As the world continues to grow more technologically advanced, we must update our methodology accordingly to meet people where they are. By adopting this digital surveying approach (in which residents receive a physical, mailed postcard invitation but complete the survey online), we can combat decreasing response rates and continue to provide you with trusted, unbiased data about your community.
FAQs!
I still would prefer to mail a traditional survey packet for my community survey—is that still available?
While the postcard-only outreach will be our default methodology, Polco will still offer a traditional survey packet mailing (containing a hard copy survey) for those who prefer! However, we will pass on the additional costs for this outreach methodology.