A systematic review of publications assessing reliability and validity of the Behavioral Risk Factor Surveillance System (BRFSS), 2004–2011 External 2013, 13:49 While many of these studies look at particular topic areas, the annotated list of publications below provides information across topics and illustrates the future of the BRFSS as a reliable and valid source of information on health related issues: There have been numerous studies that have examined issues related to the reliability and validity of the BRFSS and the system’s ability to provide both valid national estimates, within state estimates and comparisons across states ( see bibliography). We expect them to know it’s being addressed and know that by selecting to work with PureSpectrum that they are choosing a platform that has made a commitment to data quality” Menig said.Ĭonnect with a team member or schedule a demo with us today to learn more about PureSpectrum’s commitment to data reliability.BRFSS Data Quality and National Estimates “At PureSpectrum, we don’t expect researchers to have to worry about data quality. All behavior affects a respondents’ PureScore to ensure optimum data reliability. Further, all respondent behavior on the platform, including but not limited to items like answer consistency and length of interview etc, are monitored on an ongoing, real-time basis. Once Marketplace suppliers are vetted and onboarded onto the platform, monthly performance evaluations are conducted to confirm that only high quality sample is being delivered to surveys. However, this is one of the built-in benefits of using the PureSpectrum Marketplace. Quality control in online surveys can be difficult if not impossible to ensure manually. Continued data validity and reliability remain essential to researchers. As the PureSpectrum system continues to adapt to changing respondent and fraud trends, it allows researchers to meet the changing needs of the industry. The model processes millions of transactions in real time, allowing it to learn ideal respondent behavior. PureScore, a respondent level scoring system, is driven by advanced Machine Learning technology. That’s the tug of war, and ongoing nature of it that requires the attention and the innovation to keep yourself protected” Menig said. “Just because you think you have stopped them, they are going to look for a way around what you’ve done. PureSpectrum has worked to address these issues and presented PureScore in early 2020 to lead the quality initiative in the market research industry. In order to stay ahead, quality-ensuring methods need to be constantly evaluated and improved. Fraudsters adapt and stay ahead of data quality initiatives, continuing to fool researchers and collect incentives. The morphing nature of fraud is evident in market research. “A lot of times within market research, people will use the analogy of a virus for data quality fraud and how it will mutate and become resistant to the techniques or treatments that have been developed against it” Mark Menig, PureSpectrum Chief Product Officer, said. This consistent monitoring ensures the reliability and integrity of the data collected on the PureSpectrum platform. We reached a point where we generated so much data that we had to not only scale up our way of computing but also the way of looking at respondent behavior on our platform” Vasudevan said.Īny inconsistencies or negative respondent behaviors are immediately flagged and penalized. “As PureSpectrum grew one of the things we saw was the kind of data we were generating. Respondents with a PureScore of 5 and under are blocked from participating in surveys while those with a PureScore above 5 are allowed to participate in surveys on our platform. The system will then assign them a score from 0 to 10. The model based approach to PureScore takes into account three things: the profile of the respondent, their past behavior, and their current behavior on the platform. PureScore is a machine learning driven model that is designed to evaluate individual respondent level quality on each and every transaction or interaction that a user has on the PureSpectrum platform over time.
#Data validity and reliability how to#
So we continuously think about how to ensure reliability of data” Sushma Vasudevan, VP of Data Science and Analytics, said. “One of the major challenges we face in online primary data collection methods is data reliability and preserving the integrity of the data collected.