Understanding why surveys are marked with a red flag and what the response integrity feature indicates
When collecting data electronically, and especially in large amounts, it is common to encounter issues in the quality and integrity of the data from some respondents.
The response integrity feature addresses the concern of data quality control. As surveys are being completed, PDP monitors each response. If one or more of the integrity factors are met, we flag the survey for response integrity. The red flag icon displays on pages that list surveys and reports (see View > View All as shown below left and View > View Applicant Matches & Career Paths) and in report delivery emails.
Clicking on the survey row that has a flag displayed in the far right of column will open the detail panel that will indicate which factors were potentially problematic.
What will cause a survey to be flagged?
PDP studied the responses of 29,082 ProScan and Applicant ProScan Surveys (read full research study here) and identified five response integrity items. Here's a description of what's being checked by each.
|
Response Integrity Item |
Description |
Recommendation |
| Speeder | Abnormally short completion time | Highly questionable whether the respondent took the survey seriously; results may be invalid. |
| Distracted | Abnormally long completion time | Survey results are questionable; if you conduct a one-on-one feedback session, listen for confirmation of accuracy to confirm the validity of the results. |
| Patterns | Systematic response pattern (e.g., 12345) | Highly questionable wether the respondent took the survey seriously; results may be invalid. |
| Inconsistencies | Nonsensical response (e.g., 5 and 1 for the same trait) | Survey results are questionable; if you do conduct a one-on-one feedback session, listen for confirmation of accuracy to confirm the validity of the results. |
| Retry manipulation | Retaking a survey within a short period of time to get different result or a better score (e.g., applicant match percent score) | If the survey was reissued due to feedback response, then ignore the retry flag. If respondent retook the survey to get a different result, consider the results invalid. |
Note: Since Speeder and Distracted are both based on the time a respondent takes to complete the survey, a flagged survey may have one or the other but never both.
Note: Survey respondents do not receive the flag on their reports. It is only flagged to the survey sender and in the account.
Flagging a response for any of these issues initiates the next step in quality assessment: further investigation, consideration (of whether the survey should even be used), or resurvey administration.
So the next time you question whether an individual took the survey seriously or not, check for the Response Integrity flag to satisfy your curiosity.
Performing such rigorous response integrity evaluations helps ensure that the ProScan survey instrument remains a trusted and reliable source for trait assessment.
Where to find:
View > View All
or
View > View Applicant Matches & Career Paths