Protecting your insights: how Askable ensures participant quality

At Askable, we understand that maintaining participant integrity is essential for delivering high-quality, actionable insights. As we expand globally and grow our participant panel, we proactively tackle the challenges that come with this growth. 

While most participants provide valuable feedback, some attempt to exploit the system. Compromised participant activities can skew research results and undermine the process. To ensure our platform remains reliable and secure, we’ve implemented robust measures to prevent such activities.

The impact of compromised activities on researchers and genuine participants

For researchers, compromised participant activities can lead to unreliable data and diminished insight quality.

For genuine participants, it means fewer opportunities and lower engagement, resulting in churn and lack of trust.

Common methods of compromised activities

Through extensive research, we identified several common methods used to exploit our system:

• Falsifying location information: using VPNs to disguise true locations.
• Automated bots: employing bots to complete surveys or tasks.‍
• Multiple accounts: creating numerous accounts from the same IP address to access more studies.

Our measures to enhance participant integrity

We employ a range of tools and techniques to combat compromised activities and enhance the quality and integrity of participants. 

Advanced detection tools

• Blocked locations and VPN detection:
  We block participants from unsupported countries and prevent access from participants using VPNs, which can disguise their true location. This helps us ensure everyone is connecting from a legitimate location, maintaining data accuracy.
• Advanced bot detection:
  Our system quietly analyses user behaviour in the background to distinguish between real people and bots based on interaction patterns. Unlike traditional CAPTCHAs, this process doesn’t disrupt the experience for genuine users.
• Multiple IP address detection:
  We monitor for multiple participants using the same IP address, which can signal that someone is trying to game the system by creating multiple accounts. Our system flags these instances and prevents these accounts from participating.
• Reconnecting with PayPal for enhanced security:
  To ensure that only genuine participants receive rewards, we require participants to re-connect via PayPal. This process leverages PayPal’s security infrastructure, helping us identify and prevent any compromised activity. This extra layer of security ensures that the right person gets paid, enhancing overall trust in our system.

Artificial Intelligence (AI) analysis tool

Our internal AI screener analysis tool is designed to detect patterns in how participants answer screener questions across various studies. 

By analysing these patterns, the tool can identify inconsistencies and detect participants who might be providing dishonest answers to qualify for more studies. 

For example, if a participant gives conflicting answers to similar screener questions over time, the AI flags this behaviour for further review. 

This proactive approach helps us maintain a high standard of data integrity by ensuring that only genuine, honest responses are considered.

Investment in resources

We’ve invested in a dedicated team whose primary responsibility is to ensure participant quality and promptly address any feedback. 

This team is equipped with advanced tools to effectively identify and remove participants who engage in undesirable behaviours, such as speeding through surveys, professional test-taking, and creating multiple accounts. 

Our tools are backed by industry-leading standards like ISO 27001:2022 and ISO 27701:2019 for data security. For more information, please visit the Askable Trust Centre. 

What you can do as researchers

While Askable has robust measures in place, researchers can also take steps to protect their research from compromised participant activity. 

The most effective strategy is to create highly effective screener questions.

Craft comprehensive screeners

Consider these tips when designing screener questions:

• Avoid yes or no questions: These are easy to guess.
• Avoid leading questions: These can guide participants to certain answers.
• Refresh questions regularly: Keep your screeners updated to stay ahead of imposters.
• Include ‘dummy’ answers: Add options that help identify inattentive or dishonest participants.

For more in-depth guidance, check out our blog post on common mistakes when writing screener questions and a help article on how to write effective screeners in Askable. 

Analyse and report issues

When selecting participants for your study or during the analysis phase, watch for signs of compromised data. 

Look for brief, shallow, or inconsistent responses which may indicate problematic feedback.

If you encounter such issues, promptly report them to our support team. 

You can do this by providing a participant rating or using our support and feedback options. 

Our team will swiftly investigate and take action, including replacing the participant or refunding the amount charged, while conducting an internal review of the compromised participant activity.