AI Questionnaire Review at Machine Speed

Bad survey questions produce bad data. Everyone knows this. But questionnaire review is tedious, subjective, and usually happens under time pressure. A 60-question survey goes through three rounds of revision, the project manager flags two issues, the methodologist catches three more, and the client insists on keeping a leading question because "we've always asked it that way."

This tool doesn't replace the methodologist. But it catches the stuff that gets missed when you're reviewing at 6 PM before a Monday morning launch.

What it flags

The Claude API analyzes each question for seven bias categories: leading language, double-barreled construction, loaded or emotional wording, assumption violations (assuming the respondent has done something they might not have), unbalanced scales, ambiguous phrasing, and acquiescence bias setups. For each flag, it explains the specific issue, rates the severity (minor, moderate, severe), and generates a rewritten version.

The rewrites are the part that saves the most time. It's one thing to know a question is double-barreled. It's another to have a clean alternative ready to go. The AI-generated rewrites aren't always perfect, but they're a solid starting point about 80% of the time.

I've been surprised at how well it catches assumption violations specifically. "How satisfied are you with our delivery service?" assumes the respondent has used the delivery service. That seems obvious when I point it out, but it's the kind of thing that slips past human reviewers constantly because we read questions with our own context, not the respondent's.

Tech stack

React 18.2 + Babel CDN. Claude API (Anthropic) handles the analysis and rewrite generation. The bias taxonomy is defined in the system prompt with examples for each category. API calls go directly to Anthropic from the browser. This is one of two AI-powered tools in the kit, along with the LOI Estimator.