Designing human judgement into an AI research tool
How a painstaking coding task became a working prototype, and why the most useful part of the tool is the step AI doesn’t do.
Caption: Sample of accept / reject / edit review of a proposed label. The top of the validation card on the right repeats the source phrase in the AI-generated text on left. The tool assigns sub-category label for the phrase and presents rationale for the label. Reviewer accepts, rejects or edits the information.
AT A GLANCE
The problem. No taxonomy existed that combined credibility cues and anthropomorphic cues in AI responses, and coding by hand was slow.
What I built. A series of prototypes, ending in a tool where AI proposes labels and a human reviews every one.
Still unknown. Whether the labels are reliable, and where the tool is genuinely useful.
The problem I didn't want to solve.
In late 2025, I was researching how conversational AI might influence human judgement. I was particularly interested in how anthropomorphic cues, which invite us to project human characteristics and motivations onto non-human systems, might interact with the credibility cues we routinely use to evaluate information from other people. I hoped this would help me better understand the features of AI-generated responses that make them seem credible, even when that credibility may not be warranted.
I wanted to understand these features so I could design mitigation strategies and solutions that would help people (especially me) use AI more safely and effectively. I believed then, as I do now, that AI is a powerful technology that has exciting potential to enhance our lives, and exploit our vulnerabilities. Much of today’s AI is based on prediction, and we humans are very predictable.
But I hit an obstacle.
There was established research on the cues people use to assess the credibility of human-generated information, but I couldn't find an equivalent taxonomy for conversational AI that incorporated both credibility and anthropomorphic cues.
So I had to develop one.
As part of my Master's research, I undertook an exploratory qualitative study of AI-generated responses, identifying and categorizing observable cues associated with sociality and expertise (3 prompt-response pairs, 3 rounds of coding, 5 coding rubric iterations…it took 2 months). The result was a provisional coding rubric and a conceptual framework I called the Believability Effect.
It was interesting research. But the manual coding was painstaking, and I couldn't help thinking there must be a better way.
Could I outsource the coding to AI?
I was spending hours identifying and labeling text. And I knew identifying patterns and classifying information are exactly the kinds of tasks AI could help with.
What if I could give an AI system my rubric and have it identify and label the cues for me?
I didn't want to outsource the interpretation or validation of the research. But I saw little value in personally doing every repetitive step.
So, I started experimenting.
My first prototype, BE Prompt Mirror, tackled a relatively straightforward mechanism: identifying where an AI response repeats or closely paraphrases language from the user's prompt. It was a useful place to start because the evidence could be compared directly across two pieces of text.
That led to a series of increasingly ambitious prototypes: BE Classifier → BE Cue Finder → BE Cue Detector → BE Diagnostic.
Each iteration explored what could be automated, what instructions the AI needed, and what the resulting output made possible.
Along the way, something unexpected happened.
When people saw the cues highlighted in AI-generated text, they became interested in what those cues might mean for their own conversations.
That suggested a different application. Rather than simply making research coding more efficient, perhaps cue detection itself could become an intervention, helping people notice features of AI responses they might otherwise overlook. The BE Cue Detector emerged from that possibility.
Four iterations of BE tools
BE Classifier: Labels a prompt-response pair and flags where the response mirrors the prompt.
BE Cue Finder: Applies the rubric to find sociality cues in an AI response.
BE Cue Detector: Shows anyone where an AI is being social, sounding expert, or mirroring them.
Designing human judgement into the process
The research application raised a different challenge.
I wanted AI to do the identification and initial labeling. But deciding whether those labels were appropriate? That I didn’t want to outsource.
Automating the work was useful only if I could still evaluate what the system was doing.
The current BE Diagnostic makes that division of labour explicit.
AI identifies candidate cues, assigns labels using the rubric, and provides a rationale. A human reviewer examines each proposed label alongside the original text and can accept, reject, or edit it.
The tool also supports CSV export, including coder information and review details, creating a record that can be used for subsequent analysis and audit.
This isn't simply a human approval button at the end of an automated process. Human evaluation is a defined part of the workflow, applied to individual classifications.
It's a small example of something I'm increasingly interested in: designing AI-enabled processes where human judgement has a clear purpose, rather than assuming that keeping a human somewhere in the loop is enough.
A working prototype, and more questions
The BE Diagnostic is now a working research prototype. It brings together the coding rubric, AI-assisted cue identification and labeling, human validation, and exportable results.
It hasn't been validated as a research instrument, and the Believability Effect itself remains an exploratory framework. There is more work to do before making claims about the reliability of the classifications or their relationship to how people actually judge AI-generated information.
The experiments have opened up possibilities beyond my original problem: Could a more mature version of the rubric and labeling process help researchers compare conversational AI systems? Could it inform vendor assessment or AI governance? Could human-reviewed cue datasets contribute to model evaluation or improvement?
And could the user-facing Cue Detector help people become more aware of the ways AI presents information, without making every interaction an exercise in suspicion?
Limits and open questions
I'm not yet convinced that developing the Believability Effect rubric into a mature evaluation instrument is the best next investment. That would require further research, validation, and a clearer understanding of where it would be useful.
The original problem was how to make a laborious research task manageable. Building the solution raised other questions. Where should we remove friction to make work more efficient? And where should we deliberately introduce it, and how much, to encourage more thoughtful human evaluation?
Sometimes the most interesting outcome of an experiment isn't the tool you set out to build. It's what the tool makes you think about next.